Testing and validation of the web portal design
The first phase of user testing, which involved aesthetic evaluation of static images of the portal’s homepage, took place throughout July. It was a crucial task aimed at validating the visual development of the front end. This task was part of Work Package 7 and was carried out by HATCH consortium member Eurecat.
In order to provide valuable data reflecting the opinions of potential HATCH web portal users (as exemplified by the User Personas created), Eurecat carried out the following:
identification of relevant evaluation metrics, derived from the requirements and use case definitions
definition of methodologies and the general testing strategy
experimental design of the lab tasks and questionnaires
realization of the piloting tasks
evaluation of perceived aesthetics of the homepage
Two types of evaluation
In broad terms, two different kinds of evaluation of the proposed web portal design were carried out.
Layout and design tests
These tests involved look and feel questions. The goal was to test the aesthetic perception of a given design. The tests were based on self-reported evaluation of aesthetic appeal (with both open and structured questions) and visual tests such as a first-impression test.
Effectiveness and efficiency tests
These tests involved having participants execute different actions identified as relevant in the use case definitions and requirements, registering how well they did and how long it took them to complete each task. They also included self-reporting of satisfaction and evaluation of navigation, information complexity and interactivity.
The first web portal wireframes made it possible to test and validate the layout and design. Effectiveness and efficiency were also tested, although these factors will become more relevant towards the end of the project, when the actual functionalities are ready. The general strategy involved comparing HATCH to well-established websites that are similar in scope and/or thematic content. User requirements and use cases were applied to extract human factor requirements.
Later on, the web portal’s visual design (colours, fonts, headlines, etc.) was tested. The participants represented a broad target audience, including professionals, non-professionals and students, both male and female. The results can be taken as evidence that HATCH’s visual design fully meets the desired aesthetic standards. It is perceived as clear, clean and visually pleasing – key aspects in making the site attractive, especially for non-professional users. Evaluations of design and colouring, as evidenced by the first impression tests, also put Hatch at the level of top-notch websites, especially when compared to those with similar thematic content.
The test results show that the web portal’s visual design meets the expectations of the pre-defined User Personas. The needs and values of the stakeholders were well understood, translated and implemented within the design