Availability and User Testing

Usability is defined as the ease with which a product, system, or service can be used by specific users to achieve specific goals with effectiveness, efficiency, and satisfaction, as stated in the ISO 9241-11 standard. This definition does not view usability as merely a matter of technical functionality; it also incorporates the user’s perception and context of use. Effectiveness refers to the accuracy and completeness with which users achieve specified goals; efficiency denotes the effectiveness of resources and effort expended in reaching those goals; and satisfaction expresses the absence of discomfort and the presence of positive attitudes during use.

User testing is a collection of methods that involve evaluating a product or prototype through realistic tasks performed by end users. User tests can be applied during the early, middle, and late stages of the design process, offering direct observation of how the target audience perceives the design and where difficulties arise. These tests aim to uncover challenges that users may encounter in real interactions, even when these issues seem intuitively obvious to designers or developers.

Types and Approaches to Usability Testing

Usability tests are generally divided into two main categories: formative and summative tests. Formative tests are conducted during the design and development process with the goal of identifying problems early and developing solutions. For example, the accessibility of color contrast on a website can be evaluated at early design stages using paper prototypes. Such tests are recommended to be conducted not only by developers but also by usability experts and real users.

Summative tests, on the other hand, are performed after the development process is complete but before the product or website is released for actual use. The aim here is to identify remaining issues and assess the product’s overall performance one final time. Failing to include summative tests early in the process or conducting them inadequately can lead to encountering more problems at the final stage.

In terms of approaches, four primary methods stand out: guideline-based, expert-based, user-based, and model-based.

• Guideline-based approach: Evaluates interface consistency and accessibility through checklists and standards. The Government Internet Sites Guide prepared for public websites in Türkiye provides extensive examples in this regard and is based on international standards.
• Expert-based approach: Involves usability experts reviewing the interface and making intuitive assessments. Jacob Nielsen’s Ten Usability Heuristics is one of the most widely used guidelines in this field.
• User-based approach: Involves real users participating in tests, with evaluation based on observations and feedback. Tools such as eye-tracking devices are frequently used in this approach.
• Model-based approach: Involves analyzing user behavior through cognitive or physical models. The GOMS model and communication models fall within this scope.

These approaches are not mutually exclusive; they are often used together in a complementary manner.

Objectives and Methods of User Testing

User testing is applied to identify issues in the user experience and determine where improvements to the design are needed. User testing methods include usability testing, surveys, A/B testing, focus group interviews, and beta testing.

• Usability testing measures how easily a product can be used to complete specific tasks. It can be conducted in moderated or unmoderated formats. In moderated testing, a facilitator accompanies the user and collects feedback. In unmoderated testing, user interactions are recorded and analyzed later.
• Surveys provide both quantitative and qualitative data about users’ experiences.
• A/B testing compares the impact of two or more alternative designs on users.
• Focus groups are used to gather qualitative feedback from specific user groups.
• Beta testing involves testing the final version of a product with users before its public release.

Outputs and Evaluation Criteria of Usability Tests

The main outputs of usability tests are identified usability problems and performance data. Problems are classified according to their number, severity, and type. For instance, in one study, problems were analyzed in categories such as content, page structure, usage flow, menu categorization, interactive elements, system status feedback, and aesthetics-visual design. Performance data are evaluated using metrics such as success rate and task completion time.

Studies have shown that inexperienced users often uncover more usability problems. In particular, inexperienced users working with low-fidelity prototypes tend to identify more issues related to content and page structure. In contrast, high-fidelity prototypes reveal more problems related to aesthetics and visual design. Experienced users generally complete tasks faster and with higher success rates.

Factors of Prototype Fidelity and User Expertise

The outputs of usability tests are influenced by the fidelity level of the tested prototype and the expertise level of the users. Low-fidelity prototypes, such as paper prototypes, are sufficient for testing core functions, while high-fidelity prototypes, which are interactive and close to the final version, facilitate the evaluation of details in visual and interaction design.

User expertise is also a significant factor. Inexperienced users provide valuable insights by detecting problems in usage flow and interface logic that may be overlooked by others, while experienced users deliver better results in performance metrics. One study found that the interaction between prototype fidelity and user expertise has a particularly meaningful effect on problems in the aesthetics and visual category.

Implementation Process

The general steps followed in implementing user testing are as follows:

1. Determining the product, prototype, or feature to be tested.
2. Selecting the testing method to be used (e.g., usability testing, A/B testing).
3. Selecting the target user group and identifying participants.
4. Preparing test scenarios and setting up the test environment.
5. Conducting the test, observing participants, and recording feedback.
6. Analyzing the collected data; problems are classified by type, severity, and frequency.
7. Reporting findings and integrating them into the design process.

Conclusion and Evaluation

User testing and usability testing are integral components of design and development processes. Formative tests help resolve issues early in the process, while summative tests reveal how well the final product meets user expectations. Guideline-based evaluations, expert assessments, user observations, and model-based approaches can be used individually or in combination. Factors such as prototype fidelity and user expertise play a crucial role in interpreting test results. Low-fidelity prototypes offer cost and time advantages in early stages, while high-fidelity prototypes provide detailed and realistic feedback. Inexperienced users may uncover more problems, whereas experienced users tend to achieve better performance metrics.

In conclusion, usability and user testing constitute a systematic and objective set of methods that support design decisions with data throughout the product development cycle. Tests conducted with proper planning, appropriate participant selection, meaningful scenarios, and careful analysis are indispensable for improving user experience and enhancing product success.