Product usability evaluation

From WikID

What Is Product Usability Evaluation?

Example of a product usability evaluation (from student report)
Example of a product usability evaluation by means of Emo-cards, developed by Pieter Desmet (from student report)
Example of a product usability evaluation (from student report)

A product usability evaluation is an evaluation intended to validate the product-user interaction. Product usability evaluation, or usability testing helps us to understand the quality of your designs (ideas or concepts) according to usage. Usability is defined as the effectiveness, efficiency and satisfaction with which specific users achieve specific goals in a specific context of use. Product usability evaluation is primarily done by means of observation techniques. Users are invited to complete tasks while talking out loud, or discussing their motivation with the researcher, rather than showing users a rough draft and asking, “Do you understand this?”.

Setting up a usability test involves carefully creating a scenario of use tasks, or a realistic situation, in which the person performs a list of tasks using the product being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires can be used to gather feedback on the product being tested. The aim is to observe how people function in a realistic manner, so that you can see problem areas, and what people like. It is important to set up usability evaluations systematically, and to approach the evaluations as formal research projects.

An important aspect of product usability evaluations is verifying presumptions regarding the use of a product. Presumptions that are investigated in product usability evaluations are the product characteristics (materials and shapes) that provide the users with “hints” as to how to use the products. These product characteristics are also called use cues. Use cues are meaningful product characteristics that are given to products to show users what functionalities a product has and how these functionalities can be used. Some use cues are deliberately designed, and some use cues are discovered in the usability evaluation. One of the goals of product usability evaluations is to test designed use cues, and discover the unobvious use cues.

The elements of a product usability evaluation are the product, the respondent, the test setting and the type of evaluation. The product is often a prototype, either with limited functionality or almost full functionality. The choice of respondents depends on how they represent the user group. As product usability evaluations are very time-consuming, often a limited number of respondents is chosen. The test setting can be either a controlled environment such as a laboratory, or one in which users act in their natural environment. The type of evaluation could be self-completion reports, where users are asked to report on their usage of the product (for example by thinking aloud, or through retrospective interviews). Evaluation can also take the form of asking questions to users while performing tasks, or even measuring human characteristics (for example eye tracking).

When Can You Use Product Usability Evaluation?

In the design process, product usability evaluations can be conducted at several moments. The nature of the usability evaluations depends on the moment in the design process:

  1. Evaluation of the use of existing products, which typically takes place in the beginning of the design process to analyse existing, analogous products.
  2. Evaluation of simulated use of concepts, which typically takes place with the use of sketches of ideas and concepts, and with scenarios, role-playing, or storyboards.
  3. Evaluation of use of final designs, which typically takes place with threedimensional models that have a limited functionality. These types of evaluations take place during the design process.
  4. Evaluation of use of prototypes, which typically takes places at the end of the design process. These types of evaluations make use of almost fully functioning prototypes.

During usability testing, the aim is to observe people using the product in a situation that is as realistic as possible, so as to discover errors and areas of improvement.

How to Use Product Usability Evaluation?


Starting Point

The starting point of product usability testing is the need to investigate the usage of existing products or verify (test) the usage and ease of use (usability) of new product ideas and concepts.

Expected Outcome

The outcome of product usability evaluations with existing products is often a list of requirements with which the new product must comply. Product usability evaluations with new products result in a list of useful aspects and issues about the use of the new product and improvements that could resolve those issues.

Possible Procedure

  1. Determine the research objective.
  2. Describe the presumptions, in other words: in what way do the use cues designed by the designer help the user in using the product? Describe the presumptions very explicitly. Presumptions are not predictions, though!
  3. Formulate research questions.
  4. Design your research. Think about: what type of models are you using (scenarios, storyboards, prototypes), the research environment, make instructions, determine the type of evaluation.
  5. Do the observations. Record the usability evaluations.
  6. Analyse the results. You can choose to do a qualitative analysis or a quantitative analysis.
  7. Redesign the product on the basis of the results. Often improvements are suggested in the evaluation.
  8. Communicate the results.

Tips and Concerns

  • You may include a limited number of qualitative questions that will help inform future design research, but don’t let these questions sidetrack the users from their primary tasks.
  • Employ guerrilla testing techniques if money and time are limited. You don’t always need formal recruiting or testing facilities. Use your personal network to find unbiased people to test. Use a conference room as a test lab. Any testing you can do is better than no testing at all.

References and Further Reading

  • Roozenburg, N.F.M. and Eekels, J. (1995) Product Design: Fundamentals and Methods, Utrecht: Lemma.
  • Roozenburg, N. and Eekels, J. (1998, 2nd ed.) Product Ontwerpen: Structuur en Methoden, Utrecht: Lemma.
  • Van Raaij, W.F. et al. (1999) Product en Consument, Utrecht: Lemma.
  • Schoormans, J. and de Bont, C. (1995) Consumentenonderzoek in de productontwikkeling, Utrecht: Lemma.
  • Van Eijk D. (2004) Dictaat Ontwerpergonomie, Delft: Industrieel Ontwerpen.
  • Kanis H. (2005) ‘Observation as a design tool’, In: Dictaat Design Observation, Delft: Industrieel Ontwerpen.
  • Kanis H. et al. (1993) ‘Use cues in the Delft Design Course’, In: Dictaat Design Observation, Delft: Industrieel Ontwerpen.


Personal tools
Aspects & Domains