Product concept evaluation

From WikID

What Is Product Concept Evaluation?

A product concept evaluation is a type of evaluation in which the product concept developed so far is reviewed by the user group. Generally, these evaluations are aimed at selecting or optimising product concepts on the basis of the preferences of the user group and other stakeholders. The product concepts that are evaluated can have different forms (descriptions, drawings or prototypes). Typically, these evaluations take place in a controlled environment, where a panel of people judges product concepts based on a list of predetermined issues. These evaluations serve different purposes: concept screening, concept optimisation, and go/no-go decisions (Schoormans and de Bont, 1995).

Example of product concept evaluation by means of a design sketch and a scenario/storyboard (Nomad/Zilver Product- ontwikkeling, 2001)

Concept screening is aimed at selecting worthwhile product concepts. It is necessary when a large quantity of product ideas or product concepts has been generated. From these product ideas and concepts a selection has to be made for further development. Often, it is experts (managers, engineers, marketers) that are invited to do a concept screening rather than representatives from the user group, because it often involves evaluating product ideas and concepts in light of the formulated requirements.

Concept optimisation is aimed at determining which aspects of product ideas and concepts need further improvement. These tests are not directed at judging the total concept, but rather parts or elements of product ideas and concepts. The assumption is that preferred aspects or elements of the individual product concepts can be connected with each other, yielding a concept that is regarded as optimal.

Product concept evaluations for go/no-go decisions are aimed at validating important design decisions. These decisions often involve the choice between two or three product concepts. Designers can make decisions based on the programme of requirements, but sometimes it is necessary to have these decisions validated by the user group.

Types of concepts, selection of respondents, types of evaluation

Example of a mock-up (student work)

The types of concepts that you can use for product concept evaluations are the following:

  1. Textual concepts: descriptions of the product idea, which generally consist of a description of what you can do with the product idea. There are roughly two forms of textual descriptions: a scenario of how a person can use the product, or an enumeration of the aspects of the product idea.
  2. Pictographic concepts: visual representations of the product ideas. Depending on the design process, these visual representations are highly detailed visualisations or simple representations. Since sketching and visualisation are so important during the design process, pictographic concept evaluations are most common. In recent years, computer graphics have enhanced pictographic concept evaluation through easy manipulation of the perspective of the visualisation.
  3. Animations: moving visual representations of the product idea. Thanks to computer graphic software it has become quite easy to make a simple animation of how the product can be used in a particular context.
  4. Mock-ups (dummies): three-dimensional, tangible representations of the product idea. Mock-ups are a kind of prototype that only shows the external (form) characteristics of a product idea.
Example of a three-dimensional model used in a panel evaluation (student work)

The selection of respondents is an important aspect of product concept evaluations. Respondents that are invited belong to one or more of the preformulated user groups. You can make a selection based on socio-cultural characteristics or on demographic characteristics. An important issue to be taken into account is the respondents’ level of knowledge of the product category. To assess this level of knowledge, you could simply ask respondents about their experiences with similar products. Another important issue when selecting respondents is related to psychological aspects such as tolerance and innovativeness. Questions that are important are: how tolerant are the respondents towards new products and new situations? How innovative, or conservative, are the respondents? Such psychological aspects have a big influence on the results of the product concept evaluations.

Example of an animation; a car-simulator (student work)

Different types of evaluations can be used for product concept evaluations. One of the commonest methods used is the personal (individual) interview. Another form can be focus groups, or discussion groups. In focus groups, a product concept evaluation takes place with a small group of people, and has the form of a group discussion. Product concept evaluations are structured according to preformulated lists of questions. In the evaluation of the product concept, the respondents are asked about their judgments. Respondents can give their judgments using rating scales, or ranking scales. When rating product concepts, respondents attribute scores to several aspects of the concepts. When ranking, respondents are asked to order the concepts according to their preferences. Product concept evaluations often take place in a controlled environment such as a laboratory. The reason for this is to ensure that there is as little distraction as possible. The evaluations are recorded using video and audio equipment. Often questionnaires are used to capture the evaluations.

When Can You Use Product Concept Evaluation?

Product concept evaluations take place throughout the design process, based on the purpose of the evaluations. Concept screenings usually involve large numbers of product ideas and concepts, and therefore are more frequent in the beginning of the design process. Concept optimisation takes place near the end of the design process, when aspects of the concept need to be improved and optimised.

How to Use Product Concept Evaluation?


Starting Point

The starting point of a product concept evaluation is a number of concepts to be judged (with a minimum of two), and a reason for conducting the evaluation. The reasons determine the type and purpose of the product concept evaluation.

Expected Outcome

The expected outcome is a validated choice between a number of concepts in case of a concept screening or a go/no-go decision, or a better understanding of what aspects require improvement/optimisation.

Possible Procedure

  1. Describe the goal of the product concept evaluation.
  2. Determine what type of product concept evaluation you want to conduct.
  3. Gather or create the appropriate concepts for the evaluation.
  4. Create a plan for the product concept evaluation. This plan should include: the goal(s) and type of evaluation, a description of the respondents, questions you want to ask the respondents, aspects of the product concept that need to be evaluated, a description of the test environment, the means of recording the evaluation, a plan of how you are going to analyse the results.
  5. Search for and invite respondents to the evaluation.
  6. Set up the test environment, including recording equipment.
  7. Conduct the concept evaluation.
  8. Analyse the results, and present the results concisely, using either a report or a poster.

Tips and concerns

  • Make sure that you search for a valid representation of the user group when inviting respondents (don’t forget to provide them with some form of compensation).
  • Make sure you structure the evaluation systematically with the questions you want to ask the respondents.

References and Further Reading

  • Roozenburg, N.F.M. and Eekels, J. (1995) Product Design: Fundamentals and Methods, Utrecht: Lemma.
  • Van Raaij, W.F. et al. (1999) Product en Consument, Utrecht: Lemma.
  • Schoormans, J. and de Bont, C. (1995) Consumentenonderzoek in de productontwikkeling, Utrecht: Lemma.


Personal tools
Aspects & Domains