The Science of Product Testing

Dec 24 2011

Market research and product testing

The history of product testing is long and can trace its path back to many diverse fields along a similar timespan to that of market research (back to the 1920s when the science was first introduced to manage product quality). Sensory science developed at a fast rate in the 1950s and 1960s, when different techniques for the ‘descriptive analysis’ of products were developed, starting with the flavor profile method, in the fast developments of the consumer goods industries which followed the end of the second world war. The science was boosted by the publication of Stevens’ landmark textbook on psychophysics in the mid-1970s and has come a long way in the last 50 years.

Sensory analysis is now a huge industry in itself, although the majority of product testing, especially that by the mainstream market research industry, continues to have much to learn about the science of product testing. Most product tests continue to focus on short-term tactical questions, comparing two products (and sometimes focusing on one product only) to provide very limited feedback on performance which is always limited by the narrow frame of reference of the research. Such testing also relies on consumer feedback on product characteristics, when consumers are generally poor at articulating reasons for their preferences (sensory panels are much better for this).

There’s much more to product testing

Marketers and researchers spend much time and money ‘segmenting’ the market with respect to sometimes abstract attitudes, beliefs and associations which, in my experience, are often difficult to implement in terms of specific product strategies and targetable audiences. Consumers differ in their tastes for product characteristics, and more importantly seek different performance from products according to the specific context of use (the ‘job’ that needs to be done for a specific occasion).

With limited product testing, which focuses on finding a single ‘optimum’ product and compares performance with a limited selection of other products (benchmarks), brand leaders become ‘bland leaders’ converging on an average product profile which is often only optimum for an imaginary average consumer, but actually meets the needs of no one. Another failing of such testing is that it usually relies on consumers to provide feedback on product characteristics, something most consumers are very poor at doing (we know if we like something, but cannot usually say why).

Building for the future

Strategic product testing takes a different approach, testing according to a ‘planned’ approach which covers a range of possible options sometimes outside the current confines of the product category (and often using experimental design approaches). With this systematic approach to ‘designing’ test options and surveying the potential landscape, comes big opportunities to map product development strategy for years to come. It also removes the need to ask consumers the reasons for their preferences, as the description of the products and differences between them is already clearly defined.

Firstly, with sufficient sample size, segmentation can reveal the range of different ‘jobs to be done’ which consumers experience and identify potential opportunities in the market place where no product currently meets demands. Malcolm Gladwell discussed this approach in some detail in a famous TED talk (link here) about spaghetti sauce. Do you like thin sauce, chunks or spicy? This is only possible when you look outside the current product space and go beyond simple comparisons with immediate competitors. Such segmentation can help create a picture of the different people as well as different products, enabling identification of key ‘drivers’ of liking in a category.

Secondly, the data from such an exercise enables the construction of detailed ‘models’ of consumer behaviour, which can provide a reference point for years to come in mapping the territory of the category and where short and long term opportunities lie. Such modelling can not only incorporate product features and consumer liking, but can link this information to business variables such as product cost and nutritional profile. This means that the model can answer a wide range of potential questions:

  • which is the optimum recipe for my product for target group X?
  • if I had to remove the sugar from this product, what would the optimum profile look like, and how could I make an alternative which was most similar to what I currently have?
  • if the cost of sugar increases, how can I maximise product acceptance within the constraints of a fixed production cost?

I look forward to the day when such approaches become more standard across all industries. Ultimately, the product itself is the most important asset any company owns and deserves a strategic approach as much, if not more so, than the branding which surrounds it.

[Note: although sensory science has been dominated by the food and drink industry, the ideas discussed here apply equally to the evaluation of any kind of brand experience.]

Product optimisation in three dimensions!


Psychophysics: Introduction to its Perceptual, Neural and Social Prospects by S.S. Stevens (1975)

Sensory Evaluation Techniques by Meilgaard, Carr & Civille (2006)

Sensory Evaluation: A Practical Handbook by Kemp, Hollowood & Hort (2009)

‘The 21st Century Development of Products’ by Moskowitz, Krieger & Lieberman in Leading Edge Marketing Research (2011)

No responses yet

Leave a Reply