Home |
Marketing Research Services | Product Optimization (or Product Testing)
Product Optimization (or Product Testing)
Product testing is, perhaps, the single most important type of research any company ever conducts. Achieving clear-cut product superiority in a
category is the surest way to build brand share, engender customer loyalty, and boost profitability. Better products tend to command higher prices and be more responsive to advertising investments.
Product testing methods vary by product category. For consumer packaged goods, in-home usage testing is typically the recommended method. For durable goods,
however, product clinics are generally the preferred method. For services, some type of mystery shopping is recommended. Regardless of product category, the following principles generally apply.
Human tastes and preferences evolve over time. Fashion and style trends affect every industry (clothing, shoes, automobiles, foods, lawnmowers, washing machines,
jet engines, etc.), even though the rate of change varies greatly by industry. Competitive actions can redefine a product category in a matter of months.
That’s why product testing and optimization must be viewed as a strategic, ongoing activity. The human race’s preferences and proclivities are a moving target.
“Real environment” testing (i.e., testing a product the way it is typically used) is almost always the most accurate method of product evaluation.
For example, having consumers evaluate food products in their homes rather than in a laboratory or test kitchen is usually recommended.
Products should be tested using a standardized system so that each product is tested in exactly the same way. Decision Analyst designs and operates product testing
systems customized to the products, goals, and needs of each client. A “systems” approach is essential, so that every product is tested in precisely the
same way. Normative data can then be accumulated to aid in the interpretation of subsequent product tests.
Optima® Product Testing
Decision Analyst’s principal product-testing system is Optima®, a monadic, normative, product-testing system comprised of modules of standardized
questions. The system is tailored to each client’s product category and requirements. The essential features of Optima® are:
- Monadic Design. Each product is tested alone. This provides the most accurate evaluation of the product and the best diagnostic
feedback about how to improve the product.
- Standardized Systems. The sampling, data collection, data preparation, and data tabulation methods and procedures (i.e., the systems)
are standardized for each product. It is essential that every product be tested in precisely the same way.
- Standardized Questions. The standardized questionnaire is modular in structure and flexible in design. Decision Analyst owns copyrights
on all of the questionnaire modules.
Based on internal diagnostics, normative data, and analytical models, these standardized questions tell us whether the product is optimal or not, and indicate
what needs to be changed to improve the product. The primary analytic model is our Product Improvement Index (Pii®), a unique mathematical model to help optimize product formulation or design.
How Does Optima® Work?
Typically, a representative sample of category users (150 to 200 households) are given a test product to use in home for a few days. Then these consumers are
asked a series of standard questions about the product. The Optima® core questionnaire consists of the following modules:
- Screening/Qualification questions
- Overall rating of product
- Likes
- Suggested product improvements
- Diagnostic ratings of product attributes
- Ratings of product components
- Purchase intent, priced
- Expected purchase frequency
- Value rating
- Demographics
If the product test is a part of a new-product volumetric forecast, then additional modules are added to the core questionnaire.
Based on internal diagnostics, normative data, and analytical models, these standardized questions tell us whether the product is optimal or not, and indicate
what needs to be changed to improve the product. The primary analytic model is our Product Improvement Index (Pii®), a unique mathematical model to
help optimize product formulation or design.
Product Improvement Index® (Pii®)
Decision Analyst developed the Pii® mathematical model to help guide product development efforts for new products and the reformulation of existing products.
Pii® was developed primarily because of problems encountered in using various types of regression models in product-testing analyses. Regression models assume
that all of the input variables are independent (i.e., not intercorrelated in any way). The reality is, however, that virtually all of the input variables
that might explain a product’s performance are typically intercorrelated. The result is a regression equation that omits important input variables. For
example, if the color and the sweetness of a product happen to be highly correlated with each other, the regression equation would omit one of the variables. We
might think we had a color problem when in fact we had a color and a sweetness problem.
The Pii® model was designed to circumvent the “missing variables” problem associated with regression. This model is based upon a type of correlation,
using “dummy” variables, to examine the relationship between the diagnostic ratings (e.g., too sweet, about right, not sweet enough; or, too
much salt, about right, not enough salt; and so on) and the consumer’s overall rating of the product. The overall rating is typically measured with
an 11-point scale.
The output of the Pii® model is a table of important explanatory variables along with the Pii® rating and the indicated action, as illustrated here.
| Diagnostic Variable |
Pii® Score |
Indicated Action |
| Too sweet |
18.65 |
Reduce sweetness |
| Too dark in color |
14.72 |
Make product lighter |
| Too soft |
12.95 |
Make product firmer |
| Not enough salt |
9.48 |
Add some salt |
| Not enough crunch |
5.23 |
Make product crunchier |
Generally, any Pii® score greater than 4.0 indicates that some modification of the product might be necessary. The greater the Pii® score, the more
important that variable is and the more that variable should be modified.
Optimization Methods
In addition to Pii® analyses, response-surface and choice-modeling analyses are the primary optimization techniques. Experimental designs and simulation models
are employed to optimize products. By testing chosen subsets of product possibilities, response surface and choice modeling can simulate and predict consumer preferences
for hundreds of product possibilities, as defined by variations in ingredients, features, elements, or packages. The resulting equations are used to build an optimization
simulator so that “what if” products can be fully explored and understood. The goal of optimization can vary. It might be maximizing consumer preference,
or maximizing the profit margin without losing market share, or maximizing sales potential. The optimization simulator also helps reveal “cause and effect”
as inputs are changed and outcomes vary.
The value of Optima® product testing is illustrated by its many uses:
- To evaluate and improve existing products.
- To measure the threat posed by competitive products.
- To continuously improve product performance over time (i.e., to optimize products).
- To evaluate cost-reduction formulations while maintaining product superiority.
- To measure the effects of aging upon product quality (shelf-life studies).
- To provide guidance to R&D in developing or upgrading products.
- To monitor product quality from different suppliers and/or different factories.
- To implicitly evaluate marketing variables (i.e., packaging, pricing, sizing, etc.).
- To predict the success of new products.
Nonfood Product Testing
The concepts, methods, and techniques of product testing can be adapted and applied to almost any product category. Our staff has evaluated:
- Airliner seats
- Calculators
- Cellular phones
- Comforters
- Computers
- Educational toys
- Electronic games
- Film & film processing
- Food service products
- Frying pans
- Game prototypes
- Gaming devices
|
- Glass windows
- Hotel rooms
- Microwave ovens
- Pagers
- PDAs
- Personal computers
- Restaurant entrees & side dishes
- Restaurant exteriors & interiors
- Retail store layouts
- Software
- Washing machines
- Websites
|
Decision Analyst designs and implements comprehensive testing systems based on package invitations. The purpose of on-package or in-package testing systems is
typically quality assurance or quality control. Every nth package invites consumers to answer the basic question, “How good is this product?”
Respondents log in to a website and rate the product. This provides an economical way to monitor the quality of products over time, and monitor the quality of
products from different factories. It’s also a viable method of product testing when large numbers of different products must be tested on an ongoing basis
(e.g., private-label products).
Product Clinics (Durable Goods)
For durable goods, central-location product clinics are an important technique. Automobiles, washing machines, and refrigerators are examples of products that
lend themselves to clinics. In a product clinic, the test product (say, a new car) is placed in a central-location facility along with major competitive automobiles.
Potential buyers of that type of new car are recruited to view, examine, and evaluate all of the cars on display. Decision Analyst is adept at planning and executing complex product clinics.
Service Industries
Services can be improved through “product testing.” Services are nebulous and vaporous compared to physical products. Services can be evaluated by
various types of mystery shopping, or through monadic ratings by customers (compared to the same types of ratings for competitors). Decision Analyst designs
and operates research systems to monitor and improve the quality of services in service industries.
Decision Analyst maintains and operates one of the largest product-testing panels in the world. This online consumer panel (American Consumer Opinion®,
with 8 million members and significant numbers of panelists in over 150 countries) is used for Optima® in-home-usage product testing. Test products are shipped
by manufacturers to Decision Analyst’s main offices, where the test products are then boxed, labeled, and shipped to panelists for testing in their homes.
Labeling and Shipping Test Products
Decision Analyst operates a large mail-processing facility for the explicit purpose of labeling test products, packing those products in shipping containers,
labeling the boxes, and shipping all over North America, Europe, and elsewhere for in-home product testing. This facility’s staff can handle complex
study designs, rotation schemes, and labeling requirements.
Qualitative methods should not be used to test products or services in a quantitative sense. Qualitative methods are best at diagnostics. What words and phrases
do consumers use to describe the taste of a product? What are the keys to making a product better? What consumer perceptions and motivations underlie reactions to
a product? How could the product or service be refined? Depth interviews and focus groups are the primary diagnostic methods to help identify possible product
improvements. Naturally, quantitative testing must be employed to verify the qualitative hypotheses.
Product Optimization Services
Decision Analyst is a recognized leader in consumer product testing and optimization. Its staff has evaluated more than 1,000 foods, beverages, and other products
during the past 3 decades. The firms has over 50 staff members with extensive experience in the conduct and analysis of product testing and optimization studies.
The company is a leader in the development of analytical techniques to enhance product testing and optimization.
If you would like more information on product testing, please contact Jerry W. Thomas, President/CEO (jthomas@decisionanalyst.com),
or call 1-800-ANALYSIS (262-5974) or 1-817-640-6166.
Additional Resources From Decision Analyst
Product Testing Services
New Product Research Brochures
Product Testing Case Histories
Product Testing White Papers
