Optimal Experimental Design of Human Appraisals for Modeling Consumer Preferences in Engineering Design

Human appraisals are becoming increasingly important in the design of engineering systems to link engineering design attributes to customer preferences. Human appraisals are used to assess consumers’ opinions of a given product design, and are unique in that the experiment response is a function of both the product attributes and the respondents’ demographic attributes. The design of a human appraisal is characterized as a split-plot design, in which the respondent demographic attributes form the whole-plot factors while the product attributes form the split-plot factors. The experiments are also characterized by random block effects, in which the design configurations evaluated by a single respondent form a block. An experimental design algorithm is needed for human appraisal experiments because standard experimental designs often do not meet the needs of these experiments. In this work, an algorithmic approach to identify the optimal design for a human appraisal experiment is developed, which considers the effects of respondent fatigue and the block and split-plot structure of such a design. The developed algorithm seeks to identify the experimental design which maximizes the determinant of the Fisher Information Matrix, labeled as the D-criterion of a given design. The algorithm is derived assuming an ordered logit model will be used to model the rating responses. The advantages of this approach over competing approaches for minimizing the number of appraisal experiments and model-building efficiency are demonstrated using an automotive interior package human appraisal as an example.