Evaluation of Algorithms for Combining Independent Data Sets in a Human Performance Expert System

A major problem facing system designers today is predicting human performance in: 1) systems that have not yet been built, 2) situations that have not yet been experienced, and 3) situations for which there are only anecdotal reports. To address this problem, the Human Performance Expert System (Human) was designed. The system contains a large data base of equations derived from human performance research reported in the open literature. Human accesses these data to predict task performance times, task completion probabilities, and error rates. A problem was encountered when multiple independent data sets were relevant to one task. For example, a designer is interested in the effects of luminance and front size on number of reading errors. Two data sets exist in the literature: one examining the effects of luminance, the other, font size. The data in the two sets were collected at different locations with different subjects and at different times in history. How can the two data sets be combined to address the designer's problem? Four combining algorithms were developed and then tested in two steps. In step one, two reaction-time experiments were conducted: one to evaluate the effect the number of alternatives on reaction time; the second, signals per minute and number of displays being monitored. The four algorithms were used on the data from these two experiments to predict reaction time in the situation where all three independent variables are manipulated simultaneously. In step two of the test procedure, a third experiment was conducted. Subjects who had not participated in either Experiment One or Two performed a reaction-time task under the combined effects of all three independent variables. The predictions made from step one were compared to the actual empirical data collected in step two. The results of these comparisons are presented.