The design, application, and evaluation of man-machine systems are limited by our ability to measure system performance in a reliable and sensitive manner. Without adequate performance measures, there is no way to test system designs, to plan and execute training systems, or to effectively evaluate operational systems. Typically, measures are manually produced by selecting a set of candidate performance measures which are subsequently tested for reliability and validity. Since the measurement value of a given candidate measure is not known until these tests are complete, this process, which may be an iterative process, can be both time consuming and costly. Also, since only a few candidate measures can be investigated manually, there is a high probability that superior measures are not even considered. Automating at least some of the manual operations required can result in improved performance measures in less time and at lower cost. The performance measurement generating processor described in this paper accepts demonstration data representing various levels of performance, and under user control, analyzes the data to provide candidate performance measures. The processor also conducts validation tests and orders candidate measures according to their measurement value. Output from the FORTRAN IV processor includes results from validation tests and specifications for objective performance measures.
The Research of Reliability and Validity on Computer Courses Network Test System
