The PIC-TDD Framework of Test Data Design for Pattern Recognition Systems

In this paper, a new approach is proposed for the design of test data for pattern recognition systems. In the theoretical framework put forward, performance on the population of data is viewed as expectation of a random variable, and the purpose of test is to estimate the parameter. While the most popular method of test data design is random sampling, a novel approach based on performance influencing classes is proposed, which can achieve unbiased estimation and the variance of estimation is much lower than that from random sample. The method is applied to the evaluation of systems for broadcasting news segmentation, and experimental results show the advantages over the random sampling approach.