A Comparison of Variety Metrics in Engineering Design
This paper focuses on comparing and contrasting methods for assessing the variety of a group of design ideas. Variety is an important attribute of design ideas, because it indicates the extent to which the solution space has been explored. There is a greater likelihood of successfully solving a design problem when a more diverse set of ideas is generated in the early stages of design. While there are three existing metrics for variety, it has not been established how well they correlate with each other, so it is unknown whether they provide similar assessments of variety. This uncertainty inspired our investigation of the three existing metrics and, eventually, the development of a new variety metric — all of which we compared statistically and qualitatively. In particular, 104 design ideas collected from 29 sophomore mechanical engineering students were analyzed using the existing and new variety metrics. We conducted correlation analyses to determine if the four metrics were related and to what degree. We also considered the qualitative differences among these metrics, along with where they might be used most effectively. We found varying levels of statistically significant correlations among the four metrics, indicating that they are dependent. Even so, each metric offers a unique perspective on variety and may be useful in different situations.