3D Interaction with Scientific Data Through Virtual Reality and Tangible Interfacing

Three-dimensional (3D) interaction with scientific data is still an immature topic. It involves studying visualization methods to faithfully represent data, on the one hand, and designing interfaces that truly assist users in the data analysis process, on the other hand. In this chapter, we study how the human computer interface influences performance in specific scientific visualization tasks. Although a wide range of virtual reality (VR) systems are in use today, there are few guidelines to help system and application developers in selecting the components most appropriate for the domain problem they are investigating. Using the results of an empirical study, we develop guidelines for the choice of display environment for four specific, but common, volume visualization tasks: identification and judgment of the size, shape, density, and connectivity of objects present in a volume. These tasks are derived from data analysis questions being asked by domain specialists studying Cystic Fibrosis (CF). We compared user performance in three different stereo VR systems: (1) a head-mounted display (HMD); (2) a fish tank VR (fish tank); and (3) a fish tank VR augmented with a haptic device (haptic). HMD participants were placed inside the volume and walked within it to explore its structure. Fish tank and haptic participants saw the entire volume on-screen and rotated it to observe it from different perspectives. Response time and accuracy were used to measure performance. The results show that the fish tank and haptic groups were significantly more accurate at judging the shape, density, and connectivity of objects and completed the tasks significantly faster than the HMD group. Although the fish tank group was itself significantly faster than the haptic group, there were no statistical differences in accuracy between the two. Participants classified the HMD system as an inside-out display (looking outwards from inside the volume), and the fish tank and haptic systems as outside-in displays (looking inwards from outside the volume). Including haptics added an inside-out capability to the fish tank system through the use of touch. We recommend an outside-in system, since it offers both overview and context, two visual properties that are important for the volume visualization tasks we studied. In addition, based on the haptic group’s opinion (80% positive) that haptic feedback aided comprehension, we recommend supplementing the outside-in visual display with inside-out haptics when possible. Based on the results from this user study, we further investigated the 3D interaction tasks from the design perspective of tangible interfaces. Since participants using the fish tank VR system performed better than the other groups in terms of time and accuracy, we asked the question whether or not the user performance could be further improved by adding tangible elements to the interface. In particular, we designed tangible interfaces for performing clipping-plane operations. Because of the dense nature of the data, we believe that adding a tangible clipping plane and an intersection image can help the user to better understand the complex data set. The computing platform and tangible interfaces are described to clarify the different design options. An experimental study is planned to quantitatively measure the added value of different aspects of the tangible interface.