Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretability

During a research project in which we developed a machine learning (ML) driven visualization system for non-ML experts, we reflected on interpretability research in ML, computer-supported collaborative work and human-computer interaction. We found that while there are manifold technical approaches, these often focus on ML experts and are evaluated in decontextualized empirical studies. We hypothesized that participatory design research may support the understanding of stakeholders' situated sense-making in our project, yet, found guidance regarding ML interpretability inexhaustive. Building on philosophy of technology, we formulated explanation strategies as an empirical-analytical lens explicating how technical explanations mediate the contextual preferences concerning people's interpretations. In this paper, we contribute a report of our proof-of-concept use of explanation strategies to analyze a co-design workshop with non-ML experts, methodological implications for participatory design research, design implications for explanations for non-ML experts and suggest further investigation of technological mediation theories in the ML interpretability space.

Spatially resolved transcriptomics in immersive environments

Spatially resolved transcriptomics is an emerging class of high-throughput technologies that enable biologists to systematically investigate the expression of genes along with spatial information. Upon data acquisition, one major hurdle is the subsequent interpretation and visualization of the datasets acquired. To address this challenge, VR-Cardiomicsis presented, which is a novel data visualization system with interactive functionalities designed to help biologists interpret spatially resolved transcriptomic datasets. By implementing the system in two separate immersive environments, fish tank virtual reality (FTVR) and head-mounted display virtual reality (HMD-VR), biologists can interact with the data in novel ways not previously possible, such as visually exploring the gene expression patterns of an organ, and comparing genes based on their 3D expression profiles. Further, a biologist-driven use-case is presented, in which immersive environments facilitate biologists to explore and compare the heart expression profiles of different genes.

Explorative Visual Analysis of Rap Music

Detecting references and similarities in music lyrics can be a difficult task. Crowdsourced knowledge platforms such as Genius. can help in this process through user-annotated information about the artist and the song but fail to include visualizations to help users find similarities and structures on a higher and more abstract level. We propose a prototype to compute similarities between rap artists based on word embedding of their lyrics crawled from Genius. Furthermore, the artists and their lyrics can be analyzed using an explorative visualization system applying multiple visualization methods to support domain-specific tasks.

Musical Information Visualization System

The visualisation systems are spread widely as personal computer’s software. The system, that are processing audio data are presented in this article. The system visualizes the ratio of spectrum amplitudes and has fixed frequency binding to colours. The technology of audio signals processing by the device and components of the device were considered. For the increasing information processing speed was used 32bit controller and graphic equalizer with seven passbands. Music visualization it is function, that are spread widely in mediaplayer’s software, on a different operation systems. This function shows animated images that are depends on music signal. Images are usually reproduced in the real time mode and synchronized with a played audio-track. Music and visualization are merges in the different kind of art: opera, ballett, music drama or movies. Dependencies of auditory and visual sensations are used for increasing the emotional perseption for ordinary listeners . In the systems, that are currently being actively promoted, are used several tools for personal computers, such as: After Effects – The Audio Spectrum Effect, VSDC Video Editor Free – Audio Spectrum Visualizer, Magic Music Visuals. The software, that are mentioned above, has a one disadvantage: the using of streaming video is not possible with the simultaneous receipt of audio and requires processing and rendering of the resulting video series. The purpose of the work is to determine the features of spectral analysis of music information and taking into account real-time data processing. Propose a variant of the music information visualization system, which displays the spectral composition of music and the amplitude of individual harmonics, and filling the LED-matrix with the appropriate color depending on the amplitude of the audio signal, with the possibility of wireless signal transmission from the music source to the visual effects device. The technology of frequency analysis of the spectrum with estimation of amplitude of spectrum’s components of the musical data, that is arriving on the device is chosen for this project. The method is based on the analysis of the spectrum in the selected frequency bands, which in turn simplifies the function of finding maxima at different frequencies. The proposed variant of the musical information visualization system provides display on the LED-matrix of colors that correspond to the frequencies spectrum’s components in the musical composition. Moreover, the number of involved LEDs is proportional to the ratio of the amplitudes of the signal’s frequency components. The desired result is achieved by using a Fast Fourier Transform and selecting Khan or Heming windows for providing a better analysis results of the signal spectrum. The amplitudes of the individual components of the spectrum are estimated additionally and each frequency band has its own color. The work of the system is to analyze the components of the spectrum and frequency of musical information. This information affects the display of colors on the LED matrix. The using of a 32-bit microcontroller provides sufficient speed of audio signal processing with minimal delays. For the increasing the accuracy and speed up the frequency analysis, the sound range is divided into seven bands. For this purpose was used seven-band graphic equalizer MSGEQ7. Music information is transmitted to the system via Bluetooth, which greatly simplifies the selection and connection of the music data source.

Art Product Design and VR User Experience Based on IoT Technology and Visualization System

Based on the Internet of Things technology, this paper analyzes the concept and basic characteristics of visualization technology and builds a visualization digital art product information system based on VRML virtual reality technology. This article first explains the background and significance, analyzes the research status at home and abroad, introduces the application of virtual realization technology, and puts forward the technical route and main research content on this basis. After field inspection of art products, the system was analyzed according to the actual situation. After analyzing the needs and goals of the system, the overall structure of the system was proposed. Under the overall framework, this paper designs the system and designs the system’s visualization and real-time interaction modules. At the same time, combined with VRML technology to realize the visual 3D modeling of virtual art products, we establish the 3D model of the entire art product. Users can choose between role roaming and camera roaming. Finally, we made a summary statement to the full text and looked forward to the application prospects and development trends of information visualization technology in digital art products. During the experiment, by calling the HTC method in the Unity 3D system, this article uses the Unity 3D engine to implement UI scene loading and VR rendering technology optimization and realizes design knowledge through the integration of art product design models and IoT data interactive functions and dynamic loading of scenes. The effect of the model on the quality prediction is analyzed through experiments. The results show that the performance of the former is better than the latter two models, and it has a good performance on unbalanced data processing, which verifies the performance and effectiveness of the algorithm model.