Musical Interaction Design for Real-Time Score Recognition towards Applications for Musical Learning and Interactive Art

'4th Skin' is a term I coined in 2013, describing metaphorically the visual representation of a layer of multimedia skin covering a dancer’s moving body, changing its appearance in real-time, on stage. It is also the title of one of my Interactive Art projects and the name of a technological platform for video mapping on moving bodies, produced for the technical implementation of the project.The first skin is generally accepted to be our body's skin. Our clothing is very often defined, in direct and transportable meaning, as our second skin. The third skin is the ‘architecture, technology and environment’ that surround us, as Scott Drake defines in his book from 2007 [1]. What I call 4th skin in the context of Media Art, is a digital, real-time video skin, showing different actions, relations and experiences, captured with a wireless web camera attached to the performers body, transforming and placing the character in different dramaturgical contexts.While creating and implementing the multiple formats of this project, my main artistic goal was to achieve what I call 'Intermedia Oneness' between different media and performers on stage. Intermedia Oneness could be defined as a state of immersive artistic experience, governed by a horizontal structure of relations, where no media, nor human or digital performer takes the lead. They are all connected in real-time with the help of technological platforms (hardware and software). The performing body becomes an interface, which I call the 'Intermedia Body', generating or manipulating audio-visual content through its actions or presence in certain areas on stage. I initiated, co-created and co-produced the multidisciplinary artistic project 4th skin in the context of Media Arts and Interactive Dance, at the Interface Cultures Department (University of Art and Design, Linz). A group of creatives from the fields of Contemporary Dance, Sound Art and Interaction Design were involved.The different formats of this project were presented in diverse artistic and educational contexts in Europe (Austria, Romania, Spain, England, Germany, Portugal) and USA (Cleveland, Ohio), between 2013 and 2015.We created two interactive dance performances. A performative installation with public participation was implemented in a gallery. Numerous interactive improvisation sessions and workshops with dancers and media artists took place.Conceptually and artistically, the project was developed together with Dolma Jover - a Spanish choreographer, dancer and dance instructor. I invited Cristian Iordache, a Romanian interaction designer, long term collaborator of KOTKI visuals Studio in Bucharest, to realise the interactive platform for video mapping on moving bodies following my interaction concept. KOTKI visuals was the main project funder and supporter. The creative and performative contribution of Sorin Paun aka Randomform, a Romanian sound artist, Viltė Švarplytė (Estonia), Juan Camilo Herrera (Columbia), dance performers and Henning Schulze, a German interaction designer, were crucial for the realisation of the different project versions.

Download Full-text

4TH SKIN THE PERFORMING BODY AS INTERFACE FOR INTERMEDIA ONENESS IN STAGE-BASED INTERACTION

Knowledge International Journal ◽

10.35120/kij29082399m ◽

2018 ◽

Vol 28 (7) ◽

pp. 2399-2404

Author(s):

Mihaela Kavdanska

Keyword(s):

Real Time ◽

Interaction Design ◽

Sound Art ◽

Contemporary Dance ◽

Interactive Art ◽

Horizontal Structure ◽

Art And Design ◽

Web Camera ◽

Technological Platform ◽

Moving Bodies

'4th Skin' is a term I coined in 2013, describing metaphorically the visual representation of a layer of multimedia skin covering a dancer’s moving body, changing its appearance in real-time, on stage. It is also the title of one of my Interactive Art projects and the name of a technological platform for video mapping on moving bodies, produced for the technical implementation of the project.The first skin is generally accepted to be our body's skin. Our clothing is very often defined, in direct and transportable meaning, as our second skin. The third skin is the ‘architecture, technology and environment’ that surround us, as Scott Drake defines in his book from 2007 [1]. What I call 4th skin in the context of Media Art, is a digital, real-time video skin, showing different actions, relations and experiences, captured with a wireless web camera attached to the performers body, transforming and placing the character in different dramaturgical contexts.While creating and implementing the multiple formats of this project, my main artistic goal was to achieve what I call 'Intermedia Oneness' between different media and performers on stage. Intermedia Oneness could be defined as a state of immersive artistic experience, governed by a horizontal structure of relations, where no media, nor human or digital performer takes the lead. They are all connected in real-time with the help of technological platforms (hardware and software). The performing body becomes an interface, which I call the 'Intermedia Body', generating or manipulating audio-visual content through its actions or presence in certain areas on stage. I initiated, co-created and co-produced the multidisciplinary artistic project 4th skin in the context of Media Arts and Interactive Dance, at the Interface Cultures Department (University of Art and Design, Linz). A group of creatives from the fields of Contemporary Dance, Sound Art and Interaction Design were involved.The different formats of this project were presented in diverse artistic and educational contexts in Europe (Austria, Romania, Spain, England, Germany, Portugal) and USA (Cleveland, Ohio), between 2013 and 2015.We created two interactive dance performances. A performative installation with public participation was implemented in a gallery. Numerous interactive improvisation sessions and workshops with dancers and media artists took place.Conceptually and artistically, the project was developed together with Dolma Jover - a Spanish choreographer, dancer and dance instructor. I invited Cristian Iordache, a Romanian interaction designer, long term collaborator of KOTKI visuals Studio in Bucharest, to realise the interactive platform for video mapping on moving bodies following my interaction concept. KOTKI visuals was the main project funder and supporter. The creative and performative contribution of Sorin Paun aka Randomform, a Romanian sound artist, Viltė Švarplytė (Estonia), Juan Camilo Herrera (Columbia), dance performers and Henning Schulze, a German interaction designer, were crucial for the realisation of the different project versions.

Download Full-text

The Effect of Real-time Score Feedback on L2 English Learners’ Pronunciation and Motivation in an ASR-based CAPT System

Korean Journal of Applied Linguistics ◽

10.17154/kjal.2021.12.37.4.7 ◽

2021 ◽

Vol 37 (4) ◽

pp. 7-50

Author(s):

Reece Randall ◽

Yeonjung Hong ◽

Hosung Nam

Keyword(s):

Real Time ◽

English Learners ◽

Time Score

Download Full-text

A seamless solution for 3D real-time interaction: design and evaluation

Virtual Reality ◽

10.1007/s10055-014-0255-z ◽

2014 ◽

Vol 19 (1) ◽

pp. 1-20 ◽

Cited By ~ 9

Author(s):

Franck Hernoux ◽

Olivier Christmann

Keyword(s):

Real Time ◽

Interaction Design ◽

Time Interaction

Download Full-text

Social Interactive Art through Real-Time Media Sharing

CONTENTS PLUS ◽

10.14728/kcp.2015.13.06.069 ◽

2015 ◽

Vol 13 (6) ◽

pp. 69-85

Author(s):

Sangwook Kim ◽

◽

Jungeun Lee ◽

Jaeha Lyu ◽

Seungeun Jang ◽

...

Keyword(s):

Real Time ◽

Interactive Art

Download Full-text

danceroom Spectroscopy: At the Frontiers of Physics, Performance, Interactive Art and Technology

Leonardo ◽

10.1162/leon_a_00924 ◽

2016 ◽

Vol 49 (2) ◽

pp. 138-147 ◽

Cited By ~ 6

Author(s):

Thomas Mitchell ◽

Joseph Hyde ◽

Philip Tew ◽

David R. Glowacki

Keyword(s):

Real Time ◽

Interactive System ◽

Energy Landscapes ◽

Interactive Art ◽

Art Installation ◽

Art And Technology ◽

Human Energy ◽

Performance System ◽

Physics Performance ◽

And Performance

danceroom Spectroscopy is an interactive audiovisual art installation and performance system driven by rigorous algorithms commonly used to simulate and analyze nanoscale atomic dynamics. danceroom Spectroscopy interprets humans as “energy landscapes,” resulting in an interactive system in which human energy fields are embedded within a simulation of thousands of atoms. Users are able to sculpt the atomic dynamics using their movements and experience their interactions visually and sonically in real time. danceroom Spectroscopy has so far been deployed as both an interactive sci-art installation and as the platform for a dance performance called Hidden Fields.

Download Full-text