Natural Human-Computer Interaction with Musical Instruments

Digital Technology ◽

Physical World ◽

Musical Instruments ◽

Natural Interaction ◽

Professional Musicians ◽

Acoustic Instruments ◽

The Rich ◽

Online Stores ◽

The playing of a musical instrument is one of the most skilled and complex interactions between a human and an artifact. Professional musicians spend a significant part of their lives initially learning their instruments and then perfecting their skills. The production, distribution and consumption of music has been profoundly transformed by digital technology. Today music is recorded and mixed using computers, distributed through online stores and streaming services, and heard on smartphones and portable music players. Computers have also been used to synthesize new sounds, generate music, and even create sound acoustically in the field of music robotics. Despite all these advances the way musicians interact with computers has remained relatively unchanged in the last 20-30 years. Most interaction with computers in the context of music making still occurs either using the standard mouse/keyboard/screen interaction that everyone is familiar with, or using special digital musical instruments and controllers such as keyboards, synthesizers and drum machines. The string, woodwind, and brass families of instruments do not have widely available digital counterparts and in the few cases that they do the digital version is nowhere as expressive as the acoustic one. It is possible to retrofit and augment existing acoustic instruments with digital sensors in order to create what are termed hyper-instruments. These hyper-instruments allow musicians to interact naturally with their instrument as they are accustomed to, while at the same time transmitting information about what they are playing to computing systems. This approach requires significant alterations to the acoustic instrument which is something many musicians are hesitant to do. In addition, hyper-instruments are typically one of a kind research prototypes making their wider adoption practically impossible. In the past few years researchers have started exploring the use of non-invasive and minimally invasive sensing technologies that address these two limitations by allowing acoustic instruments to be used without any modifications directly as digital controllers. This enables natural human-computer interaction with all the rich and delicate control of acoustic instruments, while retaining the wide array of possibilities that digital technology can provide. In this chapter, an overview of these efforts will be provided followed by some more detailed case studies from research that has been conducted by the author's group. This natural interaction blurs the boundaries between the virtual and physical world which is something that will increasingly happen in other aspects of human-computer interaction in addition to music. It also opens up new possibilities for computer-assisted music tutoring, cyber-physical ensembles, and assistive music technologies.

Cognitively Informed Multimedia Interface Design

Multimedia Technologies ◽

10.4018/978-1-59904-953-3.ch033 ◽

2008 ◽

pp. 465-471

Author(s):

E. M. Alkhalifa

Keyword(s):

Cognitive Psychology ◽

Interface Design ◽

Present Knowledge ◽

Pivotal Role ◽

Educational Systems ◽

Multimedia Interface ◽

The Rich ◽

Made In ◽

The rich contributions made in the field of human computer interaction (HCI) have played a pivotal role in shifting the attention of the industry to the interaction between users and computers (Myers, 1998). However, technologies that include hypertext, multimedia, and manipulation of graphical objects were designed and presented to the users without referring to critical findings made in the field of cognitive psychology. These findings allow designers of multimedia educational systems to present knowledge in a fashion that would optimize learning.

DETEKSI GERAKAN KEPALA DAN KEDIPAN MATA DENGAN HAAR CASCADE CLASSIFIER CONTOUR DAN MORFOLOGI DALAM PENGOPERASIAN KOMPUTER UNTUK KAUM DIFABLE

JURTEKSI ◽

10.33330/jurteksi.v5i1.273 ◽

2019 ◽

Vol 5 (1) ◽

pp. 29-36

Author(s):

I Komang Setia Buana

Keyword(s):

Computer Technology ◽

Interaction Model ◽

Human Head ◽

Head Movements ◽

Interaction Techniques ◽

Entertainment Education ◽

Natural Interaction ◽

Haar Cascade ◽

Abstrack: Diffable or the word that has defenition is “ Different Abled People” the term for disabled people. One example of disabled is people who do not have hands, so to write even have to use their feet. Along with the increasingprogress of computer technology, the role of computer technology has also increased for the benefit of humans. One of them is the field of human and computer interaction ( IMK) or also called Human Computer Interaction ( HCI) . Although computer technology equipment is accurate and reliable, but the interaction model that is carried out is not natural as humans interact with each other, the use of such equipment to operate it requires direct contact between user and the computer. For people who dissabilities who do not have hands, it will be difficult to do so. Computer vision based interaction techniques are candidates for natural interaction techniques. The human head can also be used to replace the function of a mouse that can be used to move the cursor up and down left or right for and to click on the mouse using the blik an eye. Detection using head movements has been widely applied including in the fields of entertainment, education, and security. The camera is a tool used to make head recognition. The camera is used as a sensor to detect head movements. Head motion detection is implemented by using opency phython. Keyword: difabel, head, webcame, opencv pythonAbstrak: Difabel atau kata yang memiliki definisi “Different Abled People” ini adalah sebutan bagi orang cacat. Salah satu contoh kaum difabel adalah orang yang tidak mempunyai tangan, sehingga untuk menulispun harus menggunakan kaki. Seiring meningkatnya kemajuan teknologi komputer, peranan teknologi komputer juga semakin meningkat yang digunakan untuk kepentingan manusia. salah satunya adalah bidang interaksi manusia dan komputer (IMK), atau sering disebut Human Computer Interaction (HCI). Keyboard, mouse, dan joystick merupakan salah satu perangkat keras yang sering digunakan untuk interaksi antara manusia dan komputer yang bersifat mekanis. Meskipun peralatan-peralatan tersebut akurat dan handal (reliable), tetapi model interaksi yang dilakukan tidak bersifat alami sebagaimana manusia berinteraksi dengan sesamanya, penggunaan peralatan-peralatan tersebut untuk mengoperasikannya membutuhkan adanya kontak langsung antara user dengan komputer. Untuk kaum difabel yang tidak mempunyai tangan, akan susah melakukan hal tersebut. Teknik interaksi berbasis visi komputer menjadi kandidat teknik interaksi yang bersifat alami. kepala manusia bisa juga digunakan untuk menggantikan fungsi mouse yang bisa digunakan untuk menggerakan cursor keatas kebawah kekiri maupun kekanan dan untuk melakukan klik pada mouse menggunakan kedipan mata. Pendeteksian menggunakan gerakan kepala telah diaplikasikan secara luas diantaranya pada bidang hiburan, pendidikan serta keamanan. Kamera (webcam) merupakan alat yang digunakan untuk melakukan pengenalan kepala. Kamera ini digunakan sebagai sensor untuk mendeteksi pergerakan kepala. Pendeteksian gerakan kepala diimplementasikan dengan menggunakan opencv python.

Handbook of Research on Ubiquitous Computing Technology for Real Time Enterprises ◽

Advanced Hands and Eyes Interaction

10.4018/978-1-59904-832-1.ch019 ◽

2011 ◽

pp. 445-469

Author(s):

Michael Weber ◽

Marc Hermann

Keyword(s):

Ubiquitous Computing ◽

User Interfaces ◽

General Idea ◽

Interaction Techniques ◽

Natural Interaction ◽

Multi Scale ◽

Computer Scientists ◽

Calm Technology ◽

This chapter gives an overview of the broad range of interaction techniques for use in ubiquitous computing. It gives a short introduction to the fundamentals of human-computer interaction and the traditional user interfaces, surveys multi-scale output devices, gives a general idea of hands and eyes input, specializes them by merging the virtual and real world, and introduces attention and affection for enhancing the interaction with computers and especially with disappearing computers. The human-computer interaction techniques surveyed here help support Weiser’s idea of ubiquitous computing (1991) and calm technology (Weiser & Brown, 1996) and result in more natural interaction techniques than in use of purely graphical user interfaces. This chapter will thus first introduce the basic principles in human-computer interaction from a cognitive perspective, but aimed at computer scientists. The humancomputer interaction cycle brings us to a discussion of input and output devices and their characteristics being used within this cycle. The interrelation of the physical and virtual world as we see it in ubiquitous computing has its predecessors in the domain of virtual and augmented realities where specific hands and eyes interaction techniques and technologies have been developed. The next step will be attentive and affective user interfaces and the use of tangible objects being manipulated directly without using dedicated I/O devices.

Human-Computer Interaction. Multimodal and Natural Interaction

10.1007/978-3-030-49062-1 ◽

2020 ◽

Keyword(s):

Natural Interaction ◽

Electronic Musical Instruments: Experiences of a New Luthier

Leonardo Music Journal ◽

10.1162/lmj.2007.17.9 ◽

2007 ◽

Vol 17 ◽

pp. 9-16 ◽

Cited By ~ 14

Author(s):

Bert Bongers

Keyword(s):

Architectural Design ◽

Musical Instruments ◽

General Field ◽

Computer Interaction ◽

Video Performance

The author reflects on his experiences as a designer of new electronic musical instruments, which have led to further insights and applications in other domains such as video performance, architectural design and knowledge applied in the general field of human-computer interaction.

"The Magic of Science:" Beyond Action, a Case Study on Learning Through Socioenaction

10.5753/cbie.wie.2019.501 ◽

2019 ◽

Cited By ~ 1

Author(s):

Emanuel Felipe Duarte ◽

Vanessa Maike ◽

Yusseli Lizeth Méndez Mendoza ◽

Camilla Brennand ◽

M. Cecília Baranauskas

Keyword(s):

Children And Adolescents ◽

Digital Technology ◽

Empirical Studies ◽

Recent Advances ◽

The Subject ◽

Shed Light ◽

Alternative Theories ◽

Recent advances in Human-Computer Interaction (HCI) can significantly affect technology-enhanced educational contexts. Our evolving relationship technology is a challenging topic of investigation, but alternative theories to cognition and socially aware empirical studies can shed light on the subject. In this paper, we explore "The Magic of Science" workshop, conducted in an educational museum context. With a background on learning through action and enactivism, our objective is to observe how people can individually and socially experience pervasive digital technology in educational contexts. Our study included 15 participant children and adolescents, who explored an exhibit of three interactive artworks and then built an interactive artifact from scratch during the workshop. We observed how these interactions took place and collected feedback on the experience of the workshop. Our results indicate that new ways of interacting with pervasive technologies allow us to expand the concept of learning through action, towards learning through socioenaction.

Sketching in Knowledge Creation and Management

Encyclopedia of Knowledge Management, Second Edition ◽

10.4018/978-1-59904-931-1.ch137 ◽

2011 ◽

pp. 1438-1447

Author(s):

Fernando Ferri ◽

Patrizia Grifoni

Keyword(s):

Knowledge Creation ◽

Communication Tools ◽

Natural Interaction ◽

Schematic Representation ◽

New Ideas ◽

Communication Modalities ◽

A sketch is a schematic representation of an image containing a set of objects or concepts. When people need to express and communicate a new idea, they often sketch a rough picture to represent it. Drawing a sketch helps to develop and explore new ideas and enables useful reflection on an idea, elaborating possible alternatives and promoting its evolution. The development of different interaction and communication tools has produced new attention to more natural interaction and communication modalities, including sketching. Hand-drawn sketching is easy and intuitive to use to communicate with others, and human-computer interaction is also simplified.