Video Research Instrument

In the next few chapters, we will examine some larger projects covering a number of topics in music education. In this chapter, we will look at a research instrument designed to measure the time when a participant responds to some stimuli while watching videos. 1. Open the file STI.maxpat from the Chapter 18 Examples folder This patch allows a researcher to show participants one or more video files and ask them to press the space bar in response to whatever the researcher wants to observe. For example, the researcher might show a video of an orchestra performing Samuel Barber’s Adagio for Strings and ask participants to press the space bar whenever they experienced “thrills” or “chills.” The program would then record the exact times, with respect to the video, that each participant experienced such a reaction. The researcher could then compare the times from that participant’s session to the times from another participant’s session. Let’s walk through the process together, though, unfortunately, instead of seeing an orchestra, you’ll have to endure a homemade movie from my trip to Colorado (it’s short—don’t worry). While you are watching the video, pretend that you are experiencing “thrills” and press the space bar a few times. To start the video, press the Return/Enter key; this will display the video in fullscreen. When the video comes to an end, press the esc key to close the fullscreen view. There’s not much audio in this video, but you can click the “test audio” button to ensure that your sound card is working (the ezdac~ should be “on” by default with the icon colored red). 2. Press the Return/Enter key to start the video in fullscreen view.

Audio Playback and Pitch Tracking

In this chapter, we will look at some of the ways that you can play back and record sound files. As you know, Max lets you design the way you control the variables in your patch. We will apply these design concepts to the ways we control the playback of recorded sound. We will also look at some ways to track the pitch of analog audio and convert it into MIDI numbers. By the end of this chapter, you will have written a program that allows you to play back sound files using a computer keyboard as a control interface as well as a program that tracks the pitch you’re singing from a microphone and automatically harmonizes in real time. We will create a simple patch that plays back some prerecorded files I have prepared. Please locate the8 “.aif ” audio files located in the Chapter 13 Examples folder. 1. Copy these 8 audio files to a new folder somewhere on your computer 2. In Max, create a new patch 3. Click File>Save As and save the patch as playing_sounds.maxpat in the same folder where you put these 8 audio files. There should be 9 files total in the folder (8 audio and 1 Max patch) 4. Close the patch playing_sounds.maxpat 5. Re-open playing_sounds.maxpat (the audio files will now be in the search path of the Max patch) We can play back the prerecorded audio files we just copied using an object called sfplay~. The sfplay~ object takes an argument to specify how many channels of audio you would like the object to handle. For instance, if you are loading a stereo (two channel) file, you can specify the argument 2. Loading a sound file is easy: simply send the sfplay~ object the message open. Playing back the sound is just as easy: send it a 1 or a 0 from a toggle. Let’s build a patch that plays back these files.

Control Interfaces Continued

In this chapter, we will look at some innovative ways to control music making as we develop musical instruments. We will look at using your computer keyboard and mouse as performance instruments as well as discuss the use of videogame controllers in your patches. Designing your own custom musical instruments is a great way to tailor the controls to the specific physical abilities of users while allowing them to focus on certain specific musical concepts like pitches, scales, and harmony/chords. 1. Click on Extras>EAMIR from the top menu to view the main menu of the EAMIR SDK 2. In the umenu labeled Examples, click the third item 3.EAMIR _ASCII_Keyboard_Control.maxpat Unlock the patch that opens and look at its basic structure. As you can see, the patch is really just 4 bpatcher objects, 3 of which refer to patches we’ve already looked at. The newest bpatcher, at the top of the patch, is basically just a patch with a key object, a select object, and some fancy graphics—all things you learned to use in Chapter 3. Lock the patch and 3. Type your full name using your computer keyboard. Note that uppercase letters and lowercase letters trigger different buttons 4. Press the number keys 1–8 as these are mapped to message boxes containing numbers used as diatonic chord functions Without the top bpatcher, your patch generates chords in any key simply by clicking the message boxes. The top bpatcher is just a control interface that maps something (keys) to something else (message boxes). 5. Ctrl+click (Mac) or right click (Windows) the top bpatcher and select Object>Open Original “EAMIR_keyboard.maxpat” from the contextual menu This patch is set to open in Presentation mode. Unlock the patch and put it in Patching view. The contents of the patch are as I described: a key object, as well as a keyup object, are connected to two gigantic sel (select) objects containing the ASCII numbers for all the available characters on the computer keyboard nothing you couldn’t already do. In fact, the most impressive part of this patch, in my opinion, is the graphical part of it.

Working with Audio

In this chapter, we will discuss MSP, a collection of objects that work with audio signals. Unlike the Max objects we’ve discussed so far that handled data like numbers (including MIDI) and text, the MSP objects can handle actual sound recordings, like audio from a microphone, as well as generate signals. As we will see, MSP objects have a ~ after their name and slightly different colored patch cords signifying that they are handling some sort of audio signal. By the end of this chapter, you will be able to get a microphone signal in and out of Max, generate timbre according to the harmonic series, and build your very own synthesizer from that timbre. Let’s begin by building a basic patch that takes input from a microphone and sends it out to your speakers. Your computer is a digital device; you, as I hope you know, are not. Your voice is an acoustic instrument which, in the early days of recording, used to be recorded using analog recording techniques to represent qualities of the acoustic instrument. In order for your voice to be recorded and represented by a computer, there needs to be some conversion of the analog signal to a digital signal. For this reason, your computer’s sound card has an analog to digital converter, otherwise known as an A to D converter or, simply an ADC. Depending on the type of sound card you have, some AD converters will do a better job than others of supplying enough digits to represent the analog sound being recorded. Think about it like this: if you were allowed 50 words to describe your favorite food, the description would be a lot more articulate than if you had only 15 words to describe your favorite food. In the same regard, in a digital recording system, the more digits you have to represent the analog signal you are recording, the closer the representation will be to the original source you can hear with your ears. For now, we will leave this discussion of analog to digital conversion.

Building Stand-alone Applications

In this chapter, we will analyze a “Chord Namer” application that allows a user to enter a chord name and see the notes on a MIDI keyboard. Unlike the other patches we’ve worked on thus far, we will “build” this patch as a stand-alone program that can be used on any computer even if it does not have Max installed. Stand-alone programs are a great way of distributing your work to people for educational or commercial purposes. Open the file chord_namer.maxpat from the Chapter 11 Examples folder. This patch allows users to type in the name of a chord (C, for example) and see the chord displayed on a large kslider. Users can then play the chord on their MIDI keyboard while looking at the visual example. The letter name of each note appears on each chord tone when it is highlighted. For taller chords, a user may enable more chord tones to be added than simply just a root, third, and fifth. For example, a user wanting to play a Cdom7#9 chord could simply enable 7ths and 9ths to be displayed by checking the appropriate toggles, typing Cdom7#9 into the space provided, and pressing the return or enter key. 1. Type C into the text box at the top left and press the return or enter key 2. Play a C chord on your MIDI keyboard This patch could be useful for helping people perform a piece for which they have only a lead sheet with chord names. Let’s take a look inside the patch. The patch is currently in Presentation mode. Unlock the patch and put it into Patching mode. The patch is rather large in size so you may need to zoom out on the patch (⌘for Mac or ctrl for Windows ). Now that the patch is open, you may be surprised to see that there is only a small number of objects inside. Take note of the 3 bpatchers in the patch that generate chords, handle MIDI output, and, to the right, above the kslider, handle MIDI input.

Working with Time

In this chapter, we will discuss aspects of time, rhythm, and the sequencing of events. When used in combination with what you know about generating pitch material, this chapter will help you to create interactive performance and composition systems, as well as create patches that demonstrate rhythmic complexity. By the end of this chapter, you will have created patches that can record and loop MIDI sequences as well as a number of patches that work with notes over time. Digital Audio Workstations (DAWs) are complex programs that generally all allow users to sequence music in some way by recording MIDI and audio, and make edits to the recording data. Popular commercial DAWs include Apple Logic and GarageBand, Digidesign Pro Tools, Sony Acid, and Ableton Live, to name a few. The last thing you’d probably want to do in Max is write a patch that operates like one of these DAWs. In fact, one of the great things about Max is that since it’s not a DAW but a programming language, it’s so unlike traditional DAWs that it allows you to write any kind of program you want (like our synth that used the wheels to play pitches in Chapter 2). However, there are some aspects of these DAWs that we will want to incorporate into our patches. In particular, recording MIDI and playing it back is made possible in Max through an object called seq (as in sequencer). 1. Create a new patch 2. Create a new object called seq The seq object can record and play back raw MIDI data. Since we’re dealing with raw MIDI data as opposed to just pitches, we’ll use the midiin and midiout objects. In fact, just to protect ourselves, let’s also include the “raw MIDI” version of flush, midiflush, to avoid any stuck notes.

Compositions and Perception Tools

In this chapter, we will examine some ways to interact with audio processing objects in formal compositions. Examples of traditional instrumentalists interacting with Max patches in concert performances are common. In the interest of copyright availability, we will examine a composition of mine for E♭clarinet and computer (a Max patch). The remaining example patches in this chapter will deal with audio processing as it relates to hearing and some aspects of perception. In this composition, discourse, the clarinetist plays from a score while the Max patch “listens” to the performer (using a microphone) and processes the clarinet sound in predetermined ways. The Max patch follows a time-based “score” of its own for performing the effects on the clarinet sound and, thus, processes the audio signal the same way each time the piece is performed. Our purpose in exploring this patch has less to do with the effects that are used or any aesthetic you get from the piece than with the implementation of a usable timeline that both the clarinetist and the computer can perform to. 1. Open the file discourse.maxpat from within the folder discourse located in the Chapter 20 Examples folder When the space bar is pressed, a clocker within the patch will begin triggering the events in the Max patch; this is like the score for the computer. These events assume that since the user has pressed the space bar, the patch can expect to hear the notes of the score played back at tempo to coincide with the different audio processing taking place within the patch. Unless you happen to have your E♭clarinet handy (a PDF of the score is also available in the discourse folder), we will use a demo sound file of a synthesized clarinet playing this piece in lieu of actually performing it. This will give us a sense of what the piece would sound like if we were to perform the clarinet part live.

Control Interfaces

In this chapter, we will examine some premade patches demonstrating a few techniques for designing diatonic musical instruments. We will review some of the basic ins and outs of MIDI, learn some ways to program more efficiently, and discuss a number of control options for your patches. As we examine some working patches and encounter many new objects, please read carefully and, in your mind, follow the flow of data from one object to the next as the process is described in the text. Remember that it will be beneficial to look at the Help file for any objects you may have forgotten about or do not fully understand as you encounter them in this chapter. In the last chapter, you installed the EAMIR SDK (Soft ware Development Kit) which, in addition to putting the Modal Object Library into the Max search path, put a bunch of patches I’ve created into the path as well. In fact, if you select Extras from the top menu, you will see an item marked EAMIR among the other extras. 1. Click on Extras>EAMIR from the top menu to view the main menu of the EAMIR SDK 2. In the umenu labeled Examples, click the first item 1.EAMIR _MIDI_Basics.maxpat If you did a manual install of the EAMIR SDK, you will need to locate the EAMIR_SDK folder on your computer and open the file EAMIR.maxpat. A patch will open containing two rectangular boxes. Unlock the patch to see that these two rectangular boxes have inlets and outlets just like the objects we’ve been working with, except that they, in themselves, contain other objects. These two boxes are called bpatchers. A bpatcher is an object that allows an existing Max patch to be loaded into a viewable window; that window is the bpatcher object itself. 3. Open the Inspector for the upper bpatcher Note that the line Patcher File within the Inspector displays the filename of the Max patch currently loaded in the bpatcher: EAMIR_MIDI_in.maxpat. This is just a Max patch that is being displayed in the bpatcher.

Interactive Ear Training

In this chapter, we will review some of the objects and concepts we’ve learned so far and make an interactive ear-training program. We will build our ear-training program using primarily objects you already know. In designing a program with a specific purpose, like an ear-training program, the worst technique you can take is to begin by arbitrarily inserting objects into a patch. Instead, it’s best to conceptualize the task at hand and “divide and conquer.” Divide the task into smaller steps and think through what actually needs to get done to accomplish the goal of the patch. Begin a new blank patch we will use to build an ear-training program in which diatonic and chromatic intervals are randomly performed by the patch. Users will have to correctly select the name of the interval they heard from a menu. The soft ware will give immediate feedback based on their response. To begin, create a new patch and 1. Create a button 2. Create 2 new object boxes both called random with the argument 12 3. Position the objects next to each other horizontally 4. Connect the button to the first inlet of each random 12 object 5. Create 2 number boxes beneath each random 12 object 6. Connect the outlet of each random 12 object to the number box beneath it 7. Lock the patch and click the button to generates 2 random numbers between 0 and 11 We will use these two random numbers to build our intervals. The random number on the left will be our starting pitch and we’ll add the second random number as the added interval. As you may have realized, giving the argument 12 to random restricts the number of pitches to 12 allowing us to keep our intervals within one octave. Let’s add the two numbers together.

Audio Effects and Processing

In the last two chapters, we addressed ways to get live audio and sound files to play in your patch. In this chapter, we will address implementing audio effects into the patch. Many DAWs (Digital Audio Workstations) allow you to add effects such as delays, reverb, and chorus into the audio signal path. Max, however, gives you complete control over the effects that you add since you build them yourself. For example, if you want to build a delay effect that delays the incoming audio signal by quarter notes for 5 seconds, sixteenth notes for 2 seconds, and eighth notes for 11 seconds, you may find that this very specific task is easier to carry out in Max than in the automation window of most DAWs. Similarly, if you wanted to perform some sort of filtering on audio in a buffer, you have complete control over the audio in your patch and the way that actions are carried out on it. Since the nature of building effects is rather complex, we will focus on the concept of implementation using some simpler effects in the process. We will begin discussing delays, that is, combining an audio signal with a duplicate of the signal delayed by some amount of time. In addition to being a useful effect in itself, delays are the basis for other effects such as reverb, chorus, and flanging. Let’s build a patch that will allow us to audition a few effects that we will build together. We will use some of the sound files that were installed on your computer when you installed Max (like jongly.aif ) and sfplay~ to play these files back. As you should know, you can just as easily implement these effects into a patch that uses live audio.