In research on interactive music generation, we propose a music generation method in which the computer generates music under the recognition of a humanmusic conductor’s gestures. In this research, generated music is tuned by parameters of a network of chaotic elements which are determined by the recognized gesture in real time. The music conductor’s hand motions are detected by Microsoft Kinect in this system. Music theories are embedded in the algorithm and, as a result, generated music is richer. Furthermore, we constructed the music generation system and performed experiments for generating music composed by human beings.