The MIST project uses modern gestural technology to elicit musical feedback, and is
intended for use in a clinical setting with children with moderate
disabilities. The system is a software/hardware package that is designed to be
an inexpensive, user-friendly, portable tool for music therapy. The Microsoft
Kinect, a commercial 3D motion capture gaming device released in late 2010, was
chosen as the sensing device because of its low cost, small size, and simple
interface, yet retaining superb sensing capabilities. The system
uses open source code to gather data from the Kinect, and software written by
the author in Max/MSP to parse movement data into sound. There are two modes of
the system: the first sonifies specific movement tasks like kicking, reaching,
and balancing, based on the theory of Patterned Sensory Enhancement (Michael Thaut, 2005)
to motivate movement and retrain the brain; and the second is an interactive
musical playspace that reacts to movement within the space in direct but not
immediately obvious ways. The music occurs in response to various movements of
the body, including location and amount of motion. The goal of the second mode
is to encourage exploration of space and the body, musical expression, play,
and social interaction.
