Digitally Extending the Optical Soundtrack

The optical soundtrack has a long history in experimental film as a means of image sonification. The technique translates image luminance into amplitude along the vertical axis, enabling the sonification of a wide variety of filmed patterns. While the technical challenges of working with film preclude casual exploration of the technique, digital implementation of optical image sonification allows interested users with skill sets outside of film to access this process as a means of sonifying video input. This paper presents an overview of the workings of optical image sonification, as well as a basic implementation in the Max/MSP/Jitter environment. The benefits and drawbacks of the technique in its digital and analog forms are discussed, as well as the greatly improved control over the sonification process afforded by digital image processing. An example of these expanded possibilities in the context of audiovisual composition is presented, and supplementary code is given to provide a basis for users to test and apply the technique.