Michael Casey

From exposing Jackson Pollock forgeries to clarifying the sections of the Federalist Papers written by Alexander Hamilton, computational analysis and machine learning have proven to be powerful tools in the study of authorship. Film scholar Warren Buckland used the statistical analysis of shot lengths and shot types to make a persuasive claim that Tobe Hooper, and not Steven Spielberg as rumored, directed Poltergeist (1982). However, Buckland and other scholars using Cinemetrics have had to manually enter data for these elements.

With an optimal network topology and tuning of hyperparameters, artificial neural networks (ANNs) may be trained to learn a mapping from low level audio features to one or more higher-level representations. Such artificial neural networks are commonly used in classification and re-gression settings to perform arbitrary tasks. In this work we suggest re-purposing auto-encoding neural networks as musical audio synthesizers.

Much of music neuroscience research has focused on finding functionally specific brain re-gions, often employing highly controlled stimuli. Recent results in computational neuroscience suggest that auditory information is represented in distributed, overlapping patterns in the brain [4] and that natural sounds may be optimal for studying the functional architecture of higher order auditory areas [3]. With this in mind, the goal of the present work was to decode musical informa-tion from brain activity collected during naturalistic music listening.

Abstract. The ability to reconstruct audio-visual stimuli from human brain activity is an important step towards creating intelligent brain-computer interfaces and also serves as a valuable tool for cognitive neu-roscience research. We propose a general method for stimulus reconstruc-tion that simultaneously learns from multiple sources of brain activity and multiple stimulus representations.

Our previous work (Casey, Thompson, Kang, Raizada, and Wheatley 2012) investigated decoding hemodynamic brain activity in the feed-forward pathways involved in music listening with rich stimuli. Our current work investigates top-down music processing via auditory imagery with an imagined music task. Most previous work on auditory imagery (e.g. Zatorre 2000; Zatorre, Halpern, and Bouffard 2010) used familiar tunes, such as nursery rhymes, that have associated lyrics which elicit activation of language areas in the brain.

A new branch of systematic musicology, “normative musicology,” is proposed and its practice demonstrated. Normative musicology is the study of optimal (“norma-tive”) expectations about future musical signals, given some corpus of past signals. It is a formalization of many “statistical learning” approaches (e.g. [1]) and may be considered a computational counterpart to empirical musicology.

The cognitive representations that support our experience of pitch perception and imagery are not well understood and they generally focus on tonotopic organization of neural columns in the brain (place-based coding of absolute frequency). From prior behavioural studies, we understand musical pitch space to be relative to a reference key, and hierarchically organized. Our current study uses a new between-subject common representation of spatio-temporal multivariate population codes to identify the representational space of musical pitch.

The use of machine learning methods in functional neuroimage analysis has demonstrated an increased sensitivity to cognitive function compared to previously used univariate methods (Kilian-Hütten 2011, Naselaris 2011). This, coupled with the continued progression of cognitive neuroscience research, has led researchers to employ more ecologically valid experimental procedures and more complex stimuli.