Audiovisual Resynthesis in an Augmented Reality

ACM Multimedia
"Resynthesizing Perception" immserses participants within an audiovisual augmented reality using goggles and headphones while they explore their environment. What they hear and see is a computationally generative synthesis of what they would normally hear and see. By demonstrating the associations and juxtapositions the synthesis creates, the aim is to bring to light questions of the nature of representations supporting perception. Two modes of operation are possible.

Decoding Absolute and Relative Pitch Imagery in the Auditory Pathway

CCN Colloquium

Michael Casey - Decoding Pitch Imagery in the Auditory Pathway

Our previous work (Casey, Thompson, Kang, Raizada, and Wheatley 2012) investigated decoding hemodynamic brain activity in the feed-forward pathways involved in music listening with rich stimuli. Our current work investigates top-down music processing via auditory imagery with an imagined music task. Most previous work on auditory imagery (e.g. Zatorre 2000; Zatorre, Halpern, and Bouffard 2010) used familiar tunes, such as nursery rhymes, that have associated lyrics which elicit activation of language areas in the brain.

Normative Musicology: Automatic Tonal Induction via Entropy and Rational Expectation

Milestones in Music Cognition Workshop
A new branch of systematic musicology, “normative musicology,” is proposed and its practice demonstrated. Normative musicology is the study of optimal (“norma-tive”) expectations about future musical signals, given some corpus of past signals. It is a formalization of many “statistical learning” approaches (e.g. [1]) and may be considered a computational counterpart to empirical musicology.

How Humans Hear and Imagine Musical Scales

Decoding Population Responses Workshop
The cognitive representations that support our experience of pitch perception and imagery are not well understood and they generally focus on tonotopic organization of neural columns in the brain (place-based coding of absolute frequency). From prior behavioural studies, we understand musical pitch space to be relative to a reference key, and hierarchically organized. Our current study uses a new between-subject common representation of spatio-temporal multivariate population codes to identify the representational space of musical pitch.

Reconstructing Musical Audio Features From Continuous Single-Trial EEG

The Neurosciences and Music-V: Cognitive Stimulation and Rehabilitation
The use of machine learning methods in functional neuroimage analysis has demonstrated an increased sensitivity to cognitive function compared to previously used univariate methods (Kilian-Hütten 2011, Naselaris 2011). This, coupled with the continued progression of cognitive neuroscience research, has led researchers to employ more ecologically valid experimental procedures and more complex stimuli.