Parag Mital

"Resynthesizing Perception" immserses participants within an audiovisual augmented reality using goggles and headphones while they explore their environment. What they hear and see is a computationally generative synthesis of what they would normally hear and see. By demonstrating the associations and juxtapositions the synthesis creates, the aim is to bring to light questions of the nature of representations supporting perception. Two modes of operation are possible.

The cognitive representations that support our experience of pitch perception and imagery are not well understood and they generally focus on tonotopic organization of neural columns in the brain (place-based coding of absolute frequency). From prior behavioural studies, we understand musical pitch space to be relative to a reference key, and hierarchically organized. Our current study uses a new between-subject common representation of spatio-temporal multivariate population codes to identify the representational space of musical pitch.