Exploring Film Auteurship with the ACTION toolbox

From exposing Jackson Pollock forgeries to clarifying the sections of the Federalist Papers written by Alexander Hamilton, computational analysis and machine learning have proven to be powerful tools in the study of authorship. Film scholar Warren Buckland used the statistical analysis of shot lengths and shot types to make a persuasive claim that Tobe Hooper, and not Steven Spielberg as rumored, directed Poltergeist (1982). However, Buckland and other scholars using Cinemetrics have had to manually enter data for these elements. Film and media scholars have not yet been able to use computers to algorithmically read and analyze a film’s formal elements. In this paper, we will introduce ACTION (Audio-visual Cinematic Toolbox for Interaction, Organization, and Navigation), an open source platform that supports the computational analysis of film and other audiovisual materials. Funded by a NEH Digital Humanities Start-up Grant, the ACTION toolkit includes features extraction, multi-feature pattern analysis. and machine learning tools. These tools include color features, motion features, structural segmentations, audio features, and analyses based on automatic labeling of the data via machine learning. ACTION provides a work bench to study such features in combination with machine learning methods to yield latent stylistic patterns distributed among films and directors. As such, ACTION is a platform that advances new research methods in the study of film and media history. One specific benefit of ACTION is that it performs automatic segmentation, eliminating the need for researchers to manually record shot lengths. ACTION’s OpticalFlow feature generates analysis data of general motion on screen, tracking salient points of image data across consecutive frames. What ACTION does not replace, however, is the need for research questions and human interpretation. After introducing the software, we will show its application to several Hitchock films. The goal here is not to solve any mysteries about who authored any films. Instead, we show how ACTION illuminates shot color and framing patterns across Hitchock films that would be difficult for a single viewer to detect. These patterns open up new questions that allow us to reconsider films and a director who we think we know.