Tuning machines: an approach to exploring how Instagram’s machine vision operates on and through digital media’s participatory visual cultures

ABSTRACT The work of training machine vision systems is diffused into the participatory cultures of social media. As we use social media platforms to express ourselves we assemble databases and train algorithms; and these algorithms in turn shape our everyday cultural practices. In this article, we describe a machine vision system that we built to undertake an unsupervised classification of 13,000 images posted to Instagram from Splendour in the Grass, a large Australian multi-day music festival with over 40,000 attendees featuring international musical acts and arts performances. We demonstrate how unsupervised approaches operate as open-ended queries, rather than definitive classifications. Once a machine vision system has ‘learned’ the unique numerical feature vector associated with an art object, brand activation or gendered pose, it can be used to search for other similar users and moments. We critically explore how the capacity of machines to cluster and classify these Instagram images is interdependent with the mediatized enclosures of popular cultural events and their participatory cultures, and hence represents continuities with the longer history of experience capitalism. Where unsupervised machine vision is used on an advertiser-funded platform like Instagram it points us to the prospective nature of digital advertising, driven not only by specified targeting of pre-labelled consumer preferences, but also by continuous pattern-mining and prediction, sometimes of patterns that seem impervious to symbolic labels. We argue for the importance of critical approaches that explore the open-ended and prospective interplay between culture and machine vision. We need to investigate the feedback loops between the design and use of our cultural spaces, the creativity of participants and users, and the development of platforms’ technologies and business models.