Data as Expression

Abstract

ABSTRACT In machine learning literature, the concept of expression is seldom addressed as a general term in relation to information construction and data. The term more often carries a narrower meaning and refers to human bodily, emotional, or artistic dimensions that are recorded to train a model. However, this paper discusses a view in which expression is understood as any human sensory realm that can bring incidents explicit through the act of “pressing out,” as the early etymology of expression indicates. Also, all the sensory data that are used to train a machine learning model and the data the trained model gives as an output are considered meaningful through their expressive mediality—a form that is actively produced and can therefore be subjected to critical phenomenological analysis. This paper contrasts nonverbal expressions with categorical or linguistic expressions and asks, “What do eye movements express when they are used in training a machine learning model? What kind of expression arises in linguistic models? What could be considered aesthetic data (and what would it express)?” For philosopher John Dewey, instinctive reactions in human behavior that for example exhibit mere discharge of an emotion should be separated from purposeful expressions. However, in machine learning, the two Deweyan positions (instinctive and intentional) collapse, as artificial expressions are purely simulations of learned logic. Therefore, the phenomenological question of this paper is traced back to the original stimulus and its human annotators. In this paper, philosopher Don Ihde’s experimental phenomenology explains how appearances can be attended to without subsuming them under any assumptions, whereas art theory—arising especially from philosopher Dieter Mersch’s thinking—provides an understanding of the mediality of expression. This paper introduces examples from machine learning that are not generally considered expressive but are used for regular tasks, such as object detection, and provides an alternative approach from aesthetics and artistic research that understands these modalities as expressive. If data are understood as expressive, it can be critically assessed how current machine learning models constitute knowledge.