Signers and Speakers Show Distinct Temporal Kinematic Signatures in Their Manual Communicative Movements.

Abstract

Using our hands to move a stick along a path differs in systematic ways from using our hands to communicate about moving the stick. Kinematic signatures (e.g., enlarged moving trajectories) have been found to mark a movement as communicative, relative to its non-communicative counterpart. But communicative movements are frequently embedded within an expressive system and might differ as a function of that system. For example, deaf signers move their hands when they communicate with sign language, which is a linguistic system. Hearing speakers also move their hands-they gesture along with speech-but those gestures do not form a linguistic system unto themselves. Do the communicative movements signers and speakers use to describe the same event differ as a function of the expressive systems within which they are embedded? Because some signs are highly iconic, researchers often assume that movements in these signs have the same properties as speakers' gestures. To test this assumption, we compared spontaneous hand gestures produced by hearing speakers when they talk (co-speech gesture) to productive iconic hand signs produced by deaf signers when the signs superficially resemble co-speech gestures (classifier signs). We used motion tracking and kinematic analyses to disentangle the spatial and temporal kinematic patterns of communicative movements in 33 English-speakers and 10 American Sign Language (ASL) signers, using each group's non-communicative movements as a control. Participants copied a movement on an object performed by a model (non-communicative movement) and then described what they did with the object (communicative movement). We found no differences between groups in how non-communicative movements related to communicative movements for spatial kinematics. However, for temporal kinematics, speakers' co-speech movements were less rhythmic and jerkier than their non-communicative movements, but signers' communicative movements were more rhythmic and smoother than their non-communicative movements. We thus found differences in the temporal aspects of co-speech gestures vs. classifier signs, leading to 3 conclusions: (i) Communicative movements do not always have the same kinematic signatures but depend on the expressive system within which they are embedded. (ii) Since signers' and speakers' communicative movements have different kinematic features, even highly iconic signed movements cannot be considered entirely gestural. (iii) We need fine-grained techniques to measure communicative movements, particularly when trying to identify the gestural aspects of sign. Communicative movements, even when superficially similar, differ as a function of the system they are part of.