{"title":"Beyond Disfluency","authors":"Loulou Kosmala","doi":"10.1075/ais.11","DOIUrl":"https://doi.org/10.1075/ais.11","url":null,"abstract":"This book pioneers a tridimensional approach to (dis)fluency, evaluating fluency across three different dimensions, mainly speech, gesture, and interaction. Drawing from an extensive video dataset covering different languages and speech genres in French and English, the present research goes beyond traditional production-oriented models of so-called ‘disfluency’ phenomena, and aims to unravel the complexities of human multimodal production and interactive processes. Designed for linguists, communication scholars, and researchers, this work resonates with the latest trends in different fields (Second Language Acquisition, Interactional Linguistics, and Gesture studies). It introduces a fresh perspective on disfluency by integrating visual-gestural features, such as hand gestures, gaze, and facial expressions, captured in situated interaction.","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"19 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139389715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 6. Quantifying the interplay of gaze and gesture in deixis using an experimental-simulative approach","authors":"Thies Pfeiffer, Patrick Renner","doi":"10.1075/AIS.10.06PFE","DOIUrl":"https://doi.org/10.1075/AIS.10.06PFE","url":null,"abstract":"","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127209328","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 7. Gaze and face-to-face interaction","authors":"G. Bailly, A. Mihoub, Christian Wolf, F. Elisei","doi":"10.1075/ais.10.07bai","DOIUrl":"https://doi.org/10.1075/ais.10.07bai","url":null,"abstract":"This chapter describes experimental and modeling work aiming at describing gaze patterns that are mutually exchanged by interlocutors during situated and task-directed face-to-face two-ways interactions. We will show that these gaze patterns (incl. blinking rate) are significantly influenced by the cognitive states of the interlocutors (speaking, listening, thinking, etc.), their respective roles in the conversation (e.g. instruction giver, respondent) as well as their social relationship (e.g. colleague, supervisor). This chapter provides insights into the (micro-)coordination of gaze with other components of attention management as well as methodologies for capturing and modeling behavioral regularities observed in experimental data. A particular emphasis is put on statistical models, which are able to learn behaviors in a data-driven way. We will introduce several statistical models of multimodal behaviors that can be trained on such multimodal signals and generate behaviors given perceptual cues. We will notably compare performances and properties of models which explicitly model the temporal structure of studied signals, and which relate them to internal cognitive states. In particular we study Semi-Hidden Markov Models and Dynamic Bayesian Networks and compare them to classifiers without sequential models (Support Vector Machines and Decision Trees). We will further show that the gaze of conversational agents (virtual talking heads, speaking robots) may have a strong impact on communication efficiency. One of the conclusions we draw from these experiments is that multimodal behavioral models able to generate co-verbal gaze patterns of interactive avatars should be designed with great care in order not to increase the cognitive load of human partners. Experiments involving an impoverished or irrelevant control of the gaze of artificial agents (virtual talking heads and humanoid robots) have demonstrated its negative impact on communication (Garau, Slater, Bee, & Sasse, 2001).","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115442505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 2. Eye gaze as a cue for recognizing intention and coordinating joint action","authors":"F. Amati, S. Brennan","doi":"10.1075/AIS.10.02AMA","DOIUrl":"https://doi.org/10.1075/AIS.10.02AMA","url":null,"abstract":"","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130595749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 4. Weaving oneself into others","authors":"Rick Dale, Michael J. Spivey","doi":"10.1075/AIS.10.04DAL","DOIUrl":"https://doi.org/10.1075/AIS.10.04DAL","url":null,"abstract":"We review a range of findings that show how eye movements (and other body movements) exhibit correlated behavior across two or more people during natural interactions. We then synthesize these different results into a more general account of how people’s cognitive, sensory and motor systems become coordinated with one another during natural dialogue. We argue that treating conversants as parts of one integrated system is a useful explanatory strategy for understanding interaction. We end by describing explicit quantitative conditions for seeking “systemhood” in human interaction. These conditions motivate future research questions on social eye movements and other behaviors.","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"144 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125848285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 11. Mobile dual eye-tracking in face-to-face interaction","authors":"Anja Stukenbrock","doi":"10.1075/AIS.10.11STU","DOIUrl":"https://doi.org/10.1075/AIS.10.11STU","url":null,"abstract":"","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131156112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 3. Effects of a speaker’s gaze on language comprehension and acquisition","authors":"P. Knoeferle, H. Kreysa, M. Pickering","doi":"10.1075/AIS.10.03KNO","DOIUrl":"https://doi.org/10.1075/AIS.10.03KNO","url":null,"abstract":"","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125519064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 9. Gaze, addressee selection and turn-taking in three-party interaction","authors":"P. Auer","doi":"10.1075/AIS.10.09AUE","DOIUrl":"https://doi.org/10.1075/AIS.10.09AUE","url":null,"abstract":"Abstract: In this paper, I argue that gaze behavior in multiparty interaction is essential for two aspects of turn-taking: for addressee selection and for next-speaker selection by current speaker. The two conversational tasks are related, but – at least in longer turns – not identical and should be distinguished analytically. In multiparty interaction, addressee selection by gaze is a non-trivial issue, as most bodily arrangements make it hard or impossible for the current speaker to look at all (intended) addressees at the same time. Rather, current speakers alternatingly look at the co-participants they want to address. Further details of this pattern of gaze alternation are discussed.","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126853855","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Chapter 8. Automatic analysis of in-the-wild mobile eye-tracking experiments using object, face and person detection","authors":"Stijn De Beugher, Geert Brône, T. Goedemé","doi":"10.1075/ais.10.08beu","DOIUrl":"https://doi.org/10.1075/ais.10.08beu","url":null,"abstract":"","PeriodicalId":261135,"journal":{"name":"Advances in Interaction Studies","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133340237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}