Languages and Modalities最新文献

Graphological and semantic foregrounding as affecting gaze and speech of impulsive and reflective readers Graphological和语义前景影响冲动和反思性读者的凝视和言语

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.78721

A. Izmalkova, A. Rzheshevskaya

{"title":"Graphological and semantic foregrounding as affecting gaze and speech of impulsive and reflective readers","authors":"A. Izmalkova, A. Rzheshevskaya","doi":"10.3897/lamo.2.78721","DOIUrl":"https://doi.org/10.3897/lamo.2.78721","url":null,"abstract":"The study explores the effects of graphological and semantic foregrounding on speech and gaze behavior in textual information construal of subjects with higher and lower impulsivity. Eye movements of sixteen participants were recorded as they read drama texts with interdiscourse switching (semantic foregrounding), with features of typeface distinct from the surrounding text (graphological foregrounding). Discourse modification patterns were analyzed and processed in several steps: specification of participant/object/action/event/perspective modification, parametric annotation of participants’ discourse responses, contrastive analysis of modification parameter activity and parameter synchronized activity. Significant distinctions were found in eye movement parameters (gaze count and initial fixation duration) in subjects with higher and lower impulsivity when reading parts of text with graphological foregrounding. Impulsive subjects tended to visit the areas more often with longer initial fixations than reflective subjects, which is explained in terms of stimulus-driven attention, associated with bottom-up processes. However, these differences in gaze behavior did not result in pronounced distinctions in discourse responses, which were only slightly mediated by impulsivity/reflectivity.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132215771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Meaning construal through multimodal clusters in the theatrical discourse 戏剧话语中多模态集群的意义解释

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.78572

E. Loginova

{"title":"Meaning construal through multimodal clusters in the theatrical discourse","authors":"E. Loginova","doi":"10.3897/lamo.2.78572","DOIUrl":"https://doi.org/10.3897/lamo.2.78572","url":null,"abstract":"The paper presents the results of an empirical study of monomodal and multimodal data collected from the plays by Russian playwrights of the second half of the 20th century (A. Vampilov, A. Kazantsev, N. Sadur) and the theatrical performances based on these plays. The main aim is to find out what semiotic modalities are more frequently involved in the process of recurrent semiosis and, as such, in the multimodal meaning construal. Following the classical as well as contemporary works in the field explored, the research sets out to elaborate the cognitive-semiotic approach to the analysis of the theatre-specific means of multimodal meaning construal to get insights into dynamicity, embodiment and figurativity of artistic expression typical of theatrical performativity and inaccessible to other arts. The data are explored by means of a complex methodology that comprises quantitative and qualitative methods with the diagraph analysis as a key analytical tool which serves to recognize meaningful correlations within and across modalities resulting from mappings between dialogically juxtaposed independent utterances. This paper demonstrates that the involvement of nonverbal resources is mainly manifested in the use of hand and head gestures, facial movements and body repositioning. Next to the quantitative results, the research shows that co-occurring conceptualizations through verbal and nonverbal components, gestures in particular, can trigger off the amplification of meaning or bring about the emergence of new conceptual projections.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121717117","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

The generic features of frontpage news items 头版新闻条目的一般特征

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.e72256

E. Kosichenko

{"title":"The generic features of frontpage news items","authors":"E. Kosichenko","doi":"10.3897/lamo.2.e72256","DOIUrl":"https://doi.org/10.3897/lamo.2.e72256","url":null,"abstract":"The article considers the role of frontpage news items in creating a political or a social event, and with reliance on the BBC Web news bulletin seeks to prove that frontpage news items are a special type of multimodal text with recognizable generic characteristics: a headline, a picture and an abstract. Structurally, the present paper consists of three sections, each of which is devoted to a certain aspect of the problem in question. Firstly, the author looks into the history of scientific investigations of genres. Considering both fundamental works and the latest achievements in this field, the author stresses the social role of genres and maintains the view that different types of texts appear as a response to social expectations. Secondly, the issue of multimodality is brought up, and frontpage news items are described as texts written in two semiotic modes. The combination of a written text and a picture serves the purpose of constructing a social or a political event informing and influencing the reader at the same time. Thirdly, analysis of seven BBC frontpage news items is done to prove that frontpage form of news presentation is a special genre that has both technological and social roots. This genre has recently evolved from a traditional way of introducing news and has a number of distinctive features, like a short and meaningful headline provided with an abstract different from the headline of the linked article, an image that often conveys a meaning different from that of the headline. The contradictory meanings serve to construct events and form opinions; suggestion is made that the more meanings there are, the more politicized the event is.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"39 11","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132644110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

From visual perception to comprehension: Variations in construal and gaze behavior 从视觉感知到理解:解释和凝视行为的变化

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.78693

Yana Novikova, M. Kiose

{"title":"From visual perception to comprehension: Variations in construal and gaze behavior","authors":"Yana Novikova, M. Kiose","doi":"10.3897/lamo.2.78693","DOIUrl":"https://doi.org/10.3897/lamo.2.78693","url":null,"abstract":"The study explores the construal shifts from VISUAL PERCEPTION to COMPREHENSION image schemas (Grady, 2005) which the readers may face when they deal with the contexts employing the verb of visual perception видеть (see) in the Russian language. The aim of the study is to find out how different image schema types affect the readers’ gaze behavior and default interpretation. It develops a two-stage procedure which involves the corpus-based semantic analysis followed by the oculographic experiment and participants’ reports. The corpus-based search and semantic annotation help define a range of linguistic attractors (Gibbs, 2006) contingent on the image schema types of visual perception and comprehension with the latter exhibiting a more abstract character (Grady, 2005). To proceed, we appeal to the morphological, lexical, and syntactic attractors of the verbs of perception and abstract semantics outlined in (Kustova, 2004; Baicchi & Digonne, 2018; Béligon et al., 2019) and apply them for further samples annotation and contingency analysis. The typical attractors for VISUAL PERCEPTION image schema involve the verb in the perfective, visual semantics in pre-position, direct object in post-position, negation marker, animated direct object in post-position; whereas the COMPREHENSION image schema attractors are scarce and are restricted to attributes of evaluation in pre-position and several adverb types. The oculographic experiment tests the effects of perceptual construal priming and the construal entrenchment on gaze behavior and default interpretation. The experiment did not reveal significant schema type contingency on interpretation, however in terms of gaze behavior COMPREHENSION image schema was more demanding.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"43 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114781800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Why do we laugh? A semiotic analysis of British comedy duo sketches Why我们会笑吗?英国喜剧二人组小品的符号学分析

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.78375

Stephen Pax Leonard

{"title":"Why do we laugh? A semiotic analysis of British comedy duo sketches","authors":"Stephen Pax Leonard","doi":"10.3897/lamo.2.78375","DOIUrl":"https://doi.org/10.3897/lamo.2.78375","url":null,"abstract":"Comedy has long been analysed from a pragmatic perspective with the predictable conclusion that we laugh because one of the four Gricean maxims has been violated. However, the wording of Grice’s maxims is so loose and flexible that more or less any joke would violate one of his maxims and thus the ‘Cooperative Principle’. So, we are still left mediating the meta-pragmatic question of what it is that lies behind verbal incongruity that makes us actually laugh? This article analyses the notion of incongruity from a Peircean semiotic perspective and focuses exclusively on a selection of British comedy duo sketches whose humour is derived overwhelmingly from discursive, lexical and socio-phonetic incongruity. On the basis of classic British comedy due sketches at least, there is some mileage in perceiving incongruity as a semiotic misalignment or ‘indexical shock’ which subverts our basic social expectations by indexing non-presupposed contexts. We laugh because our verbal norms are not only challenged, but are turned upside down and torn apart. Moreover, we laugh because the social identities that the speech acts aim to index non-referentially often clash or conflict immediately with those of his or her interlocutor’s.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114320131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Aesthetic multimodality of speech and gesture: Towards its functional framework Aesthetic语言和手势的多模态:走向其功能框架

Languages and Modalities Pub Date : 2022-05-17 DOI: 10.3897/lamo.2.78840

M. Kiose, A. Leonteva, Olga Agafonova

{"title":"Aesthetic multimodality of speech and gesture: Towards its functional framework","authors":"M. Kiose, A. Leonteva, Olga Agafonova","doi":"10.3897/lamo.2.78840","DOIUrl":"https://doi.org/10.3897/lamo.2.78840","url":null,"abstract":"The study develops a functional multimodal approach to speech and gesture behavior to explore aestheticism in more and less staged discourse of cinema and interview. We hypothesize that cinema and interview employ the same communicative functions; however, these functions constitute different frameworks which contribute to the higher aesthetic potential of cinema. This approach allows to study the aesthetic via communicative functions frameworks in multimodal discourse.\u0000 To establish the function frameworks in cinema and interview, we apply a contrastive functional analysis of speech and gesture in the highly ranked actors’ argumentative and descriptive monologues. With the help of variance and regression analysis, we explore the distribution of pragmatic and discourse-structuring functions (with sub-functions) in speech as contingent on pragmatic, deictic, representational and adaptive functions of gestures. The study confirms that cinematic discourse exploits fewer deictic, representational and adaptive gesture functions, whereas pragmatic gesture functions (especially emphatic ones) appear more frequently and are contingent on several pragmatic and discourse-structuring functions of argumentative and descriptive speech. Interview function frameworks display lower predictability, which shows higher spontaneity of gestures; however, there are specific gestures typical of interview (self-adaptors) which may serve as indicators of pragmatic functions of argumentation. The study also manifests individual variations within the function frameworks. Overall, cinema and interview display variance in replication and regularity of speech and gesture functions, which presumably helps create higher and lower aesthetic effects.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117185576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring the role of the body in communicating ironic stance 探讨身体在表达讽刺姿态中的作用

Languages and Modalities Pub Date : 2021-10-25 DOI: 10.3897/lamo.1.68876

Clarissa de Vries, Bert Oben, Geert Brône

{"title":"Exploring the role of the body in communicating ironic stance","authors":"Clarissa de Vries, Bert Oben, Geert Brône","doi":"10.3897/lamo.1.68876","DOIUrl":"https://doi.org/10.3897/lamo.1.68876","url":null,"abstract":"Performing and understanding conversational irony requires a complex management of multiple viewpoints. To communicate and negotiate these intricate viewpoint shifts, speakers (and addressees) often use nonverbal means (e.g. gaze shifts, shrugs, shifts in body orientation, hand gestures, etc.) next to verbal viewpoint strategies. In the present paper we zoom in on the perspective of the speaker and try to describe and quantify bodily behavior in ironic utterances compared to non-ironic ones. To this end, we use data from a video-corpus of three-party interactions with participants wearing mobile eye-tracking devices that allow for precise eye gaze data. Our results show that speakers display more of the multimodal resources under scrutiny in ironic cases compared to non-ironic cases. More specifically, the involvement of bodily resources is mainly manifested in the use of laughter, head movements and body repositionings. We further show how those resources cluster into certain multimodal packages, and how the exact timing of the bodily behavior is relevant (i.e. the gaze behavior at the end of an ironic segment differs most notably from the end of a non-ironic one). Next to a quantitative analysis of the resources used in ironic talk in interaction, we also illustrate our findings with qualitative descriptions of relevant examples.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123873888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the diachronic aspect of gestural communication 论手势交际的历时性

Languages and Modalities Pub Date : 2021-10-25 DOI: 10.3897/lamo.1.70667

S. Savchuk

{"title":"On the diachronic aspect of gestural communication","authors":"S. Savchuk","doi":"10.3897/lamo.1.70667","DOIUrl":"https://doi.org/10.3897/lamo.1.70667","url":null,"abstract":"Modern studies of gesturing conclusively prove that a gestural communication system accompanies oral speech the units of which, like linguistic ones, can be described using a limited set of reproducible features assigned to certain classes and correlated with certain contexts of use.\u0000 The diachronic aspect of gesturing has been little studied, although there is an understanding that human gesture behaviour, like speech, changes in space, time as well as under the influence of changing sociocultural conditions. Changes in the gestural system usually refer to innovations, the emergence of new gestures. It is much more difficult to describe the gestures that have gone or are going out of use since due to the lack of video recording they have to be restored from descriptions preserved in literature.\u0000 Examples of gestures that have recently entered the Russian gestural system as well as examples of gestures that have gradually become obsolete are considered in this work. The data of the Multimodal Russian Corpus and some other corpora within the Russian National Corpus (RNC) used in this survey enable a clarification of the semantic and pragmatic characteristics of the gestures and changes in their use to be tracked.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133730708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The body between meaning and form: kinesiological analysis and typographical representation of movement in Sign Languages 意义与形式之间的形体:动作学分析与手语运动的排版表现

Languages and Modalities Pub Date : 2021-10-25 DOI: 10.3897/lamo.1.68149

Léa Chèvrefils, Claire Danet, Patrick Doan, Chloé Thomas, Morgane Rébulard, Adrien Contesse, J. Dauphin, Claudia S. Bianchini

{"title":"The body between meaning and form: kinesiological analysis and typographical representation of movement in Sign Languages","authors":"Léa Chèvrefils, Claire Danet, Patrick Doan, Chloé Thomas, Morgane Rébulard, Adrien Contesse, J. Dauphin, Claudia S. Bianchini","doi":"10.3897/lamo.1.68149","DOIUrl":"https://doi.org/10.3897/lamo.1.68149","url":null,"abstract":"Most of the research on Sign Languages (SLs) and gesture is characterized by a focus on hands, considered the sole body parts responsible for the creation of meaning. The corporal part of signs and gestures is then blurred by hand dominance. This particularly impacts the linguistic analysis of movement, which is described as unstable, even idiosyncratic. Boutet’s Kinesiological Approach (KinApp) repositions the speaker’s body at the core of meaning emergence: how this approach considers and conceptualizes movement is the subject of this article. First, the reasons that led SLs researchers to neglect the analysis of the sign signifying form, focusing on the hand, are exposed. The following part introduces KinApp which, through a radical change of point of view, allows revealing the simplicity and stability of movement: understanding the cognitive and motor reasons for this stability is the subject of research whose methodology is described. Setting the body at the center of analysis requires a descriptive model capable of accounting for the SLs signifying form, thus going beyond existing transcription systems. The last part is devoted to the presentation of Typannot, a new transcription system, aimed not only at a kinesiological description of SLs but also at assisting researchers to modify how they understand and analyze movement.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115044290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

“Of thee I sing”: An opening to Dominique Boutet’s kinesiological approach to gesture “我为你歌唱”:多米尼克·布特(Dominique Boutet)对手势的运动学方法的开场

Languages and Modalities Pub Date : 2021-10-25 DOI: 10.3897/lamo.1.68148

Aliyah Morgenstern, Léa Chèvrefils, Marion Blondel, C. Vincent, Chloé Thomas, Jean-François Jégo, D. Boutet

{"title":"“Of thee I sing”: An opening to Dominique Boutet’s kinesiological approach to gesture","authors":"Aliyah Morgenstern, Léa Chèvrefils, Marion Blondel, C. Vincent, Chloé Thomas, Jean-François Jégo, D. Boutet","doi":"10.3897/lamo.1.68148","DOIUrl":"https://doi.org/10.3897/lamo.1.68148","url":null,"abstract":"In this tribute to Dominique Boutet and the kinesiological approach he founded, the authors have tried to make their memories of the scientific collective projects they worked on together resonate with the written work of this extraordinary scientific partner, gathered in his published articles and his habilitation document (Boutet 2018). His approach based on an intimate knowledge of the biomechanics of the human body is centered on the structuring role of the body in gestures (and signed languages). It is form-based: the form of gestures shapes their meaning or function. Gestural units are described on the basis of their formal characteristics and physiological constraints rather than their imagistic iconicity.\u0000 The article presents the foundations of the approach, a synoptic description and some examples of its application. The originality of the kinesiological approach lies in the double revolution that it allows us to operate: on the one hand, gesture is not simply an appendix of speech; on the other hand; it is shaped by bodily physiology. The approach is based on the movements of the human body analyzed from a biomechanical point of view. The meaning of our gestural productions is the produce of our body, as it is naturally articulated, imprinted as it is by our past experiences.","PeriodicalId":121794,"journal":{"name":"Languages and Modalities","volume":"214 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117338297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2