Language Use and Multimodal Communication - Prof. Dr. Cornelia Müller

We welcome you to study with us how gesture, language and audiovisual media work.

With the ring gesture Obama expresses the preciseness of the points he is making. The meaning of the gesture is an embodied meaning, it is derived from the practical action of picking up small objects. In our team, we investigate how gestures mean, how they are used with spoken language and how their perception is shaped by their orchestration in audiovisual media. Currently, we investigate how body movement and speech work together to express affective stance (multimodality of speaking) and how the audiovisual staging of those speakers creates an affective experience for the viewers (audiovisual multimodality). We study affective stance in German and Polish parliamentary debates. Moreover, we work on a textbook on Gesture and Language. This includes a cross-disciplinary account of how gestures work as a bodily mode of expression, the integration of gestures in multimodal utterances and multimodal interaction, an overview of the field of gesture studies and various methods of gesture analysis.

Cornelia Müller works on multimodal forms of language use, focusing on embodied, affective, and dynamic processes of meaning making in gestures and in audio-visual media. She has published on many facets of gesture as a medium of expression and on multimodal metaphor. She has investigated gestural mimesis, emergent proto-linguistic gestural forms (sedimentation, conventionalization) and developed Methods for Gesture, Film and Metaphor Analysis. Together with Adam Kendon she has launched and co-edited the international journal GESTURE and the book series GESTURE Studies (Benjamins) from 2000 to 2010. With Hermann Kappelhoff, she has developed a transdisciplinary (film studies and linguistic) approach to the experiential and affective dynamics of metaphorical meaning in speech, gestures, and audiovisual media.

Her most recent publications include:

Müller, C. (2025). Sprache-Sprechen, Hören, Sehen, Fühlen. Eine medienästhetische Perspektive auf die Multimodalität des Sprechens in audiovisuellen Medien. In R. Vallentin & D. Horst (Hrsg.) Sprache entgrenzen. Beiträge zu einer kulturwissenschaftlichen Linguistik. transcript Verlag.

Müller, C. et al (2024). Affectivity as stance: Multimodal stance-taking in audiovisual documentations of Polish and German parliamentary debates. Frontiers in Psychology, 15, 1467185. 

Müller, C. (2024). A toolbox for methods of gesture analysis. In: Alan Cienki (ed.) The Cambridge Handbook of Gesture Studies, 182-216. Cambridge University Press.

Müller, C. (2024). Dead and Alive, Sleeping and Waking Metaphors: The Spectrum of Metaphor and the Multimodality of Discourse. In: Anders Örtenblad (ed.) The Oxford Handbook of Metaphor in Organization Studies, 70-84. Oxford: Univ. Press.

Müller, C. (2024). Gestural mimesis as ‚as-if‘ action. In: Przemysław Zywiczynski, Johan Blomberg and Monika Boruta-Zywiczynska (eds.) Perspectives on Pantomime, 217-241. Amsterdam/Philadelphia: John Benjamins.

Müller C. (2023). Language from the Body – Dynamic relations between gestures and signed language. In: Terry Janzen and Barbara Shaffer (eds.), Signed Language and Gesture Research in Cognitive Linguistics, XIII-XVI. De Gruyter Mouton.

Language Use and Multimodal Communication

Picture (from left to right): Teresa Weigand, Clara Kindler-Mathôt, Iris Franke, Jeanette-Christine Bauer, Jana Katharina Junge, Cornelia Müller