A Multimodal Language Faculty

Natural human communication is multimodal. We pair speech with gestures, and combine writing with pictures from online messaging to comics to advertising. This richness of human communication remains unaddressed in linguistic and cognitive theories which maintain traditional amodal assumptions about language. What is needed is a new, multimodal paradigm.
This book posits a bold reorganization of the structures of language, and heralds a reconsideration of its guiding assumptions. Human expressive behaviors like speaking, signing, and drawing may seem distinct, but they decompose into similar cognitive building blocks which coalesce in emergent states from a singular multimodal communicative architecture. This cognitive model accounts for unimodal and multimodal expression across all of our modalities, providing a "grand unified theory" that incorporates insights from formal linguistics, cognitive semantics, metaphor theory, Peircean semiotics, sign language, gesture, visual language, psycholinguistics, and cognitive neuroscience. Such a perspective reconfigures how we understand linguistic structure, diversity, universals, innateness, relativity, and evolution. A Multimodal Language Faculty directly confronts centuries-old notions of language and offers a compelling reimagination of what language is and how it works.

Neil Cohnis Associate Professor of Communication and Cognition at Tilburg University, the Netherlands. He is the author of The Visual Language of Comics (2013), Who Understands Comics?(2020), and editor of The Visual Narrative Reader(2016).

List of Figures List of Tables Part I: Foundations 1. Reimagining Language Part II. Modalities 2. What is a Modality? 3. Interfacing Between Modalities Part III: Meaning 4. Conceptual Structures for Multiple Modalities 5. Multimodal Semantic Interactions Part IV: Grammar 6. The Complexity Hierarchy 7. Interactions Between Combinatorial Schemas Part V: Multimodality 8. Unimodal Expressions in a Multimodal Model 9. Independent Multimodal Interactions 10. Substitutive Multimodal Interactions Part VI: A Multimodal Language Faculty 11. Consequences of a Multimodal Language Faculty 12. Evolution of a Multimodal Language Faculty 13. Towards a Multimodal Linguistics References Index