Track topics on Twitter Track topics that are important to you
Human communication is naturally multimodal, and substantial focus has examined the semantic correspondences in speech-gesture and text-image relationships. However, visual narratives, like those in comics, provide an interesting challenge to multimodal communication because the words and/or images can guide the overall meaning, and both modalities can appear in complicated "grammatical" sequences: sentences use a syntactic structure and sequential images use a narrative structure. These dual structures create complexity beyond those typically addressed by theories of multimodality where only a single form uses combinatorial structure, and also poses challenges for models of the linguistic system that focus on single modalities. This paper outlines a broad theoretical framework for multimodal interactions by expanding on Jackendoff's (2002) parallel architecture for language. Multimodal interactions are characterized in terms of their component cognitive structures: whether a particular modality (verbal, bodily, visual) is present, whether it uses a grammatical structure (syntax, narrative), and whether it "dominates" the semantics of the overall expression. Altogether, this approach integrates multimodal interactions into an existing framework of language and cognition, and characterizes interactions between varying complexity in the verbal, bodily, and graphic domains. The resulting theoretical model presents an expanded consideration of the boundaries of the "linguistic" system and its involvement in multimodal interactions, with a framework that can benefit research on corpus analyses, experimentation, and the educational benefits of multimodality.
This article was published in the following journal.
Using multi-atlas registration (MAR), information carried by atlases can be transferred onto a new input image for the tasks of region of interest (ROI) segmentation, anatomical landmark detection, an...
Opioids have been the mainstay for postoperative pain relief for many decades. Recently, opioid-related adverse events and death have been linked to postoperative dependency. Multimodal approaches to ...
Multimodal rehabilitation interventions delivered in late phase of stroke recovery involve physical (motor and sensory), social, and cognitively challenging activities. Horseback riding can be incorpo...
Prompt, accurate, objective assessment of concussion is crucial, particularly for children/adolescents and young adults. While there is currently no gold standard for the diagnosis of concussion, the ...
Opioids are the mainstay of pain management in critically ill trauma patients. However, the risks of opioid use mandate a different approach. Multimodal analgesia employs a combination of opioid and n...
Pragmatic cluster-randomized clinical trial. An open, multicentric, parallel-group trial with a control group, and with a follow-up period of 1 year.
To explore the neural and cognitive mechanisms of depression (including unipolar depression and Bipolar disorder) and anxiety using multimodal MRI techniques.
Stroke, traumatic head injury, subarachnoid hemorrhage and cerebral anoxia are main causes of a coma condition implying severe brain damage and thus, poor prognosis. Clinicians are often i...
FRench Acute cerebral multimodal imaging to select patient for MEchanical thrombectomy is a prospective multi-center study to determine if multimodal imaging could identify patients who ma...
Pain management after shoulder surgery remains to be a major challenge. Reliance on a single agent (most commonly opioids) results in increased incidence of side effects. Multimodal pain m...
The use of combination of imaging techniques or platforms (e.g., MRI SCAN and PET SCAN) encompassing aspects of anatomical, functional, or molecular imaging methods.
A disease marked by repeated episodes of increased bone resorption followed by excessive attempts at repair, resulting in weakened, deformed bones of increased mass. The resultant architecture of the bone assumes a mosaic pattern in which the fibers take on a haphazard pattern instead of the normal parallel symmetry.
The art and science of designing buildings and structures. More generally, it is the design of the total built environment, including town planning, urban design, and landscape architecture.
Works consisting of drawings of architecture and architectural projects, whether the project was executed or not. (Art & Architecture Thesaurus, 1990, v.2)
Works on the art and science of designing buildings and structures. More generally, works on the design of the total built environment, including town planning, urban design, and landscape architecture.