See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/371134046

A Multimodality in Discourse Analysis

Article · October 2022

CITATIONS READS

0 1,927

1 author:

Sawsan Qasim
Al-Mustansiriya University
23 PUBLICATIONS 12 CITATIONS

SEE PROFILE

All content following this page was uploaded by Sawsan Qasim on 29 May 2023.

The user has requested enhancement of the downloaded file.

 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

A Multimodality in Discourse Analysis

Tuqa Kadhim Falih
Mustansiriyah University - College of Arts
Tuqa77000@gmail.com
Assist. Prof. Sawsan A. Qasim
Al-Mustansiriyah University - College of Arts
Sawsan.am@uomustansiriyah.edu.iq
Abstract
There are multiple ways in which language can convey different realities or truths
in linguistics and philosophy. Modality relates to this. For example, the modal
statement may suggest that something is likely, desirable, or authorized. Modal
adverbs like "possibly" and "necessarily" are common, as are modal auxiliary verbs
like "could," "should," and "must." Modal adjectives like "conceivable" and
"probable" are also common. Modes have been discovered in a wide range of
natural language expressions such as counterfactuals and propositional attitudes.
From a variety of perspectives, modality has been extensively studied. Using cross-
linguistic typological study, it has been demonstrated that the processes used to
detect modality are heterogeneous among languages, with a particular emphasis on
their link with tense–aspect–mood marking. Theory linguists have used modal
logic approaches to examine the propositional content of modal statements as well
as the ramifications of modal statements in the context of the conversation.
Metaphysical ideas of certainty and possibility can be seen via the prism of
linguistic modality in philosophy. Whether it's a grand piece of art or a basic daily
routine like curling one's hair, all forms and levels of semiotic production are
incorporated in multimodal meaning-making as humans communicate and interact.
Multimodal meaning-making has been the subject of several studies.

1. Introduction
The term "multimodal meaning-making" refers to the processes and outcomes of
semiotic creation. A crucial idea in social semiotics and multimodality because it
stresses contextual meaning rather than abstract reality. All forms and degrees of
semiotic production are included in multimodal meaning-making since it is what
humans do with their available resources when they communicate and interact in
daily life. We'll focus on three different ways that multimodal meaning-making has
been articulated in the literature. Action, enactment, and choice in multimodal
meaning-making.
Individuals use semiotic resources to generate meaning in social circumstances
through multimodal meaning-making. The myriad actions that social actors engage
in are closely linked to the construction of meaning. Discourse, design,
manufacturing, and dissemination are all discussed by Kress & van Leeuwen
(2001) in relation to how meaning is formed (see entry on multimodal articulation).
Teaching, web design, and other professions need mastery of a wide range of social

‫ج‬

1992
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

practices, such as those found in these four categories of practice. Individuals'

social interactions are dynamic and constantly re-negotiate meaning, according to a
practiceoriented approach to the construction of meaning.
Another way to describe multimodal meaning-making is as [a set of] options (s).
The construction of meaning by the selection of semiotic resources from a pool of
available options in semiotic systems is known as meaning-making. To define a
semiotic system, think of it as a network of meaning-making resources, each with a
set of criteria for selection and combination. "A text is the outcome of ongoing
selection within a huge network of systems": "multimodal meaning-making"
(Halliday & Matthiessen 2004: 23). A critique of meaning-as-selection may be
found in Kress (2010).
2. Multimodality
It is difficult to grasp meanings & its evolution in the world of modern media
without first knowing multi-modality. This is due to communications are modified
when they go from medium to media, genre to genre, & languages to languages.
Furthermore, due to advances in technology, it is increasingly usual to include
media like photographs & hyperlinks into written texts. This communication
method includes a variety of elements that are clearly important & send a specific
message to the audience (Alduhaim, 2017: 79).
The popularity of photography & videography in evaluating human communication
raises curiosity in new modalities of communication. Other than speaking &
writing, the new modalities are nonverbal. In many instances, the vision & verbal
modes are seen as meaningsmaking prospects that need additional investigation.
The function of technology in multimodal communication & the construction of
meanings is shaped. Various semiotic resources, like languages or code, & sensory
resources, like smell or taste, combine to create meanings.
MODES: These are the many resources that are used to define the terms mode &
modality. MODES: The notion of mode has been developed in collaboration with
individuals from a wide range of backgrounds & areas of expertise. It has a number
of criteria & characteristics (Lyons, 2016: 2-3).
Multimodality researches are important for numerous reasons. First,
communication is multimodal since it happens in a variety of modalities (Hiippala,
2017: 4), like text, speech, gestures, & so on, all of which contribute to the process
of meanings-making.
Second, meanings is produced as a result of different adjustments that take place
over the course of interaction. Finally, participants make use of items that are
culturally & socially distinct & that have been agreed upon by the participants.
M.A.K. Halliday, of the Sydney School of Semiotics, is the first to introduce & use
the term "multi-modality," which refers to the modes that possibly employed to
examine objects & words in semiotics. Furthermore, multi-modality has certain
origins & repercussions in the Prague School, as it develops in languages &
culminates in art. Scholars employ multi-modality methodologies to evaluate many
modalities like paintings, pictures, event periods, & costumes (van Leeuwen, 2015:
448).
‫ج‬

1993
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

Multimodality refers to communication channels other than speech, like gestures,

writing, tone of voice, & self-presentation, among others. Semiotic analysis
includes tactics including several modes like image, sound, & languages that are
employed in conjunction with other established technologies. (Ibid)
Kress (2012: 38) defines multi-modality as an interdisciplinary discipline including
ideas & methodologies from several fields of knowledge. Linguistic is one of the
techniques that makes a significant contribution to multi-modality & its
multidisciplinary character. The investigation of these modes necessitates the
participation of other disciplines, since each has its own mode. For example, a
picture involving the history of art & languages is partly considered. Any textual
Research cannot be accomplished without the use of additional channels of
communication. Due to of the participants, the various interpretations are
subjective.
Communication in the modern globalized world is sent through many channels &
formats. Aside from languages, roadways are littered with signage. These signals
use both verbal & nonverbal resources, like pictures, texts, & images with
messages (Al-Hasnawi & Farhan, 2020: 5).
Multimodality is a branch of Research that focuses on the many resources &
locations involved in the process of producing meanings. Different disciplines of
Research, like psychology, archeology, sociology, & so on, are assembled to
develop a technique for analysing various ways of communication (Kress,
2011:38).
Social semiotics develops methodological tools for both communication &
meanings formation. Languages is one of the many distinct resources that form
meanings in multi-modality. For the process of meanings creation, the resources in
one culture are seen as a single unity.
In a multimodal approach, mode is frequently linked to other modes of
communication &, as a result, to a discipline. Modes are coupled to one another &
framed within a single unity in multimodal techniques. They are linked in a single
unity of cultural resources that has been agreed upon in a certain community.
Writing, for example, is dealt with verbally in this manner. All of these modes are
treated similarly since they all contribute to meanings formation. These modes are
dealt with differently in such a multimodal manner, & their features stem from their
differences (ibid).
All deployed resources utilized Via interlocutors to understand action are included
in multimodality: languages, lexicon, gaze, prosody, gestures, & so on. These
resources have many features, the first of which is that they are tied to coordinating
activities.
Second, multimodal resources are all equal & have no precedence over one another.
Third, certain resources are traditional, like grammar, & they are dependent on the
characteristics of the context & participants who make these resources relevant.
Fourth, they are momentarily & sequentially merged. Fifth, certain resources are
configured & are subject to the environments & its limitations (Mondada,
2018:86).
‫ج‬

1994
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

Languages as a communicative act & practice piques the curiosity of researchers

from many disciplines of knowledge both within & outside of languages.
Languages as a communicative act in sociocultural situations is of interest to
linguists, psychologists, and ethnographers of communication, discourse analysts,
& others. This is consistent with other disciplines, since languages is seen as a
means of social activity. The way languages is utilized via individuals to join
together, their surroundings, & other concerns demonstrate this
synchronicity.Multiple epistemological & academic contexts make use of the
concept of multimodality, which is defined as (like computer sciences, logistics &
transport, Analyses of Discourse, & researches of interaction). A variety of
interfaces, networks, & multimedia communication characteristics are all possible
in the field of computer science, which is known as "multi-modality." Materiality
of representation is a semiotic perspective, & it relates to diverse media & varied
types of signals that create & convey distinct semiotic outcomes, like textual
information or images on paper; graphics; or multimedia communications (Kress &
van Leeuwen 2001).
Research on human interaction, like communication theory, often uses the word to
describe how participants coordinate their actions by using a variety of instruments
including motion & gaze, facial expressions & body posture, hand gestures & even
prosody. – in order to coordinate their actions. Multi-modality is seen as
fundamental & central by the majority of the'modalities' stated in this statement. A
perspective of methods as organically interconnected, & of languages as deeply
ingrained in this diversity as the other instruments, with no preset framework, is
fostered as a result (Mondada, 2016: 337-338).
To recapitulate, multi-modality refers to the many different modes that may be used
to provide a collection of information, as well as the manner in which these modes
are presented. The modes may be shown alone or in conjunction with one another
(Farr & Murray, 2016: 511).
2.1. Systematic Practical Linguistic
Systematic Practical Linguistic (SFL), often known as critical Linguistic, is a
languages theory that emphasizes the link among languages, text, & context. Its
scope is broad in the sense that it seeks to explain how people construct meanings
Utilizing languages & other semiotic resources, as well as to comprehend the link
among languages & society. It is a linguistic strategy created Via Halliday (1985).
While many linguistic theories nowadays focus on languages as a mental activity,
SFL is intimately tied to sociology; it investigates how languages is utilized in
social circumstances to accomplish certain purposes. SFL examines the text
generated (whether spoken or written) & its settings rather than how languages is
processed or represented inside the brain. SFL emphasizes languages function
(what it is used for) above languages structure when it comes to languages usage
(how it is composed). (O'Donnell, 2011: 2) Context: Languages, DUE TO SFL,
can only be comprehended in connection to its context of usage. The context of
situation refers to the environment of languages as texts & while the context of

‫ج‬

1995
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

culture refers to the setting in which languages as a system (as opposed to their
constituent parts) may exist (as opposed to their constituent parts) (Halliday, 1978).
In Halliday's words, "the context of circumstance is an analytical concept that
describes how a text ties to the social processes in which it is situated." The
situation's context is comprised of three elements: the field, the tenor, & the
modality. Field refers to the subject of the discussion or writing in the first instance.
For example, general disciplines like science & education, military service,
medicinal, & sports are all instances of generic disciplines, while specific
disciplines like biology, microbiology, languages education, English language
education, & so on are examples of specific areas. Second, the tenor of a discussion
relates to the individuals who are taking part in it as well as their relationships with
one another. There are several instances, including power relations (e.g., unequal:
doctor & patient, teacher & student, equal: friend & friend, student & student),
formality (informal/formal), & proximity (distant/neutral/close).
Third, the mode of interaction refers to the involvement of the languages in the
interaction as well as the form that interaction takes (spoken or written). It is made
up of the Role (Ancillary: languages accompanying nonverbal activity, as when we
talk as we cook together or constitutive: the event is defined Via the languages, as
in a speech), Directionality: unidirectional or bidirectional channel directionality:
written vs. spoken, or any mix of the two directionality (unidirectional allows only
monologue, while a bi-directional channel allows dialogue), Media: +/-vision
contact (for example, -vision contact during a phone conversation); the use of
multimedia resources (video, PowerPoint, etc.), (2011) distinguishes among
spontaneous & intentional preparation, as well as among time for thinking &
hurrying through the process (O'Donnell, 2011).
2.2. Multi-Modal Discourse
Multimodality Analyses of Discourse, DUE TO O'Halloran (2004:1-2), is
concerned with the many semiotic resources, or'modes.' These semiotic resources
also play a role in the development of this multimodal discourse theory & practice.
In multi-modal texts, discourse, additionally, in addition to the functions &
meanings of vision pictures, the multi-modal takes into account the meanings that
is generated from the integrated use of semiotic resources like languages, image &
gestures, music, & architecture. Additionally, have a look at (Hyland & Paltridge,
2011:120).
DUE TO Van Leeuwen (2005:73), multi-modal discourse is concerned with
examining the 'grammars' of diverse modalities of communication & expression. It
is vital to understand how different routes of communication might achieve the
same job. As an example, the lexical coherence among words & visions, the
exchange structure in musical conversations, & the referential cohesiveness among
the action & conversation & music or movies are all examples of how various
modes may be combined in multi-modal texts.
DUE TO Johnston (2008:225), there are many significant sorts of communication
that take place in a range of settings & situations. Authors, artists, & designers have
traditionally worked with words; musicians, on the other hand, have traditionally
‫ج‬

1996
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

worked with music. Nowadays, people employ a number of strategies to do their

tasks successfully. Vision presentation is just as crucial in contemporary art music
as it has always been in other genres of music, if not more so. Many writers are
needed to show a grasp of vision design, & the same is true for graphic designers.
In light of the increasing prevalence of multi-modal communication in public life,
discourse analysts are paying close attention to the many different ways in which
discourse might be multi-modal in nature.
Hallet & Kaplen-Weinger (2010:11) underline the importance of both vision design
& language at the beginning of the multi-modal discussion. When it comes to
investigating the meaningss formed by the syntagmatic interaction among people,
places, & things shown in photographs, a multi-modal Analyses of Discourse is
intended to provide a "detailed & explicit" method. As well as representing
something, these meaningss are often characterized as interactive, that is,
performing actions for the benefit of the observer.
By making use of semiotic resources, we may Moreover, DUE TO Kress
(2012:36-37), the multi-modal text is made coherent both internally, among the
text's fragments, & outwardly, in relation to the context in which it appears. A
multi-modal approach, for example, argues that languages are simply one of many
ways in which we may communicate ourselves & transmit our ideas & emotions to
others. It is the author's overall meanings that is comprised of the collective
meanings that are formed by using all of the modes in a work.
It's possible that "to some extent" modality is realized "in a form of the message
itself," since the text provides cues about the reliability of messages, guiding
receivers in deciphering whether what they see or hear is "true, factual, real," "or
[...] a lie, a fiction," something beyond reality. According to van Leeuwen and
Kress, modality judgment is not an appraisal of a statement's truth, but rather the
degree to which the proposition is portrayed as true or real (Kress 2006: 15455)
Conclusion
Textual modality is the use of language to express statements about the world's
truth value or plausibility. MODALITY expresses an assessment of the certainty or
likelihood of what is expressed; it is "the speaker's judgment, or a request for the
judgment of the listener" in Hallidayen systemic functional grammar. The way
modality conveys probability is through "interpret[ing] the zone of ambiguity that
stands between 'yes' and 'no.'" There are several forms of modality, including
modal auxiliary verbs like "may," "will," and "must," as well as modal adverbs
such as "perhaps," "definitely," and "absolutely," as well as modal adjectives like
"likely" and "probable." Even a high-value mod (such as "absolutely" or "always")
is less decisive than polar forms if you're talking about modality: that's certainly
true. As far as John is concerned, he's not quite so sure about that; summer showers
are less predictable. You only pretend to be certain when you aren't. Adapting
Hallidayen grammar to different sign systems, social semiotics uses the term
"modality" to describe how objects are shown "as though they are genuine, as
though they truly exist in this manner, or as though they do notes like imaginations,
fancies, caricatures, etc."
‫ج‬

1997
 ‫هـ‬1443 – ‫ م‬2022 ‫) لسنة‬1( ‫) امللحق‬3( ‫( العدد‬61) ‫جملد‬ ‫جملة األستاذ للعلوم اإلنسانية واالجتماعية‬

References
Bernstein, Basil. 1971. Class, Codes and Control: Volume 1 – Theoretical Studies
Towards A Sociology Of Language. London and New York: Routledge.
Halliday, Michael and Christian M.I.M. Matthiessen. 2014. Introduction to
Functional Grammar. Fourth Edition. London and New York: Routledge.
Hodge, Roger and Gunther Kress. 1988. Social Semiotics. Cambridge: Polity.
Hudson, G. (2000). Essential Introductory Linguistic. Cambridge, Massachusetts:
Blackwell Publishers ltd.
Hyland, K. & Paltridge, B. (2011). Continuum Companion to Analyses of
Discourse. London: Continuum.
Kress, G. & Van Leeuwen, T. (2001). Multimodality Discourse: The Method &
Media of Contemporary Communication. London: Arnold.
Kress, G. (2012). Multimodality Analyses of Discourse in the Routledge Handbook
of Analyses of Discourse. (Eds.) James Paul Gee & Michael Handford. Oxon:
Routledge.
Kress, Gunther and Theo van Leeuwen. 2006. Reading Images. The Grammar of
Visual Design. 2nd ed. London: Routledge.
Lorenza Mondada (2018). Multiple Temporalities of Languages & Body in
Interaction: Challenges for Transcribing Multimodality, Research on Languages &
Social Interaction, 51:1.
Lyons, A. (2016). Multimodality In: Zhu Hua (ed.) Research Methods i
Intercultural Communication: A practical guide. Wiley-Blackwell.
Machin, D. (2007). Introduction to Multimodality Analysis. Oxford: Oxford
University Press.
Van Leeuwen, Theo. 2005. Introducing Social Semiotics. Abingdon, Oxon.:
Routledge.

‫ج‬

1998

View publication stats