languages
Article
Multimodal Transduction and Translanguaging in
Deaf Pedagogy
Michael E. Skyer
Department of Theory and Practice in Teacher Education, University of Tennessee, Knoxville, TN 37996, USA;
mskyer1@utk.edu
Citation: Skyer, Michael E.. 2023.
Multimodal Transduction and
Translanguaging in Deaf Pedagogy.
Abstract: Multimodal transduction is an interaction of teaching and learning. It traverses changes
in epistemology and ontology through judgements about axiology. Using multimodal transduction
(MT), students and teachers transcend languages and employ nonlanguage and quasi-language
modes (e.g., drawing, color, line, math, infographics, and even sculptures). This study uses qualitative
empirical data via grounded theory and case study designs to make theoretical claims about MT in a
deaf higher-educational context. The data for this multi-year project were sourced through interviews,
document analysis, observations, and stimulated recall with six university professors who are deaf.
My analysis shows that these deaf faculty-members employ MT to convert inaccessible modes to
become accessible for deaf learners. By changing modalities through MT, deaf faculty enhance
comprehensibility and equity for deaf learners. This theoretical account of MT contends, extends, and
clarifies aspects of translanguaging theory. As I argue, both operations transform power relations in
the classroom by addressing ethics through deaf-centric aesthetics. In deaf education, MT is equally
important for faculty in teaching and students’ learning. MT is widely and creatively used, owing
to its flexibility and adaptivity. MT is useful for all deaf agents, regardless of additional disabilities,
language competencies, or language deprivation. The MT process is inexplicit and undertheorized in
the literature about deaf pedagogy and in translanguaging research. My study provides empirical
support for theoretical claims about underlying mechanisms of translanguaging. One focus is to
explore how MT and translanguaging (and similar theories) align or diverge. I argue that MT is a core
mechanism that supports changes between all modes of discourse that enable information exchange,
including but surpassing languages and translanguaging. In sum, MT is an interaction whereby
deaf agents change the forms of knowledge; meanwhile, new realities and new power relations are
manifested.
Languages 8: 127. https://doi.org/
10.3390/languages8020127
Keywords: deaf pedagogy; multimodality; qualitative research; translanguaging
Academic Editors: Hannah M. Dostal,
Leala Holcomb, Gloshanda Lawyer
and Jeanine Treffers-Daller
Received: 3 August 2022
Revised: 31 March 2023
Accepted: 6 April 2023
Published: 11 May 2023
Copyright:
© 2023 by the author.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
1. Introduction
Multimodal transduction—What is it good for?
This article summarizes components of a larger research project (Skyer 2021) where I
examined deaf pedagogical praxis using a multimethod (grounded theory and case-study)
design (Easterbrooks 2017) involving six deaf educators who are themselves deaf and
teach deaf students in higher education, who I refer to in collective as “deaf faculty”.
While researching alongside these deaf faculty, I found copious evidence of multimodal
transduction (MT) (Kress 2010). The dominance of MT in my data extends current research
about how lecturers use translanguaging and multimodality in deaf higher education
(Holmström and Schönström 2018). Throughout this article, I juxtapose the theories
and practicalities of MT against translanguaging to explore my introductory question
about utility. The main contribution is that MT is the mode-transcendent mechanism that
enables translanguaging. Furthermore, my empirical analysis of deaf pedagogic MT clarifies,
complements, and closes gaps that exist in translanguaging theory and applications.
4.0/).
Languages 2023, 8, 127. https://doi.org/10.3390/languages8020127
https://www.mdpi.com/journal/languages
Languages 2023, 8, 127
2 of 40
Presently, I explore a data-grounded theoretical account of MT using qualitative
case study evidence, which is analyzed, interpreted, and contextualized against deaf
research highlighting the role of multimodality in translanguaging (De Meulder et al. 2019;
Swanwick 2017b). I begin by sketching out the context of my study and the dilemmas it
addresses. In the body of the text, I describe my study’s methodology and showcase MT’s
theoretical and practical features, meanwhile illustrating (and seeking to close) gaps and
dilemmas about the application of translanguaging theories.
My analysis of MT uses data to explain gaps in the research on translanguaging in deaf
pedagogy; I do so by decentering language-based changes (Wei 2022) and by exploring
the pivotal role of multimodal semiotics (Kress 2010) in deaf pedagogic interactions. This
shift in emphasis is congruent with some translanguaging theory that highlights multimodality but does not usually explore transduction specifically. Likewise, my examination
of MT is contextualized within extensive bodies of knowledge about the crucial role of
sign languages in general deaf pedagogy (Kurz et al. 2021) and in deaf translanguaging
pedagogy research (e.g., Swanwick 2017b; Swanwick et al. 2022). The following summary
should orient the reader but not exhaust the analysis—each subsequent section deepens
these basic claims.
MT interactions occurred extensively across and within all six cases I studied. This
study occurred in a US context. It was approved by two IRBs. All participant names are
pseudonyms. All subjects underwent informed consent. All data were member-checked
by participants during data analysis, who also had pre-print access to the final report.
Throughout this article, I focus on three cases to describe useful data that illustrate my
claims about the limitations of language in deaf pedagogy and the assets of MT that
transcend them. In the first case, Astoria converted a textual narrative into a drawing, as
supported by exchanges in American Sign Language (ASL) alongside embodied modes
such as gesture and eye gaze. Next, Howard constructed a digital image also embedded in
a video featuring ASL and English alongside whimsical toy props including hula-hoops
and foam swords. Finally, Tessa Rose explained how MT occurs while deconstructing her
students’ process of building graphical memoirs about food using creative comic-book
style illustrations that sublimate texts into sophisticated multimodal formats.
Throughout, MT is explored empirically using the supporting data and by juxtaposing
translanguaging and multimodal theories. My basic theory is summarized as follows: MT
describes all interactions where the form (e.g., mode) of knowledge changes, regardless if that change
uses or does not use language (Kress 2010; Skyer 2021). My definition is orientated toward
multimodal discourse analysis (Kress 2011), which differs in key ways from linguisticsinformed research on multimodality in translanguaging theories (Wei 2022). The primary
issue is the theoretical structure of how “language” and “multimodality” relate. We
might ask: which category subsumes the other? In contrast to Wei’s (2022) definition of
translanguaging, which subsumes multimodality under a rubric of translanguaging, I claim
the reverse and share empirical verifications for Kress’s (2010) theories. In this, MT is an
“umbrella” encapsulating all changes to modes including those not using language.
If MT includes all interactions where modes (e.g., knowledge forms) change, then,
in consequence, the theory of MT is necessary to fully explore the complex information
exchange ecologies of deaf pedagogy and to extend the theory of translanguaging, which
lacks a mechanistic explanation (Holmström and Schönström 2018; Swanwick et al. 2022).
What I attempt to resolve and describe is that MT is the mechanism that allows translanguaging (and similar language-based changes) to occur, while also constructing a primary
space where changes to modes that are not language may also co-occur. My data challenge
the primacy of language in deaf pedagogy and translanguaging and show that MT is a
“bridge” capable of linking changes both to and among language and communication
modes (nonlanguage and quasi-language modes). In contrast with translanguaging theory,
my analysis suggests that MT (not translanguaging) is the larger category in the same way
that multimodality includes but is broader than language.
Languages 2023, 8, 127
3 of 40
2. Methodological Framework
2.1. Impetus
Across a three-year time period, I observed and theorized deaf pedagogic interactions.
These were partly defined as reciprocal exchanges involving deaf collegiate students’ learning in relation to deaf faculty who used pedagogic methods and curricular materials they
designed for deaf learners at the research site. My sample included six faculty members,
all of whom are deaf and also teach deaf students in a higher education setting, traversing STEM and Humanities disciplines. I delimited my initial focus by identifying visual
pedagogic theories, tools, and methods (Easterbrooks and Stoner 2006; Rose 2012). These,
I speculated, might resolve dilemmas in deaf education teaching and research, including a paucity of empirically-substantiated teaching methods and theories (Cawthon and
Garberoglio 2017; Kusters et al. 2017; Swanwick and Marschark 2010).
My findings were interpreted through a conceptual framework about dissensus and
(Rancière 2010) axiological conflicts of deaf pedagogy (Kress 2010; Vygotsky 1993). It is
beyond the scope of this article to fully discuss the history of dissensual conflict in deaf
education (See: Skyer 2021), particularly in relation to sign languages, but it should be
noted that several dilemmas exist about the epistemic worth of deaf people’s learning and
teaching using visual modes (Skyer 2020). One element is an ongoing devaluation and
subsequent marginalization of multimodal discourses in deaf education (Thoutenhoofd
2010). As my study developed, I moved from an initial focus on visuality toward a stance
that centered multimodal theories of education (Hodge and Kress 1988; Jewitt 2008; Kress
2010). These, I reasoned, were a pragmatic means to understand the role of values in deaf
educational axiology (Skyer 2021) along two linked axes: the aesthetic values embedded in
(deaf) educational changes (Cherryholmes 1999) and the ethical values that configure the
choices made by deaf pedagogics in classrooms (Christensen 2010a, 2010b).
2.2. Questions and Design
The overall study addressed two broad research questions: (1) What modes, tools,
discourses, and interactions are present in deaf multimodal-visual pedagogy (DMVP)? And, (2)
How do deaf faculty understand, construct, apply, and evaluate DMVP?
To adequately address the queries, I used a multimethod (Easterbrooks 2017) approach
with two mutually supporting qualitative designs in tandem. The first was collective case
study (Enns 2017; Stake 1995, 2005). I used this method mainly to construct my sample
and devise and plan data collection procedures. The second design was grounded theory
(Charmaz 2014; Konecki 2011; Timmermans and Tavory 2012). Grounded theory methods
provided a formal approach to data analysis that I used in my quest to construct explanatory
theories about the data. Ultimately, I constructed five major theories about the forms and
functions of visual and multimodal pedagogy in deaf higher education contexts by using
the case data. The theories ranged in content and scale from descriptions of teaching
methods to detailed analyses of operations of power. In the context of this paper, my
focus is on one data-grounded theory (about MT). Overall, I work to show that MT is
a core interaction that links other modes and tools in deaf pedagogy. My theory of MT
addresses the first research question in the sense that MT is a primary interaction of DMVP.
I addressed the second question by exploring my MT theory with my participants who
confirmed the accuracy of my analysis then offered new insights.
2.3. Population, Site, Sample
I identified participants by using a sampling plan derived at the point of overlap
between case study and grounded theory (Charmaz 2014; Mertens 2020). By using theoretical
sampling, I purposefully oversampled for deafness and other relevant markers of social
difference (Enns 2017; Stake 1995). The six cases represent the diversity of the research site
and wider deaf populations, well-known for fractal heterogeneity (Luckner 2018).
I support this claim with the following narrative of how the site, sample, and population relate. First, all participants met the following baseline criteria. All participants were
Languages 2023, 8, 127
4 of 40
credentialed faculty in deaf education with five or more years teaching experience in deaf
higher education. Two were from STEM disciplines and four were from Humanities fields.
Four participants self-identified as “d/Deaf” and two as “hard of hearing” (DHH). All
were ASL–English bilinguals. All overtly professed to use some form of “multimodal-visual
pedagogy” specific to deaf education. All had earned master’s degrees, and one participant
had earned two. Another participant was working toward a terminal degree during data
collection. Yet another had earned an EdD. The remaining three had earned PhDs prior to
data collection.
Numerous individual differences between participants were noted. My participants
were aged between 30 and 70. One participant was a child of deaf adults/parents (CODA).
Two were siblings of another DHH person. Three were parents of DHH children. Where
data could be located, demographic statistics for the site are included and compared
with descriptions of the participant sample. At the time of data collection, published
institutional data about site personnel were limited. Readers may note that my sample is
representative of the site in some ways but not in others. For example, my sample had
three women and three men, which closely matches the site’s reported gender distribution
(51% female to 49% male). At the site, only 47% of site faculty are DHH, whereas my
sample is entirely comprised of DHH faculty. Other categories lacked institutional data
for comparisons. For example, the site does not subdivide deaf and hard of hearing
groups but my participants did. Alternately, whereas the site does not publish these
data, two participants identified with underrepresented sexual orientations and gender
expressions. The reported demographic data from the site about race and ethnicity explicitly
subdivide faculty into some racial and ethnic minority groups (8% Black, 4% Hispanic)
but either neglect to include others (e.g., Asian) or the statistics show that there were no
faculty identifying with another group (e.g., Indigenous or Pacific Islanders). Two of my
participants explicitly identified with one or more of these named minoritized racial or
ethnic groups.
This study was conducted in the US against a sociopolitical backdrop of increasing
conflicts between (a) White supremacy and anti-minority fascism and (b) pro-diversity
movements, such as Black Lives Matter and disability rights campaigns. As a result, I
encountered a particularly challenging choice about either highlighting or masking social
categories of difference (e.g., race, gender, and disability) as there are good reasons ethically
and methodologically for revealing or concealing the identities of individual deaf people
who share membership with another minoritized group. First, there is the issue of increased
recognizability that is concomitant with increased description—meaning, in small deaf
communities, the act of indicating any one visible social trait such as race rapidly increases
the risk for a loss of anonymity (Young and Temple 2014).
In another instance, research shows that it is problematic to “Whitewash” the contributions of Black deaf educators and scholars (Moges 2020); however, in contrast, there
is a contemporaneous discourse about how communities and researchers must protect
vulnerable deaf people who face intersectional oppression; that is, people who face multiple
and interlocking forms of oppression (Moges-Riedel et al. 2020). Deaf-forward forms of
intersectionality include persons from any of the following groups: indigenous people
who are deaf, deaf people who are queer, deafblind, deaf-immigrants, deaf-and-disabled
persons, or any other deaf person who also belongs to one or more marginalized groups,
including any other disability. My final choice erred toward protecting confidentiality
at the expense of complete description. By describing racial differences and gender diversity at the sample-level, I am able to avoid describing people at the participant-level,
which could raise the probability of harm. In the end, I elevated some aspects of social
difference out of obscurity but not so much as it could appreciably elevate the danger
of subsequent racially-motivated targeting or discriminatory reprisal, including bullying
based on ableism, harassment based on gender-expression, or anti-Indigenous xenophobia.
Languages 2023, 8, 127
5 of 40
2.4. Data Collection
Four methods were paramount: (1) observations of deaf faculty teaching deaf students,
(2) interviews with deaf faculty about pedagogic processes, (3) document analysis, sourced
from curriculum artifacts and objects built by deaf learners, and, (4) stimulated recall. In
grounded theory and case study, observations and interviews (Miles et al. 2020; Corbin and
Strauss 2008), and document analyses (Phillips and Carr 2014) are well-known data sources,
widely used by general qualitative researchers (Charmaz 2014), multimodal discourse
analysts (Kress 2011), and deaf translanguaging researchers (Holmström and Schönström
2018; Swanwick et al. 2022). Each “raw” type of data was documented with still-images and
video recordings. Quantities and data-capture methods of my primary dataset included
51h of video (captured using a GoPro 5 and processed on a MacBook) and 162 documents
and images (captured using digital photography and scanning technologies). The data are
stored in external hard-drives using encrypted digital processing. They are archived for
purposes including data audits and member-checking.
Stimulated recall (SR) is not widely cited in the deaf pedagogy research literature
(Perniss 2015). SR proved both complex and valuable. SR is an elicited-response task
where a segment of one participant’s data becomes a prompt for subsequent data from the
same participant. As Perniss (2015) relates, researchers may begin by showing images or
videos to deaf participants, who then describe what they see using sign language, gestures,
or other relevant modes. In SR, a participant and I would examine their data, which
included video-clips from recordings of classroom observations, transcribed segments from
their interviews about teaching, or documents they or their students produced. With my
supporting questions, participants offered insights about the data that were previously
inexplicit. SR tasks were dialogic in the sense that they were a conversation among nearcolleagues and were multimodal in the sense that they produced new data that combined
language, images, gestures, etc. I describe two SR examples in the present article, one from
Howard’s data, including a co-analysis of a multimodal video he produced, and another
from Tessa-Rose, where she and I co-analyze a multimodal transcript I built.
2.5. Data Analysis
This study generated 1.38 terabytes of multimodal data, which I analyzed to form
taxonomies, narrative case studies (Stake 2005), and explanatory grounded theories using
visual representations (Konecki 2011). Primary data analysis included writing observational
field notes and analytic memos about data collection events. Raw data were coded and
analyzed using MAXQDA digital software (Version 2018.1). This program allows the
researcher to directly code (analyze) texts, images, and videos. In so doing, I produced 18K
specific codes, organized in an extensive codebook, color-categorized into hundreds of code
families. The data were compiled (built) using abductive reasoning, constant comparisons,
and analytic memos (Charmaz 2014; Saldaña 2012; Timmermans and Tavory 2012). This
trio of methods supports ongoing analysis, which juxtaposes insights stemming from the
data alongside insights sourced from the researcher. All findings were interpreted within
a superstructure of multimodal educational discourse analysis (Kress 2011). Theories of
multimodality were paramount in my quest for meaning-making, yet this theoretical bias
may be thought as a limitation for some readers.
Through my research, theory and data were always juxtaposed. No claims were made
that lacked empirical support. All findings were either triangulated (good) or crystalized
(better) and verified in member-checking activities. Triangulation and crystallization are
comparable metaphors that communicate the basic idea that multiple converging data
sources make for more-robust claims (Charmaz 2014). During focused data analysis, I
used constant comparisons (Corbin and Strauss 2008) to problematize my ideas. I actively
sought out disconfirming data and explored contrary explanations. If I had a hunch, I
would verify it in the data or with participants and then modified my ideas based on these
interactions. If a hunch lacked data, it was discarded.
Languages 2023, 8, 127
6 of 40
I used multiple stages of member-checking (Marshall and Rossman 2016) where I
invited my participants to read or view selections of my analysis and offer competing
or complementary explanations. In the end, I deemed that the theories I produced were
saturated (Charmaz 2014). Member-checking is a process recommended for both case
study and grounded theory, but saturation is unique to grounded theory. Both techniques
were vital to ensure that my research was deemed trustworthy by the people it describes
(Enns 2017; Charmaz 2014). Saturation is a main criterion of value in grounded theory
construction, which shows that theories are robust and potentially transferable to other
sites. I discuss transferability in my limitations section, below. All participants were
supplied with the full report upon its completion and had ample time to reflect on it prior
to publication. No major refutations have been proposed by my participants in the 2 years
since I supplied my initial report to them.
That is to say: my theory of MT was conceived as suppositions borne out of my
interpretations of the data, but it was also confirmed by each participant who evaluated
my theories and found them adequate. The data not only support my theoretical claims
but comprise them.
3. A Theory of Multimodal Transduction in Deaf Pedagogy
3.1. Metatext
MT was present in all six cases. Section 3 complies general features of my theory of MT,
which I present as a class-wide phenomenon of deaf pedagogy, expertly used by the deaf
faculty I studied. This overview presages an examination of empirical evidence from three
cases, which I explore after (Section 4). The same pattern continues within the three case
descriptions that follow—a broad claim is made and then data that support the claim come
afterward, with increasing particularization and granularity (Stake 2005). In this section
and in the Discussion (Section 5), I consciously work to construct and review a bricolage
of theories with the express purpose of exploring what my theory of deaf pedagogic MT
contributes to the literature about deaf translanguaging studies and deaf pedagogy broadly.
In so doing, I critique the literature as I review it; meanwhile, I construct new theories using
data that fit into noted gaps or that address specific problems.
To orient my research, I cite three definitions. First is Kress’s (2010) definition of
multimodal transduction—a change “moving across modes” (p. 124), which includes
but transcends languages. Notably, this definition predates translanguaging (c.f., Lewis
et al. 2012). Second, Holmström and Schönström (2018) describe translanguaging in deaf
education as “a process in which two languages are used together in meaning-making
[across] the whole language repertoire” (pp. 90–91). This definition offers a broad definition
of multimodal language, which is in line with most current translanguaging research.
Finally, Kusters et al. (2017) acknowledge a key problem in deaf research between methods
focused on (1) changes to languages modes and (2) changes via multimodal semiosis
extending beyond language. Exploring the second category is my main interest.
In the context of this Special Issue of Languages about translanguaging in deaf education, it is important to distinguish how and why I use the terms transduction and
multimodality in the context of translanguaging. First, multimodality is a characteristic of
discourse. Second, transduction represents all changes to knowledge and reality. Overall,
my claims converge on the following idea: MT is a broadly useful mechanism that supports
and enables translanguaging. Specifically, MT allows for the transcendence of language.
This is the most important function of MT. I explore this claim in many ways in this article.
In part or in sum, MT need not rely on, reference, or use any language modes at all. This
shift supplants the focus of language and subverts several original definitions of translanguaging, including those adjacent to deaf pedagogy theory, which attempt to subsume
multimodality in a language-based conceptual frame (García 2009; Wei 2022). My account
of MT reverses this operation and aptly places language as one classification of modes
within a plurality of discourses (Bezemer and Jewitt 2010; Kress 2010).
Languages 2023, 8, 127
7 of 40
3.2. Multimodal Transduction in Deaf Pedagogy
As I theorize it, MT is an interactive, intercreative process. MT mainly occurs between
agents (deaf faculty and students) and curriculum content. It is and requires epistemic
operations that have corresponding ontological functions. As deaf agents change the forms
of knowledge, they concurrently restructure reality. These changes generally increased
accessibility and simultaneously improved the aesthetics and ethics of deaf pedagogy. Deaf
students and deaf faculty use MT for general and particular purposes. When deaf faculty
use MT in teaching, deaf students often emulate the process, and MT reappears in their
learning products, which are both visual and multimodal in form. I also found that deaf
agents use MT for creative aesthetic experimentation. Not only is MT useful for teaching
and learning, it is also fun and deeply enjoyable. Because deaf faculty change knowledge
modes to enhance accessibility, MT is an ethical process of change (Kress 2010). Deaf faculty
mostly use MT to construct accessible paths to curriculum content, to make knowledge
comprehensible for deaf learners, or to enhance educational interactivity. MT’s mechanism
is also its purpose—to change the forms of knowledge without substantially changing
content (Kress 2010). This is what Kress (2010) describes as a “(usually total) rearticulation”
of modality (pp. 124–25, parens. original).
The most basic form of MT entails two sets of modes, two stages, and one major
change. Usually, mode/s present in stage one change into new mode/s in the second
stage, but many stages are possible, and many changes to modes are not only feasible but
desirable. When using MT, deaf faculty aim to maintain or expand epistemological concepts
while dramatically changing the form of knowledge—in one case, this involved literal
sculptures. Via MT, epistemic forms are constructed, deconstructed, then reconstructed;
meanwhile, meaning is preserved or even enhanced across the stages of change. In this
study, MT occurred in teaching and learning and in curriculum and assessment. MT was
present across all six cases. Its ubiquity suggests its importance. Yet, transduction (the verb
and the theory) is not widely known. It is not a term widely used among multimodalists,
translanguaging theorists, or deaf pedagogics, for that matter. Likewise, when pressed,
each of my participants, who are all highly-credentialled deaf faculty, lacked research-based
theoretical or pedagogical terminology to describe what they were doing when using MT,
even though they reasoned (and in two cases empirically proved) that such changes were
both ethically necessary and beneficial for deaf students.
3.3. Contrasting Multimodal Transduction and Translanguaging
Theorizing MT in historical and current translanguaging theory is necessary to fully explore
the discursive complexities of these subjects alone and in relation to deaf pedagogy, which
all-but requires multiple language modes, yet, also readily transcends changes to language forms
(Swanwick et al. 2022). Another area where MT and translanguaging differ is that MT does not
originate in a language-based framework. It is not limited to, focused on, or first defined by
language—sign, speech, text, etc. MT is the under-acknowledged “bridge” that links changes
among language-based, non-language, and quasi-language modes. This gap is acknowledged
(but not closed) by the leading minds of translanguaging theory. Wei (2022) explains, the
new focus of translanguaging is “a shift away from language [to] attend to a wider range of
multi-semiotic resources” (p. 2). Furthermore, while new translanguaging research emphasizes
“going beyond languages” (Wei and García 2022, italics original), “intersemiotic correspondences”
(Swanwick et al. 2022), and “intramodal translanguaging” (Holmström and Schönström 2018),
there is no clear agreement about a core mechanism that collocates these changes.
The most important features of MT are its independence and transcendence of language.
Likewise, my core emphasis on multimodality is purposeful, as multimodality is often subordinated in translanguaging (c.f., Baker 2011; Swanwick 2017a, 2017b; Swanwick et al. 2022;
Thoutenhoofd 2010). In my view, Kress’s (2010) transduction offers a more compelling understanding of how humans (deaf or otherwise) use a plurality of discourse forms (modes) in
teaching and learning and other sociocultural interactions.
Languages 2023, 8, 127
8 of 40
While it is definitely an oversimplification to suggest that translanguaging is “only” concerned with how language changes form, it is not too much to note that the theoretical genealogy
of translanguaging is overwhelmingly preoccupied with language. The name—translanguaging—
readily shows this bias. As one specific example, Wei (2022) writes,
Translanguaging reconceptualizes language as a multilingual, multisemiotic, multisensory, and multimodal resource for sense- and meaning-making . . . It has the
capacity to enable us to explore the [holistic] human mind . . . and rethink some
of the bigger, theoretical issues in linguistics. (n.p., emphases added)
Wei’s multimodality is a facet of language. Multimodality is used as an adjective for
language. The cited definition is primarily concerned with characterizing languages, which
Wei achieves by enlarging a boundary as an answer to the question: What is language? It is true
that Wei (2022) leads the recent charge to enhance multimodality in translanguaging; however,
only four years prior, Wei (2018) defined “Translanguaging as a practical theory of language”
(emphasis added). This bias is evident not only in the title of Wei’s paper but also the journal
in which it was published. This language-preoccupied approach contravenes Kress’s (2010)
definition of multimodality and confuses the link between multimodality and language. Kress
asks a different basic question: What is discourse? His answer inverts the translanguaging stance:
Kress states that multimodality is the larger descriptor of human knowledge forms, in which
language (as one major class of modes) resides. In Kress’s view, multimodality always already
transcends language.
Like Kress, my theory of transduction concerns all modalities in which information is
exchanged. I focus on how MT happens in deaf classrooms, in particular, on how deaf faculty
redress inequities by converting modes to be fully perceptible and mutually comprehensible for
all deaf students. Throughout my dataset, I found that MT occurred at one scale or another in
almost every interaction, whereas changes centered on language occurred much less frequently
(frequency counts are supplied in the Discussion). According to my participants and published
accounts in the literature, translanguaging theory is not well understood by practitioners and
researchers, which leads to misapplications in classrooms (Swanwick 2017b). Confusion has
several sources. Some stem from rapidly accruing changes to basic tenets of translanguaging,
including revisions about multimodality (García and Lin 2016; Wei and García 2022). Likewise,
confusion stems from fundamental differences between how researchers define “language”
and “multimodality” and how they relate these two terms to each other. The noted differences
between Kress (2010) and Wei (2022) shows this ambiguity. Confusion is also a result from
translanguaging theorists who offer inexplicit analysis of transduction, which is the gap I
intend to close presently.
Many new deaf studies of translanguaging are nominally inclusive to images, yet non/quasilanguage modes such as laughter, color, line, and graphic design are afforded less status compared to language-based modes such as text and sign language. Most importantly, empirical
data (Holmström and Schönström 2018; Swanwick et al. 2022) are essential but often lacking
in studies of deaf translanguaging (Kusters et al. 2017). Whereas language is exalted in these
studies, non/quasi-language modes such as mathematics, gesture, and body movement occupy
subordinate positions. To use a colloquial phrase, semiotic multimodality appears to be a
“bolt-on” element to the original translanguaging theory (Baker 2011), which focused largely
on two or more language modes in bilingual learning. Thoutenhoofd (2010) speculates that
the marginalization of multimodality in deaf studies is one element of a “persistent motivated
alignment” to subordinate deaf ways of being in modern societies (p. 217). In the discussion
section, I explore my contentions about the ample theoretical daylight between translanguaging
and MT in much more detail. Furthermore, I suggest that dissensus in theorizing MT and
translanguaging is far from harmful and is instead productive.
3.4. Interpreting Data on Multimodal Transduction
An overview of how data were interpreted by myself and my participants may clarify
some of my bold claims. All six participants were familiar with specific theories and methods
that describe changes from one language mode into another language mode. They gave
Languages 2023, 8, 127
9 of 40
me examples, including the role of ASL interpreters who change speech into sign (and the
reverse), and noted classroom practices, where they and their students translated texts into sign
language (and the reverse). They used explicit theoretical terms to describe multiple modes of
language, including sign languages, in their pedagogy; including bilingualism, plurilingualism,
multilingualism, and yes, translanguaging. This interpretation of the central role of language
is consistent with definitions of translanguaging from deaf pedagogy literatures—including
Holmström and Schönström’s (2018) definition, which, at the time of data collection, was only a
year old.
However, in sharp contrast, none of the six highly credentialed participants used the term
translanguaging to describe the equally-important and equally-common changes of modality
going beyond language modes. Not one participant, for example, called it translanguaging when
an utterance in sign language was represented as a mathematical equation using numbers,
symbols, and letterforms. Nor did they call it translanguaging if a word or phrase was transformed into a drawing or when a text was constructed into a sculpture or if an English text was
represented as an image or as an ASL poem. They did, however, characterize this kind of deaf
pedagogy as multimodal.
Once I defined MT for them, my participants and I realized that it occurred in almost every
educational interaction. Overall, we learned that MT is both extremely simple and ineffably
complex. First, a simple concept: MT is (and requires) a change from one mode or set of modes
into another mode or set of modes. At face value, deaf-centric MT usually results in increased
visuality, as noted by Holmström and Schönström (2018), Christensen (2010a), and Smith (2010);
however, and more to the point, I found that MT results in increased multimodality. MT both
supports and demands an understanding of how inter-mode and trans-mode changes occur.
While the purpose of operations constitutive of MT is acknowledged by Raike et al. (2014) and
Swanwick et al. (2022), the terms MT and transduction are not used or theorized in the research
I reviewed, aside from Kress (2010), who appears to have coined the term and whose work I
used most to advance my interpretations.
The lack of theoretical specificity is not altogether surprising as the literature about deaf
pedagogy and the literature about deaf translanguaging both lack mechanistic descriptions
of MT, including what it is, why it is applied, and how it works, let alone how MT relates to
literal translation and transcription, though they share a common prefix—trans—meaning:
across, beyond, or through. Based on this disparity, I decided to dig deeper. Interestingly,
there were an abundance of data from my study that repeated the basic misunderstanding
that translanguaging is about changes to languages. It is hard to fault classroom teachers who use
translanguaging theory incorrectly when the theory itself is constantly in flux. For example,
in my dataset, Howard—who is a sign language researcher—used translanguaging following
an observation. He wrote “I compared English and ASL to see similarities and differences
in the linguistic structure of both languages” (OBS. 1, DEBRIEF, p. 16)1 . This statement
characterizes a typical “translanguaging” code from my data analysis sessions; it also reflects
the deaf translanguaging literature in vogue at that time, which posits that translanguaging is
an applied bilingual theory of language learning not one focused on multimodal transduction
or the transcendence of language.
The more I examined MT—theoretically, in the context of data and in comparison to
the literature—the more I understood that MT interactions were not adequately represented
in any knowledge base known to me, including in the literature intersecting deaf pedagogy
and translanguaging, precisely where it is so sorely needed. While the MT phenomenon is
widespread, it is also elusive—theoretically, conceptually, and practically. While each of
them used MT, none of these experienced, widely-read, well-credentialed deaf faculty had
a name for MT or could point me to precise research to support their thinking and actions.
When I presented one of my participants with my tentative theory, they became visibly
excited. They seemed to say, Yes!—THAT! Someone finally understands!
Languages 2023, 8, 127
10 of 40
3.5. Multimodal Transduction: Power and Axiology in Deaf Education
One extremely important reason that the MT perspective is needed in deaf pedagogy
research is one that is seldom acknowledged: The discursive repertories of deaf students and
deaf educators differ. Alongside differences of discursive capability are concomitant differentials
in power (Bourdieu 1991; Kress 2010). Educators, including deaf educators, nearly always
enter pedagogic relations with some superior language skills and with other advanced semiotic
repertories. Concordantly, students including those who are deaf usually enter these interactions
with fewer skills and abilities. With this imbalance of knowledge there is an implied and
preexisting power imbalance. In nearly all interactions of language, teachers hold linguistic
power (Bourdieu 1991) over deaf students. This is always problematic, but especially so with
deaf students who are language deprived or possess language dysfluencies, specific language
disabilities, or learning disabilities affecting their ability to use language in educational or social
tasks. As my participants showed me, MT is also an effective and ethical pedagogical strategy
for these students; MT mitigates differentials in power and knowledge and dis/ability.
Class-wide data showed that deaf faculty and deaf students used MT for general and
particular purposes. The deaf faculty I studied professed to use MT for one or more of the
following dialectic reasons, each of which is linked to discursive power: first, to increase
accessibility and decrease inaccessibility; second, to enhance interactivity and reduce passivity;
third, to decrease communication breakdowns and improve the exchange of information
through mutual comprehensibility; and finally, to make educational interactions more ethical by
reducing pre-existing inequalities of power in deaf education. Because of its dominance in my
dataset, I came to understand that MT is like an umbrella subsuming a range of other changes
to modalities, including translanguaging and several other theories focused on language, like
codeswitching. To reiterate; MT unites all changes to all modes under one common rubric
linked to power. Disambiguation among these ideas, therefore, is the focus of my Discussion.
Via axiological judgements—that is, the choices that are made by deaf faculty about
deaf-positive values—MT and translanguaging share a goal of “fundamentally [transforming]
power relations” (Wei and García 2022, p. 322), but there are key differences. This is another
main reason why a theory of MT is so sorely needed: to flesh-out the axiological values
and aesthetic processes about multimodal components of translanguaging theory, which are
inexplicit, misunderstood, ignored, implied, or latterly added. While language is almost always
relevant when theorizing power in deaf pedagogy, the active suppression or marginalization
of multimodality is a specific source of harm that reifies and reinforces discursive inequities in
deaf education (Thoutenhoofd 2010).
Generally, while observing my participants in deaf educational interactions and again
when co-analyzing the data with them, I saw that the changes they made to discourses via MT
enabled mutual information exchanges, which were principally performed to redress preexisting
imbalances of power. To do this, deaf faculty worked to change modes and support not only
mutual perception but also mutual comprehension for all curricular and pedagogic content
and for all deaf agents involved. As my participants and I defined it, MT involves the ethical
manipulation of aesthetics in deaf pedagogy; it involves both domains of axiology (ethics and
aesthetics), including their operationalization in value-based decision making in teaching and
in response to specific learning events. In the discussion section, I enlarge on this definition
and further explore how and why MT differs from translanguaging and other similar linguistic
theories that share a common genealogy in the social uses of language by learners, including
languaging and chaining, among others. Said differently, the origin of MT is the pedagogic act
of leveraging of multimodality, and it finds focus and direction in the desire to enhance equity
and redress imbalances of power through ethics and aesthetics.
4. Data—Multimodal Transduction in Use: Three Case Analyses
4.1. Introduction to Data about MT
Overall, the data taught me that MT is the core mechanism enabling epistemological operations and corresponding ontological functions that are manifested when deaf
agents apply positive values about deafness in education contexts. What I mean is that
Languages 2023, 8, 127
11 of 40
transduction is a positive change to improve the form of modes in which knowledge is
shared to be more amenable to the deaf learner’s cognitive and social characteristics. The
MT mechanism requires two simultaneous operations: the first is the change in knowledge
itself (e.g., the epistemology), and the second is a change in the form of reality (e.g., the
ontology). When these cooccurring changes happen, deaf faculty members work to keep
the epistemic content the same or highly similar, meanwhile, radically changing the form,
thus, creating a new reality that is more just because it is more aesthetically aligned and situated. These cooccurring shifts often required visual modes in conjunction with embodied
or kinetic modes, or all three, and are, therefore, multimodal in character and transcendent
of language, as the evidence I will explore demonstrates.
The data showed that in the context of deaf pedagogy, MT was necessary, beneficial,
and even beautiful. Before showing a close analysis of three cases, I provide a short
introduction to MT in the full dataset. This is warranted to illustrate the depth and breadth
of MT’s applications within not only the three cases but in the entire class. The data-based
list (below) shows the broad applicability of MT in diverse disciplines of deaf higher
education and shows how deaf agents concurrently change forms of knowledge or reality
using the entirety of their multimodal semiotic toolkits in ways that readily surpass language
repertoires (Kusters et al. 2017). Most examples entered my corpus as observations. A few
were synthesized or adduced (TERTIARY ANALYSIS MEMO SEPT. 1 2020, pp. 158–59).2
Examples of MT in the disciplines of deaf higher education include when:
•
•
•
•
•
•
•
•
•
•
•
•
a print word fixes the meaning of an image in an English class on new media forms
two mathematical formulae are drawn on the board prior to a chemistry experiment
a gesture explicates the connotation of a print word in a composition/rhetoric course
an ASL narrative is metonymized with imagery in a science laboratory infographic
an image captures “the feeling” or ethos of an era in a history or philosophy lecture
a diagram is explained in detail using a descriptive text in the science lecture hall
a drawing provides contextual cues for locating keywords in the computer lab
a visual tool simplifies commonly used ASL phrases in the linguistics classroom
a photograph documents a correct result in a science laboratory procedure
a Google image search illustrates concepts in developmental writing classes
a gesture links a textual definition with its applications in math word problem sets
an ASL poem is “back-translated” from ASL into English in a deaf literature class
Because MT is best understood in context, I now present an analysis of specific casebased empirical data. Each example begins with a main claim, which is accompanied by
evidence and the corresponding theoretical implications.
4.2. Case-Analysis 1—Astoria
4.2.1. Main Claim
Multimodal transduction is mode-independent and language-independent.
4.2.2. Evidence
In the observation that follows, Astoria used MT in conjunction with her deaf students’
social critical thinking. As we discussed in her SR task, deaf students are more capable of
sophisticated critical analysis when it is presented as a social rather than individual task.
In this example, Astoria co-constructed a hand-drawn, multimodal image based partly
on a text and partly on an extended discussion in ASL. MT, therefore, is cognitive but
involves aesthetic, cultural, and interpersonal components. Astoria’s data support aspects
of Holmström and Schönström’s (2018) theory of translanguaging: “Deaf lecturers in higher
education settings create a visually based learning environment . . . by using their whole
repertoire of semiotic resources (e.g., different languages, gestures, pointing, pictures, etc.)
. . . to create an accessible learning environment” (p. 90).
Astoria began by displaying a textual narrative about a Bento Box lunch to a developmental writing class, where many students, she told me, have diagnoses of language
deprivation. Next, she added semiotic overlays, including circles and yellow highlighter,
Languages 2023, 8, 127
12 of 40
to draw focus to terms about spatial orientation, the focus of her lesson. Next, she sketched
elements on the whiteboard, which included ovals, circles, line drawings, and text. Then,
she asked questions in ASL to her students. The students read the text and described their
interpretations in ASL as well as fingerspelled words. They also used their own gestures,
such as pointing and eye-gaze. Astoria changed their form by using transduction, where she
converted English and ASL into new hand-drawn images. Below, two selections of my
data from Astoria’s case are shown. One is an excerpt from a field note; then, two stages
of their collaborative process, including the multimodal text and multimodal drawing are
depicted in Figure 1 (below).
Astoria’s Data Begin.
Field Note:
Astoria and her students read the text first (Figure 1, left). Then, using ASL, they interpreted its meaning, focusing on
the vocabulary that Astoria had highlighted with a yellow marker. These were words about spatial arrangements.
Astoria asked questions about the reading and supplied cultural background knowledge if the students got stuck—
she used Google to search for images of sushi, onigiri, and tempura. Projected alongside the text and drawing were
photographs of Bento Boxes and color swatches (not shown here).
Next, they collaborated on a complex hand-drawn diagrammatic representation of the Bento Box using color, shapes,
numbers, and language modes, including text (Figure 1, right). Note the words: “spatial” and “oval.” Linguistic meaning is discussed, but ideas are only affixed in the hand-drawn image. They literally draw meaning out of the text.
Text
Diagram
Figure 1. Astoria’s Multimodal Transduction.
Astoria told me she wanted to explore spatiality. She used language as data to describe spatial arrangements. She creatively uses
ASL classifiers, the in-progress diagram, and complex gestures to ask questions about the particularities of spatial arrangements.
She asked: “How is a bento box arranged? How is it eaten, from the center to outside or outside to center? Can circles represent fried zucchini?
May small ovals characterize onigiri? Can triangles show winter squash?” In addition, Astoria used other tools from her semiotic toolkit:
arrows, shapes, typewritten digital text, handwritten English, and images. Notably, she used ASL classifiers to illustrate spatial
concepts from the text and ask questions. The English word “perched,” was a focus as she asks: “How might we depict a ‘flower
perching on an egg’?” Her query was accompanied by an ASL classifier skit, showing how birds alight on a wire.
As the class closely analyzed and unpacked the text, it became an act of social critical thinking. Together, they signed, drew, wrote,
and gestured until the spatial arrangements and drawn elements generated a cohesive image. By using close reading and
unpacking the image and the text, members of the class determined the placement and arrangement of items and answered the
following questions: “What goes on top, to the left, right, center?” Astoria and her students collectively decided where to add an
element to the diagram. Astoria physically drew the visual representations but under guidance from her students. Hypothetical
arrangements were discussed, tested, and agreed upon (or changed) before setting them down in ink. Spatial arrangements were
discussed with English, ASL with classifiers, and gestures but they were affixed by the visual image. Options were shown with
language, but meaning was determined by drawing.
Astoria’s Data End
(OBSERVATION 1, FIELD NOTE, pp. 5, 9).
Languages 2023, 8, 127
13 of 40
In this example, text (a language mode) is basal, adjoined by ASL (another language
mode), then the text is overlaid with color, line, and shapes (nonlanguage modes) to alert
students to relevant words about spatial orientation (another nonlanguage mode). Then,
segments of the text are translated into ASL (another language mode) as augmented with
other language modes including classifiers and fingerspelling, co-presented with nonlanguage modes such as color, the Google Image search results, and gestures—to depict
objects in relation to other objects (e.g., “a flower perches on each egg”). Then, Astoria added
more questions and statements in ASL, which were also modified by using gestures or
facial expressions (quasi-language modes). Then, she reverted to illustration.
There were over a hundred isolated instances of MT in her 90-min lesson. Her pedagogy explored language modes in dialogic interactions. However, the centerpiece was
the image. It contained multiple visual design modes, sketched in colorful markers. During points of transition, Astoria constructed joint visual attention and deaf mutual gaze
interactions. Her teaching uses translanguaging, assuredly, but it supersedes a linguistic
focus—in fact, her lesson’s objective was about visualizing spatial arrangements, not about
any language.
4.2.3. Implications
Astoria’s MT reverses modal constancy. Modal constancy is the tendency to meet an
utterance in a first mode with a response in the same mode. For instance, if a student
asks a question in ASL, the teacher generally answers in ASL (not speech or writing).
While Astoria often used modal constancy, multimodal inconstancy was observed with two
key variations: (1) within-language inconstancy and (2) trans-mode inconstancy. Withinlanguage inconstancy occurs when changing from text to ASL (translation) or ASL into
written text (transcription). In another observation with Astoria, an “oral” deaf student
replied to Astoria’s ASL using speech, which was interpreted by another agent back into ASL.
These are within-language forms of modal inconstancy while also being clear examples of
translanguaging as it was commonly understood by my participants.
Trans-mode inconstancy gives MT its most potent powers. It can traverse all modes
quickly and effectively. It can enhance meaning and make knowledge more accessible
and, thus, supports a pedagogy that is equitable for students lacking fluency in language,
such as Astoria’s language deprived students. This description of trans-mode inconstancy
supports and enlarges the original definition of translanguaging, something that is commonly misunderstood by teachers (Wei and García 2022). However, the multitude of
trans-mode changes in Astoria’s pedagogy shows that MT is not restricted to languagebased changes. Trans-mode changes and modal inconstancy give MT its most awesome
powers. This finding shores up weak aspects of general translanguaging theory, which
deemphasizes non/quasi-language modes in theory or neglects them empirically. Notably,
MT traverses all modes—MT is what happens when any one mode is converted into any
other mode, regardless of the inclusion of language. For example, Astoria watched her
students’ gestures, then, her pedagogic response was to construct a ovoid drawing. There
were many other examples of trans-mode inconstancy in Astoria’s Bento Box lecture that
eschewed language: Astoria changed a photograph into a drawing, a gesture into a line,
and a digital color swatch into an analog marker line using a similar color. Ultimately, she
transduced a multimodal assemblage from myriad parts, many of which had nothing to
do with language or did not go through a language process. In Astoria’s pedagogic MT, a
language-based question was often answered using a non/quasi-language mode. I asked
her why. In reply, she wrote:
I am a visual person and do a lot of drawing myself. I often create concept maps,
charts, graphs, or other images to make sense of new content. So, this [change]
is natural for me. I just go through my toolbox . . . and try different methods—
drawing, images from Google . . . There are instances where I ask students to do
this (draw, find images/content, etc.)—to depict their understanding and how
Languages 2023, 8, 127
14 of 40
they would apply the content they learned. This helps me gauge their learning
process (MEMBER CHECKING, p. 4, brackets added, parentheses original).
Astoria’s MT allows her to traverse the entirety of her own semiotic toolkit (Kusters
et al. 2017) and to assess her students’ ability to do the same. She can teach, they can
learn, and she can assess their learning using MT. To guide her students, she relies on
her own experiences as a deaf learner. This shows that MT is a compelling form of “deafsame” (Kusters and Friedner 2015) and intergenerational deaf pedagogy (Kusters 2017;
Holmström and Schönström 2018). She used MT to affix meaning visually, multimodally,
and experientially; in a word, via aesthetics. Astoria’s pedagogy is characteristic of deaf
educational aesthetics, which are artful forms of knowledge achieved by way of centering
deaf ethics. Astoria’s actions and designs redress inequity. Thus, throughout a range of
intermediary steps, new forms of knowledge and reality and power are generated. A whole
deaf-centric axiology is constructed in the process of MT.
MT is of utmost importance in deaf education, especially for deaf students with
language deprivation, such as Astoria’s developmental composition students, who, I
observed, lacked for some language-based semiotic resources but were able to compensate
with a wealth of other modes at their disposal. This phenomena is only marginally explored
in deaf research (Pollard and Fox 2019). Sacks (1990), for example, describes a young deaf
boy who has no language whatsoever but is a drawing virtuoso. Vygotsky (1993) also shows
that modes other than language can be used to great effect in deaf pedagogy, including
to enhance sociocultural development in multiply disabled deaf youth. To neglect intact
non-language and quasi-language semiotic abilities in deaf students is to simultaneously
imperil the efficacy of educational interactions and disempower deaf learners.
4.3. Case-Analysis 2—Howard
4.3.1. Main Claim
Deaf pedagogy is multimodal and requires multimodal semiotics.
4.3.2. Evidence
During SR, I asked participants to bring examples of deaf-centric visual tools (Easterbrooks and Stoner 2006) that they had designed for instructional purposes. Then, we
co-analyzed them. Howard brought two diagrams and a video. One of the diagrams
reappeared in the video, suggesting its importance. I analyze both the visual tool and the
video to illuminates features of MT. By analyzing these data, I learned that MT allows deaf
educators to construct multiple forms of representation that extend embodied learning
through non/quasi-language means, including visuality, spatiality, and temporality.
First, I describe the diagram, which I also call a visual tool. It is shown in Figure 2
(below). The diagram is multimodal. The most immediate component is an icon representing a human body. It is surrounded by rings much like the planet Saturn. The rings are
labeled with letters (P, M, D, E) and numbers (0, 1, 2, 3). There are four text boxes, one per
quadrant, explaining technical terms that Howard derived from ASL linguistics research.
Howard’s diagram encodes morphemes of ASL graphically and in text, alongside design
elements such as boxes, brackets, color, layout, and the deft use of negative space.
Howard designed the diagram to solve a specific problem: his ASL linguistics students
(who were both deaf and hearing) could not “visualize” how sign languages used embodied
space. This lack of understanding frustrated classroom progress. The diagram solved the
problem by helping students think visually (Howard’s term). I argue its efficacy is dependent
on the affordances of MT. The diagram helps them think multimodally. Deaf faculty who
use MT maintain or enhance content knowledge across changes in modality. Doing so
positively affects learning by increasing interactivity and accessibility. Both goals are clearly
seen in Howard’s diagram and an excerpt from a field note (below), which also use multiple
modes in support of my claims.
Languages 2023, 8, 127
15 of 40
Howard’s Data 1 Begin.
Figure 2. Howard’s Visual Tool.
Field Note:
I asked Howard: “What is the visual tool? What is its purpose?”
Howard commented: The diagram represents ASL’s embodied spatial locations and body production sites. I designed it to visualize ASL
morphemes like palm orientation, touch sites, and physical movements. The diagram represents spatial location in 2D. In addition to changing
the form of knowledge, I simplified and reduced complexity and simplified the dimensions.
I considered Howard’s diagram and narrative, then, constructed a drawing of my own (Figure 3, below). After, I wrote a memo to
illustrate what I thought to be three primary kinds of knowledge: spatial, visual, and multimodal for two deaf agents engaged in a
simple sign language educational interaction. These comprise an early draft of my theory of multimodal transduction, which
explores four distinct forms of knowledge, each discussed in turn.
1. SPATIAL knowledge
Like Howard’s tool, my sketch has icons representing people. On the left is a person signing in ASL, which is known for its
embodied use of space as a form of knowledge. To know and use ASL is to describe signs in four spacetime dimensions. While ASL
is temporal, here (as with Howard’s diagram), we pause time to focus on 3D axes: X, Y, and Z (height, width, depth). Simply, to sign
in ASL is to use SPATIAL and embodied knowledge in language production.
2. VISUAL knowledge
To the right is a second person, whose gaze rests on the signer at left. The icon is labeled VISUAL for its use of visual sensory
systems to learn from spatial-embodied knowledge. The optic system “flattens” 3D signs into 2D neurological visual images. If
signing production (“describing”) is spatial, then reading signs (“descrying”) by other agents is primarily a visual
phenomenological event. Howard’s tool is also VISUAL. It reduces complexity by flattening 3D ASL production to a 2D surface.
The visual tool eliminates the dimension of “time”, further simplifying the information exchange.
Languages 2023, 8, 127
16 of 40
Figure 3. Multimodal Transduction Diagram.
3. MULTIMODAL knowledge
A bracket connects SPATIAL and VISUAL. The bracket classifies the total interaction as MULTIMODAL. Multimodality combines
spatiality and visuality and much more. Multimodality refers to the supersystem in which spatial and visual interactions co-occur.
More explicitly, deaf multimodality is social and cognitive and dependent on human perceptual and cultural relationships and uses
for specific modes (epistemology) that frequently undergo states of change affecting reality (ontology), in which forms are changed
but meaning is conserved. This requires multimodal transduction.
4. Knowledge via MULTIMODAL TRANSDUCTION
For an observing teacher, student, or researcher, spatial-embodied ASL knowledge is perceived as visual, whereas for the signer, it
is perceived as spatial/embodied. Both are correct, relative to the point of view. This shows that Howard’s pedagogy is multimodal
overall and requires numerous forms of transduction that supersede language-based changes. MT is an interaction; it makes explicit
linkages both across and between sets of modes; this distinction obviates the change of mode beyond language. By establishing
relationships among discourse forms in pedagogical interactions, both knowledge and reality changes. How it changes is different
for Howard and for his students as relative to perspective and agent position. Likewise, educational forces, such as power and
self-determination, are relevant in spurring the operation. This is what prompted Howard to make the tool—an ethical urge to
support equity and redress inequality.
Howard’s Data 1 End
(ELICITATION TASK, FIELD NOTE, p. 8)
Because ASL signs are built on and off the body, Howard determined there was a need
to create a multimodal tool to externalize, apportion, and effectively disembody ASL. To
properly teach ASL, Howard needed to supersede language. When students encounter the
Saturnine diagram, Howard reflects, they say: “Oh! This is math!” (ELICITATION TASK,
p. 8)—referring to the three-dimensional axes. As we discussed it, the diagram provides
concrete referents that abstract the spatial body. This occurs by reducing complexity
and eliminating a dimension. Both are achieved with MT as the broad mechanism and
are supported by translanguaging, including Howard’s explicit analysis between ASL
and English.
Languages 2023, 8, 127
17 of 40
Beyond these two named languages, Howard used aesthetic design elements, such
as: line, icon, number, color, layout, font, and so on, that exist alongside language modes
such as text and sign. Howard’s tool uses MT to represent embodied sites into a concrete
form that is quite different from ASL. Where ASL is embodied, the diagram is disembodied.
Where ASL is temporal and fleeting, the visual tool is static and durable. When his students
use the tool, they reverse the MT changes and construct their own meaning, from image to
sign. These MT changes support Howard’s goals: to visualize embodied sign production,
maintain content knowledge, and increase accessibility and interactivity.
4.3.3. Implications
MT reduces cognitive load (Christensen 2010b; Smith 2010) for deaf learners by effectively focusing visual attention then dividing it. Howard’s pedagogy reduces the volume of
data by abstracting some aspects and concretizing others. Concurrently, he slows down
the delivery of information, disaggregates complexity, and reduces dimensionality. His
students gain conceptual understanding with increasing sophistication over time as they
use his tools and pedagogy in piecemeal fashion. This is a general feature of MT; embedded
knowledge becomes overt to novices through adding, subtracting, dividing, or multiplying
layers of representation (See the Discussion for more on these four modal logics). As
Howard used it, MT reduces complexity by abstracting and reducing dimensionality, then,
it rebuilds complexity back over time. Howard told me he did not adapt the visual tool
from elsewhere but created it independently based on his knowledge of the research and
understanding of what his students needed to learn.
Howard embedded the same “Rings of Saturn” diagram into a video we co-analyzed
in the SR task. He used the video-based pedagogy to effectively unpack the diagram using
another layer of meta-pedagogical expository ASL. Together, our co-analysis constituted a
complex form of methodological MT featuring scores of additional transductions. Howard
leverages the affordances of video technologies to frame and analyze knowledge forms
that are mutually sensible, interpretable, and comprehensible for deaf learners and deaf
faculty alike. The video extends MT’s additive logic to multiply and increase forms of
multimodality. Howard re-introduces the fourth dimension: time, which he had previously
removed. Howard’s original video is 35 min long. Juxtaposed against the depth and
breadth of his ASL linguistics research and knowledge, it is notable for its sense of whimsy,
and fanciful toy props.
Howard demonstrates ASL morphology by juxtaposing erudite terms with concrete
aspects of the visual tool using tables and charts, his ASL description, English text, and of
course, a foam sword and three hula-hoops for good measure. In the video, Howard wears
these variously-sized hula-hoops suspended off of his body that function identically to the
“The Rings of Saturn” in the diagram. Both the toys and the visual tool are juxtaposed in
the video. The precise parallel between both sets of rings, one graphic and one physical,
emphasizes the role of MT in the interaction. The encoded labels (P, M, D, E, 0–3) on the
rings are identical and further assist students to link meaning through MT. By dividing
aspects of visual complexity in piecemeal fashion, MT reduces cognitive burdens and
increases accessibility while also making the content relatable, fun, and even funny.
Wei and García (2022) emphasize that “bilingualism is a lot more complex than simply
having a first language and a second language” (n.p.). In kind, transduction is more
complex than just interactions between two language modes or a language mode and
a nonlanguage mode. The literature readily supports this claim; when one looks past
language, there are wider discursive networks at play—First, both Howard and deaf
math pedagogics use numbers and dimensions which are not languages but function with
rule-bound structures and require the ability to navigate a complex intersemiotic circuit
(Pagliaro and Kurz 2021). Second, deaf artists and educators use visual tools, information
behaviors, and aesthetic epistemologies that supplant language yet remain bound by rules
and conventions and result in infinitely creative artistic forms (Lupi 2016; Schiff 2010).
Languages 2023, 8, 127
18 of 40
4.4. Case-Analysis 3—Tessa Rose
4.4.1. Main Claim
Multimodal transduction constructs, simulates, and augments reality.
4.4.2. Evidence
Tessa Rose’s teaching was astoundingly multimodal. While we discussed pedagogy
in several ways, in this section, I pivot my analysis to focus on the interactive impacts of
teaching on deaf students’ learning. The data demonstrate how deaf students capably use
MT when they are prompted and supported in doing so. In one SR session, Tessa Rose
and I focused on how and why MT appeared in her deaf student’s multimodal learning
artifacts. In one assignment, Tessa Rose directed her students to write about a food with
special meaning; one that evoked a childhood memory, was culturally important, or was
otherwise significant for them in terms of family or place. Two selections of data are shown.
Both describe the preparation of bahn tet—a Vietnamese rice dish. The first comes from an
act of document analysis involving her student’s work. In Figure 4, both compositions were
created by the same deaf student, whose family emigrated from Vietnam. The left panel of
Figure 4 (below) showcases a recipe–memoir. The right panel shows a graphic-novel form
of the same memoir.
Tessa Rose explained, as students illustrated and wrote about the foods that represented parts of their cultural identities, the first text’s design also mimicked the visual
grammar of a recipe (Kress and Van Leeuwen 2006). This complex design was only stage
one. Next, her students used additional forms of MT to create another newly derived composition, this one using the visual grammar of comic books and graphic novel genres. In
very different ways, these students used the same basic content but diverging assemblage
processes to represent their cultural identities. On the whole, this process of learning is
multimodal, yet it also requires many intermediary stages of MT. As shown in Data 1 and
2, Tessa Rose’s students write and draw concurrently and sequentially. Each mode informs
the other. When the stages are complete, Tessa Rose asks: What changed? Was anything
gained or lost?
Tessa Rose’s Data 1 and 2 Begin.
Recipe Composition
Graphic Memoir Composition
Tessa Rose’s Data 1 and 2 End.
Figure 4. Tessa Rose’s “Student Translations”.
In the SR session, I asked Tessa Rose, How do you and your students change between
modes? I wanted to clarify her metaphorical term “translate”, which she used to describe an
Languages 2023, 8, 127
19 of 40
analytic process of breaking texts down and reconstructing them as images. Tessa Rose was
shy about watching herself on video and asked that instead of co-analyzing video data, as I
did with other participants, that we analyze a transcript instead. To do so, I needed to first
produce a multimodal–multilingual transcript. These were the most complex data from
my entire study—in terms of production and analysis. Some of what follows stands on its
own (e.g., Tessa Rose’s description of the power of multimodality for rich, multisensory
learning), while other areas need explicit interpretation (e.g., her comments about what MT
is or does).
The data analyzed next were gathered in the SR session, where I asked Tessa Rose
questions about the transcript I built that depicts her classroom interactions. The transcript
was intended to use language modes; however, my transcript shows the limits of language
to represent multimodal data; it also shows why MT is useful and at times necessary in
data collection. While analyzing the data, I found it necessary to devise my own forms
of MT. In the SR session, Tessa Rose also used simultaneous communication—this is an
idiosyncratic mix of ASL and speech. Furthermore, she used gestures that are simultaneous
with the two named languages so she can refer to non-present objects and images such as
papers and comic book panels. To represent these as text, I used color, arrows, and other
lines. I also used italics to describe her facial expressions, prosody, and other extra-lingual
information. Midway through our discussion, I describe my tentative theory of MT. In
response, Tessa Rose readily agreed that yes, this is what occurs. After, she offered her own
new insights about how MT is an interactive phenomenon.
Languages 2023, 8, 127
20 of 40
Tessa Rose’s Data 3 Begin.
a.
Skyer:
What does “break it down” mean?
b.
Tessa Rose:
Here is the breakdown, step by step by step. We start here with
writing. Then, the students use several scenes for the drawing, using a graphic-novel
organizer.
Using ASL classifiers, paper, and gestures, Tessa Rose sets up a sequence of shapes on the
table in front of us. There are three “stages,” which she represents using three box-like shapes.
She “places” the boxes starting at right and moves left, on the table surface in front of us.
The boxes look like this:
e.g., [box] [box] [box]
c.
Tessa Rose:
We start here, in box 1.
Tessa points at the first box.
*Herein, I use orange color codes to represent the changes she describes. Please see below—
e.g., [box] [box] [box]
d.
Tessa Rose:
In one section … the students pick it up and then place it here.
Referring to the classifier shapes, she “picks up” and transfers the first ‘scene’ and moves it
to the middle space, between boxes…
e.g., [box] [box] ≪ [box]
Without speaking or signing, she traces the shape of a rectangle on the right side of the table,
then “lifts” it into the air, and recreates it in mid-neutral space. Looking through it, at me.
e.
Tessa Rose:
And then it becomes the first panel in the comic.
She touches the table with her hand, corresponding to the center box.
e.g., [box] [box] [box]
f.
Tessa Rose:
Then, they use what was previously written, which becomes the
illustrated panel. The writing is translated into the visual panel of the comic. There is
a transfer that happens.
g.
Tessa Rose:
Well, maybe I should not, say “transfer” maybe change is what I
mean, or when I am teaching it, I say “translate.” I talk about translation when I explain it to students.
She again, ‘picks up’ the concept from the far Right, and ‘puts it down’ in the middle.
e.g., [box] [box] ≪ [box]
e.g., [box] [box] [box]
h.
Skyer:
It’s not “copy/paste” right? Something changes. Correct?
i.
Tessa Rose:
Yep. Yes. Something happens. Something changes.
Languages 2023, 8, 127
21 of 40
j.
Tessa Rose:
So, what happens in this moment is that we are changing the words
from here [far right box] as we move them here and [change/translate/transfer] then into
the visual [middle box]. And then, at the end, we go back and change them back into
words again here [far left space].
e.g., [box] ≪ [box] ≪ [box]
e.g., [box]
k.
l.
[box]
[box]
Tessa Rose:
And then we go back again. During that phase, I say, “Ok, look at
your comic. Now, tell the story again. Don’t look at this [right/text], look at this [middle/image].” And then they compose a new paragraph that explains what happens
during the change from image to text.
e.g., [box] ≪ [box] [box]
Tessa Rose:
After, we compare this [right/text] and this [left/text]. I ask: “How
is it different?” This is to encourage the “translation” process.
e.g., [box] [box] [box]
m.
Tessa Rose:
Then, I ask, “Is something missing here?”
e.g., [box] [box] [box]
n.
Skyer:
I’m working on a concept called “Multimodal Transduction.” It
means that the initial content of knowledge remains close if not the same in meaning
but the form of knowledge changes completely.
o.
Tessa Rose:
Saying nothing, she furrows her eyebrows. Thinking.
p.
Skyer:
Does that seem accurate?
q.
Tessa Rose:
Yep.
r.
Tessa Rose:
I’m also noticing from what you say that I am really invested in
teaching students and thinking about the process of thinking and seeing metacognitively. And about making. Essentially, that creation itself is a process of change. Building. It’s additive. I so obsessed with change, with “translation”? Like why? How is it
helping them? Why am I teaching this?
Brows again furrowed, looking off into the distance. Then, she drums her fingers on the table
for a moment. After a pause, she continues…
s.
Tessa Rose:
Most of the time I try to think of things that are fun for them …interesting… but maybe, like I said already, it’s the process that is important. I am making them think for themselves, making them step out and become, well, metacognitive. Teaching them to step out and look at or self-analyze.
t.
Tessa Rose:
My students might say: “Look I’m making something when I am
writing. I am sequencing. Arranging.” That’s the process that is going on. Sometimes
it requires ‘mixing modes,’ but it is not a static format/formula.
Languages 2023, 8, 127
22 of 40
u.
Tessa Rose:
i.
ii.
v.
(SIMCOM)
In Spoken English: I’m trying to think of a way to make use of translation, of
communicating that…
In ASL: Translating. Change. Translating. Change.
Tessa Rose:
Tessa Rose’s Data 3 End
That’s interesting. I think that’s why I like multimodal assignments.
(ELICITATION TASK, 11052019b.mp4 -09:58-14:56).
The remaining data on pages 22 and 23 are all sourced from Tessa Rose’s memberchecking document. In it, Tessa Rose’s words and my own are juxtaposed in an asynchronous conversation. Member checking data interrelate information from several sources
located in Tessa Rose’s case study, which are interspliced with my commentary and new
questions, resulting from the interactive member-checking process. This set of highlycomplex data perhaps suggests why MT is inexplicitly and insufficiently described in the
deaf translanguaging research, as a research-based methodology or as a theoretical term
of pedagogy.
After I described my theory of MT to her in the SR session, in the member checking
section, I asked Tessa Rose point blank: How do you transduce? She explained that in her
use of MT, cognitive and poetic processes co-occur. These intercreative processes affect deaf
students and deaf faculty in different ways, depending on the modes used, the purpose for
the MT, and the deaf agent’s prior experiences with MT. Often, deaf faculty have wider and
deeper semiotic toolkits, and more ways to use the tools within them. This allows them to
support a range of differentiated tasks and processes for specific learners and content areas.
Following my request to explicate the processes embedded in multimodal transduction,
her reply describes MT using metaphors, in ineffable and evocative ways.
She wrote:
I see this [MT] change as an expansion/distillation dynamic—breathing in and
out, almost—[because] often ASL into English requires added language—expansion—
English into ASL requires a distillation of meaning—almost like going from prose
to poetry—keeping the essential oil if you will—ASL into drawing could be
a much closer adaptation/translation = from poem to poem, almost—from
ideogrammatic language to ideogrammatic language—so more “word for word”
in a way [it requires a] cinematic grammar [in the way that H.D-L.] Bauman
[writes about].
When I asked why?, she linked multimodality and multisensory data to deaf cognition
and memory-building. As an example, she cited annotating as a form of notetaking that
physically uses handwriting as a semiotic overlay atop another printed text. She claimed
that deaf agents who manually interact with texts create deeper, more lasting memories
because of the uniquely multimodal and embodied dimensions of multimodal deaf learning.
“This is a motor action that triggers [processes in] the brain.” As she theorizes it, deaf
students’ composition process is multimodal; likewise, the product requires multimodality.
Images and texts are enmeshed in a common matrix. Her pedagogy, likewise, shows and
tells. Tessa Rose notes what she did not do:
I could have scanned the materials into a PowerPoint, but I didn’t for at least two
reasons: First that it was just not doable in a way that would preserve the sensory
information that I wanted to share with them, and second, it would be too time
consuming and [would] not [be] worth the effort.
4.4.3. Implications
For Tessa Rose, multimodal transduction is the alpha and omega of deaf pedagogy.
Not language, not sign language, languaging, or translanguaging. “[They are] not rich
enough,” she says. They lack modes. Images alone, Tessa Rose judged, would not suffice.
Languages 2023, 8, 127
23 of 40
Words alone do not suffice. Even sign language pedagogies alone are insufficient. Likewise,
static reproductions or single modes that lack valuable information. Images of books lack
the rich smells and the texture of paper. During one of Tessa Rose observations, I saw a
deaf student scraping a fingernail against a swatch of India ink on vellum. Even the tactile
qualities of ink were a source of fascination. This, I observed, was noted by many of her
deaf students, who touched, smelled, closely inspected, were absorbed with, and artfully
interacted with Tessa Rose’s MT. Watching her students’ keen interest directly supported
this view. For Tessa Rose, visuality in pedagogy is a bare minimum but not an end goal.
Changes to epistemological forms and ontological realities are enmeshed in MT. In
MT, meaning is “carried across” the change, just like in metaphors. This abstraction affects
deaf agents, who often increase visuality as only one stage toward more sophisticated
multimodal simulacra. In one of Tessa Rose’s lectures, her MT process was prompted with
the phrase, “look, look, look!” Tessa Rose used this interjection to guide her students’ gaze.
The gaze was only the beginning of their interactions with the materials on the table, which
contained books, final images, sketches, and written drafts of the memoir project. When
I again asked Why did you use this phrase?, she countered that it was about focusing all
of their attention. She recognized that the approach is based on enhancing the students’
understanding through multiple, converging forms of knowledge, in multiple modes, and
through multiple senses, which are evoked by the materials themselves, along with the
ideas they represent. This demands that non-language and quasi-language modes such as
sculptures and odors take on equal epistemic weight relative to language modes.
For Tessa Rose, deaf pedagogic MT interactions are multisensory and multimodal,
they require tactile, kinetic, spatial, visual—even olfactory knowledge: “Sometimes I am
old fashioned,” she says, “I like old books, real papers, and writing with pen. I like the smell
of old books,” she told me. For Tessa Rose, tangible materials offer fully immersive sensory
experiences, which include visuality but goes well beyond visuality or language. Notable
here is that Tessa Rose’s class is ostensibly a writing class. However, as they write, Tessa
Rose invites her students to draw, to sculpt, to Tweet, to perform, and to be fully-engaged
deaf learners—to “see that” or “do that” for themselves. She commented: “I’d like my
students to experience the real thing, too”. When I yet again asked Why?, she expanded
on the idea of deaf ontologies in deaf pedagogy and the concept of the real. Teaching and
learning in deaf education can be more real or “closer to reality”. According to Tessa Rose,
MT can even supersede the limits of raw perceptual reality. Tessa Rose wanted to build
simulacra that are close to the real, perhaps better than the real. These make for more rich—but
carefully delimited—sensory environments.
5. Discussion
5.1. Theorizing Multimodal Transduction
MT is a mechanism and process involving changes between forms of knowledge that
supersede language. MT is at once epistemic and ontological—these co-occur. Kress (2010)
states that MT results in a complete “rearticulation of [epistemic] meaning [and a] near
total . . . change in ontological orientation” (pp. 124–25). When knowledge changes, new
realities are built and socially shared; importantly, doing so changes the dynamics of power
in deaf classrooms. Etymology demonstrates that transduction is a movement across or
through stages. Because of this, MT uses the same underlying logic as scientific and poetic
metaphors (Lakoff and Johnson 1980; Sfard 1998). This lends MT its ineffable characteristics.
Tessa Rose explained, “something happens” between the phase-change. In this section of
my article, I attempt to explain what happens, why, and how. Likewise, I work to explore
MT in the context of several more-or-less similar theories and methods.
All six deaf faculty in my study were observed using MT flexibly. When deaf faculty
used MT, they aspired to retain meaning, enhance accessibility, and extract or amplify
the most salient features of knowledge to benefit learning in deaf students. MT supports
the transfer of knowledge between modes (or assemblages of modes) via intermediary
forms, processes, operations, and functions that reconfigure deaf students’ sensory access
Languages 2023, 8, 127
24 of 40
to reality; meanwhile, the essentials of meaning are “carried across” the stages of change.
MT is not only an educational interaction between agents and knowledge but an aesthetic
form of cognition endemic to deaf pedagogy. MT is creative, artful, and artistic; those who
use MT act as artists and poets do.
In deaf pedagogy, MT supports flexible changes from any one mode or set of modes
into any other mode or set of modes. MT can happen all at once in one major change or
reoccur across a sequence of linked changes. MT can use language at the start, intermediate,
or end stages of change; however, MT need not use any language modes at all. When
deaf faculty use MT in teaching, it usually results in end-products or by-products that are
appreciably visual; however, on the whole, MT is multimodal in character (Kusters et al.
2017; Tapio 2013).
Observations of deaf pedagogic MT nearly always emphasized visuality over orality,
bilingualism over monolingualism, and multimodality over monomodality. As the dea
faculty and students I interacted with showed me and told me, MT may occur in conjunction
with translanguaging but is separate from it. MT is capable of interactive pedagogies
and learning events that are not just bilingual or bimodal–bilingual but, overall, use
multimodal-multilingualism (Kurz et al. 2021). Here, the term multimodality refers to three
things: multimodal forms of language (Wei 2022), multimodal semiotics (Kress 2010), and
multimodal discourse analysis (Kress 2011).
5.2. Disambiguating Multimodal Transduction
This discussion section aims to disambiguate MT with several more-or-less similar
theories and forms of epistemic and ontological change already present in deaf education. I mainly analyze similarities and differences between the theories of languaging,
translanguaging, code-switching, chaining, and bi/multilingualism. I also construct new
comparable ideas that extend the theory of MT as I have posited it thus far. As a central
pedagogical method observed in all cases of my study, MT is a class-wide phenomena.
As I observed it, MT is used to construct or reconstruct knowledge toward deaf ways of
being. MT represents an “umbrella,” that can organize related changes dominated by
language and those driven by communication modes (e.g., non/quasi-language modes).
Below (Figure 5), I propose a nested diagram to depict how these varied terms might be
organized. In my diagram, the largest category is MT. Under the umbrella, I subdivide two
large categories, one focused on changes using language and the other focused on changes
with communication modes.
Given the context of this Special Issue about translanguaging in deaf spaces, I give
special attention to exploring disambiguating and contrasting MT and translanguaging in
deaf pedagogy research contexts. My rationale is enlarged in a narrative that comprises a
synthetic and critical literature review (Boote and Beile 2005), which highlights the need for
a theory of MT in general research on deaf pedagogy. Because of widespread confusion,
disambiguation is necessary to describe how MT and translanguaging are similar but
different from each other, and how language-dominant and communication-driven forms
of MT differ but also relate. Doing so fulfills a practical need to describe teaching methods
for preservice and practicing deaf educators who may find similar or often-changing
theories hard to distinguish, especially as their basic definitions shift (Wei 2022). It should
also assist researchers of deaf education for similar reasons (Swanwick and Marschark
2010), including a need for empirically-grounded teaching theories. As I argue, the unique
aspects of my theory of deaf pedagogic MT are assets for general translanguaging and
multimodal theorists also, who may explore MT as a mechanism and means to correlate
the distinct concepts of: language, communication, and discourse, more broadly.
Languages 2023, 8, 127
25 of 40
Figure 5. The MT Umbrella.
Seen in this way, MT is a theoretical concept that acts as a basic substrate to enable other
kinds of educational interactions, as represented across the major theories and paradigms
already commonly used in deaf education. My taxonomy lists four forms of languagedominant modal transductions. In the image, they are presented from least to most
complex or demanding for semiotic resources. In order, they are listed as: codeswitching,
languaging, chaining, and translanguaging. Thereafter, I describe four comparable forms
of MT dominated by communication modes. In the diagram, communication modal logics
also track from least complex to the most complex and borrow mathematics terms. They
are: additive, subtractive, multiplicative, and divided visual attention. I call all eight of
these approaches “modal logics,” borrowing phrasing from Kress (2010).
Generally, all eight modal logics are distinguished based on the number of modes and
intensity of their use. Furthermore, language transductions must begin or end or begin
and end with language modes but sometimes involve digressions into communication
modes as supports for language interactions. Language transductions retain an emphasis
on language, and usually use within-language transductions or modal constancy, where
the mode never changes—even if the language does. On the other side, using intentional
designs, communication transductions decenter or entirely eschew language. Communication transductions usually highlight trans-language changes and require modal inconstancy.
In communication transductions, language may be employed as a support, but need not
be the focus. It should also be noted that the relatively-rigid classifications I propose are
data-based theoretical categories. They are artifacts of the constant comparison method,
and are also designed to highlight the differences between the modal logics within and
across categories; however, in actual practice, deaf faculty were observed combining and
remixing these approaches with aplomb and joy.
Languages 2023, 8, 127
26 of 40
5.3. Language Transduction: Four Modal Logics
Language often changes form in deaf education. This operation defines much of what
is known in deaf research (Young and Temple 2014). As I see them, language transductions
are a major subset of MT and have been the primary preoccupation of deaf research since
time immemorial (Rée 1999; Sacks 1990). Language transductions entail within-language
changes to modes but support limited trans-mode changes (e.g., shifting from text to speech,
in which both italicized modes are forms of language). I discuss four examples of language
dominant transductions, called modal logics, in bold, italic typefaces below. I begin with
languaging to emphasize its formative role in the theories of translanguaging; thereafter, I
explain how codeswitching and chaining fit between these bookends in the classification
scheme. Readers may note this narrative diverges from the ordering listed in Figure 5.
Whereas the figure illustrates a sequencing based on relative complexity, the text that
follows begins with contemporary theories and then tracks backwards to examine theories
that comprise important historical precedents.
Languaging is the theory that language is not something that is but rather something
one does (Bagga-Gupta 2019; Swanwick 2017a). Languaging theory is rooted in the idea
that deaf students need to actively use multiple languages in several language modes
within social contexts. However, the idea that originally described social habits was quickly
adapted to describe classroom-based learning; thereafter, it was again adapted and applied
as a teaching framework (Swanwick 2017a). The theory of languaging in deaf education is
originated in and articulated through linguistics research focused on a social framework for
language-based interactions (Ortega 2009). Languaging theory emphasizes a multiplicity
of uses and contexts for language. Its origins are rooted in bilingual traditions, but it has
been since used to support bimodal–bi/multilingualism in deaf education (Kurz et al. 2021;
Swanwick 2017a). While the theoretical stance about social language interactions may
appear relatively benign, it represents a significant departure from the popular cognitivist
theories of language that continue to hold sway in deaf research (Knoors and Marschark
2014). It is also notable that the languaging theory was only in vogue for a short time before
being replaced by translanguaging.
Translanguaging extends languaging but retains many of its key characteristics. In
deaf pedagogy contexts, Swanwick’s (2017b) initial definition states that “translanguaging
represents an additive view of bilingualism and multilingualism of deaf learners [that]
focus on language as a social phenomenon” (p. 233). Although other differences exist, the
chief one to consider here is that languaging and translanguaging differ in the number of
languages and language modes and the intensity of language changes that occur. Both
theories emphasize actions with language modes. Translanguaging takes a broader view of
linguistic repertories of deaf and hard of hearing people and aims to support a positive
and heteroglossic ideology of multilingualism (García 2009; Kusters et al. 2017).
Translanguaging is and requires competence in two (or more) languages (Baker 2011;
Holmström and Schönström 2018). However, these languages need not be known equally,
nor does it require different modes—for example the rapid intertwining of spoken English
and spoken Spanish may comprise one form of translanguaging. Often this implies persons
from two (or more) sociocultural groups, which may be unequal in status or power (García
and Wei 2014). Translanguaging theory offers inborn support for changes among modes of
language, as with the change between written English and spoken English. In deaf education,
this offers wide utility, as deaf students and deaf educators often make changes not only
between two languages but also two language modes. For example, moving between signed
ASL and printed English, or Swedish Sign Language and written Swedish (Holmström
and Schönström 2018). Translanguaging is useful for teachers in deaf education because it
may support the overt change between two oral modes (e.g., English to Dutch), two sign
language modes (e.g., ASL to Langue des Signes Québécois), or between one sign mode and
one oral mode (e.g., ASL to spoken English).
Languages 2023, 8, 127
27 of 40
Swanwick (2017b) shows that translanguaging occurs as an output of socialization
or learning, but more recently, the theory is supported for use by teachers in pedagogy or
as curriculum (García and Lin 2016; Swanwick 2017b; Swanwick et al. 2022). Swanwick
(2017a) states: “Translanguaging [is the] active and purposeful use of language for learning.
From this theoretical perspective, language is transformed from a noun into a verb. The
ideological roots [are] located in the shift to a more positive view of bilingualism” (pp.
82–83, emphasis added). Teachers in deaf education may exploit these features by calling
attention to the changes between language modes. At the outset, it is important to note
that translanguaging theory is rapidly changing, especially in the years since my data
were collected. More recently, general translanguaging theorists such as Wei, Garcia and
Lin and theorists of deaf translanguaging such as Kusters, Tapio, Swanwick and others
have explored the role of non-language modes and take a broader view of multimodality;
however, in general, language remains the major focus. I discuss the differences between
these ideas and my theory of MT in the final section of this article.
Codeswitching follows the language-dominant pattern of changes to modes but on a
much smaller scale than languaging or translanguaging. Codeswitching occurs when an
utterance made in language one is briefly interrupted by an utterance made in language
two; thereafter, the conversation returns to language one, where it generally remains. In
most nondeaf cases, the language modes also remain the same; for example, an utterance
in English is interrupted by one word or phrase in Spanish, but both English and Spanish
are spoken throughout. This kind of code switching is within-mode and within-language
transduction. Codeswitching has social, learning, and pedagogic value, but codeswitching
is generally functional not transformative of power relations. Codeswitching is rooted
in monoglossic, monolingual norms that may tacitly or explicitly support subtractive
bilingualism (García 2009; García and Lin 2016; Swanwick 2017b). For an extended account
of differences between codeswitching and translanguaging, see García and Lin (2016), who
briefly discuss deaf educational codeswitching. This gap provides impetus to discuss these
ideas at some length.
Generic code switching by nondeaf bi/multilinguals is seldom mode-switching; in
contrast, for deaf people, code switching is very often mode-switching (Kuntze 2016). In
deaf education, codeswitching usually involves trans-mode changes that are (at minimum)
bilingual and bimodal (García 2009). Kuntze (2016) writes that while generic code switching “retain[s] phonological and morphological structure, [deaf code switchers do] not
conform to characteristics of spoken-language code switching” (n.p.). Like languaging and
translanguaging, codeswitching is derived from social language uses. Its origins are social
and functional; while it may be used subversively, it is not generally transformative of
power relations or inequities thereof.
While it may be pedagogically utilized (this term is used deliberately)—codeswitching
is not always valued similarly by students, teachers, or institutions (García and Lin 2016). As
Guardino et al. (2018) explain, codeswitching is a phenomenon of learning, of sociocultural
exchanges that can be exploited by deaf educators. It is “an instructional technique [to]
utilize ASL where [deaf agents] alternate between English and ASL [to] illustrate the
differences between the languages” (Guardino et al. 2018, p. 227). Codeswitching has a
limited use-value in deaf pedagogy. I found only nine events of clear codeswitching in
my data corpus. For example, Louis used one or two English words in the middle of an
ASL utterance to teach a student with a cochlear implant who preferred speech modes.
Elsewhere, Sarah Jo used codeswitching to emphasize grammatical differences between
suffixes in English by using Signed English.
Finally, Chaining deliberately links sign language modes with fingerspelling alongside
written texts. Overall, the method supports print literacy as the primary goal. The aim
is important here: signacy is valued as a means to an end, not as an end itself. Guardino
et al. (2018) define chaining as “an instructional strategy [that maps] ASL to English print
Languages 2023, 8, 127
28 of 40
by fingerspelling a word, pointing to it in written form, and showing the ASL sign for the
word” (p. 226). The method was coined by Humphries and MacDougall (2000) as a way to
pedagogically link visual language modes, including text, sign, and fingerspelling. Per their
empirical description, chaining may use undefined “media” as auxiliary supports, but these
are deemphasized theoretically and empirically. A variant form is called “Sandwiching.”
Sandwiching is even more limited—it focuses on changes that begin and end in the same
mode (“the bread”), with one change of mode in-between (“the filling”). A “sandwich”
may look like this—sign, image, sign, or text, sign, text, etc.).
In Chaining, multimodality is optional to a pedagogy dominated by and focused on
language. As Humphries and MacDougall (2000) write:
[Chaining identifies] the ways that ASL and English interact with each other in
various forms. Specifically:
•
•
•
•
•
•
how teachers make connections between signing and print
how teachers introduce/talk about English words
how teachers use fingerspelling and initialized signs
how teacher[s] introduce new words/concepts
how teachers use different media to [connect ASL] with print
other types of language interplay that teachers use (p. 87).
Chaining is dominated by language. Language modes are mentioned 13 times in this
definition and “media” just once, and in relation to languages. Chaining is linear changes to
language modes. Chaining’s inputs and outputs are language modes and move in a specific
sequence. Chaining supports literacy. “Media” are not required, nor do they appear to be
valorized or thoroughly described in the original article, nor do Humphries and McDougall
deign to detail their forms, functions, or aesthetics. There is a conspicuous lack of detail
about “media”, suggesting that researchers and teachers may not value them. In this way,
chaining may also tacitly be a subtractive bilingual process (García 2009) that inadvertently
maintains imbalances of power. I identified 17 instances of chaining, suggesting it has some
use-value in deaf higher education, but it remains fundamentally limited by its overreliance
on and fealty to language modes and transductions.
5.4. Communication Transduction: Four Modal Logics
Communication transduction is different. These modal logics deploy modes flexibly
in as-needed, dynamic combinations to create complex web-like representations. There is
no upper limit to the number or kind of modes used. There are no restrictions for which
mode begin or ends, how many changes can or should occur, or what discipline it can be
used with. Modes overlap in transit, work together, or work at cross-purposes (Smith 2010).
The data show intermediary stages, events, digressions, and additional processes. Like
the four language logics, communication transduction has four independent modal logics
(below, Table 1). There are at least these; other datasets would likely reveal more nuance
and variation. Each differently alters the forms and dimensions of knowledge modes that
configure reality for the deaf agents who interact with them.
MT seeks mutually comprehensible information exchanges and is broadly in support
of multimodal educational interactions. Languaging, translanguaging, codeswitching, and
chaining originate as techniques or theories about learning and originate from the use of
social languages. MT is pedagogical in origin and involves the widest-range of semiotic
resources (Kusters et al. 2017; Tapio 2013). MT readily extends into other domains such
as curriculum, assessment, feedback, and so on. In MT and communication transduction,
modes very often interpose, are indeterminate, or are purposefully juxtaposed within or
atop one another. In reverse (as it is commonly applied), translanguaging is language transduction that focuses on language modes with other modes in subordinate positions. The
results and processes of MT are multimodal clusters of meaning, where hard distinctions
between modes, be they language or communication, are neither needed nor sought out.
Languages 2023, 8, 127
29 of 40
Table 1. Communication Transduction: Four Modal Logics.
Modal Logic:
Additive
Subtractive
Multiplicative
Divided
Visual Attention
Definition
Deaf agents add one or
more new mode/s or
multimodal
assemblages in a slow,
methodical fashion.
Deaf agents
purposefully remove
single or multiple
modes, or reduce the
intensity or volume of
multimodal
assemblages.
Deaf agents deploy
multiple modes or
several assemblages
quickly by means of
simultaneous or
near-simultaneous
events.
Deaf agents divorce or
disaggregate complex
multimodal
assemblages into
components or split
their gaze resources
purposefully.
Purpose
. . . to increase
explanatory power
with the addition of
new modes or to
conserve time or effort.
. . . to augment reality
or change knowledge
forms by reducing
dimensions or to clarify
a salient concept using
fewer modes.
. . . to simulate the
depth, breadth, and
intensity of multiple
senses acting in concert
or to simulate a
complex, multimodal
reality.
. . . to emphasize
relationships between
plural entities,
including as an overt
analysis of
language-based or
communication-based
multimodal
assemblages.
Corpus
Examples
1. Louis draws an
arrow between two
columns of biochemical
data, drawing his
students’ focus to links
between data in one
column and an applied
mathematical formula
in the second column.
1. Sarah Jo comments:
English is 2D and linear,
whereas ASL is 3D and
spatial. She explains,
textual language
imposes limits on
expansive discursive
dimensions that are
latent in ASL.
1. Edward’s video
lecture uses text, sign,
image, speech, layout,
the movement of the
body in physical space
[MOTBIPS], gesture,
visual tools, and other
modes in concert.
1. Sarah Jo’s student
uses a smartphone app
to translate and decode
her English
instructions by using
traditional Chinese
pictographic
characters.
2. Through elaborate,
multimodal dialogues
and social critical
thinking, Astoria and
her students create a
new drawing using
visual design principles
to depict spatial
arrangements, sourced
from a text.
2. Howard
disembodies and
schematizes ASL
modes using visual
tools he designed to
reduce the number and
intensity of embodied
modes students
encounter at one time.
2. Tessa Rose’s table
lecture deploys
numerous graphic
memoirs in various
stages of completion.
She prompts students
to “look, look, look,”
then, guides the
students in multimodal
analysis.
2. Edward’s student
sits with a notebook
and his textbook to the
left of his computer
monitor; as he
manipulates the visual
tool software suite, he
also refers back to his
hand-constructed
visual tools and textual
notes in a lengthy cycle
of learning.
5.5. Situating MT and Translanguaging in Deaf Pedagogies
It is important not to be overly reductive. In this section, I attempt to describe how MT
and translanguaging are similar to and yet still very different from one another. The main
issue at hand is: which modes of discourse that enable information exchange are (tacitly or overtly)
the focus and which modes are considered subordinate? Deaf translanguaging pedagogy, theoretically and practically, mostly deals with language transductions; however, deaf educational
translanguaging studies are increasingly written to be nominally inclusive to multimodality,
including the “modes of image, sound . . . gesture . . . gaze, body posture” (Swanwick et al.
2022, p. 1). Changes of modalities from language-to-language are relatively common and
are assuredly important in deaf pedagogy. To my knowledge, no transduction mechanism
has been described in this literature to date, and the theoretical focus of translanguaging
remains on language. It is not a coincidence that Swanwick et al. (2022) and Holmström
and Schönström’s research (2018), the objective of translanguaging is learning language.
However, language transduction, including modal constancy and within-language
transduction is only a fraction of the total forms of MT that occur in deaf pedagogy. When I
coded my data in 2019, the most popular theory of deaf pedagogic translanguaging available
Languages 2023, 8, 127
30 of 40
was Swanwick (2017b), whose theory advances the claim that translanguaging requires
teachers to match the language repertories of their deaf students. In my total dataset, I identified
just 32 instances of language transductions; and yet only 6 fit this popular definition
of translanguaging. Importantly, this signals a problem that has been unaddressed in
translanguaging research, which I discuss next.
5.5.1. Linguistic Overdetermination and Multimodal Superfusion
From the theory’s inception, translanguaging is keenly focused on language in (deaf)
students learning (Swanwick 2017b). A canonical, foundational text begins this way:
“Translanguaging is the process of making meaning, shaping experiences, gaining understanding and knowledge through the use of two languages” (Baker 2011, p. 288, italics
added). Lewis et al. (2012) find it necessary to disambiguate translanguaging with two
not-quite-synonyms, including codeswitching and bilingual education. These show the
genealogical roots of translanguaging, which place it squarely in the context of language
research. Wei and García (2022) express consternation about the ongoing confusion about
translanguaging theory—“Translanguaging is often interpreted simply as enabling students to go across the two languages of instruction” (p. 314). The idea that translanguaging
transcends language is relatively recent (Wei 2018, 2022), and is only partly analyzed by
current empirical research in deaf studies (Holmström and Schönström 2018; Swanwick
et al. 2022). While translanguaging is “moving away from a focus on linguistic behavior to
embrace the multimodal and multisensorial aspects of communication” (Swanwick et al.
2022), translanguaging in and outside deaf education remains dominated by references
into, out of, or in near proximity to languages.
The translanguaging generalist’s focus on language interactions affects deaf research in
specific ways. While most deaf translanguaging research rightly champions sign language
modes and their important role in development and cognition and to redress imbalances
and asymmetries of power in teaching (De Meulder et al. 2019; Kusters et al. 2017), excusing
select hyper-current studies (Swanwick et al. 2022), most current research (De Meulder et al.
2019; García 2009; Swanwick 2017b) about deaf translanguaging exhibits linguistic overdetermination. Overdetermination is the notion that within a given set of co-existing and
plural causes and effects, contradictions appear because causes and effects are restricted by
a given frame of reference (Althusser 1962; de Alba et al. 2000). By linguistic overdetermination, I mean that (at least some foundational) translanguaging theorists have (perhaps
inadvertently) constructed a “round peg and square hole” dilemma, whose methodology
generally attempts to force communication modes (that are not language modes) into a
framework expressly built to theorize language. Doing so distorts reality, subordinates
multimodality and, reinforces inequities of power in deaf education (Thoutenhoofd 2010).
Linguistic overdetermination is an unacknowledged limitation to translanguaging
theory. Perpetuating linguistic overdetermination is perhaps unwise. Here are three
other examples from foundational deaf translanguaging research that show why linguistic
overdetermination is a practical dilemma. First, García (2009) writes, “Deaf people in the
United States translanguage between American Sign Language, mime, written English, and
often International Sign Language” (p. 135). This example names specific languages then
vaguely refers to “mime” to fill the gap where multimodal aspects like gesture might fit.
Second, García and Lin (2016) write, “bimodal bilingual translanguaging of deaf children [is
a] useful means of conceptualizing their language practices and the ways in which they use
their language repertories” (p. 9, emphasis added). This characterization affords language
privileged status. Garcia does excellent other work in many other volumes related to deaf
bilingualism, but her theory of deaf translanguaging in teaching is limited by its dedication
to languages.
Likewise, as a final example, De Meulder et al. (2019) carefully include multimodal
semiotics and gesture studies in their theoretical frame, but, in the following quote—
“translanguaging and (multimodal) language repertories” (p. 898)—they literally bracket
multimodality, which is subordinated by parenthesis. Elsewhere in the same study, they
Languages 2023, 8, 127
31 of 40
offer a strong critique, which gives another view of linguistic overdetermination, which I
quantify next. Their passage is lengthy, but instructive:
Originally, translanguaging described [how] a minority language was used in the
classroom along with a majority language (Lewis et al. 2012), but since then, it has
become a ‘terminological house with many rooms;’ (Jaspers 2018, p. 2). An oftencited recent definition is that translanguaging is ‘the deployment of a speaker’s
full linguistic repertoire without regard for the watchful adherence to the socially
and politically defined boundaries of named . . . languages (Otheguy et al. 2015).
García and Lin (2016, p. 19) suggested that we are witnessing a ‘translanguaging
turn’ with the term now referring to both the complex language practices of
plurilingual individuals and communities, as well as the pedagogical approaches
that use those complex practices . . . Translanguaging is currently used in both
descriptive and prescriptive ways. It can be used to refer to a bilingual pedagogy,
multilingual spontaneous language practices, everyday cognitive processes, a
theory of language in education, as well as a process of personal and social
transformation” (p. 893).
This lengthy excerpt surveys a decade of studies, yet in it, there is no mention of modes
other than language or citations related to multimodality. Instead, 14 references are specific to language or language modes. Other examples of linguistic overdetermination in
translanguaging could be given, but largely, the pattern holds. While Jaspers (cited above)
describes translanguaging as a house with many rooms, to me, it appears that the house is
overfull and its contents are spilling into the yard.
Linguistic overdetermination reinforces discursive inequity and contributes to the
marginalization of deaf ways of being and knowing by minimizing the role of non/quasilanguage modes, including images and artforms, mathematical modes, and embodied
modes such as kinetic discourses, proxemics and eye gaze, which are critically important
in deaf pedagogic contexts. Affording multiple language modes privileged status above
multimodality, including gestures or drawings—discounts or denigrates intact abilities in
deaf learners (Skyer and Cochell 2020). This reinforces problematic power relationships
and results in subpar information exchange, further impairing educational processes. The
MT perspective could support and clarify examples present but undertheorized in studies
of deaf pedagogy performed by Swanwick et al. (2022) and Holmström and Schönström
(2018). In Swanwick’s British study, the authors note “language becomes dominant” and
overtakes in importance “the use of pointing, gaze, and body orientation” (p. 7). The
British team also notes that pedagogic “decisions [are] dominated by a reliance on spoken
language” (pp. 12–13). Echoing this completely, the Swedish team writes, “our analysis
focuses on languages” (p. 98). Later, the Swedes reveal, “translanguaging in general is only
possible if the interlocutors share two or more languages” (p. 109).
This is the key to understanding why linguistic overdetermination is so problematic—
which is perhaps only revealed by the inclusion of deaf perspectives in translanguaging
research. What I mean is that the persistence of language deprivation is widespread in deaf
populations; this total or partial lack of language fluency is not adequately dealt with in translanguaging theory or practice. At least according to the Swedes above, translanguaging should be
conceptually impossible with language deprived deaf learners. In actual practice, this means
that deaf students, such as students in Astoria’s class do not have equal access to the two or
more languages that translanguaging requires. This disempowers deaf learners in immediate
classroom interactions and marginalizes their intact abilities in research and teaching. Linguistic
overdetermination is a round peg and square hole dilemma that must be resolved.
The incoherence of the round peg and square hole method is problematic, because deaf
education as a whole is probably best characterized not as a phenomena of bimodality in
languages but as a multimodal superfusion of discourses. My research showed this multimodal superfusion as 500 discrete modes, which I classified into 11 categories, alongside
6 specific sensory systems (Skyer 2021). It is perhaps understandable that my participants
used and cited multimodality much more often than translanguaging. Empirically, the
Languages 2023, 8, 127
32 of 40
data I collected are univocal—MT traverses the entirety of the multimodal superfusion to
transcend languages entirely.
5.5.2. Reclassifying Translanguaging as Multimodal Transduction
In addition to a longstanding emphasis on multiple modes of language, recently,
translanguaging has embraced a semiotic approach to multimodality. First, Wei (2018)
writes “Human beings think beyond language, and thinking requires the use of a variety of
cognitive, semiotic, and modal resources of which language [is] only one.” (p. 18). Second,
Wei and García (2022) have emphasized that translanguaging research is a “transcendence
of named languages” (p. 313). Finally, Swanwick et al. (2022) claim, “a lot could be learnt
from the close observation of even short interactional episodes to map out the classroom
layout, positions and resources of the participants, analyze the auditory and visual attention
demands of the setting and the coordination possibilities of these intersecting [multimodal]
resources” (p. 14).
As a result of the data I have shown and in response to the literature and theories I
have reviewed, I propose two major taxonomic reclassifications: first, that translanguaging
be understood as a subset of MT that is mainly defined by language transductions with
multimodal semiotics playing a supporting role; second, that there are (at least) two major
sub-classifications of MT—those that are focused on language transductions (including
traditional translanguaging) and those inclusive to communication modes such as nonand quasi-language semiotic forms. Neither language transductions or communication
transductions are better than the other; both have value in the deaf classroom. There are
too few empirical studies examining how translanguaging and multimodality are used in
deaf classrooms. Those available focus on language in learning (Holmström and Schönström 2018; Swanwick et al. 2022), not communication transduction, multimodal pedagogy,
multimodal discourses, or MT mechanisms. Importantly, my study empirically addresses
each of these gaps. The two strongest differences between the new classifications and
those explored in traditional translanguaging theory are that (1) MT and communication
transduction are not restricted to or dominated by language, and (2) communication transduction and MT are not originated in social learning theories or principally about or defined
by linguistics concepts. Instead, communication transduction and MT are pedagogic in
origin and orientation and readily bypass the limits of languages, specifically the limits
of language in deaf pedagogy and its research traditions. While multimodal discourses
can be mutually comprehensible for all deaf agents (Skyer 2023), if they are devalued in
research, they will also be devalued in pedagogy, curriculum, assessment, learning, and in
the administration of deaf education.
My stance about reclassifications is an empirical affirmation of Wei’s (2022) claim
that researchers and teachers must “refus[e] to privilege particular modes and methods of
meaning making over others” (p. 1). My proposed shift is foundationally oriented toward
multimodal discourse analysis, as opposed to linguistics, and configures multimodality
(not language) as the fundamental basis for ethical pedagogy and curriculum (Kress 2010)
in deaf education (Skyer 2020, 2022). My study of MT empirically describes what much
of translanguaging theory only suggests. It assists in theorizing with increasing precision
how language relates to communication via MT and shows both as equally important parts
of a superstructure of multimodal discourses. My study is one small contribution, but as
my participants told me often and showed me in many ways: the MT stance is critically
needed, it is novel and represented something they wanted but could not find—even when
they looked to deaf pedagogic translanguaging.
6. Limitations and Applications
6.1. Limitations and Transferability of MT
As with all research, this study was limited. Mainly, it was limited by data, by available
participants, and my own biases and interests (Timmermans and Tavory 2012). A key area
of interest to grounded theory and case study researchers is the probability of transfer. The
Languages 2023, 8, 127
33 of 40
transfer of findings based on my theories to other sites and populations must be carefully
considered. Ultimately, decisions about transference fall under the purview of consumers
(not producers) of research. I have attempted to extract useful data and construct useful
theories, but my analysis and data are partial. It is worth noting again, this article excerpts
data from a three-year study (Skyer 2021). Interested readers are encouraged to see how my
findings of MT fit the larger study, which includes other forms of multimodal data and an
expanded discussion of methodology, theory, and interpretation related to multimodality
and visuality.
More empirical-based theorizing, such as other grounded theory studies, is necessary,
along with new datasets and other participants, researchers, and research teams. These
should be lead by researchers and teachers who are deaf. Doing so could make both
theories, MT and translanguaging more comprehensible and applicable for researchers and
teachers in the classroom or lab. My research setting was deaf higher education—this may
not be the same as yours. Although not observed directly, deaf pedagogies based on MT
may have transferable use-value for deaf educators and stakeholders in early childhood
education, such as parent–infant programs (PIPs), or sites of early-detection and earlyintervention relative to deafness and other disabilities. MT may support deaf andragogy,
that is, studies of teaching and learning with adults, or deaf gerontology, defined as studies
of living related to elder deaf persons. MT could link domains including: Graham’s (2015)
work on early-childhood deaf gaze interactions, Simms et al.’s (2013) deaf early-childhood
visual communication competencies, Kuntze et al.’s (2014) and Kuntze and Golos’s (2021)
visual literacies, and Raike et al.’s (2014) work on deaf adult uses of multimodality in deaf
education teacher preparation. MT could be useful for deaf mental health interventions
with language deprived persons (Glickman and Hall 2019; Pollard and Fox 2019). MT
might fill the gap where “deaf dual coding theory” should reside, where “the verbal” and
“the visual” coincide, something current cognitivist theorists of deaf learning cannot or will
not explain (Knoors and Marschark 2014; Marschark et al. 2017).
MT is built for teaching. It is a tool that supports deaf faculty metacognition and supports
detailed analysis of pedagogy (Swanwick et al. 2022). This can include the traditional focus
on the efficacy of teaching and the relationships between teaching and learning. However, MT theory can also support new analyses of the overall axiology of deaf education,
including ideas about what makes deaf pedagogy enjoyable, good, or beautiful. MT is
flexible, extensible, and intercreative. It is the multipurpose use of the full semiotic toolkit.
It is an eclectic, adaptable approach to interaction in deaf multimodal and visual pedagogies. However, let me be exactingly clear: Eclecticism is not random tools for random
jobs. MT is not a relativist grab-bag or a hasty mishmash of “anything goes” methods. As
prior deaf pedagogy philosophies have shown us, an unorganized “totality” can lead to
more problems.
6.2. Limitations of Translanguaging
I claim that “translanguaging” as it was represented by many foundational studies
and as popularly understood by my participants and by others is better defined as language
transduction. If true, then translanguaging is just one subset of MT. Given the context of
this special issue, my stance is not likely to be fully accepted at face value; however, my
perspective could be supported or refuted by other datasets and other analyses. I invite
others to examine these issues with me.
As Swanwick et al. (2022) and Wei (2018) are adamant about noting, new analyses
about the transcendence of language can change how we understand even the most basic
aspects of deaf pedagogy, curriculum, learning, cognition, and even language itself. MT
research should be seen as a point of entry into a new problem space that aims to improve
and better align the ethics and aesthetics of teaching and learning in deaf education. MT
research could, for instance, deepen and clarify general translanguaging research. For
example, Wei and García (2022) discuss equity and power, with respect to aesthetic modes
such as calligraphy and musical knowledge, which are important in many cultures, but
Languages 2023, 8, 127
34 of 40
are aspects that are implied but not explicated in their paper. Beyond the limitations of
the current literature, my data demonstrate the limits of language in deaf pedagogy and
curriculum and the limits of language as research data. They also suggest how MT can
provide tools and concepts to transcend them.
Deaf languaging pedagogies (Bagga-Gupta 2019; Swanwick 2017a) and deaf translanguaging pedagogies (García and Cole 2014; Snoddon 2017) represent educational theories
originated to describe the language habits of learners. If communication modes such as
color or image are used, they are afforded less status, play marginal supporting roles, and
are minimally explored. Translanguaging began as a descriptive theory of what learners
do with languages in social settings, but it rapidly became a prescriptive pedagogical koan
(De Meulder et al. 2019). These changes and disjunctions flustered stakeholders, including
those in my study, who needed the theory to do something it could not yet do. To support
multimodal deaf pedagogy, they looked to deaf translanguaging research and could not
find what they needed. Several times, translanguaging researchers have changed the theory in an attempt to transcend the focus on languages but problems of interpretation and
application remain. As I have explored them, these problems are rooted in linguistic overdetermination. At base, I speculate that the dilemma is one of focus—where translanguaging
research’s roots are in linguistic and social uses of languaging and learning, MT’s roots are
in multimodal discourse theories focused on pedagogy and curriculum. Specifically, these
latter domains are those that teachers can influence directly.
6.3. On Discourse and Axiology
MT is an input and an output with aesthetic and ethical affordances and constraints
(Kress 2010). Developing the broadest possible pedagogical-discursive toolkit is the only
way to differentiate for the heterogenous discursive toolkits of deaf learners. All deafperceptible modes are equipotential and necessary for teaching diverse deaf learners in
an interaction framework (Skyer 2020; Skyer and Cochell 2020; Swanwick et al. 2022). My
description of MT elevates non-language and quasi-language modes to be on par with
language modes and suggests that communication transduction is of equal importance to
language transduction. This distinction may appear pedantic, but it is not. Acknowledging
that modes such as drawing, sculpture, and body movement are equally important as
language modes such as speech, text, and signing, which my data show unequivocally, is
an empirical reflection of the theoretical assertion that the composition of deaf students and
their teachers’ semiotic toolkits differ (Kusters et al. 2017; Kurz et al. 2021).
Likewise, if the focus on MT and translanguaging studies continues to be on equity
and resolving imbalances of power (as it should be), researchers and teachers should be
open to the ethical potentiality of multimodality, especially of modes that are not linguistic
in origin or focus (Kress 2010). Moreover, subordinating communication below language
perpetuates inequity and fosters discursive injustice. The MT stance suggests that deaf
education is defined not by bimodalism or bimodal languages but instead by a multimodal
superfusion, in which hundreds of language and communication modes are co-present,
valid and potentially equally important.
MT research has only begun. It is imperative that we continue these lines of research.
The characteristics of MT offer wide utility in the classroom and in research contexts about
deaf pedagogy and deaf curriculum. The MT realignment I suggest demonstrates new
potentialities for the beneficial growth for all deaf agents. Deaf faculty use MT for a range
of pedagogical interactions. They leverage not only visuality (Holmström and Schönström
(2018) but an abundance of other intact sensory systems that coincide with multimodal
semiotic networks of affect and effect. MT supports the change in form and the retention of
meaning from any mode into any other mode. MT can be used for any content area, in any
discipline, for cognition and metacognition. It can even be used for its own sake—just for
the aesthetic pleasure afforded by change. MT is motivated (Kress 2010). Its deployment is
not universalizing, but situated (Gee 2004). MT uses specific tools for specific jobs. As deaf
Languages 2023, 8, 127
35 of 40
faculty make decisions about MT based on its axiological use-value, they must consider
both valences of deaf axiology, its ethics and its aesthetics.
7. Conclusions about Deaf Pedagogies of Multimodal Transduction
7.1. MT Is Literally and Metaphorically Transformative
MT occurs when knowledge changes form, when modes of knowledge transform, or
when tacit knowledge is obviated. MT extends far beyond language-based modes (e.g.,
sign, script, speech, text, etc.). Instead, MT relates languages to communication modes
such as quasi-language and non-language modes (e.g., gesture, spatiality, embodiment,
image, etc.). MT includes changes from any one mode or set into any other mode or set,
including when the initial, middle, or final modes circumvent language to explore modes
including mathematics, color, line, layout, non-language sounds, proxemics, etc. MT differs
from but supports the ways that deaf learners and teachers do language transduction,
which I defined using four modal logics. These including code switching, which is a
temporary language changes; chaining, which is a set of linear language representational
changes; languaging, which is the active use of languages; and translanguaging, which
describes broadly systemic changes to language modes. My theory of MT provides four
new modal logics. The additive logic shows how new modes are included methodically.
The subtractive illustrates how modes are taken away deliberately. The multiplicative logic
shows how a plethora of new modes can rapidly accrete and result in complex simulacra.
Finally, the divided visual attention logic supports how discrete assemblages are used in
sequence or in close proximity. These communication transductions extend the modal
logics already present in language transduction and flesh out what is absent in traditional
translanguaging theory.
As this study’s participants demonstrated abundantly, MT entails simultaneous
changes to knowledge and reality that deaf faculty use to preserve or enhance meaning throughout formal changes to knowledge. MT is an ethical approach to interactions
that explore both communication and language in deaf pedagogy. Deaf faculty make realtime changes when using MT, based partly on their positive initial values about deafness
and partly based on student feedback, and adjust their tactics if needed. MT is extensible
and used iteratively in cycles until, as Astoria suggests, a tipping point occurs, and both
faculty and students are satisfied that conceptual fulfilment has occurred. MT is valorized
for aesthetic expressivity. In addition to fostering language learning, developing communication abilities, conveying curricular content, and transforming disciplinary knowledge,
MT is an aesthetic change that is ethical, enjoyable, and joyful.
7.2. Situated Deaf Pedagogies
Deaf faculty enact MT in situated contexts where they interact with MT alongside contentarea knowledge, general pedagogical knowledge, and knowledge specific to deaf education,
including but extending beyond knowledge about Deaf Culture, ASL-based pedagogies, and
the movement of the body in physical space (MOTBIPS). MT occurred in learning, teaching,
pedagogy, curriculum, assessment, and feedback. It appears to be a case-independent, classwide phenomenon that offers a common underlying structure for the praxis of deaf education.
Scholars increasingly refer to deaf students as bimodal–bilinguals (De Meulder et al. 2019;
Knoors and Marschark 2014; Leigh and Andrews 2017), but this focus on the ontology of language
mischaracterizes the complex multimodal ontology of deaf students and deaf educators
(Holcomb et al. 2021; Kurz et al. 2021). MT provides a superstructure for analyzing all of
these domains.
By decentering language, deaf agents can creatively and flexibly use the widest possible range of modes and discourses. MT supports a comprehensive understanding of the
complex forms of information exchanges common in deaf education’s many disciplines
and sub-disciplines. Additional study is warranted, particularly with cultural, racial, or
gendered subpopulations and niche areas of study, for instance: how do deaf archeologists use
literal artifacts in conjunction with gestures and sign language when training new students in field
Languages 2023, 8, 127
36 of 40
studies? How do deaf human geographers use maps and embodiment to explore diaspora? How do
deaf indigenous storytellers use animal bones and feathers in place-based narratives?
7.3. On Disability
Holcomb et al. (2021) suggest that deaf pedagogical translanguaging theory lacks
an accounting of disability and deafness. My findings show that MT may circumvent the
aporetic problems of using language-based discourses in the education of deaf pupils who
are language deprived (Skyer 2021). I mentioned before the incoherence of the square peg
and round hole methodology. This dilemma is exacerbated by linguistic overdetermination,
where, as one example, Wei (2022) attempts to redraw a boundary by asking what is
language? Perhaps instead of new definitions, new questions are warranted.
In our SR session, I asked Tessa Rose, how can a deaf student without an intact
language make use of any language. She said she did not know. Neither did I. The theory
of translanguaging breaks down in the face of language deprivation. In its place, MT may be
useful for educators trying to make progress with deaf students diagnosed with language
deprivation or who live with other language disorders, sensory disabilities, or learning
disabilities, such as deafblindness, auditory-processing disorders, or autism (Gargiulo and
Bouck 2021; Gulati 2019). Perhaps work on MT could be useful to frame and construct
an approach to teaching that entirely bypasses language deficits and to work with intact
communicative and discursive assets instead. This would emphasize a pedagogy using
intact abilities as a situated means to compensate for biosocial disabilities (Skyer 2020;
Vygotsky 1993).
7.4. Use-Value of Multimodal Transduction
MT’s instructional utility depends on the deaf pedagogue’s judicious use of a wide,
deep semiotic toolkit, which needs to be purposefully aligned to deaf learners’ toolkits to
enhance their own inner sensory and discursive abilities. This toolkit cannot be limited
to language alone for our reality is not limited to language. Said another way, our understanding of reality may even be limited by language. MT accounts for intra-cohort similarities
and individual differences among deaf students and faculty. It subsumes translanguaging theory and supersedes the focus on language that has long defined the horizons of
deaf research.
MTs support a wide range of discursive, metacognitive, and developmental processes
that are educationally useful for deaf students and deaf faculty. In my study (Skyer 2021),
Louis had no name for MT, but he used it consistently. He demonstrated MT to me by twisting his key lanyard. Alongside his ASL narrative, the nylon strap was transfigured into an
elegant visualization of DNA’s double-helix structure. Howard explicated elements of ASL
morphology using MT, meanwhile, his students constructed notations of ASL signs using
visual templates, where MT drove the representational logic. Sarah Jo stated, “I cannot
conceive of [deaf pedagogy] without signing, visual tools, and visual aids. [Deaf] students
rely on visual [and] multimodal means . . . to learn in an interactive setting.” (MEMBER
CHECKING, p. 26). Edward also encouraged his students to construct visual tools and
videos to deeply encode their learning using MT at each stage of change. Astoria extoled,
“multiple ways of teaching [that are] visual, experiential, multimodal” (OBSERVATION
MEMO, p. 1). Tessa Rose encouraged her students to playfully explore MT. Under her
tutelage, her students reconstructed texts as three-dimensional sculptures using Joseph
Cornell’s assemblage method.
These examples show that MT is creative and expressive and social and situated. MT
is an artform of deaf education. MT is also based in empirical science. MT’s characteristics
are at once recognizable and ineffable. MT is metaphoric (Lakoff and Johnson 1980; Sfard
1998). Furthermore, like metaphors, MT supports the simultaneous poetic and pragmatic
transference of knowledge between radically dissimilar forms. MT transcends language.
MT is the purposeful augmentation of form to preserve meaning and build increasingly
realistic simulacra and experiences. MT is deaf educational synesthesia3 .
Languages 2023, 8, 127
37 of 40
A final note is warranted—in multimodal transduction, deaf-centric discourses are
valued for their own sake, for their inherent beauty and functionality; for the pleasure
they afford the agents who create them. MT is not a base steppingstone. This warrants
additional study in the context of Cherryholmes’s (1999) assertion that pragmatic educators
must “bring about beautiful results in the midst of power and oppression and ignorance”
(p. 5). MT is affected by axiological properties; its ethics and aesthetics suggest a plausible
resolution to the teleological tension between adaptation and struggle in deaf education.
I will close by asking precisely the same question I began with: Multimodal transduction. What is it good for?
Absolutely everything.
Funding: This research received no external funding.
Institutional Review Board Statement: The study was conducted in accordance with the Declaration
of Helsinki. It was approved by two IRBs, including the IRB of record located at the Office for Human
Subject Protection of University of Rochester (Protocol code STUDY00003233, date of approval: 20
December 2018).
Informed Consent Statement: Informed consent was obtained from all participants involved in the
study, but no signatures were collected; instead, assent was recorded using video procedures during
data collection to minimize the possibility of data breach.
Data Availability Statement: Data are identifiable and are, therefore, not publicly available. A thorough data audit was conducted and found satisfactory by the project PI and overseeing coordinators.
Secondary data are available on request. See cover for email correspondence.
Acknowledgments: I acknowledge the moral support of my dissertation committee members, and
my employer who supported my study, while I, meanwhile, taught a full load of courses.
Conflicts of Interest: The author declares no conflict of interest. There were no external funders
who had a role in the design of the study; in the collection, analyses, or interpretation of data; in the
writing of the manuscript; or in the decision to publish the results.
Notes
1
The citations with “ALL CAPS” formatting are references to the dataset. These are included for the purpose of data audits.
2
Some are simple, others complex. Many may occur bidirectionally. To use the first as an example, a print word may become an
image, but an image may also be likened to a word, and so forth.
3
Aside from the pedagogic, MT includes “exotic” processes such as: synesthesia, phronesis, ekphrasis, and aisthesis. Synesthesia
is sensory to sensory crossover, as discussed by Kress and C. Spence; where, for example, one can see sounds or taste colors.
Phronesis is the change from theoretical to practical knowledge guided by ethics, as described by Aristotle and Flyvbjerg more
recently; where, for instance, a student of education enacts a theory of social literacy in the classroom. Ekphrasis is the practice of
changing a visual artform into words, as discussed by Plato in ancient times and, more recently, by critics such as C. Greenberg;
where, for example, an educator shares a work of art and then describes its formal composition for novices. Aisthesis converts
the umwelt to text, seen in Rancière and J. Morrell; where, for example Thomas Wolfe encodes robust sensual experiences into
poetic, textual narratives. MT also includes transcription, translation, and yes, translanguaging. Among these disparate changes,
only one process is constant: meaning is deconstructed then reconstructed—in a word—transduced. MT undergirds numerous
epistemological operations and ontological functions present in and beyond deaf pedagogy. MT is motivated and purposeful; it
is a processual, flexible continuum of nonlinear interactions that are emergent and contingent.
References
Althusser, Louis. 1962. Contradiction and overdetermination. In For Marx: Notes for An Investigation. Translated by B. Brewster.
London: Penguin Press. Available online: https://www.marxists.org/reference/archive/althusser/works/formarx/althuss1
.htm (accessed on 2 August 2022).
Baker, Colin. 2011. Foundations of Bilingual Education and Bilingualism, 5th ed. Clevedon: Multilingual Matters.
Bagga-Gupta, Sangeeta. 2019. Languaging across time and space in educational contexts. Language studies and deaf studies. Deafness
and Education International 21: 65–73. [CrossRef]
Bezemer, Jeff, and Carey Jewitt. 2010. Multimodal analysis: Key issues. In Research Methods in Linguistics. Edited by L. Litosseliti.
London: Continuum, pp. 80–97.
Languages 2023, 8, 127
38 of 40
Boote, David N., and Penny Beile. 2005. Scholars before researchers: On the centrality of the dissertation literature review in research
preparation. Educational Researcher 34: 3–15. [CrossRef]
Bourdieu, Pierre. 1991. Language and Symbolic Power. Cambridge: Harvard University Press.
Cawthon, Stephanie W., and Carrie Lou Garberoglio. 2017. Introduction. In Research in Deaf Education. Edited by Stephanie W. Cawthon
and Carrie Lou Garberoglio. New York: Oxford University Press.
Charmaz, Kathy. 2014. Constructing Grounded Theory, 2nd ed. Los Angeles: SAGE.
Cherryholmes, Cleo H. 1999. Reading Pragmatism. New York: Teachers College Press.
Christensen, Kathee Mangan. 2010a. Where do we look? What do we see? A framework for ethical decision making in the education of
students who are deaf or hard of hearing. In Ethical Considerations in Educating Children Who Are Deaf or Hard of Hearing. Edited by
Kathee Mangan Christensen. Washington: Gallaudet University Press.
Christensen, Kathee Mangan, ed. 2010b. Ethical Considerations in Educating Children Who Are Deaf or Hard of Hearing. Washington:
Gallaudet University Press.
Corbin, Juliet, and Anselm Strauss. 2008. Basics of Qualitative Research: Techniques and Procedures for Grounded Theory, 3rd ed. Thousand
Oaks: SAGE.
de Alba, Alicia, E. Gonzalez-Gaudiano, Colin Lankshear, and Michael Peters. 2000. Curriculum in the Postmodern Condition. New York: Lang.
De Meulder, Maartje, Annelies Kusters, Erin Moriarty, and Joseph J. Murray. 2019. Describe, don’t prescribe. The practice and politics
of translanguaging in the context of deaf signers. Journal of Multilingual and Multicultural I 40: 892–906. [CrossRef]
Easterbrooks, Susan R. 2017. Conceptualization, I, and application of research in deaf education: From phenomenon to implementation.
In Research in Deaf Education. Edited by Stephanie Cawthon and Carrie Lou Garberoglio. New York: Oxford University Press,
pp. 1–34.
Easterbrooks, Susan R., and Melody Stoner. 2006. Using a visual tool to increase adjectives in the written language of students who are
deaf or hard of hearing. Communication Disorders Quarterly 27: 95–109. [CrossRef]
Enns, Charlotte. 2017. Making the case for case studies in deaf education research. In Research in Deaf Education. Edited by Stephanie
Cawthon and Carrie Lou Garberoglio. New York: Oxford University Press, pp. 203–24.
García, Ofelia. 2009. Bilingual Education in the 21st Century: A Global Perspective. Malden: Wiley-Blackwell Publishers.
García, Ofelia, and Angel M. Y. Lin. 2016. Translanguaging in bilingual education. In Encyclopedia of Language and Education; Bilingual
and Multilingual Education. Edited by O. Garcia. New York: SAGE.
García, Ofelia, and Debra Cole. 2014. Deaf gains in the study of bilingualism and bilingual education. In Deaf Gain: Raising the Stakes for
Human Diversity. Edited by H-Dirksen L. Bauman and Joseph J. Murray. Minneapolis: University of Minnesota Press, pp. 95–111.
García, Ofelia, and Li Wei. 2014. Translanguaging: Language, Bilingualism, and Education. London: Palgrave-MacMillian.
Gargiulo, Richard M., and Emily C. Bouck. 2021. Special Education in Contemporary Society, 7th ed. Thousand Oaks: SAGE.
Gee, James Paul. 2004. Situated Language and Learning: A Critique of Traditional Schooling. New York: Routledge.
Glickman, Neil S., and Wyatte C. Hall. 2019. Language Deprivation and Deaf Mental Health. New York: Routledge.
Graham, Patrick J. 2015. Examining the need of attention strategies for academic I in deaf and hard of hearing children. Journal of
Education and Human I 4: 16–21. [CrossRef]
Guardino, Caroline, Jennifer S. Beal, Joanna E. Cannon, Jenna Voss, and Jessica P. Bergeron. 2018. Case Studies in Deaf Education: Inquiry,
Application, and Resources. Washington, DC: Gallaudet University Press.
Gulati, Sanjay. 2019. Language deprivation syndrome. In Language Deprivation and Deaf Mental Health. Edited by Neil S. Glickman and
Wyatte C. Hall. New York: Routledge, pp. 24–46.
Hodge, Robert, and Gunther Kress. 1988. Social Semiotics. Ithaca: Cornell University Press.
Holcomb, Leala, Hannah M. Dostal, and Gloshanda Lawyer. 2021. Translanguaging Resources: An Informational Guide. Unpublished manuscript.
Holmström, Ingela, and Krister Schönström. 2018. Deaf lecturers’ translanguaging in a higher education setting. A multimodal
multilingual perspective. Applied Linguistics Review 9: 89–111. [CrossRef]
Humphries, Tom, and Francine MacDougall. 2000. Chaining’ and other links: Making connections between American Sign Language
and English in two types of school settings. Visual Anthropology Review 15: 84–94. [CrossRef]
Jaspers, Jürgen. 2018. The Transformative Limits of Translanguaging. Language & Communication 58: 1–10.
Jewitt, Carey. 2008. Multimodal discourses across the curriculum. In Encyclopedia of Language and Education, 2nd ed. Vol. 3: Discourse and
Education. Edited by M. Martin-Jones, A. M. de Mejia and N. H. Hornberger. Berlin: Springer Science+Business Media, pp. 1–11.
Knoors, Harry, and Marc Marschark. 2014. Teaching Deaf Learners. New York: Oxford University Press.
Konecki, K. T. 2011. Visual grounded theory: A methodological outline and examples from empirical work. Revija za Sociologiju 41:
131–60. [CrossRef]
Kress, Gunther R. 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. New York: Routledge.
Kress, Gunther R. 2011. Discourse analysis and education: A multimodal social semiotic approach. In Critical Discourse Analysis in
Education, 2nd ed. Edited by Rebecca Rogers. New York: Routledge.
Kress, Gunther R., and Theo Van Leeuwen. 2006. Reading Images: The Grammar of Visual Design, 2nd ed. New York: Routledge.
Kuntze, Marlon. 2016. Code-switching. In SAGE Deaf Studies Encyclopedia. Edited by G. Gertz and P. Boudreault. New York and
London: SAGE.
Languages 2023, 8, 127
39 of 40
Kuntze, Marlon, and Debbie Golos. 2021. Revisiting rethinking literacy. In Discussing Bilingualism in Deaf Children: Essays in Honor of
Robert Hoffmeister. Edited by Charlotte Enns, Jonathan Henner and Lynn McQuarrie. New York: Routledge, pp. 99–112.
Kuntze, Marlon, Debbie Golos, and Charlotte Enns. 2014. Rethinking literacy: Broadening opportunities for visual learners. Sign
Language Studies 14: 203–24. [CrossRef]
Kurz, Christopher, Debbie Golos, Lon Kuntze, Jonathan Henner, and Jessica Scott. 2021. Guidelines For Multilingual Deaf Education
Teacher Preparation Programs. Washington, DC: Gallaudet University Press.
Kusters, Annelies, Massimiliano Spotti, Ruth Swanwick, and Elina Tapio. 2017. Beyond languages, beyond modalities: Transforming
the study of semiotic repertoires. International Journal of Multilingualism 14: 219–32. [CrossRef]
Kusters, Annelies Maria Jozef, and Michele Ilana Friedner. 2015. Introduction: Deaf-same and difference in international deaf spaces
and encounters. In It’s a Small World: International Deaf Spaces and Encounters. Edited by Michele Friedner and Annelies Kusters.
Washington, DC: Gallaudet University Press, pp. ix–xxx.
Kusters, Marieke. 2017. Intergenerational responsibility in deaf pedagogies. In Innovations in Deaf Studies: The Role of Deaf Scholars.
Edited by Annelies Kusters, Maartje De Meulder and Dai O’Brien. New York: Oxford University Press.
Lakoff, George, and Mark Johnson. 1980. Metaphors We Live By. Chicago and London: The University of Chicago Press.
Leigh, Irene W., and Jean F. Andrews. 2017. Deaf People and Society: Psychological, Sociological, and Educational Perspectives, 2nd ed. New
York: Routledge.
Lewis, Gwyn, Bryn Jones, and Colin Baker. 2012. Translanguaging: Origins and I form school to street and beyond. Educational Research
and Evaluation 18: 1–14. [CrossRef]
Luckner, John L. 2018. Foreword. In Case Studies in Deaf Education: Inquiry, Application, and Resources. Edited by Caroline Guardino,
Jennifer S. Beal, Joanna E. Cannon, Jenna Voss and Jessica P. Bergeron. Washington, DC: Gallaudet University Press.
Lupi, Giorgia. 2016. From language to representation. In Mind, Maps, and Infographics. New York: Moleskine.
Marschark, Marc, Allan Paivio, Linda J Spencer, Andreana Durkin, Georgianna Borgna, Carol Convertino, and Elizabeth Machmer.
2017. Don’t assume deaf students are visual learners. Journal of Developmental and Physical Disabilities 29: 153–71. [CrossRef]
[PubMed]
Marshall, Catherine, and Gretchen B. Rossman. 2016. Designing Qualitative Research, 6th ed. New York: SAGE.
Mertens, Donna M. 2020. Research and Evaluation in Education and Psychology, 5th ed. Thousand Oaks: SAGE.
Miles, Matthew B., A. Michael Huberman, and Johnny Saldana. 2020. Qualitative Data Analysis, 4th ed. New York: SAGE.
Moges, Rezenet Tsegay. 2020. From White Deaf people’s adversity to Black Deaf Gain: A proposal for a new lens of Black Deaf
educational history. Journal Committed to Social Change on Race and Ethnicity 6: 69–99.
Moges-Riedel, R., S. Garcia, O. Robinson, S. Siddiqi, A. Lim Franck, K. Aaron-Lozano, S. Pineda, and A. Hayes. 2020. Open Letter in
ASL about Intersectionality. [Video]. YouTube. Available online: https://youtu.be/JcTYXpofAI0 (accessed on 2 August 2022).
Ortega, Lourdes. 2009. Understanding Second Language Acquisition. London: Hodder & Stoughton.
Otheguy, Ricardo, Ofelia García, and Wallis Reid. 2015. Clarifying Translanguaging and Deconstructing Named Languages: A Perspective from Linguistics. Applied Linguistics Review 6: 281–307. [CrossRef]
Pagliaro, Claudia M., and Christopher Kurz. 2021. Using ASL to navigate the semantic circuit in the bilingual math classroom. In
Discussing Bilingualism in Deaf Children: Essays in Honor of Robert Hoffmeister. Edited by Charlotte Enns, Jonathan Henner and
Lynn McQuarrie. New York: Routledge.
Perniss, Pamela. 2015. Collecting and analyzing sign language data: Video requirements and use of annotation software. In Research
Methods in Sign Language Studies. Edited by Eleni Orfanidou, Bencie Woll and Gary Morgan. West Sussex: Wiley Blackwell.
Phillips, Donna Kalmbach, and Kevin Carr. 2014. Becoming a Teacher through Action Research: Process, Context, and Self-Study, 3rd ed.
New York: Routledge.
Pollard, Robert Q., and Meghan L. Fox. 2019. Forensic evaluation of deaf adults with language deprivation. In Language Deprivation
and Deaf Mental Health. Edited by Neil S. Glickman and Wyatte C. Hall. New York: Routledge, pp. 101–35.
Raike, Antti, Suvi Pylvänen, and Päivi Rainò. 2014. Co-design from divergent thinking. In Deaf Gain: Raising the stakes for human
diversity. Edited by H-Dirksen L. Bauman and Joseph J. Murray. Minneapolis: University of Minnesota Press, pp. 402–20.
Rancière, Jacques. 2010. Dissensus: On Politics and Aesthetics. London: Bloomsbury Academic.
Rée, Jonathan. 1999. I See a Voice: Deafness, Language, and the Senses—A Philosophical History. New York: Metropolitan Books.
Rose, Gillian. 2012. Visual Methodologies. London: SAGE Publications.
Sacks, Oliver. 1990. Seeing Voices: A Journey into the World of the Deaf. New York: Vintage.
Saldaña, Johnny. 2012. The Coding Manual for Qualitative Researchers. London: SAGE Publications.
Schiff, Debra. 2010. Information Behaviors of Deaf Artists. Art Documentation: Journal of the Art Libraries Society of North America 29:
44–47. Available online: http://www.jstor.org/stable/27949552 (accessed on 2 August 2022).
Sfard, Anna. 1998. On two metaphors for learning and the dangers of choosing just one. Educational Researcher 27: 4–13. [CrossRef]
Simms, Laurene, Sharon Baker, and M. Diane Clark. 2013. The Standardized Visual Communication and Sign Language Checklist for
Signing Children. Sign Language Studies 14: 101–24. [CrossRef]
Skyer, Michael E. 2020. Invited article: The bright triad and five propositions: Toward a Vygotskian framework for deaf pedagogy and
research. American Annals of the Deaf 164: 577–91. [CrossRef]
Skyer, Michael E. 2021. Pupil ⇄ Pedagogue: Grounded Theories about Biosocial Interactions and Axiology for Deaf Educators.
Doctoral Dissertation, University of Rochester, Rochester, NY, USA.
Languages 2023, 8, 127
40 of 40
Skyer, Michael E. 2022. Power in Deaf Pedagogy and Curriculum Design Multimodality in the Digital Environments of Deaf Education
(DE2 ). Przeglad
˛ Kulturoznawczy (Arts and Cultural Studies Review) 3: 345–81. [CrossRef]
Skyer, Michael E. 2023. The deaf biosocial condition: Metaparadigmatic lessons from and beyond Vygotsky’s deaf pedagogy research.
American Annals of the Deaf, in press.
Skyer, Michael E., and Laura Cochell. 2020. Aesthetics, Culture, Power: Critical Deaf Pedagogy and ASL Video-Publications as
Resistance-to-Audism in Deaf Education and Research. Critical Education 11: 1–25.
Smith, Melissa Beth. 2010. Opening our eyes: The complexity of competing visual demands in interpreted classrooms. In Ethical
Considerations in Educating Children Who Are Deaf or Hard of Hearing. Edited by Kathee Mangan Christensen. Washington, DC:
Gallaudet University Press.
Snoddon, Kristin. 2017. Uncovering translingual practices in teaching parents classical ASL varieties. International Journal of
Multilingualism 14: 303–16. [CrossRef]
Stake, Robert E. 1995. The Art of Case Study Research. London: SAGE publications.
Stake, Robert E. 2005. Qualitative case studies. In The Sage Handbook of Qualitative Research, 3rd ed. Edited by Norman K. Denzin and
Yvonna S. Lincoln. Thousand Oaks: SAGE, pp. 443–66.
Swanwick, Ruth. 2017a. Languages and Languaging in Deaf Education: A Framework for Pedagogy. Oxford: Oxford University Press.
Swanwick, Ruth. 2017b. Translanguaging, learning and teaching in deaf education. International Journal of Multilingualism 14: 233–49.
[CrossRef]
Swanwick, Ruth, and Marc Marschark. 2010. Enhancing education for deaf children: Research into practice and back again. Deafness
and Education International 12: 217–35. [CrossRef]
Swanwick, Ruth, Samantha Goodchild, and Elisabetta Adami. 2022. Problematizing translanguaging as an inclusive pedagogical
strategy in deaf education. International Journal of Bilingual Education and Bilingualism. [CrossRef]
Tapio, Elina. 2013. A Nexus Analysis of English in the Everyday life of FinSL Signers: A multimodal View on Interaction. Doctoral
Dissertation, University, Jyväskylä, Jyväskylä, Finland.
Thoutenhoofd, Ernst D. 2010. Acting with attainment technologies in deaf education: Reinventing monitoring as an intervention
collaboratory. Sign Language Studies 10: 214–30. [CrossRef]
Timmermans, Stefan, and Iddo Tavory. 2012. Theory construction in qualitative research: From grounded theory to abductive analysis.
Sociological Theory 30: 167–86. [CrossRef]
Vygotsky, Lev Semyonovich. 1993. The Collected Works of L. S. Vygotsky: The Fundamentals of Defectology (Abnormal Psychology and
Learning Disabilities). New York: Plenium Press, vol. 2.
Wei, Li. 2018. Translanguaging as a practical theory of language. Applied Linguistics 39: 9–30. [CrossRef]
Wei, Li. 2022. Translanguaging as method. Research Methods in Applied Linguistics 1: 100026. [CrossRef]
Wei, Li, and Ofelia García. 2022. Not a first language but one repertoire. Translanguaging as a decolonizing project. RELC Journal 52:
313–24. [CrossRef]
Young, Alys, and Bogusia Temple. 2014. Approaches to Social Research: The Case of Deaf Studies. Oxford: Oxford University Press.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.