Text in the world of synergized codes

2020 ISSN 1509-5304 PRACE JĘZYKOZNAWCZE DOI 10.31648/pj.5826 XXII/4 219–236 Jacek Warchala Uniwersytet Śląski w Katowicach ORCID: https://orcid.org/0000-0002-9160-9270 e-mail: jacek.warchala@us.edu.pl Text in the world of synergized codes Tekst w świecie synergii kodów Abstrakt Artykuł porusza problem granic badań filologicznych w związku z ewolucją filologii w kierunku antropologii języka oraz przekraczania granic tradycyjnie pojmowanego tekstu w epoce multimedialności. Głównym przedmiotem refleksji jest tu komunikat reklamowy jako swoisty tekst wielokodowy, multimodalny, w którym sens powstaje w wyniku interferencji poszczególnych systemów semiotycznych (kodów): werbalnego, ikonicznego (wizualnego), muzycznego czy, w szczególnych przypadkach, kodu zaaranżowanej przestrzeni. Artykuł rozważa problem wzajemnej zależności kodu werbalnego i ikonicznego (słowa i obrazu) w obrębie pojedynczego tekstu reklamowego. Wskazuje na trzy rodzaje zależności: metaforyczną, metonimiczną i tautologiczną. Słowa kluczowe: relacja słowo-obraz, reklama, tekst multikodowy/multimodalny, wizualizacja, kultura wizualna, perswazja Abstract This study deals with the problem of limits of philological research in the context of the evolution of philology towards anthropology of language and transgression (in the multimedia age) of limits of a traditionally understood text. Its main subject is advertising communication conceived as multicodal and multimodal text in which meaning results from the interference of particular semiotic systems (codes): verbal, iconic (visual), musical or, in some instances, the code of arranged space. The study considers the issue of mutual dependence of the verbal and iconic codes (or word and image) within a single advertising text. Three types of dependence are pointed out: metaphorical, metonymical and tautological. Keywords: word-iconic relations, advertisement, multicodal and multimodal text, visualization, visual culture, persuasion The notion of text – introductory remarks. In linguistic and anthropological studies alike, the term has veered so far away from its original meaning that its users now need to clarify what they actually 220 Jacek Warchala mean by it. “Text” has come to denote not only the written word but also all cultural phenomena spreading beyond the realm of art and literature. Consequently, a painting by Picasso is a text as well as a sonnet, a billboard, a shop window, or an architectural detail, as long as they are all conceived in semiotic terms as sign-based messages or statements (Gołębiewska 2003: 235). In this sense, what can be “read” as a text includes not only narrowly defined works of art and non-literary texts, like documents, but also certain social phenomena such as outfits, house décor, store design, or a furnished supermarket space. That is because they were all created upon the intent of textuality: (a) they have the interpretable property of being coherent and cohesive, which allows us to grasp the supermarket space as consciously designed and intelligible, or to understand the purport of a colour combination of someone’s outfit as a telling composition conveying (in a conscious or unconscious way) a message, for instance: “I’m dressed in black because: I’m in mourning / I’m a neofascist / I’m a member of the ‘goth’ subculture”; (b) they have the property of being intertextual (they are read in the context of other texts/artefacts); (c) they have the property of being intentional (they reveal the sender, the creator, the architect, the decorator); and finally (d) they reveal a reference to a definite, known, and knowable reality (Geertz 2000). In this broad sense, a text is an eloquent manifestation of transgressing the limits of the philological category of text, and an instance of crossing or blurring the border between genres and disciplines1 such as philology and sociology or architecture (Czapliński 2010: 34−35); their interpenetration and peculiar affinity is based on the use of some helpful categories they have in common, such as wholeness/coherence2. The idea of wholeness/coherence. This is a principle shared by verbal and visual works of art – from the traditional realist and figurative art to the most radical avant-garde feats. The principle of “filling” the blanks and vague areas left in a message having the high communicational entropy coefficient pushes us to interpret the verbal or visual artefact in question in accordance with our sense of coherence, some presumed symbolic convention, our general knowledge, our experience or a certain rule organizing our comprehension of the world. One example of such a principle which allows us to understand the world as a “logical” construct 1 Geertz uses the notion of blurred genres, which he explores in a separate essay (Geertz 2000: 17). 2 I resort to the notion of wholeness, drawn from art theory, as an equivalent of the philological notion of cohesion/coherence; see: (Gombrich 2009). Text in the world of synergized codes 221 is the principle of order defined by Ernst Gombrich (Gombrich 2009: 107−108). Gombrich has even contended that we have an innate sense of order which makes us fill in the gaps and aporias, connect the spots, complete the outlines, and see a whole in a draft or a model. This resembles the process of completing or harmonizing the lines of a caricature, where if you know the relevant elements you can fill it with redundant details, like in the famous caricature of Alfred Hitchcock: A text, regardless of its form, never loses its constitutive properties such as the aforementioned and evident category of coherence and cohesion, but also those of intentionality, informativity and acceptability. Therein lies the question: how do the multicodal or multimodal products functioning as multimedia preserve the fundamental structural property of text, that is cohesion/coherence? The traditional understanding of text3 refers chiefly to its written/printed form, but not its electronic or spoken/dialogical form. In this traditional understanding, for instance, the text is always examined as a complete whole, rather than a process continuing and developing over time like a conversation; it is never assumed to be a text-formative process 3 This understanding has been established in the Polish tradition of textological research by the school of M. R. Mayen, who upheld the sign-based concept of a text as a completed, coherent whole of information, produced through a text-formative action of the sender. A broader understanding, although still grounded in this framework, was offered by two researchers from Lublin: Jerzy Bartmiński and Stanisława Niebrzegowska-Bartmińska, who defined their Tekstologia [Textology] as anthropology of the word; they frame the text broadly, considering the theory of speech acts, new conditions for virtual text on the Internet, and introducing the notions of hypertext and intertextuality. However, apart from a few cursory remarks, they fail to consider concepts such as visuality or, in a broader perspective, visual anthropology or the anthropology of the image, even though they do mention the so-called visual turn; when discussing advertising, they consider it as text but without elaborating on the functioning of image within its framework. That is why I place their concept within the limits of the linguistic concept of text as verbal entity (Bartmiński, Niebrzegowska-Bratmińska 2009). 222 Jacek Warchala revealed in the course of an interaction, but regarded as the “final product”. Furthermore, this understanding fails to consider the opposition between a text that exists online (process) and offline (mind representation ); nor does it assume that a text could be precisely the product of a joint effort the “wholeness” of which is rather vague – as is the case with any modern Internet hypertext (Wikipedia), the so-called web fiction, a spoken dialogue, or an advertisement. This latter, in my view, is the most representative example of a multicodal/multimodal text nowadays, since any interpretation of its meaning may be undertaken only if we transgress traditional limits in the understanding of the written text, and shift towards the idea of co-existence of word and image, that is the visual, emotional, subjective character of reception and its social limitations; in other words, if we stop equating the meaning of a given text with a set of literal meanings arising from a collection of words (Warchala 2018). Additionally, in this framework, a text is not only – perforce – a product of the sender, but also a product constructed in reference to the receiver. It exists only insofar as the receiver comprehends it as a text, that is recognizes in it a certain ground of cohesion/coherence. Furthermore, the notion of text includes not only verbal constructs, traditionally put in the written form (written down, recorded, inscribed), but also the dialogical oral texts and iconic messages – pictures, photographs, films, conventional writings for various occasions, signboards, factories, museums, churches and libraries, supermarkets and parks, since they have the structure of a text, as long as the receiver recognizes them as such (Warchala 2018). We will deal with multimedia texts (involving the use of various media in the process of communication) and multimodal texts (involving the use of various semiotic systems/modes/forms within a text) exploiting many genetically different codes with incompatible morphologies, such as word and image, music and word, but nevertheless creating a coherent, meaningful message. This message is based on a bundle of data which, though not always mutually translatable or referential, have a synergetic effect, as they together amplify (build up) the common sense of the message, and, which, above all, can be received and subjected to interpretation. The sense is conveyed not only by conventional signs but also elements of reality which produce their meanings in the process of semiosis; the visible world conveys a text (or texts) which we – the receivers – can interpret; and conversely, the text contains a world that can be reconstructed as an idea or message (Olivier 2010: 83). A text is an informational structure which only opens the field of syncretic meanings by synergizing fragmentary meanings introduced into the text with the use Text in the world of synergized codes 223 of multiple codes and sign systems. In this sense, the text loses its status as a formally uniform entity, a written work. It becomes something potential, for which the ultimate decisive instance is the receiver who combines the multicodal, multimodal, heteronomous bundles of data into a whole, that is a meaningful and coherent message. Multicodality – the textual state of matter. Multicodality should be understood as the capacity – exhibited either by the entire text or its components (colour, light, shadows), the musical or verbal stratum – to function not only as elements in a certain sense external to the verbal sphere but as structural elements of a multimodal text, assuming there is a principle which organizes the absorption of heteronomous modes. My take on this issue follows contemporary research on the visual culture by Mieke Bal or Georges Didi-Huberman, for whom the visual is neither a simple translation of images into words, i.e. the ability to describe an image with words or illustrate words with an image (e.g. in an interpretation of a literary scene), nor just an ability to provide a faithful description of an image, as in the case of ekphrasis, but a phenomenon of structural synaesthesia. This broader understanding of the visual involves the synergetic nature of meanings and the synesthetic nature of perception as structural principles of text formation (Skudrzyk: 2005; Dziadek: 2004; Skwara: 2007)4, particularly in the case of multimodal messages. Let us illustrate this phenomenon by the example of sunlight at the Ronchamp Cathedral – it never stops changing on the inside of the temple, and continuously affects the hilltop building from the outside; the light is a structural element of the cathedral, as intended by its creator Le Corbusier who used light as a modus. The sun penetrating the tiny windows sets the rhythm of the constantly changing interior; the light is a modus which dictates the dynamics of change. The resulting tension between light and darkness creates a theological sense which, albeit perceived differently by each individual, directs us towards an image of a relationship between God (personalized in the light shining through) and Man, between the light of the eternal God and the darkness symbolizing evanescence of Man. These constant changes, visualize according 4 Synaesthesia in the spoken and written text was discussed in more detail by Aldona Skudrzyk (Skudrzyk 2005); an interesting study on ekphrasis within the framework of literary science was undertaken by Adam Dziadek (Dziadek 2004) and Marek Skwara (Skwara 2007); in addition, let us note that ekphrasis itself has evolved from its ancient forms and become more and more a translation of the content read by the receiver, an image of the receiver’s interpretation. 224 Jacek Warchala to the artist’s design the sense of religious life, its dynamics and vicissitudes on the road from the profane to the sacred. As a result, the interior of the cathedral becomes a text read in the spirit of Catholic theology. The ontology of a multimodal text. The expansion of the term “text” took place outside traditional philology, when cultural anthropologists and semioticists have come to treat as texts such non-verbal entities as cities, gardens, or an architectural form, which may be perceived as messages having a discernible property of cohesion/coherence – the fundamental prerequisite for all products exhibiting the structural properties of being a text. The products thus far considered non-semantic, such as musical works, architectural sites, and the entire urban complexes with gardens and streets, undergo semiotization. We can “read” these artefacts in the same manner as we “read” the forms of Gothic architecture or a cathedral floorplan which serves the purposes of rhetoric and persuasion. The perception of a city as a polylogical game of codes and a synchronic co-existence of multiple overlapping and colliding semiotic systems (Łotman 2008: 297−298) allows it to be framed as an urban palimpsest, discovered and read layer by layer by its residents or tourists equipped with walking guides (Szalewska 2012: 130)5. The metaphor of the urban palimpsest epitomizes the potential understanding of a city in the semiotic categories of a hidden, fragmentary text which has yet to be revealed, bared of its outer layer and penetrated down to the heart of the form and its hidden meaning. In other words, we need to make sense of what we discovered, read the text anew and perform its interpretation, considering the knowledge and the sensitivity of the receiver and the hints left by the walking guide’s author who weaves some sort of a city narrative. Additionally, this means that the city as text resembles a dialogue constituted by two equal subjects: the sender (the builder, the architect, the planner, or finally, the guide) and the receiver; that the receiver creates senses in the same way as the sender creates signs. The receiving subject /tourist/resident should recognize this semiotic character and construct a coherent, meaningful text imbued with the property of acceptability, that is, openness to interpretative activity, as defined by de Beaugrande and Dressler in their already classical concept of text (de Beaugrande, Dressler 1999). Thus, there are common interpretative categories which allow a description and analysis of both literary texts and the (urban) artefacts discussed above. 5 Krystyna Szalewska notes: “The palimpsest formula, which allows to link time and space – the great semantic figures of urban narratives – already tends etymologically towards text as a source” (Szalewska 2012: 130). Text in the world of synergized codes 225 Significant changes in the understanding of text are also due to some developments in music theory. As early as at the beginning of the 20th century, Albert Schweitzer (in 1905) and Arnold Schering (in 1908) were the first theoreticians to observe the impact of rhetoric on Baroque music; according to Schering theories on rhetorical figures were used and analysed in the Baroque study of composition. These discoveries marked the beginning of the research in “musical rhetoric”, and in 1941 a catalogue of rhetorical figures used in music was even compiled by Hans-Heinrich Unger (Unger 1941; Korpanty 2008)6. This approach considered musical works as interrelations of two modes: speech and word, though instead of regarding the verbal and the musical layer as parallel, it explored the unique process of absorbing rhetorical devices typical for verbal communication into a musical structure, of speech being “composed into” the musical layer of the work. This resembled the understanding of a metaphor as “composition” of the image into the verbal layer, as subordination of the image to the combinations of words, and escape into image when words prove insufficient: the Polish poet Bolesław Leśmian called it “[capturing] the elusive in two adjoining words” (Poet). Incidentally, it is worth noting that Schweitzer viewed Bach’s music as figurative art, emphasizing its plastic and picturesque character (two other semiotic systems/modes incorporated into the musical layer of the work by way of synesthesia). An advertising campaign as text. One example of a multicodal/ multi- or at least bimodal, and multimedia text is provided by an advertising campaign regarded as a hypertext with a syncretically understood sense produced by at least two semiotic systems: image and word. The element of coherence/cohesion and the determinant of the whole is the claim7, common to all media productions, and the semantically related image which visualizes, for example, the idea behind the product, the identity 6 Katarzyna Korpanty writes: “In composition textbooks, German theoreticians invoked rhetorical devices to point to the similarities in the process of composing a musical work and producing speech. The composition process was viewed through the lens of rhetoric, and five disciplines of linguistic rhetoric were used in reference to music. Inventio was the general vision of a musical work, involving the selection of text, modus, tempo, and casting. Dispositio was used to determine the concept of the musical form (the selection of tact, technique). Elocutio concerned the selection of musico-rhetorical figures. The last two disciplines, memoria and pronuntiatio were of secondary importance and concerned the execution” (Korpanty 2008: 39). 7 The claim is a short verbal message accompanying a brand, supposed to build company and brand images in the long run ; often included in the logotype it resembles a slogan, but refers to the entire brand rather than individual promotional campaign. 226 Jacek Warchala of the sender, the promise to the target group (the collective receiver of the advertising message) contained in the slogan and the main message of the campaign. In general, every campaign involves the use of many media: billboards, television and radio channels, newspapers and magazines, graphics on company cars, etc. The sense of any single text production, such as a billboard or a television spot – the hypertext of the campaign – is constituted as a compound of at least two elements: visual (e.g. a product photo) and verbal (e.g. a text), with the verbal channel also exploiting the effect of visualization (e.g. a logo usually visualizes the sender’s identity) by the use of meticulously selected typography. In addition, an advertisement is an example of a persuasive text whose primary conative function organizes (coherence at the level of language functions) all others – which is why we can speak of the secondary persuasiveness of the aesthetic sphere, shared references and uniform image of the sender (the expressive function). The pragmatics of an advertisement boils down to effectively communicating the largest possible number of components or modes and optimized media channels to the predetermined target receiver. Thus, assuming that to understand an advertising message as a global/syncretic text, we need to make sense of it by considering at least two modes: image and word (as mentioned above), the relationship between the visual and verbal layer becomes an interesting issue in terms of text ontology. How to understand the process of visualization? It seems interesting to understand visuality in a way that explains how an image can contribute to text formation as an element incorporated in the structure of a multicodal text, rather than as a something external in a sense like an illustration. To even begin to understand the advertising text such as a billboard, which at first glance may look like an illogical or chaotic jumble of elements, we need to recall a series of connotations which provide its semantical or sense-formative background and constitute a prerequisite for its coherence. There is a process here which I would like to describe as an atrophy of a verbal text, which does not vanish or lose its importance but rather “allows visuality into its structure”. This mechanism by means of which the image does not oust or supplant the word but becomes absorbed, and the logocentric perception gives way to a visual one, is, in my opinion, the central problem of visual culture (Warchala 2014). Let us remember that for Mieke Bal, now a leading researcher in visuality, looking is an act of reading (Bal 2006; Didi-Huberman 2011). It is worth explaining that the notion of visualization as a process of synesthetic perception may be understood in two ways. First, as a co- Text in the world of synergized codes 227 existence of image and word, with the image regarded as a transference of the object in the form of its visual representation; second, as a revelation of the image through tropical processes such as metaphorization, understood as mental “seeing” of images triggered by words. If the image of an object (product) is to serve the persuasive function, which is the primary function of an advertising text (such as billboard), it visualizes the will/necessity to possess a given object/product by the potential client. A frosted bottle of a refreshing drink refers us to the product (let us call it “Reference 1”) but more importantly, visualizes the physical sense of thirst (“Reference 2”). Through visualization, what hides inside is experienced as overt and open to visual, auditory, tactile, or gustatory perception. In cognitive terms, we can speak of the conceptualization of an abstract fantasy by making it visible or, to borrow a handy term from Leonard Talmy, “palpable” (Talmy 2000; Tabakowska 2011). Thus, visualization is an interpretative proposition, a suggestion for the receiver on how to make sense of the meaning of the elements making up the image – such as composition, colour, texture, or context – or, in other words, how to actualize the individual meanings into a whole/coherent sense of the message. This is probably also how we should understand Gillian Rose’s words that visuality “refers to the way in which vision is constructed in various ways: ‘how we see, how we are able, allowed, or made to see, and how we see this seeing and the unseeing therein’” (Rose 2001: 6). This brings us to the problem of the difference between looking and seeing. Seeing is an act that brings order to the chaos of the outside world; looking perpetuates the chaos, while seeing tames it8 when we notice in its midst the elements of order for the receiver; by looking, we select the elements of the world that will serve to create a story which, by the force of its inescapable narrative scheme, requires choosing and ordering events across the timeline. Visualization is based on the assumption that it is the receiver who makes sense of the image, while the sender only gives her a reason to do so. We will avoid the overly obvious constructivist approach by using the handy term “visual field” created by the tension between three entities: 8 Interviewed by Łukasz Głombicki for “Gazeta Wyborcza”, Tomasz Tomaszewski, a photographer and lecturer in photography, describes it as follows: “The same goes for photography. There are around a dozen photographers who know how to handle this magic wand, who have attained the nirvana and unraveled the mystery shrouding the difference between looking and seeing. They don’t uncover the world before our eyes, but miraculously uncover the relationships between things present in this world. They put the surrounding chaos to order, giving us consolation” (Głombicki 2013). 228 Jacek Warchala the image – which, in W. J. T. Mitchell’s formulation, “wants” something from us, implying that it is an element dynamically forming our attitude towards ourselves – the receiver/enthusiast/expert/client, with their sociocultural background and the conventions of seeing informed by different texts, images, and practices of reception and, finally, the reality surrounding both the object and its receiver. In the modern visual culture, seeing is a cultural activity which generates the power and the strategy of images, that is, their power in the culture of visuality, our current practices of “looking” and our cultural context. Michael Fred quoted by W. J. T. Mitchell observed the power of visual representations in their capacity to catch t viewers’ attention, stop them in their tracks and immobilize them in wonder (Fred 1980; Mitchell 2013: 72). In this context, Mitchell writes about a transposition of a desire revealed by both the representations and the receivers themselves. The impact of an advertising poster relies on its capacity to catch the viewers’ attention and force them to put the elements of the represented world in order, to find the right sense through interpreting and revealing/reading the viewers’ wishes. It is the classic AIDA model used by ad makers: get the receivers to pay ATTENTION to your message; then stop them and catch their INTEREST by asking why the object is represented in this way and not another, create a DESIRE, and get them to take ACTION. The principle of visualization in an advertising text. In advertising, visuality manifests itself as the visualization of the “idea behind the project” – it is revealed by denotation, i.e. by referring the image used on a medium such as billboard to the reality familiar to the viewer, and thus inspiring possible connotations related to the marketed product. The advertisement transfers meaning from one sign (or a series of signs acting as attributes) to another, that is from one image playing the role of a denotative reference to another, created or conjured up as a connotative reference (Barthes 1985). This seems to be the fundamental operating principle of any persuasive text, modern or otherwise, such as an advertisement: it refers to the principle of analogy between the image provided and the image suggested, for example between the image presented on a billboard and the conjured image that overlays the marketed product. It seems that we are dealing with the basic principle of understanding and perceiving the relationship between the denotation of the properties of a model and the connotation of the properties of a product. Let’s take as an example denotative properties of the image used in the advertising images of the mineral water called Mother and me: A young woman + a Text in the world of synergized codes 229 new-born + the pose of a woman holding a child in a characteristic manner + bare shoulders + facial expression (e.g. a loving smile) – they all refer to the well-known empirical topos of motherhood as well as the motif of Virgin Mary with a Child derived from Christian iconography; connotative properties of the product (e.g. water) are the analogon and, at the same time, the visualized notion of motherhood: care, health, vital energy, youth, attentiveness, nutritious value (of the food), life, giving life. Yet, there is also a fundamental problem related to “thinking through analogy” or allegorical thinking when the object means something else in its denotative aspect and in its connotative sense. The basic way to frame meanings in allegorical thinking may be personification as a personifying transference (Smith 1962−1963: 122−123) – water as a life-giving mother. We can also follow Aristotle in regarding personification as a legitimation of the image, a confirmation of the sense of both the expression and the claim of veracity made by the author. Metaphor, metonymy, and tautology – three types of relationship between word and image. Research in multicodality and its special case, the word–image relationship, seeks a principle to explain the co-existence of those two forms of modality. It always seems appealing to consider the word and the image in the perspective of a grammatical relationship, with “the grammar of the image” considered metaphorically but also literally, when the word–image relationship is considered either formally, as a paratactic relationship of co-ordination and a hypotactic relationship of subordination (or superordination), or semantically as a relationship leading to the elaboration – development or emphasising – of one of its elements; this proposition was offered by such scholars as Radan Martinec and Andrew Salway (Martinec, Salway 2005; Maćkiewicz 2017: 39). However, the co-existence of word and image in a text is not an evident case either of superordination or subordination; together with all other potential semiotic systems (codes), image and word create a twisted wire where information is synergized, with some ingredients of the image, like colour, being untranslatable into language and its grammar of mutual dependencies, despite the possibility to use metaphorical expressions. The relationship between the verbal and the visual side of an advertising message was discussed by Hartmut Stöckl, who draws upon the concept of transcriptivity of two semiotic systems (modes) to describe 230 Jacek Warchala a variety of relationships and accentuate their complex and virtually unlimited combinations. Stöckl observed that, although the advertising text and social practices in advertising communication are clearly defined, the “mechanisms” of integration between language and image in advertising exhibit an infinite wealth of possibilities on both the graphical and rhetorical plane (Stöckl 2015). This observation may be regarded as an expression of doubt on the part of the author in regards to the prospect of creating any kind of relational grammar. Difficulties in this context include different perception of those two modalities. The image is usually perceived through the vector of analogy to real-life objects; it involves the search for reference and the use of our general knowledge. Reading images, despite the opinion of M. Bal, does not fully mimic the linear reading of a linguistic text: for instance, Stöckl hints at the fact that picturing does not involve such elements of language as modality (understood in linguistics as the speaker’s attitude to his/her own communication), logical implication or negation (Stöckl 2015: 117–118). Obviously, if we assume that perception is related to the processes of thinking and concept formation, logical operations such as argumentation may apply also to reading and understanding the graphical aspect of the message, even though it is not, strictly speaking, an operation of the propositional calculus and has more to do with general knowledge or metaphorical transference. According to Anthony Blaire (Blaire 2008), visual argumentation is an operation genuinely performed by the viewer in the process of perceiving an image (a photo or icon); it differs from the verbal argumentation by its evocative power and the capacity to involve the receiver in the reasoning (and, for instance, persuading) through direct participation. Even if visual argumentation cannot be explicitly translated into the process of linear formulation of opinions in a natural language, we can say that the image has the potential to enter into argumentative relations; that is because images are enthymemes with gaps that may be filled by a competent participant in the process of perception and communication – not by way of linearity, but directness and simultaneity (Warchala 2019: 137−139). The image–word relationship relies on the interpretative character of the signs making up a given code. For an image to have a meaning, it must be expressible in words, and one code must be internally translated into another. This structural translation usually relies on our efficiency in naming things and notions, our familiarity with pictorial or simply visual conventions, and finally, our cultural competence. But some capacity for expression is assumed; what we cannot speak about, we can only pass Text in the world of synergized codes 231 over in silence, which brings us to the unsaid9. Thus, one should be able ask a question about an iconic sign – and if one can ask, one can also answer. For all translation and interpretation involves the implicit question about meaning; it treats visual signs and artefacts in visual arts semiotically, as texts or interrelated bundles of data exhibiting the constitutional principle of cohesion and coherence. With this approach, we can put the image into words, but also transpose one medium to another. The question of mutual dependence between the verbal and the visual code appears again (Bagiński, Francuz 2007: 35). So what could be the relationship between word and image in a multimodal text? One answer to this question may be provided by the cognitive approach, where the semantic and text-formative function is performed by three verbo-iconic relationships: metaphor, metonymy and tautology. Metaphor and metonymy determine the relationship between word and image in terms of various forms of dependence between the interpreted sense of the image and the sense of the word. The metaphor imposes the relationship of interdependence, or mutual dependence of word and image: the image visualizes an element of the word’s semantic field, while the word narrows down the sense of the image and justifies the coherence of the elements included in the entire composition. What we have here is a metaphorical relationship of mutual replacement. Such interdependence may be found in the advertisement of Volvo cars. The connection between the picture of an arrow and the word: VOLVO 850 – the verbal representative of the physical product – is metaphorical; there is no real or physically perceptible connection between the word-sign VOLVO and the arrow, for there is no real connection between the man / animal / car / plane and the arrow. However, there is a metaphorical connection 9 This is an obviousreference to the famous statement of Ludwig Wittgenstein: “Whereof one cannot speak, thereof one must be silent” (Wittgenstein 1997: 83). 232 Jacek Warchala perpetuated in the traditional saying “swift as an arrow”, which allows our imagination to link the car with the arrow by creating a unifying reasoning scheme: the arrow is swift – the presented car has the capacity to gain swiftness – a Volvo gains the swiftness of an arrow – the arrow refers to the car through the meaning of “swift” – the arrow is a metaphor of the car – the sign “VOLVO 850” justifiesor legitimizes the cohesive relationship between word and image. The non-representation of the product combined with the requirement to link image and word in a coherent whole – given that we assume the message to be a text, rather than a random juxtaposition of two different and unrelated elements – inclines us to interpret the billboard as a coherent text which makes a syncretic and multimodal sense. A similar relationship of interdependence may be found in the social advertisement urging aid for flood survivors: [Header: HELP] [Bottom-right corner: the Polsat Foundation, Help for Flood Victims] The word is a clarion call, and the image evokes a flood which justifies the need for help; the sense of the image is contingent upon that of the word and vice versa. The typography alludes to the flood and a sense of threat, visualizing fear and danger, representing destruction and “filth”. The image inscribed in the word (“help”) becomes a metaphor of the threat by way of emotionalization, achieved through colour, blurring and, finally, the word itself, which is an exclamation bringing to mind the last, desperate cry of a drowning human being. The second relationship – metonymy – arises as the image adheres to the word. It is grounded in the similarity and adherence to reality: the word accompanies the image, which, in turn, reveals and emphasises the sense of the word – or, to be exact, one of its senses – presenting it in a way it should be imagined by us (persuasion as forcing a meaning) because this is how it really looks like. Metonymy may be illustrated by Text in the world of synergized codes 233 a poster from a well-known social campaign which warned the youth against smoking: “Cigarettes are for assholes” (illustrated by Andrzej Pągowski): [Header: Cigarettes are for assholes] The third text-formative relationship between text and image is tautology, which involves the sameness of two elements in terms of form and meaning. In general, the tautology boils down to the “endless repetition” of some elements, motifs, or senses, without adding new ones. It may be exemplified by the poster of the “Mediapol” advertising agency: [Header:] GREATER OUTREACH [Captions under the graters:] CLIENT ACQUISITION MAKES YOUR TEETH GRATE? / WE’LL SHOW YOU GRATE & STRAIGHT PATHS / TO FIND GRATEFUL NEW CLIENTS 234 Jacek Warchala We note the tautology in the meaning of the image (graters are for grating) and the word “greater outreach”. Needless to say, the appeal of this message lies in the subtle distance – ironic or self-ironic – which overlays the unifying tautological relationship between “a grater” and “greater [outreach]”. Conclusions. To understand the process of rhetorical persuasion through images, we should consider whether the iconic channel may operate autonomously and independently of the verbal channel, or whether the image can only complement the verbal message, like a caption under an illustration only complements its purport by orienting the interpretation and thus suggesting a meaning. Anthony Blair (whom we already mentioned above) asked if rhetoric could ever extend beyond words towards pure visuality and still serve persuasive and argumentative functions as it does in the realm of verbal expression (Blair 2008: 41). For when it comes to verifying truthfulness, there is an observable parallel between verbal and visual arguments: needless to say, a visual argument does not need to be true or false, but as it meets the eye, it becomes a “fact” for the receiver, which grants visual persuasion the status of rationality even when the representation alone is imprecise and ambiguous. After all, phrases in a natural language carry the same imprecision and ambiguity. As Andrzej Wajda once observed, “[…] what we can see is not only real, but even better – it’s trustworthy”. Herein lies the somewhat idealized power of the visible. In the practice of marketing, the sender (ad maker) intends to mitigate the risk of misreading the intended meaning, which could be lost in ambiguity when both word and image share the same semantic function. All the examples discussed above lead to the conclusion that image alone, without word, is too ambiguous and vague, too enigmatic to convey a persuasive message. On the other hand, word without visualization (without image) is insufficient for advertising purposes since it lacks vectors that refer its meaning directly to the field of connotations. It is the image that opens the field of connotations, whereas the word narrows down the meaning and orients the vectors of sense (understood here as construed/actualized meaning). The use of image and word combined leads to the emergence of a shared sense in a process which exploits our cognitive capacity to perceive the relationship between various phenomena in our reality through metaphor, metonymy, and tautology. Advertising reveals the essence of the modern visual culture which succeeds the oral and the verbal eras. This transition is regarded as the beginning of the modern period, that is a departure from the written culture Text in the world of synergized codes 235 in favour of the image which from now on dominates the process of perceiving and conceptualizing the world (Warchala, Skudrzyk 2010: 143−150). In this spirit, Nicholas Mirzoeff defines the culture of Western modernity as a distinct formation the distinctive characteristic of which is its imperative of visualization (Mirzoeff 2006: 357). This dominance is defined as the visual turn – a cultural breakthrough which, on a radical interpretation, would involve leaving the culture of the written word behind in favour of the image. But the visual turn does not necessarily imply the evolution of the emerging image culture into some expansive form that would effectively oust the culture of the written and spoken word. Even though cultural critics see the visual turn as a symptom of a revolution and “a new opening”, I believe it would be more reasonable to point out that technological possibilities of communication and perception of the world have led to the emergence of a new type of interdependence between the cultures of word and image; to observe that the process of conceptualizing the world has gained a new element which complements its representation with the aspect of visibility. For the importance of images as signs or collections of signs, either autonomous or sharing the function of the verbal channel, is undeniably on the rise, creating the research problem of multimodal competence in the construction and reception of modern polysemiotic texts. References Bagiński, D., Francuz, P. (2007): W poszukiwaniu podstaw kodów wizualnych. [In:] Obrazy w umyśle. Studia nad percepcją i wyobraźnią. P. Francuz (ed.). Warszawa, pp. 19–43. Bal, M. (2006): Wizualny esencjalizm i przedmiot kultury wizualnej. Translated by M. Bryl. „Atrium Questiones” XVII, pp. 295 – 331. Barthes, R. (1985): Retoryka obrazu. Translated by Z. Kruszyński. „Pamiętnik Literacki”. Vol. 3, pp. 289−302. Bartmiński, J., Niebrzegowska-Bratmińska, S. (2009): Tekstologia. Warszawa. Blaire, J. A. (2008): Rhetoric of Visual Arguments. [In:] Defining Visual Rhetorics. Ch. A. Hill, M. Helmers (ed.). University of Wisconsin Oshkosh, pp. 41−63. Czapliński, P. (2010): Tekst drugi. „Teksty Drugie”. Vol. 1−2, pp. 34−35. De Beaugrande, R.-A., Dressler, W. U. (1990): Wstęp do lingwistyki tekstu. Translated by A. Szwedek. Warszawa. Didi-Huberman, G. (2011): Przed obrazem. Pytania o cele historii sztuki. Translated by B. Brzezicka. Gdańsk. Dziadek, A. (2004): Obrazy i wiersze. Z zagadnień interferencji sztuk w polskiej poezji współczesnej. Katowice. Fred, M. (1980): Absorbtion and Theatricality. Unieversity of Chicago Press. Geertz, C. (2000): Wiedza lokalna. Dalsze eseje z zakresu antropologii interpretatywnej. Translated by D. Wolska. Kraków. Głombicki, Ł. (2013): Tomaszewski o fotografii: Odebrano mi zawód. Dziś każdy może robić zdjęcia [interview with Tomasz Tomaszeski]. „Gazeta Wyborcza” z 10 sierpnia. Gołębiewska, M. (2003): Demontaż atrakcji. O estetyce audiowizualności. Gdańsk 2003. 236 Jacek Warchala Gombrich, E. H. (2009): Zmysł porządku. O psychologii sztuki dekoracyjnej. Translated by D. Folga-Januszewska. Kraków. Korpanty, K. (2008): Jan Sebastian Bach i retoryka. Analiza muzyczno-retoryczna kantaty Sehet, wir gehn hinauf gen Jerusalem BWV 159. „Młoda Muzykologia”, pp. 36–61. Łotman, J. (2008): Symbolika Petersburga. [In:] Uniwersum umysłu. Semiotyczna teoria kultury. Translated by B. Żyłko. Gdańsk, pp. 291 – 307. Maćkiewicz, J. (2017): Badanie mediów multimodalnych – multimodalne badanie mediów. „Studia Medioznawcze”. Vol. 2 (69), pp. 33 – 46. Martinec, R., Salway, A. (2005): A system for image-text relations in new (and old) media, “Visual Communication”. Vol. 4 (3), pp. 338–371. Mirzoeff, N. (2006): Wszystko to bez sensu. „Artium Questiones”. Vol. XVII, pp. 195–214. Mitchell, W. J. T. (2013): Czego chcą obrazy. Translated by Ł. Zaremba. Warszawa. Olivier, B. (2010): Nauki o komunikacji. Translated by I. Piechnik. Warszawa. Rose, G. (2001): Visual Methodologies. An Introduction to the Interpretation of Visual Materials. London. Schweitzer, A. (2009): Jan Sebastian Bach. Translated by M. Kurecka, W. Wirpsza. Warszawa. Skudrzyk, A. (2005): Czy zmierzch kultury pisma? O synestezji i analfabetyzmie funkcjonalnym. Katowice. Skwara, M. (2007): O niektórych właściwościach ekfraza. [In]: Perspektywy polskiej retoryki. B. Sobczak, H. Zgółka (ed.). Poznań, pp. 204−210. Smith, W. (1962−1963): Prudentius’s „Psychomachia”. “Modern Philology”. Vol. 60, pp. 161−171. Stöckl, H. (2015): Czytanie tekstów językowo-obrazowych? Elementy kompetencji podstawowej. [In:] Lingwistyka mediów. Antologia tłumaczeń. I. Bartoszewicz, R. Opiłowski, J. Jarosz (ed.). Wrocław−Dresden, pp. 113−138. Szalewska, K. (2012): Pasaż tekstowy. Czytanie miasta jako forma doświadczania przeszłości we współczesnym eseju polskim. Kraków. Tabakowska, E. (2011): Co przechodzi ludzkie pojęcie? [In:] Pojęcia. Jak reprezentujemy i kategoryzujemy świat. J. Brehmer, A. Chuderski (ed.). Kraków, pp. 199−214. Talmy, L. (2000): Toward a Cognitive Semantics. Vol. 1. The MIT Press, pp. 139−141. Unger, H.-H. (1941): Die Beziehungen zwischen Musik und Rhetorik im 16.−18. Jahrhundert. Würzburg. Warchala, J. (2013): Miasto obrócone w tekst. [In:] Narracyjność języka i kultury. Literatura i media. D. Filar, D. Piekarczyk (ed.). Lublin, pp. 255−273. Warchala, J. (2014): Teksty na granicy języka. „Konteksty Kultury” 11B. Vol. 3−4, pp. 302−321. Warchala, J. (2018): Badanie tekstu jako przekraczanie granic, czyli kilka uwag o możliwości badań filologicznych nad reklamą. [In:] Polonistyka na początku XXI wieku. Diagnozy. Koncepcje. Perspektywy. M. Maciołek (ed.). Vol. III: Współczesne aspekty badań nad językiem – teoria i praktyka. Katowice, pp. 331−347. Warchala, J. (2019): Formy perswazji. Katowice. Warchala, J., Skudrzyk, A. (2010): Kultura piśmienności młodego pokolenia. Katowice. Wittgenstein, L. (1997): Tractatus logico-philosophicus. Translated by B. Wolniewicz. Warszawa.