RSSReview: Working with Multimodality
Jul 03
AUTHOR: Jennifer Rowsell
TITLE: Working with Multimodality
SUBTITLE: Rethinking Literacy in a Digital Age
PUBLISHER: Routledge (Taylor and Francis)
YEAR: 2012
REVIEWER: Stefania M. Maci, University of Bergamo
SUMMARY
“Working with Multimodality” links the field of ‘New Literacy Studies’ to
multimodality by trying to consider how literacy practices can be better
understood within wider domains. In other words, New Literacy Studies and
multimodality show how meaning-making processes exploit multiple modes in
order to create texts that are modally complex. Yet, to date, very few studies
have examined multimodal composition outside educational contexts. Therefore,
while concentrating on professional contexts involving multimodal creation,
the author attempts to offer a literacy framework for both education and
training, suggesting “a need to think far more progressively about what
literacy might mean in the future with digital and media convergence” (p. 2).
Rowsell’s belief is that working with modes allows the development of higher
levels of abstraction and universalization across discipline-specific
practices. As suggested by Halliday (1978), Rowsell defines a mode as the
textual product resulting from the “cultural shaping of a material” (p. 3).
Naturally, given the fact that a multimodal text is composed of more than one
mode, it is essential for the author to disclose how modes work. Indeed,
meaning-making processes can be transmodal (i.e. when the elements within a
text create a whole, e.g., films, where meaning is provided by both visual and
sound modes), intermodal (i.e. when elements within a text create a link
between modes, although they may exist independently from one another, while
at the same time cross-referencing, e.g., illustrations), or intramodal (i.e.
where more than one element jointly creates meaning, e.g., the clothing
industry, where colour combines with particular textiles in order to create a
more pronounced effect). The relationship between modes is dependant on the
mode-assembling process, which takes into consideration which, amongst the
available modes, is the most suitable or has the greatest aptness (Kress 2010)
with regard to text creation. This process transcends time and determines a
transformation, or transduction, of the modes under consideration.
As said above, meaning-making processes in literacy education have seldom been
investigated. The gap that the author would like to bridge within literacy
education is therefore closely related to the way in which we can work on the
creation of complex multimodal texts. Indeed, ‘new’ literacies, such as sound,
images and hypertexts, exist alongside ‘old’ literacies, such as space, dance,
movement and textiles which, in the author’s opinion, should be taken into
consideration in literacy education policies, since learning with modality
means converging both the social and subjective elements of meaning-making,
which is a process that is always culturally-based and socially-inherent
(p.4), and which we must bear in mind in any attempt to explain what literacy
may mean if linked to today’s fast-paced digital and media convergence.
The analysis, based on an ethno-methodological approach, focuses on nine
different modes: film, sound, visual, interface, videogames, space, movement,
word, and textiles. Each is examined as a case study and dealt with in nine
different chapters, starting with film, which embraces several other modes,
and then moving on to all the others, each of which is connected to the
previous one. The ethnographic means by which such modes are examined consist
of thirty open-structured interviews with creators, who range from web
designers and film producers to textile designers and dancers, and from song
writers to videogame creators and advertisers. The investigation takes a look
at storytelling as a way of creating meaning. What the author actually does is
look at the social practices involved in modal production in order to show
that design and creation are more important than the final product itself. The
analysis of case studies across modes outside the context of educational
pedagogy is, according to the author, extremely interesting, as it allows the
elaboration of a framework for modal learning. Multiple modes are thus brought
together to form an integrated theory of multimodality. The questions at the
end of each chapter provide food for thought, while key-points are dealt with
by the author’s comments on each case study presented.
Chapter one: Film
In this chapter, consideration is given to representation rather than
communication. As Kress (2010) claims, representation concerns the addresser,
whereas communication focuses on the recipient. The storytelling emerging
during interviews with Tobias Wiegand (an animator), Robin Benger (a
documentary director) and Rebecca Birch (a film producer) shows how
professionals understand (and reproduce) the representation of a perceptual
world. As they all explain, various modes, such as words, music, space, and
movement, are grouped together in films in a synaesthic way so as to represent
ideas, emotions and perceptual worlds. As such, films require a great deal of
planning. Educational movie-making projects require the development of skills
concerning the transformation of a topic into a narrative which needs to be
developed in order to provide verbal, visual and sound coherence.
Chapter two: Sound
Music is not only appreciated from a cultural viewpoint, but is also
synaesthically-constructed, as it involves different sounds, colours,
emotions, stories and meaning. This chapter examines sound as a way of
conveying and interpreting meanings, as revealed through interviews with David
Murphy (a composer), and Paul Chivers (a mixer-composer). In their opinion,
meaning results from either the combination of chords, or the linking of
lyrics to melodies, or even a remix of old music, so as to create new
compositions. Such combinations may become a part of literacy, since, for
example, music can be the equivalent of words, genres, and registers.
Chapter three: Visual
According to Rowsell, visual elements are linked to creativity, subjectivity
and intuition. This is clearly demonstrated through interviews with Ben Hodson
(an illustrator), Bany Mendi (a director), and Lee Edward Födi (a
writer-illustrator). Although visual elements are at the core of most texts,
this mode is seldom incorporated into language. This is a point to be taken
into consideration because today we live in an increasingly design-orientated
world. Therefore, the analysis and production of visual techniques should be
included in educational projects, as they require “thinking and expressing in
images what is often beyond linguistic capabilities” (p. 45). Indeed, the
visual mode is supplementary to other meaning-creating modes, as it enriches
the interpretation offered by a text.
Chapter four: Interface
Interface refers to “the face of digital environments” (p. 60), and as such,
it is the way in which content is displayed. Here, the focus is on interface
design, with particular regard to aesthetics-driven, user-friendly, mobile and
networked interfaces. This is a necessity, considering that people, in
particular, youth, ‘consume’ and produce digital media very quickly. The
emphasis, therefore, is on the way in which technologies converge to create
texts whose interface represents an ideal combination of function and
aesthetics. The results, as seen in interviews with Lisa Murphy (a web
director), Adrian Thiessen and Kristen Nater (a media president and
vice-president), and Joe Delisco (the icloud creator), aim to encourage both
educators and students to develop meta-awareness of web-search engine
keywords, as well as inspire web-design, with the latter being based on the
idea that web-texts are read by following an F-pattern (according to which a
text is first read on the left column and then on the rows on the right; cf.
p. 66) rather than the traditional Z-reading path (according to which a text
is read from left to right, from top to bottom, line by line; cf. Kress and
van Leeuwen, 2006).
Chapter five: Videogames
Rowsell regards videogames as problem-solving tools, as they expect players to
“strategize, communicate, interpret context, solve problems, analyze
characters, possess hand/eye coordination, have patience, understand semiotic
tools, use their spatial sense” (p. 79), which could be easily applied to the
context of literacy learning, from which students could benefit greatly,
considering that the skills pertaining to videogames are the same as those
used in modern and digital communication systems. Indeed, this is what emerges
from interviews with David Elton (a videogame creator), and Kevin Kee (a
videogame creator in the field of mobile phone technology).
Chapter six: Space
The importance of space in relation to place is the key element of interviews
with three architects, Anthony Robins, David Parker and Ana Lakoseljak. Space
and place have a cultural as well as a subjective meaning, and in literacy
learning, such a mode should be taken into consideration. Designing space
according to its social role is an important achievement in terms of
recognition of how space is used, thus requiring a meaningful fruition of
space.
Chapter seven: Movement
The mode of movement is the main focus here, in particular, with regard to
dance as a form of communication. Unlike all the other modes, which can be
experienced without a performer, movement is the only one where a performer is
required in “the midst of practice to communicate” (p. 110). The analysis of
this mode is offered through a report of interviews with Karin Kain (a ballet
dancer), Glenys McQueen-Fuentes (a ballet teacher), and Derek Metz (an actor).
The lesson to be learned here is that movement, be it formalized (e.g.
ballet), or free (e.g. expressive movement and dance), allows people to vent
their own personality traits. In literacy projects, movement allows transmodal
analysis and design, since it lets students express their own interests,
either in music or more visual forms.
Chapter eight: Word
Although language is the primary mode of communication, words can be limiting
if individuals rely only on oral expression. Words need to be supplemented by
other modes in order to allow effective communication. Such an analysis is
based on four case studies, i.e., interviews with Gary Bonilla (Creative
Director for Nestlé), Grant Lefleche (a journalist), Kari-Lynn Winters (an
author of books for children), and Gail Bowen (a playwright). In all cases,
words are seen as visual tools, which, together with either static or moving
images, can amplify and emphasize the basic meaning of a message. By aiding
students in their attempt to understand what word best conveys meaning, and
where a particular lexical choice does not fulfil communication objectives,
and should therefore be excluded in favour of other modes, students will be
able, on their own, to decide which modes are the most suitable to communicate
information or ideas.
Chapter nine: Textiles
The use of textiles is a mode which has very rarely been analysed in the field
of education. However, according to Rowsell, literacy teachers can include
this mode in classroom activities because textiles, by encompassing both
design and technology, fashion and imagination, and business and economics,
offer practical projects, problem-based learning and practical demonstrations.
Case studies regarding such exploitation of the textile mode in literacy
classes are offered through interviews with fashion creators Trish Ewanika and
Michelle Vanderheyden.
EVALUATION
Overall, the book is a valid introduction to literacy, as it follows a
practical rather than a theoretical approach. This places “Working with
Multimodality” among the most authoritative textbooks on New Literacy that are
currently available
A key point of the book is that it is written in a clear and user-friendly
style, and definitions of terminology are provided wherever necessary. Because
of its characteristics, “Working with Multimodality” is an invaluable resource
for teachers, trainers and students. It is suitable as further reading in a
course on literacy, particularly if the readership is comprised of students
with a very basic or limited linguistic background. It is very useful for
linguistics students and would-be language teachers and students involved in
communications studies courses. Further, individual chapters may be used as
integrated material for courses on literacy education, multimodality and
applied linguistics.
Although all chapters of the book are compelling, I would like to highlight
the part of the fourth chapter that deals specifically with web interface.
Indeed, this is a new mode of communication whereby there is no clear-cut
distinction between the text producer and the text consumer; the recipient of
the message is the ‘prosumer’, i.e., s/he is simultaneously the text producer
and text consumer. Readers of this mode are editors because they are enabled
to enter, edit and manipulate web-text while visiting a website. In addition,
to the best of my knowledge, no other books in linguistics have ever
emphasized the F-reading pattern characterizing web-texts. Overall, interface
may create interaction, and may facilitate communication, and this chapter
helps educators who are trying to develop discourse meta-awareness in
students.
Nevertheless, there are still a few criticisms to be made. Apart from a typo
on p. 56 (‘illustratorr’ instead of ‘illustrator’), in my opinion, there is
some ambiguity in the text where two terms are being defined: the concepts of
‘emic’ and ‘etic’ are mentioned on page 25, but when the term ‘emic’ first
appears (p. 9), the reader is left to his or her own devices with regard to
the interpretation of its meaning. Similarly, although on page 15 the author
provides a clear definition of ‘mode’ by referring to the one provided by
Kress, this is totally missing on page 3 under the paragraph “Modes” (where it
would be more useful). Unfortunately, the author also made a few oversights:
Kress (2010), who is quoted throughout the textbook, is not listed in the
bibliography; the same is true for Rowsell (2012), and Whorf (1929). Such
oversights are a pity, especially when considering the overall value of the
volume.
Regardless of the minor criticisms above, “Working with Multimodality” is a
valid aid for teachers, advanced students and linguists wishing to have a
better understanding of the relationship between literacy and education in our
modern digital world.
REFERENCES
Halliday, Michael 1978. Language as Social Semiotic. London: Edward Arnold.
Kress, Gunther 2010. Multimodality: A Social Semiotic Approach to Contemporary
Communication. London: Routledge.
Kress, Gunther and van Leeuwen, Teo (2006). Reading Images: The Grammar of
Visual Design. New York: Routledge.
ABOUT THE REVIEWER
STEFANIA M. MACI is a researcher of English Language and Translation at the
University of Bergamo, Italy, where she teaches English linguistic courses at
graduate and undergraduate level. She is member of CERLIS (Centro di Ricerca
sui Linguaggi Specialistici, coordinated by Prof. Maurizio Gotti), CLAVIER
(The Corpus and Language Variation in English Research Group), BAAL (British
Association of Applied Linguistics), and AIA (Associazione Italiana di
Anglistica). Her research is focussed on the analysis of the English language
in academic contexts, with particular regard to the analysis of English in
National and Professional Contexts. Amongst her recent publications are:
“Glocal Features of In-flight magazines” (2012), “Arbitration in action: the
display of arbitrators’ neutrality in witness hearings” (2012); “The
Discussion Section of Medical Research Articles: A Cross Cultural Perspective”
(2012); “Fast-Track Publications: The Genre of Medical Research letters”
(2012); “The Genre of Medical Conference Posters” (2012); “Poster Makers
Should Think as Much about Show Business as Science. The Case of Medical
Posters in a Diachronic Perspective” (2012); and the monographs “Tourism
Discourse: professional, promotional, digital voices” (2013); “The Language of
Tourism” (2010), and “The Linguistic Design of Mary Magdalene” (2008).
Recent Comments