Here are summaries of the key points of hte following books. First I summarized the gramar of visual design by Kress and vanLeeuwen and then I summarized key concepts in their Multimodality book. One important concept that Kress aludes to in both books as well as in literacy in the new media age is the concept of "readng path". I've been thinking about reading path in terms of how the students are reading films in the camp. It seems as if there are multiple paths. For example, students read temporally. They also read the images as spatial. They also look for changes as the images move. They also look for changes in expressions or uses of specific shots to depict meaning. Hmmm. What does this mean?

Kress, G., & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design: Second Edition. New York: Routledge.
In Reading Images, Kress and van Leeuwen provided a model for visual grammar. A grammar, they pointed out, is an inventory of observed regularities, used as a means of representation; not just a delineation of rules and regulations of normative correctness. They noted that the grammar of verbal texts and visual images have developed side by side. These similarities, however, should not lead one to expect a specific grammar for visual images as is found for linguistic texts.
Kress & van Leeuwen further discuss ground their theory in systemic functional linguistics. Here are some of the main features of the theory of visual grammar:
1) Narrative in visual representations: A vector is needed to make a proposition in visual media. A vector is a line or implied line that suggests direction. Elements of a composition are called ‘participants’, the participant from which a vector departs is an ‘actor’, and the arrival point is the ‘goal’. The meaning is a transaction; if this meaning is reversible it is called an interactive transaction. The geometrics of such relationships are sources of meaning. Lack of a clear ‘reading path’ can lead to ambiguity. A summary of realizations can be found on page 74-75.
2) Conceptual Representations:
• Classification: Symbolism or shape can be added to these diagrams. Vectors can also be evident in diagrams such as flow charts
• Analytic: Relation of a whole and parts that give ‘possessive attributes’ to the whole. Obvious examples are bar charts and circuit drawings, but portraits can also be structured in this way
3) Representation and interaction: The first direct gaze from the representation of a human out to a viewer is attributed to Van Eyck (1433). This is a power relationship, a powerful way of addressing the viewer—the direct gaze. Other directions of gaze are symbolic. If the subject is looking up, the subject is inferior. If the subject is looking down, she is superior. A level gaze denotes equality.
4) Modality: we prioritize an image by modality markers embedded within. In the west, high modality is signified by realism (truth). In other cultures, it may be more symbolic (religious). Markers of realism can be: detail, depth, quality of material, illumination, color, and craft design skill. Different areas of culture and different “subject” area discourses may have different coding orientations. For example, in areas of science, the modality code is the blueprint; whereas, in advertising, the modality code may require bright colors. In art, modality becomes a play of signifiers; complex and often esoteric relations between modality markers often provide an intertextual high modality. Modality is also conveyed by authenticity.
5) Composition: provides an integration through symbolic meanings of position, weight, and framing. Realizations include: centered, polarized, triptych, circular, margin, mediator, given, new, ideal, real, salience, disconnection, connection.
• Left and right denote the ‘given’ and the ‘new’ due to the broad convention in the West that relates to our custom of reading left to right. The eye tends to start at the left of the image and move right. (note: this is often different with pre-readers—salience plays a role in their visual literacy).
• Top and bottom denote ideal and real, promise and product, emotive and practical, head and foot.
• The center is the place of the ruler, harmony and symmetry. In western art and graphic design, the use of a geometrically centered image is considered naïve.
• Weight includes: size, focus, contrast, and foregrounding. The weightings of these aspects of image have a center of gravity.
• Framing: may be explicit or implied. Lack of framing implies a group identity; whereas, framing individuates.
• Rhythm: in film—time based image. In a book, flicking through a page. Rhythm in multimedia could also refer to zooms, pans, and transitions in a sequence of time.
• Salience is the degree to which an element draws attention to itself due to: size, place, overlapping of elements (color, tone, sharpness, definition,etc.).
• Connection/disconnection: the degree to which element is connected or visually separated through framing, empty space, vectors and differences/ similarities in color, shape.
6) Materiality and Meaning: Inscription: Brush strokes…also the hand made marks, marks recorded with technology, and marks synthesized in technology. Color is a semiotic mode, which carries meanings of its own (including cross cultural variations). Color also has emotions attached with it. Additionally, in certain arenas, color has textual connotations (blue text on computer symbolizes hyperlink). Color coordination can promote cohesion. Distinctive features to the semiotics of color include: value, saturation, purity, modulation, differentiation, and hue. Finally, color schemes can provide significant design qualities.













Article 4
Kress, G., & van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. New York: Oxford University Press..
In Multimodal Discourse, Kress and van Leeuwen outline a theory of communication for the age of interactive multimedia. Beginning with the concept of ‘design’ they outline an approach to social discourse where color and font play a role equal to language. They defined multimodality as the “use of several semiotic modes in the design of a semiotic product or event, together with the particular way in which these modes are combined—they may for instance reinforce each other (say the same thing in different ways), fulfill complementary roles…or be hierarchically ordered” (eg. action films where action is dominant and music adds to the presence). Furthermore, they articulated communication as a “process in which a semiotic product or event is both articulated or produced and interpreted or used” (p. 111).
In the final chapter, where they delineate a multimodal theory of communication, which concentrates on two things: a) the semiotic resources of communication (modes and media); and b) the communicative practices in which resources are used (discursive, interpretive, production, design, and/or distribution practices). They key point they made is that meaning is made “not only with a multiplicity of semiotic resources, in a multiplicity of modes and media but also at different ‘places’ within each of these” (p. 111).
One of the key elements in the novelty of multimedia discourse is the as pect of design. Discourses can be realized in different modes; each mode adds layers of meaning. Design consists of a ‘blueprint”, an overall spatial schema of a page with bits of information. This could also be used in connection with other modes (text, color, spatial arrangement, font, etc.). Therefore, on a multimodal “page” information is spatially, rather than sequentially organized. Spatial order—where elements are placed, how salient they are, in which ways they are framed, how they are connected, color harmony/disharmony—becomes a key aspect to the visual schema. Unlike a traditional text, where the reader follows a sequential order, in a visual text, the importance is suggested by hierarchies of salience.
Further elements of design are related to production and distribution. For example, the way in which separate bits of information are produced (with boxes on pages—such as a website) adds to the meaning. In this way, typography also becomes significant. The use of a handwriting font depicts a personal message, something that has become conventional. Using the premise of “Provenance” --“the idea that signs may be imported from one context into another in order to signify the ideas and values associated with that other context by those who do the importing” (p. 23)--the use of handwriting is a sign of personal address that has become conventional. However, it has not been grammaticalised; typography is still ‘lexical’ and works through connotation. Therefore, the meaning of the font is different than the meaning of the actual text, which follows grammatical rules.