Media Literacy

Chapter 2. The Aesthetics of Multimedia (published in Japan in Japanese in 1998.)

1. Grammar of Multimedia

Language and Grammar

Before the birth of multimedia, a person who could not draw pictures could never become an artist, any more than one who could not play an instrument could become a musician.  However, our modern technological civilization has produce machines that render such impossibilities possible.  In other words, computers have popularized the arts.

Yet, to the massed without access to computers, there is still no means of practicing the arts.  This "popularization" of the arts is limited to the more affluent peoples of the world.

There is yet another pitfall.  Ordinary people who do have access to computers may have past experiences in art appreciation, but most have never before created art of their own.

When pondering the issue of multimedia education, we must first consider the hard-to-define term of "media literacy".  It is correct to regard multimedia as an unknown language, but how unfortunate that a technological giant like Japan limits itself to the crude concept of media literacy as a mere understanding of hardware (i.e., an issue of whether or not one can use the machines)!

Properly speaking, media literacy cannot exist without a decoding of the context of the medium itself (software).  In order to do this, one must of course make reference to the history of the medium's establishment, as well as possibilities for forms of expression employing media.  We shall take these considerations up at another time, focusing in this chapter on an interpretation of the fundamentals of multimedia grammar (or aesthetics) from two directions:  images appealing to the sense of vision (movies, television, videos, computer images, CD-ROMs) and sounds appealing to the sense of hearing (the human voice, recordings, CD-ROMs).

The Fundamentals of Visual Images

When we consider the basics of visual images in the context of education, we find that the growth in opportunities for students to create visual images by themselves is the biggest factor setting the multimedia era apart from the past. For instance, a student could use an image of the full moon from the home page of National Geographic magazine to create a film about the moon.  But when we consider this situation carefully, we realize that multimedia cannot be used effectively without an understanding of visual composition, metaphor, symbols, and the connection between one image an the next--areas that have previously been the domain of specialists.  We also recognize that a more detailed knowledge of images was actually necessary even back in the days when all the average person had to do with visual images was behold them.  It is instructive to consider Leni Riefenstahl's film Triumph of the Will  from this standpoint.  This film, made to glorify the Nazis under Hitler's control, is a treasure-house of the conventions to which visual images lend themselves.  The film shows Hitler as if descending from a gorgeous sky, the camera taking the vantage point of people worshiping him from below, flanking both sides of a street.  It is as if Hitler were God himself.  The people below look up at Hitler against the backdrop of the sky.  In the world of film, a camera angle looking up into the heavens symbolizes hope.  Of course, history has shown just how empty the hope conveyed by this propaganda film turned out to be.

More recently, English Patient, which won nine Academy Awards, including Best Picture, used this same convention, the camera angling up into the sky as Juliette Binoche, playing the nurse,left the battlefield  at the end. Not a few viewers must have had the ominous foreboding that this film may actually have been a tribute to war.  Most frightening of all, these examples make us realize that multimedia and photography of all kinds are created under Western conceptions of the world.

Cultural Issues

Because they are under the misconception that Japan is quite Westernized, the Japanese tend to make light of differences between Japan and the West when considering visual images,  The moon in a picture is the same moon, wherever it is viewed, but there is a difference in significance between the moon viewed in a Japanese environment and the moon as Westerners conceive of it. 

For the Japanese, the tradition of moon-viewing remains in full force, whether or not rockets reach the moon, and the tale of "The Bamboo-Cutter and the Moon-Child " is a time-honored fantasy.  For Americans, the moon is just a place that has been conquered, not the stuff of fantasy.  It only leaves behind an image of craters, barren land, and desolate scenes.  In this contrast between the Japanese and Western images of the moon we see a classic example of Samuel P. Huntington's "clash of civilizations."  Thus, if we were to create an image of a Japanese fairy tale using a shot of the moon, Americans would not be able to grasp its full cultural significance.  This difficulty in cross-cultural interpretation of images forces us to consider the issue of whether Japanese multimedia education is to be based on Western aesthetics or whether we should find characteristically Japanese modes of expression in the course of compromise, with Western aesthetics studied as an adjunct to this pursuit (Since computer images themselves have come into being under Western conceptions, it is perhaps impossible to completely ignore western aesthetics).  This may seem like a potential problems that lies in the distant future, but I believe that cultural clashes of this type will occur surprisingly soon, even in the use of multimedia in the classroom.

The Aesthetics of Visual Images

The aesthetics of visual images is based on the necessary premise that the image not be unpleasant.  This premise is common to both paintings and photographs, and it is a matter of the "stability" of the image.  When it comes to movies, in particular, one could fill a book with all the pertinent rules, but the main theme boils down to not giving an unpleasant impression to the viewer.  This depends on what is to be shown in the image, but in the case of a human being, you must make it clear what exactly you are expressing about that person.  For instance, when making a shot of a girl about to vault over a gymnastic buck, you would not want to show just her face, lest the viewer fail to grasp what she is doing.  On the other hand, it would be boring to show a continuous view of the buck and the girl's entire body.  If you were to insert some shots of the details of her face (eye movement, etc.), you could enliven the image by expressing her emotions as well.  In aesthetics, one must also consider the rule of thirds, whereby one divides the image into thirds vertically as well as horizontally, positioning the subject so as to give an impression of harmony and stability.  (See Figure below)

Title of the Figure below: The Aesthetics of Visual Images And The Rule of Thirds

(Reference* T. Schroeppel, "The Basic Bones Camera Course For Film and Video." 1997)

Look at this stability of this image. 

Height of the eyes is the upper one third. 

Do not position the profile face to the center.

Sound

Let us now consider the fundamentals of expression through sound.  Sound encompasses physical attributes such as amplitude, frequency, and waveform, as well as psychological aspects such as loudness, pitch, and tone.  As stressed by Dr. D. M. Carroll of Menlo College (1998), the first issue that arises in discussion of sound quality in the classroom is, of course, the human voice.  The human voice must be carefully recorded. If, for instance, you are using a recorded voice in a presentation, that presentation could fail to achieve the desired effect if the audience is unable to make out what is said.  In order that the listener might easily comprehend the content of the presentation, it may be necessary that the speaker have training in speaking Japanese correctly.  Moreover, it can be very effective to modulate the voice at the time of recording.

Having a special room for recording is helpful in regulating the physical effects of sound, but even if a recording studio is available, such a space is essentially an audio room and will require some adjustments to function as a media room.  If possible, it is desirable to secure access to a computer with large memory capacity, as well as a sound-surround-type studio and a directional mike, but we must not forget that the minimum required recording  equipment is still just a microphone.

a kind of

2. Editing in Multimedia

Montage

How do people look at the paintings they go to see in museums? There are times when they will examine only the detail in a specific portion of the painting, but most people will at some point stand off a distance from the painting to take it in in its entirety.  They will try to discover what the painting is expressing, what tjere is about the painting that strikes a chord of empathy in themselves, and what they might imagine about it or associate with it.  They give themselves over to a kind of trance, as it were.  The famous British art critic J. Berger was once asked, "What in a painting is lost when the painting is reproduced by a camera?"  He replied that the tranquility the face of the painting intrinsically possesses is lost.  He went on to say that the original focus of the painting is diffused when a close-up shot of a painting shifts the perspective from an overall view to a portion.  Even when shown without background music, the individual portions of a landscape painting, shot by a photographer or isolated with some intention in mind, will collectively differ in the meaning conveyed when the order of presentation is varied.  Pieter Bruegel's painting The Processing to Calvary gives a detailed depiction of many people proceeding toward the hill at Calvary, where Jesus Christ was crucified.  The painting exudes an atmosphere of reverent prayer, sorrow, and suffering, but the significance of the work is totally altered when the camera intentionally focuses on individual people in the scene or shows a close-up of a figure with his back turned to the viewer.

For a more familiar example, consider a television drama or popular film that shows a close-up of a girl with her eyes looking toward the left, and the following shot again shows the same girl, we understand that the two are in love.  However, if the boy's eyes are turned toward the right, and the next shot shows a different from, then the sequence indicate a love triangle.  The problem of what the human eye sees next is closely connected with our explanation of the relationship between the objects we are observing.  Or perhaps it is related to our ability to grasp the nature of a situation, based on the knowledge that comes of human experience. 

When discussing this kind of editing of film images, anyone familiar with the history of motion pictures will inevitably mention Russian filmmaker Sergei Eisenstein's dialectic editing technique, known as montage theory.  In this age of home videos, the famous Odessa Steps scene in Eisenstein's masterpiece Battleship Potemkin (1925) demands consideration.  In 1905, when people thronged to Odessa Harbor to meet the return of the mutineers on the battleship Potemkin, they were fired upon by Czarist troops.  If that sequence had been filmed in the style of a home video, simply following the temporal flow, the inability to do retakes or interrupt the action would have made it impossible to do anything more than mark the passage of time.  However, the situation changes when this story is filmed from different angles by a number of cameras and the film is then edited by the montage method.  A mother wheeling a baby carriage at the top of the steps is shot, and as the baby carriage tumbles down the steps (of course, the baby is still inside!), swept along by the advance of the troops, it rolls past the fallen bodies of those who have been shot, gaining speed, to capsize at the foot of the steps.  Eisenstein uses insertions and repetitions to give powerful expressive force to the depictions at each point in this descent: the mother's agony, the terror of the crowd, and the ceaseless barrage of fire from the troops.  The sequence thus has an impact and persuasive force far greater than could have been achieved through a single unvaried shot of the events occurring before one's eyes.  This Odessa Steps scene is a good example of just how much can be conveyed through film images, depending upon the sequence of presentation.

Varying Meaning Through Sound

The significance of a picture can also be changed through the use of background sounds.  In a 1972 television program co-produced with the BBC, J. Berger used the example of a Caravaggio painting to illustrate how music can totally alter the meaning of a visual image.  This painting shows a group of people surrounding a dinner table laid out with bread and fruit.  At the center is a woman, and she is talking with some men seated at the same table.  At first, the program showed a sequence of close-ups of the faces of individual people in the painting, using a background of Italian opera music to make it seem as through the people at the table were engaged in lively conversation.  Next, Berger used a background of somber, requiem-like music to create the illusion that this was a religious painting.  In these examples, the individual is not free to view and asses the painting at a speed of his or her own choosing, and the effect--a suggestion to "see the image this way"--is of the same type achieved in a film sequence edited with a specific intention in mind.

Varying Meaning Through Narration

J. Berger also states that the significance of a picture can be altered through narration.  Viewed under ordinary circumstances, Van Gogh's Wheat Field with Crows, while very bold in terms of color and touch, is still just a painting of birds flying over a field.  How much greater impact could be achieved by accompanying of lamenting the camera shot of this painting with a narration stating that the painting was created a short while before Van Gogh committed suicide!  In another example, a morning rush-hour scene on a Tokyo subway line could be teamed with a elegiac narration lamenting the overcrowding of the city, or it could be turned into an image affirming the status quo if paired with a narration explaining that "This is how morning begins in the busy metropolis of Tokyo."

Internet Ethics

I had intended to discuss the degree to which media editing governs and alters human perception.  But if you are going to examine the order of image presentation in films, as I have above, you also must consider the use of subliminal techniques on the Internet.  Subliminal methods of suggestion are familiar to most from stories of how American movie theaters once experimented with inserting fleeting images of popcorn, lasting less than a second, between the sciences in feature films to increase sales of popcorn.  Psychologists are divided in their assessment of the reliability of these experiments and the credibility of the "subliminal effect" itself, but at least there is lively debate over the possibility that a stimulus of which a person is not conscious might exert an effect upon his or her behavior in some form.  To begin with, situations where a person possessed with ordinary powers of judgement has no awareness of the information confronting him are dubious from an ethical standpoint.

The issue of subliminal techniques being used on the Internet suggests to us the following.  Firstly, even if some people are abusing these subliminal techniques,there is no organization for policing them.  Secondly, the software used for these purposes is easily obtainable at adult video and magazine stores within Japan.  Or the Internet itself can be used to obtain unimaginably lascivious or violent images, without respect to national borders.  Finally, anyone with a digital video camera can upload images shot with the camera directly into a computer.

From the standpoint of the aesthetics of multimedia discussed here, we would have to say that such abuse of subliminal techniques deviates grossly from aesthetic constraints.  However this discussion may depart from this book's main themes of educational media and media literacy, or whatever political frictions it may incite, the issue of subliminal manipulation is one that cannot be ignored. 

References

Berger, J.  Ways of Seeing,1952.  See also Ways of Seeing, Berger's 1972 television program co-produced with the BBC.

Carroll, D.M. "Multimedia Sound in the Classroom" (1998).  "The Effect if Audio Fidelity as Determined by Sampling Rate of Digitally Recorded Sound on Recognition  Memory in a Computerized Audio tutorial" (1992).

Note

Samuel P. Huntington, born in New York in 1927.  Divides world into eight cultural blocs: Chinese, Japanese, Hindu, Islamic, Western Europe, Russian Orthodox, Latin American, and North American.   Holds that cultural identities give rise to patterns of fragmentation or conflict.

(ATHERTON, Suji)

Chapter 11 Exploring the Front Line of Multimedia Education