Study Guides, Notes, & Quizzes

A Study in Synchresis

Description
A Study in Synchresis
Published
of 2
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  A Study in Synchresis Daniel Wilson, Sebastian Gassner Art & TechnologyDept. of Applied Information TechnologyG¨oteborg University and Chalmers University of Technology402 75 G¨oteborg, SwedenEmail: { wilson, sgassner } @ituniv.se  Abstract —The ability to record and replay images and soundis quite new, and the combining of the two is even more recent- with cinema spending its first 35 years separated from anysynchronized sound. However, sound attracts slight notice outsideof the realm of music - becoming quickly overlooked as soon as itis combined with image. Our aim is to separate and recombine thevisual and auditory elements as a means of examining sound’sinfluence on the cumulative message that we receive from anaudio/visual source - also known as synchresis . I. I NTRODUCTION The French avant-garde composer, author and theoretician,Michel Chion, has been a pioneer in the new field of audio-visual relationships, developing a theory he calls synchresis (derived from synchronism and synthesis), which he definesas: ”The spontaneous and irresistible mental fusion, completely free of any logic, that happens between a sound and a visualwhen these occur at exactly the same time” [1]. Our aim is to explore this concept; the idea that there isan emergent property which is revealed in the union of soundand image.Historically, in the early days of sound films, it was commonpractice to record a different actor’s voice from the onethat actually appears on screen. This separation of the actualsound and the presented sound has continued - and becomeincreasingly sophisticated. Today, the audio for sound film islargely created - and totally created in the case of animation- in the postproduction studio. The audio in any given scenecan range from a partial to total construct of sounds that wereeither recorded after the fact, by Foley artists, or drawn frommassive high-quality sound libraries.Through this process of sound design we are able to bothconceal and emphasize - as well as create reassociations inthe fabrication of a dual modality reality. This is done throughthe use of sound that is often more real than the real. Whatwe hear is not simply a replacement of the sounds from theoriginal scene, but the construction of a soundscape that isdeemed most conducive to eliciting the desired psychologicalresponse. This more real than real sound is what Chion calls rendered sound  [2].Chion sees this as a means of  adding value to the image. Inother words, as the nature of synchronous sound is one thatinduces the audience to construe the image, and hence event,differently, we can view the relationship of sound and imagein film, as well as visual media in general - as John Cage andMerce Cunningham explored in their collaborations - as onethat is not merely associationist  , but synergetic .This synergetic emergent property has been explored innumerous scientific studies, where it has been found that ”Spatially and temporally coincident acoustic and visualinformation are often bound together to form multisensory percepts” [3]. This, in turn, can give rise to illusory percepts,when incongruent information is presented through these twomodalities. The McGurk effect is an example of this, usingspeech, where a spoken sound and a video of a person artic-ulating an unmatched sound gives rise to a third, imaginarysound; or BA + GA = DA [4]. Along these lines Chion had an exercise he called Forced  Marriage which involved replaying the same film segmentwith different soundtracks in order to examine the relationshipsthat are altered and created [5]. Our project is an extensionof this concept in which we examine the power and effects of synchresis by means of an interactive installation that allowsthe audience to immediately and dynamically see, hear andfeel the effects of this phenomenon.II. T HE I NSTALLATION The basic element of our study on synchresis is a short video- created with the goal of audio dynamism - shown on a screenin 5.1 surround sound. The audio for this film is recorded andmastered in a way that every single sound source has it’s ownaudio track. This allows each sound to be altered, swapped orreplaced independently.  A. Sound Templates In order to maintain a relation with the initial soundelements, each srcinal track will leave it’s template behindwhen a new track comes in to replace it. In other words, it’svolume, timing, and positioning in the 5.1 environment willremain associated with the track, and will then be applied tothe new sound (for example, a remapped bird song wouldonly be heard with each footfall - and would be at the relativevolume and pan of the srcinal footfall).  B. Sound Manipulation The second dynamic and independent variable will affect theactual sound elements, such as pitch, timbre and harmonics.  C. Modes of Operation There will also be a choice of four different initial statesfor the film’s audio:1) The actual audio that was made for the film2) A replacement of the audio with randomly chosen soundfiles of the same sound (replacing one type of footstepwith another, eg. high heels for sneakers)3) A real-time net-search for audio files using key wordsfor each audio element4) A real-time setup of microphones, connected to thecomputer, which are positioned in places that couldpotentially have the same sound as is in each of thesrcinal movie’s audio tracks.  D. Logistics The set-up for the installation would be fairly simple. Wewill supply all necessary hardware elements. The installation ishighly scalable, but a room of at least 2m * 3m, a projectionscreen or white wall, and power outlets would be required.This could be either a physical room or a room created witha frame, and covered with a sound-deadening fabric - so asto avoid disturbing other installations. There would be onecontrol panel, centered in the room.III. C ONCLUSION Through this exercise the audience will develop both ap-preciation for the synergetic properties of sound and imageas well as an awareness of the ability of the sound designerto manipulate and re-associate through careful selection. Itwill provide some insight into how we hear - and howcharacteristics of visual elements may be rendered in sound,such as ”movement, weight, size, solidity, resistance, contact,texture, temperature, impact, release, ...” [6] .Synchresis is an often-overlooked element of life - withsound generally getting overshadowed due to the primacy of image. Here we venture to pull audio back to the forefrontthrough interactive and generative methods that aim to showboth the malleability of the message we receive from visualinformation, and the power of creation and association that isthe property of the union of the auditory and the visual stimuli.R EFERENCES[1] Michel Chion, Audio-Vision: Sound on the Screen , p. xviii, New York:Columbia University Press, 1994.[2] Michel Chion, Audio-Vision: Sound on the Screen , p. 98, New York:Columbia University Press, 1994.[3] T.S. Andersen, K. Tiippana and M. Sams, Factors influencing audiovisual fission and fusion illusions , Cognitive Brain Research, 21(3), p. 301-308,2004.[4] H. McGurk and J. MacDonald, Hearing lips and seeing voices , Nature,246, p. 746-748, 1976.[5] Michel Chion, Audio-Vision: Sound on the Screen , p. 118, New York:Columbia University Press, 1994.[6] David Sonnenschein, Sound Design - The Expressive Power of Music,Voice, and Sound Effects in Cinema , p. 27, CA (USA): Michael WieseProductions, 2001.
Search
Tags
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks