Introduction to Digital Sound Design

A Coursera course by Steve Everett (Emory University)

Course summary for the course ‘Introduction to Digital Sound Design’ on www.coursera.org.
This summary was made by Jeroen Lenaerts. All copyright (unless otherwise indicated with some
images) belongs to Steve Everett, Emory University and Coursera.
 Introduction to Digital Sound Design

Table of contents
Week One: The Nature of Sound ............................................................................................................ 3
The Human Ear .................................................................................................................................... 3
The Six Properties of Sound ................................................................................................................ 4
Timbre and the Overtone Series ......................................................................................................... 5
Music and the Human Condition......................................................................................................... 6
Sonic Visualiser Tutorial ...................................................................................................................... 8
Music and the Environment ................................................................................................................ 9

All copyright belongs to their respective owners

2
 Introduction to Digital Sound Design

Week One: The Nature of Sound

The Human Ear

Path of the sound

 Vibrating air particles enter the ear

 Ear drum (attached to 3 bones) vibrates
 3 bones (ossicles) convert vibration to the cochlea
 Basilar membrane inside the cochlea vibrates in specific
place(s):
o Higher frequencies vibrate at the narrow part of
the basilar membrane
o Lower frequencies vibrate at the wide part of the basilar membrane
 Along the basilar membrane are little hairs: Cilia
 Cilia move when that part of the basilar membrane vibrates
 When cilia move, they send electrical signals to the auditory nerve
 Electrical signal goes to the brain
 We hear sound

Conversion of energy

 Air molecules vibrating in outer ear: physical energy

 Fluid molecules vibrating in cochlea (after transfer by ossicles): physical energy
 Basilar membrane picks up these vibrations
 Cilia converts these vibrations to electrical energy

Conversion from physical to electrical energy – Same principle as a microphone

 Electrical pulses from microphone get converted into a digital signal (0’s and 1’s)
 Some limitations in this conversion

Frequencies of the basilar membrane

 Basilar membrane only picks up frequencies between 16 and 20000 vibrations per second
 We only hear these sounds

All copyright belongs to their respective owners

3
 Introduction to Digital Sound Design

The Six Properties of Sound

1. Pitch (Frequency)
 Steady state of vibrations
 Amount of vibrations
(compressions/rarefactions) per second: Available under the Creative Commons CC0 1.0 Universal Public
o More vibrations: Higher Domain Dedication license
(http://commons.wikimedia.org/wiki/File:CPT-sound-physical-
o Less vibrations: Lower manifestation.svg)

2. Loudness (Amplitude)
 Height of the compressions/rarefactions
3. Timbre
 Multiple different frequencies at different amplitudes at the same time
 Acoustic: difficult to change timbre
 Electric: easy to change timbre (manipulate parameters of the sound wave)
4. Duration
 Length of a sound
 Duration of time is relative (perceived relative to heartbeat)
5. Articulation (Reverberant environment)
 First few milliseconds of a sound (are most important)
 How you move into a sound and how it dies out (attack/sustain/decay)
6. Spatial location (Diffusion)
 Where is the sound coming from?
 Evolutionary benefit
 Monophonic, Stereophonic, Quadrophonic, …

Brain can identify all of these properties of a sound, and even of multiple sounds at the same time
All of these properties of sound can be analyzed or manipulated

All copyright belongs to their respective owners

4
 Introduction to Digital Sound Design

Timbre and the Overtone Series

Timbre is combination of multiple frequencies sounding in the
brain simultaneously

Overtone series (harmonic series)

Simultaneous sounding of a particular fundamental frequency

plus divisions of that frequency in various ratios

 Depends on the nature of the instrument (resonance

in the materials)
Available in the Public Domain
 Different instruments have the same series of (http://commons.wikimedia.org/wiki/File:Harmonic_par
tials_on_strings.svg)
overtones, but with each overtone louder of softer
 Different timbre
 We don’t hear individual overtones
 Timbre: warmer or brighter sound

Example for an instrument playing Deep C (C2)

 Multiple frequencies resonate at the same time

 ( )+( )+( )+…
 Maximum (we can’t hear any higher)
 Each partial has a specific amplitude in overtone series, e.g.:
Available under the Creative
o at Commons Attribution-Share Alike 3.0
o at Unported license (Made by Hyacinth)
(http://commons.wikimedia.org/wiki/File:
o at 40dB First_eight_harmonics_vertical.png)
o
 Overtone series gives the specific spectrum or timbre for that specific instrument (and that
fundamental)

Synthesis

When synthesizing a specific timbre:

 Synthesize a sound as fundamental and different overtones

 Create and adjust overtones (filters, …)
 Change timbre

Available under the Creative Commons Attribution-Share Alike 3.0

Unported license (Made by Hyacinth)
(http://commons.wikimedia.org/wiki/File:Harmonic_spectra_theor
etical_x_y.png)

All copyright belongs to their respective owners

5
 Introduction to Digital Sound Design

Music and the Human Condition

Sound and music are very important in every human civilization

Hearing sound

 Auditory aspect: perception of electrical energy from auditory nerve by the brains
 Visual aspect: from where, from who, … can color the way we hear
 Kinesthetic aspect: musician knows what muscles to use, how to breathe, …

Music and evolution

Several theories about how music evolved:

1. Music promotes group cohesion

 Strengthens emotional bonds between mother and infant
 Music can transmit emotional information to many individuals at once
2. Socio-emotional bonding
 Music activates cognitive domains involved in speech
 Music and dance act as indicators of group stability
3. Music promotes sexual selection
 Evolution of music in humans had its roots in courtship songs (Charles Darwin)

Music in the cognitive evolution of a child

 Children recognize certain aspects of music very early on:

o Consonant vs. dissonant
o Happy vs. sad
o Pitch
o Harmony
o …
 Infants pay more attention to their mother’s infant-directed singing than to her infant-
directed speaking
 The brain from birth has the ability to process timbre very good
 White noise makes children feel comfortable (identical to sounds in the womb)

Timbre

 Timbre is perhaps most important property of sound (more basic to what we perceive
esthetically)
 Every culture has different preferences for auditory roughness

All copyright belongs to their respective owners

6
 Introduction to Digital Sound Design

Auditory roughness

Auditory roughness is an aspect of timbre

 A sound has a lot of auditory roughness when it has:

o high ratio partials (high overtones with high amplitude)
o amplitude fluctuations of the spectrum
o pitch instability
o a high degree of sonic information in-between pitch areas
 The amount of auditory roughness is experienced differently by different people
 The amount of auditory roughness influences the musical experience

All copyright belongs to their respective owners

7
 Introduction to Digital Sound Design

Sonic Visualiser Tutorial

Many programs can be found to analyze sound files

Sonic Visualiser

 Spectrum analyzer
 Dissects the sound (overtones, frequencies, tempo, key, …)
 Free software
 www.sonicvisualiser.org
o Software download
o Plugins download
o Reference Manual

All copyright belongs to their respective owners

8
 Introduction to Digital Sound Design

Music and the Environment

 We think differently about digital sound than about acoustical sound
 Different cultures approach music and sounds differently, same goes for digital music
 Digital sound/music led various people to think about sound/music in new ways

Transportable sound

 Western music more ‘transporatble’ than Japanese (Toru Takemitsu)

o Western music can be abstracted into relations between fixed pitches
o Japanese music more tied to specific timbres
 Objet Sonore/Sound Object (Pierre Schaeffer)
o Timbre not accounted for in Western music scores (Musical note doesn’t say
anything about timbre)
o Therefore use instead Sound Object (which includes timbre)
o Sound Object doesn’t indicate sound source (“this sound is coming from a violin”),
refers to sounds without seeing the causes behind them
http://silentlistening.wordpress.com/2008/05/09/the-concept-of-
%E2%80%9Esound-object%E2%80%9C-objet-sonore-by-pierre-schaeffer/)
 Acousmatic listening (Pythagoreans, Pierre Schaeffer)
o Listen to sounds without seeing the causes behind it
o More focus on content of audio perception than on physical object that causes this
audio
o In electronic music  acousmatic (don’t see source of the sounds)
 Schizophonia (R. Murray Schafer)
o Sounds/music become abstracted also from cultural context (not only from original
sound source)
o Electronic music often has no cultural links
 Timbral morphs/Linear timbral transformations (Barry Schrader)
o Timbres created digitally that constantly change over time
o Timbres morphing: one particular state to another
o Very common in digital sound design, digital composition, spectral composition

Music teleology

What is the function of music?

Organized from intra- to extra- musical meanings
Music caries no meaning at all (abstract)  Music has a very powerful meaning

1. Organic interconnection of musical elements

 Autonomous listening: listening to music, blocking out all other sounds in environment
o autonomous listening space, e.g.: concert hall (somewhat unique for Western music)
 Disinterested contemplation (Imanuel Kant): highest esthetic experience is to let music
wash over us in autonomous listening space

All copyright belongs to their respective owners

9
 Introduction to Digital Sound Design

2. Expression of emotion
 Composer expresses
 Sometimes similar, sometimes different from experience of emotion
3. Experience of emotion
 Listener experiences
 Sometimes similar, sometimes different from expression of emotion
4. Motion
 Music is embedded with motion (from beat to beat, in phrases)
 Human beings move (across planet)  link to motion in music
5. Tone painting/Sound painting
 Imitate natural phenomena
 Trying to represent experience of the world
6. Imagery and narative
 Music follows certain narrative (film)
7. Metaphor of a life experience
8. Creating a social or self-identity
 For instance national anthem

Musical affect

Many cultures will generate emotions using more or less the same technical devices

All copyright belongs to their respective owners

10
 Introduction to Digital Sound Design

Ecological approach to musical perception

 Ecological Psychology (James J. Gibson)

 Ecological approach to perception of musical meaning (Eric Clarke)
o How one perceives something depends on both environmentally available
information and on capacities, sensitivities and interests of the perceiver
o Certain objects have different affordances (qualities an object can have) once they
are taken out of their normal context
 Sound can have different affordances as well (different meanings, linked to
one sound)
o Musical sounds not fundamentally different from sounds in natural world
 We experience music and sounds from nature in similar way
o The way listener perceives music/sounds depends on its listening history
 When perceiving music we collect information from musical environment
 Through continual exposure (growing up and hearing music/sounds) we
build up a certain way of how we perceive music/sound
 Understand how listener perceives sound?  Understand their current
environment but also their entire listening history
o We tend to think that cultural and ideological components are not as important in
the perception of music as are the basic sensory and perceptual attributes
o Ecological approach offers an alternative view that embraces environmental
attributes, e.g.:
 Spatial location
 Physical source
 Structural function
 Cultural and ideological value
 …
o Ecologically modeled compositions emphasize relationships between composing
the piece and the entire context of the perception of the piece
 Different cultures have different aesthetic contexts when listening to music
o There’s a whole system of thinking in the Japanese musical tradition (not just the
superficial ideas outsiders have) (Joji Yuasa)

Thoughts on the link between music and environment

 In electroacoustic music, tend to listen teleologically (the sounds have a purpose)

o Listen with certain expectations
 Experience of art is inextricably joined with everyday experience and motion of the body in
space
o The way we experience music/sound is depending on the environment we’re in as
well as on the movement we are experiencing
 Electroacoustic art is a continuum with every other kind of experience that gives rise to
meaning

All copyright belongs to their respective owners

11
 Introduction to Digital Sound Design

 Ecological approach to electro-acoustic music analysis is useful for developing a perspective

of a work, informed by:
o Structure
o Culture
o Symbolism
o Hermeneutics (text interpretation)
o Emotions
o Environment
o Perception
 Prevalence and types of timbres used in contemporary sound design is culturally and
environmentally determined and is arguably one of the most salient qualities in perception
of sound
o Timbres are perhaps the most important part of what we are experiencing when
listening to music/sound
o In sound design we sample or synthesize sounds which we later process
(manipulating timbre, rather than focusing on pitches)
 Linear music often evokes a continuity of motion (goal oriented or non-directed)
o In contrast: Japanese music is not so much about relationships between different
sounds, it is more about space in between notes
 Distinctly nonlinear
 Responds to disintegration of linear thought in Western culture
 Each work of electroacoustic music needs to be analyzed as a distinctive system of thought
with its own ecology and traceable history of belief and aesthetic choices.
 Technologically produced art should not be tried to understood outside of traditional
aesthetic and cultural frameworks

Soundscape composition (acoustic ecology)

Movement of composers and sound artists who approach sound in a new way, emerged in 70’s

 Questions that influenced these ideas:

o How do our attitudes toward listening and sound-making shape our concepts of
music, noise and silence?
o How can we achieve a balance in our environment among animal, human and
technological sounds?
o What are the physical and emotional effects of noise?
 Modern world is filled with sound (absence of noise is hard to find)

What is soundscape composition?

 R. Murray Schafer:
o Study of the effects of the acoustic environment or soundscape on the physical
response or behavioral characteristics of creatures living within it
o Particular aim is to draw attention to imbalances which may have unhealthy or
inimical effects

All copyright belongs to their respective owners

12
 Introduction to Digital Sound Design

 Barry Truax:
o Study of sound as a component of natural and artificial environments, with effects
on health, cognition and culture
o Integrated field of study with links to anthropology (cultural attitudes) and
environmental sciences (health and culture)
o Different approach to composing music : ‘composing through sound’
 Uses processing techniques that are used to reveal the inner levels of
meaning and symbolism contained within the timbres of familiar sounds
 Recreates models of a more balanced relationship between ourselves and
the environment
 Soundscape composition is always rooted in themes of the sound environment, it is never
abstract
o Recorded environmental sounds are its ‘instruments’ and may be heard both
unprocessed and processed
o Use recorded sounds and (optionally) process them in different ways to use in a
soundscape
 Soundscape composition is as much a comment on the environment as it is a revelation of
the composer’s sonic visions, experiences and attitudes towards the soundscape
 3 elements of a soundscape (R. Murray Schafer):
o Keynote sounds:
 May not always be heard consciously
 Background noises
 E.g.: wind, water, forests, plains, birds, insects, animals, …
o Sound signals:
 Foreground sounds
 Listened to consciously
 E.g.: bells, whistles, horns, sirens, …
o Soundmarks:
 Unique to a particular area or moment
 4 characteristic principles of a soundscape (Barry Truax):
o Listener recognizability of source material is maintained
o Listener’s knowledge of environmental and psychological context of soundscape
material is encouraged to complete the network of meanings ascribed to the music
o Composer’s knowledge of environmental and psychological context of soundscape
material is allowed to influence the shape of the composition at every level
o Ideally, the work enhances our understanding of the world, and its influence carries
over into everyday perceptual habits
 Goal of soundscapes: what are the sounds in the world around us, what are their meanings
and how are they different from those of other cultures?

Sound design is not just about learning the techniques, it’s also about thinking about sound and
exploring different contexts

All copyright belongs to their respective owners

13