Musicology

Although research on synaesthesia has blossomed in the last decades, few things are known about
inherited sound-colour synaesthesia and processes underlying synaesthetic perception. In this study we
investigated how the auditory pitch, the frequency range, the complexity, as well as the structure of the
sound affected the synaesthetic visual attributes of chroma, colour brightness, colour plurality and density.
A distinct mapping was observed between sound features and visual characteristics, which suggests
that visual stimulation requires sensory auditory processing. We propose that congenital synaesthetic
perception involves two stages of processing: initially the sensory auditory information is processed by the
primary auditory cortex, and then it is distributed to the secondary and extrastriate visual brain areas,
generating visual stimulation. This process does not necessarily involve the participation of brain areas
responsible for associative thinking.

Synaesthesia is a condition, an involuntary process which occurs,
when a stimulus not only stimulates the appropriate sense, but also
stimulates another modality at the same time. In order to examine if
pitch and timbre influence the synaesthetic visual experience, induced
by sound, an experiment with sound-colour synaesthetes (N=22) was
conducted. It was found that a) high pitched sounds conclude to a
presence of hue, b) low pitched sounds to an absence of hue, c) single
frequencies cause a uni-colour sensation and d) multiple high pitched
frequencies induce a multi-colour sensation. Variation of chromatic
colour, which is present in the sensation, depends on the timbre of the
sound. These findings suggest that the synaesthetic mechanism (in
case of sound-colour synaesthesia) maps sound to visual sensations
depending on the mechanisms underlying temporal and spectral
auditory processing.

Synaesthesia is an involuntary process in which a stimulus not only stimulates the appropriate sense, but also stimulates another modality at the same time. This kind of joining can happen between all five senses, and can occur in any combination (e.g., sound-to-colour, grapheme-to-colour, smell-to-colour, etc.). In general, synaesthesia affects a minority of the population. However, there are synaesthetic-like phenomena which are labelled either as cross-modal correspondences, or as intermodal analogies, which are reported often by the common population. We are used to making descriptions such as “that has a bright sound,” or “this sound is dark”. Although brightness (bright/dark) is a property of light, we use it also to describe properties of sound.
This study focuses on sound-colour synaesthesia, and on intermodal audio-visual perception. This work sheds light on both phenomena (synaesthetic and synaesthetic-like), and its purpose is not to compare the phenomena, but to treat them as autonomic cases of audio-visual perception. The goal is to explore how sound, culture, and musical training affect audio-visual correspondences. Furthermore, it is important to explore how people justify these correspondences. An experiment with three main groups (cultural groups, musicians/non-musicians and synaesthetes with sound-colour synaesthesia) was conducted in order to examine: a) whether culture affects audio-visual correspondences, b) whether musical training affects audio-visual correspondences and c) the ways in which sound characteristics affect these correspondences and the resultant synaesthetic experience.
It was observed that three sound characteristics (pitch, structure and timbre) influence different aspects of audio-visual correspondences. Pitch influences the colour attributes chroma and brightness (sound-to-colour correspondences), sound structure influences the ways in which audio-visual analogies emerge (sound-to-patterns/textures correspondences), and timbre facilitates verbal judgments of sound (through adjectives). Additionally, it was found that different cultural groups (Germans, Chinese and South Americans) make similar audio-visual correspondences, though cognitive input accompanying these correspondences can be culture-specific. Furthermore, musical training does not affect these correspondences, but it does limit associative thinking and mental imagery.
In case of synaesthesia, it was observed that a) high-pitched sounds result in a chromatic experience (presence of hue), b) low-pitched sounds result in an achromatic experience (absence of hue), c) single frequencies cause a uni-colour sensation, and d) multiple high frequencies induce a multi-colour sensation. Variation of chromatic colour (e.g., blue, red, yellow, etc.), which is present in the sensation, depends on the timbre of sound. These findings suggest that the synaesthetic mechanism (in the case of sound-colour synaesthesia) maps sound onto visual sensations depending on the mechanisms underlying temporal and spectral auditory processing.
This study shows that intermodal correspondences are based, on the one hand, on the sensory characteristics of the stimulus, and on the other hand, on the cognitive information contained in the presented stimulus. The theory of associative sequence learning, and the theory of grounded cognition, may explain synaesthetic and synaesthetic-like phenomena: intermodal analogies could be caused by repetitive experiences in which perceptual and conceptual information is coupled. Specifically, in case of synaesthesia, both perceptual and conceptual information is combined into one perceptual experience.