| CONCEPT & DESIGN | |
| Kathi Rüll, kathiruell.com |
|
| SUPPORT & CARE | |
| CODE | Leo Hilsheimer, Haram Choi |
| MUSIC | Nils Eberhard, Immanuel Eißler |
| SCENOGRAPHY | Jannik Lang, Janina Capelle, Kamilla Murtzina |
| DESIGN & CONCEPT | James Langdon, Ivan Weiss, Michael Kryenbühl |
| TECHNICAL | HfG Karlsruhe |
Color changes how we feel. Sound transforms what we see. Smell determines what we taste.
Synesthesia — the experience of one sense evoking another — is more common than we realize. While Performing music and designing visual we need to create the overlaps and intersections of the senses.
Hearing colors, also known as chromesthesia, is one of the forms of "involuntary parallel sensations." Thereby, it is understood as being the sensation of colors as experienced by means of hearing acoustic tones and sounds. It involves subjective color-sound relationships. Corresponding to the aforementioned characteristics, these phenomena appear involuntarily among synesthetes. Every individual exhibits a unique scheme of allocation which remains constant and independent of the context of the perception throughout his or her entire life. Besides synesthetes, most people are able to spontaneously assign colors to an auditory event. They have corresponding analogy relations. However, this connection is not absolute. Instead, it experiences modifications depending on the perceptual situation.
The absolute connection of color perceptions with a certain tone or sound attribute, in the case of a color hearer, is comparable to the capabilities of hearing with perfect pitch. The tone pitch can be regarded as a visual-spatial attribute within the framework of cross-modal analogies. As a result, color perceptions arise among some people as a consequence of certain tone pitches and among others due to certain timbres, intervals, temporal structures, or other auditory attributes.
Chromesthesia could teach us to listen more detailed and look more closely to our environment. If sharpening our senses while listening to something auditive, people could realize that another's viewpoint, is equally true and real.
Scientists and artists have always been researching color scales. When Isaac Newton projected light through a prism and discovered the color spectrum around 1665, he chose to slice the rainbow into seven basic hues, believing that color must be structured like the musical scale. Newton was so convinced that color and music were analogous, he added indigo to his color wheel to make his system a better match for music.
In the seventeenth century, Aristotle already proposed a scale according to color brightness. This color scale made reference to seven equally spaced musical intervals - thus, not to tones, but to distances between the tones.
At the latest, since the experiments of Louis-Bertrand Castel in the mid-eighteenth century, the classification of sound and color has been the subject of various research studies and artistic experiments. It is also obvious, however, that there is no conclusive, nearby solution which would exclusively orient itself with respect to the color tone and also be accepted by a majority of people. Indeed, test subjects usually tend to classify colors on the basis of the color brightness or saturation.
This is based upon the finding that typical colors exhibit a characteristic brightness, which appears low in the case of blue, but with higher values in the case of yellow or white. An appropriate color scale approximately leads from black to white via blue, red, orange, and yellow.
VIDI uses three different color schemes that can be selected before playing. These scales vary between light, soft colors and dark, strong colors. Within the color schemes, the intervals of the played tones are calculated and, depending on the musical harmony, similar or contrasting colors are shown to each other.
When we describe our visual and auditory perceptions through language, we often make use of adjectives and attributes that do not come from the practice we are describing. Examples of this in German are pitch [Tonhöhe] or timbre [Klangfarbe].
Sound primes us to see. When a dog snarls or brakes screech, images rush to our minds. Musicians and audio engineers call a sound “bright” or “dull” depending on its overall frequency. The clash of a cymbal is considered wide and flat, while the voice of a flute is bright and sharp. Such terms assign visual and tactile qualities to sound.4 People with synesthesia link certain sensations in a consistent or systematic manner — as do people with more ordinary sensory responses. Metaphors such as “piercing cry,” “soft whisper,” or “coarse language” assign tactile characteristics to sounds.
„The colors of music are concrete and vivid – most spoken voices and individual sounds are as elusive as smoke.“
„I could tell that I perceive colors as tasting like toothpaste or biting like acid“
In order to describe sensations as elements which give rise to complex models of the perceived, these must be designated as perceptual characteristics or qualities.
By using non-restrictive language or open-ended attributes, VIDI can keep the playing field more open and creative for musicians and performers, rather than having to choose a specific genre.
Design* is often evaluated as qualitative if it is reduced and timeless. In contrast to music, there are few examples in design where the design gains quality through the emotionality shown by the designer. Most design is caged by the image. We “look” at design, we don't “feel,” “experience,” or “sense” it. In fact, most design is non-sense design — cold, technical, formal, and inhuman — engineered to serve business or technical functions rather than to surprise, inspire, and delight. We have allowed two of our sensory domains — sight and sound — to dominate our design imagination. In fact, when it comes to the culture of architecture and design, we create and produce almost exclusively for one sense— the visual.
The creation of design is also often practiced in a very linear and goal-oriented manner. From the first sketch and subsequent drafts often follows a result to the conclusion. What about the process, intuition and improvisation? Also, the collaboration of designers is often rather limited to the individual. We don't show off our designs but brag about our final products. Even the software we use to design is tailored to individuals. Working together on a file still remains unresolved. We sit in front of our computers and engage in painstaking pixel-shifting when editing moving images.
VIDI thinks about a more immediate practice of design and puts the moment of creation and perception at eye level. We think that it's good for graphic designers to be inspired by sources that come from somewhere else.
We link music, as well as colors very strongly with our perceptions and memories. A song can change our feelings and a color can influence us. The musical long-term memory can be seen as a mentally represented "music library".
Certain colors can also trigger negative as well as positive feelings in us. We describe colors as "friendly" or "aggressive" and thus interpret human feelings into them.
Neuroscience tells us that we are emotional decision makers, not rational spreadsheets. Most of our decisions are made live — in real time without our awareness. Let's use the language of passion to translate the intellectual into the emotional.
The gesture of the analog has changed enormously to the gesture of the digital, especially for music. Digital music lacks the tangible resonance of analog instruments. The vibration of analog sound production is diminishing and we observe musicians pressing keys in front of computers.
VIDI uses the computer as a visual and musical instrument and supports the gesture of playing music. Midi signals are recognized by VIDI and translated into moving color dots.
Design may well learn from music. Improvising and jamming can function between musicians like a self-developed language. The trial and error and unplanned creative process creates great moments of exchange.
When using VIDI, the visual output can certainly influence musical development and it gives the opportunity to be inspired during creation or improvisation.
Immediate Media: Most of the media we consume is fast and works anywhere, anytime. One might think that we consume design live, yet most of the design we absorb is staged and planned to the second. Perceiving visual design live, the second it is created, it perhaps holds new possibilities?
We need to design the design process to open the medium for full immersive experience. Live experience has the bandwidth of reality — it is fully immersive, and the visitor's body and mind are alive and available for full engagement of all senses.
Movement shows speed, spatial presence and dynamics. While music works intuitively with movement in the process of creation and the moment of perception, design often has a hard time with it. Often the only movement that design demands is the direction of the gaze on what is designed. Can't the visual challenge us in our physical movement just like music? Let's design for our sense of movement. After all, our visitors are not static viewpoints.
VIDI plays with the commonalities of visual and musical practice: movement, time & space
Music is the consciously designed, temporally structured ordering of acoustic events in social contexts. Music is therefore, along with language, a second human-specific, intraspecific phonetic communication system that establishes social bonds and generates emotions. Music perception is based on a complex interplay of processing melodic, temporal, harmonic, and dynamic structures.
When we speak of music theory, strictly speaking, we are not talking about a theory, but more about a fixed system of order. This system helps us to distinguish and explain harmonic and dissonant sounds.
VIDI uses the approaches of the tone relationships to each other to assign color contrasts to each other.
In multisensory connections, the particular proximity of spatial-visual and spectral-auditory attributes is striking, for example, the preferential perceptual connection between visual height and auditory pitch. The perception of sound is more strongly influenced by the brightness and color saturation of simultaneously presented visual stimuli than by hue itself. Unpleasant or frightening sounds exert less influence when accompanied by brightness and saturation.
Researchers sought to discover possible systematic relationships between sound qualities and visual forms. Again, no self-evident solutions can be found that would seem intuitive to many people.
Music notation is an important, traditional application of the analogy between pitch and visual form. In currently established notation, pitch corresponds to the spatial arrangement of notes on the vertical axis, while time corresponds to movement on the horizontal axis. The transformation of time corresponds to the direction of writing from left to right. Efforts to translate auditory perceptual events into the visual sphere are often based on this scheme.
VIDI uses the translation of tonal qualities, such as volume, pitch, and type of play, into visual analogs such as scalability, saturation, and visual appearance and disappearance.
Multi-sensorial Design: human communication constantly mixes sound and vision.Typography is multisensory — it is audio and visual, sound and symbol. Speech is haptic and gestural as well as aural, felt in the muscles of the face, throat, and mouth. Writing is gesture. The sign languages created by deaf communities are fully functioning linguistic systems, designed for seeing, not hearing. Studying sound through visual or tactile means is a powerful method of invention. Visualization offers a rich path for experiencing sound beyond the audible. Graphic interpretations of sound illuminate patterns and structure and generate new memories and associations. Communication that bridges our different senses helps connect us to each other and to the physical world.
The ability to understand and synthesize complex, diverse inputs across many disciplines into one compelling, immersive experience is the magic and meaning of design.
If we are to develop an all-senses design method, we will need to consciously focus on the senses and build a process of synthesis. Design for each of the senses; then synthesize. Think of each sense as a track in a complex polyphonic composition.
Using VIDI not only gives the possibility to create a synthesis between auditive and visual, it also proposes to use the generated visual as a projected light into the spacial. The synthesis is even more evident through the audio and light waves carried over the air.
Music visualizations are commonplace at clubs or concerts, where screen graphics pulse with frantic energy. Within the framework of multimedia applications, it is interesting to create visual structures which correspond as plausibly as possible to given sounds and music passages.
Today, lighting, film and computers offer various possibilities of realization. Simultaneously, the video provides broad possibilities of visualization - ranging from the presentation of moving abstract forms to the interpretation of the music in specific scenes and storylines. Thereby, nonetheless, uncommitted approaches are preferred, instead of seeking to establish a theoretical foundation of the multimedia production and to systematically assigning colors and forms to certain elements of the music.
In the course of scientific and industrial development, there was increased interest in visualizing music. There are numerous examples of audio-visual connections as designed for the arts and music. The technical possibilities of artistic configuration have been significantly expanded by virtue of lasers, cybernetics, electronics, and computers. A new dimension was reached with the vast possibilities of interactivity, thus blurring the traditional border between the active, formative artist and the inactive, simply perceiving recipient. While the technical realization of multimedia applications was increasingly made possible, the readiness to determine systematic assignments between tones, sounds and colors, however, has declined over the course of the twentieth century. Systematic concepts have made way for greater creative freedom and emotionality.
VIDI gives smaller artists, who can't afford a light show, the possibility to generate visual imagery personalized to their music and creates a contrast to prefabricated VJ settings.
RESEARCH BASED ON