Synaesthesia, a Neuroaesthetics Exhibition

This is a talk about a show I curated at the digital studio of the ICA (Institute of Contemporary Art in London) entitled “Synaesthesia, A Neuroaesthetics Exhibition”. I’d like to take this opportunity to talk about the concept for the show as well as some works included in the exhibition. I’d also like to touch upon this idea of taking something cognitive and placing it in the context of an art gallery. If nothing else, and as we saw this morning, a scientific approach to aesthetics sparks debate, and if nothing else, an aesthetic approach to science should raise a few eyebrows.

This conference aims to look at these spaces where neuroscience and aesthetics meet: be that in dj decks or in architectural forms and processes, in LSD-filled minds or in the computer of the animator of The Matrix, for example. It’s also interesting to examine how objective scientific stories can relate to subjective experience and visual or audio pleasure especially in the context of an art institution, where, as mentioned in the program notes of this conference, such stories show the position the brain takes.

The word synaesthesia is derived from the Greek syn, which means together and aesthesis, which means sensation, combining to describe the action or feeling of experiencing together. This definition can work and has worked as the legitimate point of device providing different plots and narratives some more literal than others, starting with the premise that the stimulus or event requires experience and recipients to make its essence complete. Now, neuroscientists have identified synaesthesia as a condition that occurs when the individual who receives a stimulus, for example sight, simultaneously receives a stimulus in a separate sense, for example audition, so a synaesthete may be able to hear colors or see sounds or taste textures. The existence of synaesthesia seems at odds with the common-sense world-view of five different senses channeling impressions onto our brains. The exhibition at the ICA aimed at exploring brain through different new media and digital art initiatives. It borrowed the concept of synaesthesia from the neuro-scientific world to look at works that allowed for such modality crossings but also traced back the concept to its original meaning as a joint experience, through works that engage the audience’s perceptual systems as a shared function. Why Neuroaesthetics? Well, at an existential level, neuroscientists and new media practitioners today very often share the same investigations. So issues and processes of consciousness, attention, memory, pattern recognition and mental imagery, as well as has been mentioned before, networks. Among others are topics pertinent to both disciplines. The brain itself acts as a subject or “ready-made” whose mystery and intricacy can provide infinite possibilities for artistic metaphor. This is then amplified by recent developments in neuro-imaging technologies, which not only allow for a clearer understanding of the function of different brain modules but also provide the visual representation of the different components of the brain, collectively responsible for consciousness and behavior. Now the functionality of the brain’s processing system acquires relevance in a new media dialog when someone considers contemporary technologies. So in digital media, the analysis of the information via link nodes across the Internet in the World Wide Web environments, for example, mirrors the transmission and the processing of the information by the brain’s neuro-network. Hypermedia tends to provide working and learning environments that parallel human thinking, that is, environments where users can make associations between what’s really important, to the effect that the brain’s processes and outputs on contemporary culture are the effect of contemporary invention derived by the wiring. The brain is not just a collection of neurocells and chemical associations between them, it’s also embedded in culture, whose products and processes can reshape the way cerebral neural networks are configured. So, this exhibition aimed to inform its audience of the concepts and processes shaped by digital and new media art and neuroscience, emerging from the common interest in cerebral processes of perception. A synaesthetic paradigm uniting the disciplines of neuroscience and aesthetics didn’t aim to be a pleasing demonstration of the clinical condition of synaesthesia, but it aimed to work as a metaphor pointing to the particularly idiosyncratic nature of the media practice, which is the breaking down of genres and categories. It allowed for artists from different disciplines as well as scientists engaged with the nervous system as a communication device, while providing the audience with the new interface with the internal and the external world. So the artists in this exhibition play with the processes and outputs of such media as film, photography, sound, or the visual presentation and installation, to investigate how our senses are interrelated and how our brain separates and interconnects these processes and sensations and thoughts– the physicality with which the brain separates and integrates these processes and sensations and thoughts. So the physicality of the brain was the starting point while its dynamic physical and invisible processes provided inspiration and gave rise to diverse and variable experiences both between as well as within different people.

So, I’m just gonna talk about different works. For this piece, electronic musician and media artist Stephen Vittielo proposed the question “What does light sound like?” He used a small light sensor to translate light frequencies into sound and also presented these lights visually, by pointing a thing called a photocell, which is something used by photographers to measure the light around their subjects. So, by pointing this to different light sources and feeding it through the computer, light, color and speed were translated into tone-pitch duration, producing unique and mesmerizing light effects.

Warren participated in this exhibition with some of his video and work. For the work From Your Ears to my Lips (and this isn’t actually the actual image I was to show though) the work that was included was conceived as a part of a series of images that Warren created inspired by the condition of prosopagnosia. Prosopagnosia involves the loss of the ability to recognize familiar faces, even faces of spouses or children or even one’s own face and it’s usually caused by some brain injury or brain damage. These images were inspired by stimuli psychologists used to test this problem. So, what they do is that  they first take pictures of famous people and then they manipulate them in various ways, take images that pulsate between a face and an abstract form, and that’s done to test the extent of the condition of prosopagnosia. So these images were made with touch. A small flashlight was drawn across the face of an individual in the dark. It took about one to three minutes to be created, wherein light touched different areas of the face such as the lips, the ears or the skull. So, over time, the image that emerges linked the senses of touch and sight in a synaesthetic embrace. For his video work Kiss, Warren taped a digital camera onto a blind man’s cane, blindfolded himself, and then outlined the surface of an old card by keeping pressure on the end of the stake. It’s important to note at this point that brain mapping studies have shown that blind people use the information provided by their blind cane and this cane becomes part of the somatosensory representation or map found in the pre and post central gyrus of the brain. The cane becomes, a part of the body, so as an extension of the body and its vital information to access space it also becomes part of the somato-sensory and motor maps. Now the viewer and the performer in this piece received very different sensorial information for the perception of the same object. The entire video which is over six minutes in the end produces a the perception of an entire car which takes the entire to unfold from the pieces of small abstract spaces that come in contact with the blind mans cane over time.  The artist as performer organizes the senses of touch and sound to find his way around the car, which emerges as a result of his contact with it. For his video work Conversation Mapping, and again this is a photograph on the screen from the same process, five deaf people who have lights attached to their fingers tald about the daily events in their lives and what emerged was a symphonic language organism that can mutate, glowing and growing through time.

The Neurophone. The Neurophone was an audio piece created by Dr. Sonja Grun, who is a professor and researcher of theoretical neuroscience at the Institute of Neurobiology at the Free University of Berlin. It’s worth giving a bit of background information about this, and perhaps some background with regards to neurons– basically a nervous system is an electrochemical system that receives information from the environment via the five different senses, organizes this information, organizing it with already stored data and then sends directions to our muscles and our glands to produce organized movements and secretions and thus that’s the basis for conscious experience. Electrical and chemical interactors between the brain’s constituent elements, the neurons, form the building blocks of computational processes. When we are influenced by something, be that light or sound or pressure, it emits a so-called “action potential,” an electrical charge. Neuroscientists measured this as a way to create functional maps of the brain, that means, what sort of stimuli are more sensitive and what neurons are sensitive and respond to what kinds of stimuli. So, these neurons are usually organized in functional groups, which means that neurons that are more sensitive to sound, for example, will tend to be organized together. The coordinated activity can be recorded through different means and is usually displayed in visual graphs. Now Dr. Sonja Grun has created an imaginative and creative way to visualize the difference in timing of neuronal activity. She recorded multiple neurons in parallel and assigned to each one a separate voice by giving it a different tone.  The result is a piece of neuromusic, in which each neuron contributed with its tone each time it emitted this action potential. The idea is to audiolize neuronal activity by presenting its simultaneous activity in a form compared to music.

Moving from sound to film now, another piece that dealt with the nervous system, although in a completely different manner, was Flo Rounds a Corner by Ken Jacobs, who is a pioneer figure in the history of the avant-garde film. Jacobs has been experimenting in the cinema since the sixties with his exploration of The Mechanics of the Moving Image. His nervous system demonstrations involved a technique, where basically he puts two identical filmstrips on the two projectors capable of single frame advance and freeze, and he operates these projectors effectively like a musician, allowing him to move on slightly out of synch and therefore causing a semblance of motion and very strange three-dimensional space illusions. Time appears constant yet audiences are left very disoriented with a sense of depth through space. So in this piece, the audience’s perceptual system and perception of oneself are shifted.

Fred Worden is another artist working with the medium of film. Unlike Jacobs, he is not interested in projection techniques, but in the medium of film itself. Obviously referencing Stan Brakhage, he interferes with the filmstrip by manipulating it in various ways. Spaces are manipulated through powerful abstract forms and motion, which take on the biometaphor of brain itself. So this cinematic abstraction teases and tests the eye-brain system and viewers are not sure whether the video originates on screen or in their heads, as it were.

A piece that rendered the ICA’s digital studio a more interactive space was Nina Sobell’s Thinking of You. Sobell has been developing her brain wave drawings since the seventies. For this piece, she wanted to draw upon the social nature of the brain’s electro-chemistry as it revealed a deeper understanding of an eternal dialogue. So, visitors in the digital studio were wired up with electrons around their head that measured their brain-wave activation, while web cams monitored their faces. They were able to observe and manipulate the change of mental alertness by watching the brain’s activation change in real time. And we were also able to compare this activation with other people wired up in the studio. There was also an audio component and interpretation to this piece.  Waves with larger amplitudes and moderate frequencies suggested that the brain was at a more relaxed state and produced low-pitched sounds, whereas high frequencies, which suggested that the participants were active and engaged in some sort of activity, produced high tech sounds. So, the technological advances of the past years have transformed our world in a way that we seem to be reaching sensorial information from all. I’d like to end this presentation with a question that was posed by Dr. Daniel Glaser at the talk he gave at this exhibition: What can objective scientific empirical data tell us about subjective experience or pleasure? There is a clear, at least motivational, difference between placing art in a neuroscience lab and placing the brain in an art gallery context. As a curatorial statement, its procedures and results run into a state of objective study and evaluation.