Alloaesthetics and Neuroaesthetics: Travels through Phenomenology and Neurophysiology

I want to cover lots and lots of things, and of course there isn’t enough time for everything so what I’m going to do is do it a little bit backwards, meaning I’m going to show you the things that I want to conclude with as fast as I can and then, as time allows, try to give you the context for them and then the sort of the theoretical construct that they fit in. But I do want to touch upon the liquid architecture thing, just a bit. So this is the title that’s in the conference proceedings “alloaesthetics and Neuroaesthetics, phenomenology and neurophysiology”, and it’s all quite right, but since Paul [Miller] was riffing off this thing and was using liquid architectures I thought I could take his book and play with it.

So there is this exotic product called culture, and I thought that maybe I should be speaking about shuttered science, not some rhythmic/arrhythmic culture and that would fit quite well. In some sense, what I’m speaking about is about Neuroaesthetics but it’s also about this trajectory from liquid architectures to something I’m calling transvergence, that I may or may not be able to unpack for you, but I would say that the whole topic of the conference, the whole mixture that Warren [Neidich] is providing for us, this bringing together of art and neurophysiology, is actually an instance of something that’s happening culturally, and of course Warren has caught it and brought it to us to hear of this mixture of things not previously conjoined and that are another beast and you can see that you can see that in 5, 10, 15 years there’s going to be in some place a department of Neuroaesthetics that will probably get transformed into something else, and that sense of the evolution of everything, including disciplines, seems to be the condition we built, and that’s kind of the armature within which this is happening. This little beast, I hope it’s quasi legible, is a relatively quick unpacking of what the liquid architectural reference that Paul made applies to, which stands for the kind of architecture kind of poetics and observation of culture, something about media that bring things together like liquid architecture and music, a critique of space and time, a separate being brought together into space-time and the pending observation about the liquid and the architectonic in the matrix of the worlds. So DNA is actually an architecture that is liquid, life itself has this balance of agility and fluidity and it points to a great big mess of things that we think of dualistically, but we really can’t think of dualistically, which I call the “holy grail,” but I don’t mean it religiously. I mean it probably more like Monty Python means it. Zero and one, nothingness and real, waves and particles, and all this analog and digital inventory, but there’s a whole list of things that you will try to keep separate but actually fit together and if we can’t fit them in our head then it’s our problem. Quanta do what they do, they don’t listen to us whether we understand them or not, they are indifferent to that. So anyway, we may or may not return to liquid architecture.

The thing that I really want to talk about is this thing called “transvergent beauty,” alloaesthetics and Neuroaesthetics: that would be a sort of re-spinnng of my own title into what I am going to try to describe to you. As I said, I’m going to proceed backwards, give you a kind of example of one of the things I’m working on that applies to the conference and then, as time allows, I will return to transvergence beauty, examples of pretty pictures we all like to see in a talk. I’m calling it for now “alloneuro” and you can see that fMRI, and indeed with the collaboration of people of the UCLA Brainmapping Center looking at brains I had wanted very much to have that in my brain up there, but timing disallowed that, Mark Cohen, who is in charge of the machine was in Florida and I was at the other part of the world, we couldn’t do it, silly reasons.  But we’re working on this kind of stuff. You see the canonical view of sections of the brain and activations and I’ll show you how some of this is done. Here’s the close-up. These kinds of things signify our increase of blood flows in the brain, and I’ll show you the live program because it does some things. Now, you can take the data and if you are inclined to make liquid architecture, strange things from unlike resources, you can begin to use tools for specializing various kinds of scientific data and you can make an object and then you can start thinking of it as a kind of a fantastic voyage and begin traveling inside the brain, and then you can just remove the brain eventually and then see the caverns and spaces that are in there if you see them, which turn out to be quite surprising for neurophysiologists, it just hasn’t occurred to them that it might be fun to go in there. It might be, actually, more than fun; it might be informative. Then you can strip the brain off and just leave the activation and you got these structures and space that can be more as you wished them to be in terms of their contents and space and they can become more and more world like and more and more, you know, the kind of thing you want to explore in a fantastic journey. That’s interesting, but it happens to coincide with this other thing that is called the “allosphere”– we’ll be hearing this prefix allo repeatedly. That’s the current little prefix I have adopted after trans. It means, I should say it now, it means the other in a very deep sense, and it’s the root of word alien the root of word else and the root of the word alternative and all of those things.

So I’m at the UCSB, the University of California, Santa Barbara and I’m part of three things: something called media art and technology, something called art, and then this other beast which is called the Californian Nanosystems Institute, which is for nanotechnology, and we’re building a new building that is going to contain this little beast, which is a three-story high sphere. It’s actually 2 hemispheres plus a cylinder for those who are precise in the description of things, that is made for projection in the round, full, immersive projection of stereoscopic imagery and what’s called “eyelimiting resolution” meaning there are so many pixels that your eye can no longer discern pixels and it just doesn’t care about pixels and then surrounding this thing, which is actually made of perforated aluminum, it is in a array of a 144 or more or maybe a few less separate channels of sound instantiating something called wave feel synthesis, which produces a virtual sound source of every cubic meter in the sphere, which if you do the multiplication with the body and the sphere you’ll find that it’s something like 137 or 138,000 sound sources in the space. And then there’s a bridge and people are on the bridge and on that bridge are going to be various kinds of haptic devices, of course feedback devices, censors and other kinds of things. Basically, it is a pretty nice machine for setting reality aside. Now, this is a little bit of the description of what actually goes on in there, there’s a kind of chain of command in there which is: there is a person using some interfaces sending streams of data to various kinds of clusters of computers that detect, through motion capture or video processing, what the person is doing, feed that information to a scientific visualization cluster that, unlike normal ones, is working on constantly dynamic data, sends that off to another group that does something called world making adding sound and any other processing compositing different kinds of things and then sends that yet to another piece that is driving all the different projectors and speakers and the rest of it, and then finally it all comes back and creates a sense of reality for whoever is in here, and that reality is made of grains, which are grains of video or sound or feedback or sensing, so it’s a kind of environment made of virtual, let’s say, atoms or virtual molecules, and that allows the things that form in it, unlike if were made of polygons and such, it allows everything to be completely metamorphic and this device is to be an instrument in two senses, an instrument for science, for bringing people to things like nanotechnology, or computer science or quantum physics, we’ve got the Institute of Theoretical Physics next to us or as a musical instrument or an instrument for liquid architectures or archimusic or new forms of expression that require art and science to be fused together. And so, in this thing what I’ve been working on doing, among various kinds of projects, is constructing a feedback loop whereby the subject who is in the sphere is witnessing a series of worlds, that are made from abstract n-dimensional data, that drives the construction of algorithms and generate a reality, and then that data is actually in a loop, so that the data creates the world and the world creates the stimulus that affects the brain of the person that sends the data to create the world again. So there’s a curious kind of circle that happens here. Now I’m going to show you this very quickly but I will show you something a lot less awkward that does this kind of thing in the conventional view of how the brain is looked at with fMRI and what I’m working on doing with it in this allosphere environment, which is this: normally with the subject you put inside the machine, there’s a sort of prearrangement of what stimulus is going to be shown to you and at what timing– so if you are going to be presented with sounds and they are going to be spaced two seconds apart, something like that, or something is going to play music or show you pictures of whatever. So there’s a prearranged stimulus and a prearranged time fed into the subject. There is a scan, which detects noise so you can tell what is actually happening in the brain, which is noise, which is not, a fairly complex amount of statistics trying to clear up the noise and to correlate which of these things actually correspond to something that the brain just did. From that you finally have the kind of picture that you saw, that’s the normal thing.

Now, I was at a conference on consciousness and some of these things go back quite a few years, but I was at a conference on consciousness observing all the neurophysiologists talking about the neural correlates of consciousness, ok? What is the brain doing when we’ve got consciousness, and it was occurring to me that it was a little bit like saying at a certain time of day lots of phone calls go from London to Paris because, you know, that’s the time of day when people are done with one thing here and beginning something else there. Basically you can watch a network working without ever knowing what’s being said, right? And if you wanted to find something out, you would somehow had to fuse neurophysiology, the activation of the network, with phenomenology, which would be the lived experience of what the network is doing, you have to observe the two of them at once.

In the case of what I am trying to do, as I alluded to it in the title but haven’t completely been explained yet, I’m interested in the question of, really the question of aesthetics which is the question of beauty, we’ve had various kinds of questions spoken about, but that just makes the issue of aesthetics even harder than just aesthetics, as if we all know what that is. I know for me, beauty is an important thing and I want to know more about it and so if I wanted to use this tool to investigate it I would have to find a way of watching the brain work while having the corresponding experience of what is producing the effect of meaning, or as a maker, what it is that I am doing when I am making something that is considered to be beautiful and watch what happens when I register ok, I got it. Now what does that mean, what is the physiological correlate of that? And so, in this instance, you have a different kind of arrangement of this kind of experiment, where the data of the stimulus is variable and is generated by the subject. The timing is variable and generated by the subject, it still gets planned, it still gets correlated, but now it’s getting back information that is actually the subject of him or herself saying what just happened. So, and what that is, is that basically I’m proposing to put myself, or anyone who is an artist, in the machine with their artwork. Making it or responding to it and watching in one view watching what their brain is doing as a kind of ok, that’s just laid up and on the other hand being subject to the stimulus of the artwork and saying “ok, I am going to have both of these things” and then, because, you know, happily, I do things with programs, actually having the artwork be in a form that can be objectively examined. So, it’s not about interpretation, and it’s not about mimesis, it’s not about association and it’s not about all these kinds of ineffable things, it is actually all right there. Then, it’s in a form in there, which is corresponding to the form that the brain is in in this analytical machine, so you can actually use the same tools of analysis for both. So, let me try to show you a few of these things, very very quickly, I may have to shift out of this illusion and I’ll come back to it. First just to make real this building, and it’s actually quite a bit further along, let me just show you what it is. Ok, that’s where we are, that will do for now.

[Showing slide] A little memory of Boullee and the Cenotaph for Newton, which is appropriate because it’s Newton as well, after Einstein.

That’s the sphere, you just saw a section of it, you’ll see the plan, quickly, there’s the support lab and so on. It’s in a nanotechnology building so there are lots of other labs that have to do with nanotechnology. So this is a picture of another virtual environment of mine that had to do with a conference on psychology and psychiatry, a group of psychologists and psychiatrists interested in the spaces in the mind. So, it’s an indication that this is about virtual environments but not representational ones or at least not representational ones of the outside world, but of the inner world– this is an esoscope. That block in the corner is where the sphere goes, it fills that entire corner, the support labs are in that wing, and this wing is full of molecular samplers and wet labs and centrifuges and all these things. This little space within which the sphere will fit is a catwalk that goes around it and you’ll see the bridge in a moment. So, you got the sense of the scale of this, this thing is an apparatus, that’s one little thing I wanted to show you. The other thing I wanted to show you was what the actual software that does this kind of work looks like and what the standard representations look like. So this is someone’s brain, you can see there’s a cursor, as I move the cursor in one view you can see that the corresponding sections adjudge, so basically I have all the informational data of this person’s brain right here, and I can see it in any one of these canonical views, or I can see all the slices at once and then for each voxel, each volume element, what I have is a signal, that is, there may be 200 signals or I set the sampler every 2 seconds or there may be 100 points or something like that and I can move this through, through here, and see what happened at a particular time and if I have loaded the right file, you can see now, if I hold the right button you can see that I can pick a slice and that’s what’s actually happening in this person’s brain with the noise and, by noise, by the way, noise might be noise from the scanning, or noise might be noise because the person has other thoughts in their mind, you know, if I’m speaking to you or listening to you at the same time you are thinking, you know, I want to go roller-skating. And there is another part of your brain, that’s roller-skating so that has to be factored out. Now, what I’ve been suggesting is that I want to construct a situation, whereby I can compare, in an objective sense, something subjective with something that I’ve created subjectively, artistically with what’s actually being measured here. What you see from this is that basically in some sense this is a set of frames or it’s like a movie, it’s three-dimensional it’s not two, but basically it’s a set of frames, you know, in the end, it’s like comparing one movie to another, pixels are pixels, voxels are voxels.

The next thing I want to show you is a little program such as the one that I’ve been using, and I’m saying “such” because it’s an evolving thing. I want you to pay attention, maybe afterwards you can tell me what you think. So, this is a little program that I’ve written that creates images. It’s kind of like a glorified screensaver if you needed to classify or somehow pigeonhole it you can call it that. So let me get it over the entire screen. Now, this thing does strange things. In my eyes, it does things that are interesting, most of the time, and frequently enough, things that I find quite beautiful.  I don’t know exactly when it’s going to do them, I don’t know exactly what it’s going to do, at what moment it’s going to do them, but I do know that I find myself recognizing color combinations, form combinations that mean something to me. What’s going to be happening in this sort of experiment I’m talking about is that I’ll be at the machine and we’re watching this and when I recognize something, as some of you might (we won’t let this run longer, it will just take over and it will kind of swirl itself), I would hit the button, and when I hit the button a timestamp will be made, a correlating moment in my brain would be identified, and all the data that this is made of would be made and this itself as data in, you know, in a similar kind of language of pixels that the brain is made of would be available. So then, as I correlate the noise out of the brain, I can correlate the noise out of the picture and find out what data in here corresponds to some activation that happened in my brain at a certain moment, when I, as, you know, the aesthetic measure, said “this is what I mean by beauty.” Now in a first pass through this I would do it passively, in the second pass I would do it actively, so there would be switches in the machine. I’m still in the machine but I’m playing with this, and so I’m trying to discriminate in between the passive sort of consumption of beauty from the production of it. What is it to say that I’m making something that is beautiful? Where in the brain does that originate? And, as you see, I’m going to try to now.

Let’s see if I can do it in five minutes, the rest of it. Well, I guess I do have to make a few points; I’ll see if I can make them faster. The context is something like this: in 2000 I was in the Biennale representing Greece, in 2004 just this past summer I was again in the Biennale. In 2000, I showed work that had to do with invisible architectures, which seemed absurd for architecture, and in 2004 I showed this other thing called allobio, which built upon all the kinds of things that I might have been doing including liquid architectures but added to it a nanoeffect, bioeffect, new materials, something called new atomism, neurophysiology and phenomenology, all these kinds of things and then in the end the idea that you could grow and not just build a building, you got a new species. And I was asked to submit this project, and then when I went there, I discovered that it was at the very end of the Arsenale, which was a very very long set of rooms, three very long rooms, a great axial and focal kind of arrangement, and I was at the very end, like the last word. I was curated onto the sort of hotspot. And it was a very strange thing, because what I’ve done is I’ve taken models from mathematics and biology and fused them to make these things, these allobiological things and eventually proposed, you know, this thing is a building. Instead of being called crazy, this is a recurring thing you see, instead of being called crazy someone said well, here we’ll put you at this spot. The thing is that this stuff had been done before I ever had to do with actual scientists that could do it. I mean I know some things about this, but I am not the person in molecular science or in the particle accelerator. But, I’ve had the thing as a prototype, and there is something about proposing something as strange as you can make it, and then finding it just sort of firmly having a grip on the reality that you might have thought would have rejected it. So this is where the transvergence thing comes in. This is where the old virtual stuff and this is where we are. And this is where I want to get to; this is this association of the allo.

So this is the situation we’re in. We’re all under this Moore’s Law which means that once upon a time we had a lot of time to observe a little bit of change. Now we have almost no time to observe a lot of change. And that means things like Neuroaesthetics happen, yeah, that’s what it means. It means that you have a choice (and it has to be conscious at this point) of whether or not you follow a given path or when a certain basis of situations arises, which, you know, in ancient history it was called a digital, but we’re past that now, you either along a conventional line or continue a line that is perhaps new to you but still conventional, so I won’t explain all of this completely, or you choose to do something else. If you know what light cones are, you’ll understand this reference, if not, we can talk about this later. But basically, it’s a thought process about the strategy of derailing oneself and pushing oneself outside the envelope of the known into something that is not convergence, not divergence, but it’s sort of pushing oneself into elsewhere. It’s an anticipation. This is where the rhythmic culture can come in. It’s an anticipation of what’s evident now again digital, we have the nano, we have the bio, we have the neuro, we have the quanto, we have waves of these things coming, and some of us have to choose not to always do what’s expected but to always do what’s not expected. So in each of these strategic points, we derail ourselves into this alternative path. Now it actually ties into this business about the aesthetics as we’ve been hearing, because it prohibits mimesis. It prohibits all those kinds of explanations that have to do with knowing what’s coming because it’s actually seeking what you don’t know, but I would posit that whatever the aesthetic is it is something that can deal with radical novelty, and in fact does so all the time. This little business is about speciation and how you can speciate various things, very quickly; I think you can do it in 30 seconds. You can gather existing species– we know there’s mutations, most mutations die; some mutations become regressed to the mean; some mutations are stable, but sterile, and they become hybrids; they’re the mule, let’s say, and a few remain viable and then eventually become incompatible with their own kind and become a new species. That’s how you grow diversity. I would say that I said that the title of this thing is the alloaesthetic, right? So I’m interested in a new species of sense, a new species of aesthetics, a new species of words, a new species of disciplines and how it is that in view of a cultural and global situation, where this thing is the condition and not the exception, how do we actually embrace that creatively and not retreat to my fellow Greek Plato and Aristotle, brilliant as they are, who are not exactly applicable to what we are doing. This is about, you can deal with that link and have some fun, but evolution proceeds without preconception. It makes viable things, it makes rhinoceri and such creatures, and if these things survive, they’re ok, and if they die, they die, and life goes on. So, it’s a different way of addressing the world. In it, there is a sense of something objective in beauty and if there are new senses, then there is a transvergent beauty and an alloaesthetics. I’ll just keep on showing slides, now, I won’t read these things, but the important thing about these things is that, especially these two, this is Saint Thomas Aquinas, you wouldn’t think that a Saint would say this, but he really comes the conclusion that the sense is a sort of reason. So, the whole business about trivializing and diminishing and relativizing beauty is thrown out of the window, he says it’s actually a cognitive power.

About Neuroaesthetics. It can be that art helps investigate the brain; it can be that the brain helps investigate art. In all these things the sort of strongly positive position is that there is meaning before language, meaning before taxonomy, meaning before discourse, that beauty is this multi-modal, formalism I mentioned yesterday, is actually a very, very deep thing– the mind and the body are not separate and the whole thing is about not being mimetic. I won’t mention the references, there are many many people that apply, but I guess I’ll take it to the word beauty and end with this one. But this is a kind of typical treatment that beauty is subjective, it’s relative, so it’s taste, it’s trivialized, so it’s whimsy, it’s canonified, so it’s fashion. We seem to forget that people fight wars over it, that they spend massive resources over it, make life decisions over it, so there’s something there. The alternative is that somehow it is objective and then you have to wonder where it is. There are theoretical approaches like Aquinas and Alberti that say specific things about where it might be located, and in our time there are empirical ways of approaching it, and then you can see that perhaps what we called beautiful is in the object, but then that doesn’t answer enough, or perhaps it is in the brain, but then that doesn’t answer enough because it doesn’t say enough about the object, or there might be a relation to the premise that I’ve been showing to you that beauty is neither in the thing by itself nor in the brain by itself. It’s a common feature among humans that some things fire up a certain relationship in the brain and when that relationship is fired up the person says “ah, beautiful”. Now, one person might see a lizard and say that it is beautiful; another one might see a sunset and say that it is beautiful. But both will say something of this sort, they will both report on the relationship. And there are various kinds of experiments showing that it’s very very deeply embedded, and the one that I wanted to mention today is one, where they ask babies, they had newborns, and there are six or sixty newborns or something, and of course the babies had a corresponding number of mothers of a certain age, and then they found a group of women of the same ages as the mothers, who were separately tested among some bigger pool of women to be beautiful, some group of people decided that these other women were beautiful, more beautiful than the mothers. I don’t know how they did that, but this is what they did. And then they presented the babies, the newborns, with all these women and they didn’t, of course, tell the newborns who the mothers were. And it turned out that- and this is a curious thing- the newborns had a preference for the women that had been considered beautiful. Now, I don’t mean this in a sexist way, I presume that it would also be true if they could have done with the fathers and that’s what the experiment was, but the point of the story is not the gender point, the point is that the children, the newborns, were actually detecting something already.