Perception, Selection and the Brain

So, I will take the next fifteen minutes to tell you how the brain works. Good luck. Let me try to say some things about how the brain works that I think are very relevant to the discussion today, and in particular that speak to some misconceptions about how the brain works. If I can untangle them, might make it seem a bit more comprehensible about why we can produce art. William James probably got more of this right than any other single individual in history. He understood several very critical facts: 1) that consciousness is not a thing, but a process, and 2) that the brain does not record, it constructs. That’s very important.

What we now know is that the world around us is a complete chaos of electromagnetic signals. The coherent vision of the world that we see is something that our brains construct. And that is absolutely the case. That means that physics describes the world in probabilistic terms, not as solid objects, but as probabilistic entities. We construct them as being coherent, and with things having category and entity to them. That’s what our brains do.

And how do they do that? How is it that you see that this is a cat even though in any detailed sense this has very, very limited relationship to what a cat actually looks like if you were to put a photograph next to it, but you have no doubt at all that it’s a cat. Or why is it that this elephant seems to have something funny going on in its legs?

If you were a camera you wouldn’t have that problem, you would simply record it, and that would be fine, and that would be the end of it. But the reason we have a problem with this is because of this unstoppable drive of our brains to make coherence out of what we see, hear and sense. One of the great misconceptions of the contemporary world is that the brain works like a computer. In no sense does the brain work like a computer. Computers record, and computers have things stored in specific places that are stable. Our brains do none of that.

When the great chess master Gary Kasparov lost to Big Blue everybody said “Ah ha, this machine can think!” Big Blue was not thinking. Big Blue was simply replaying the entire history of chess. That’s not the way that Gary Kasparov or any human being plays chess. We do pattern recognition. Even though we are capable of logic, our brain does not operate by the principles of logic. It operates by selection of pattern recognition. It’s a dynamic network. It’s not an “if-then” logic machine.

Here’s an example. In an experiment that we did at our Institute on a phenomenon called “binocular rivalry” we showed subjects, of which I was one, this pattern, wearing cellophane glasses with red and blue.

One eye sees only the red and the other side sees only the blue, at the same time. Your brain cannot stand that. You do not see both of them at once, when one eye can only see one thing, and the other eye can only see another other thing. Instead you alternate. Your perception alternates out of your control by this “rivalry,” as it’s called. So you see the red then you see the blue then you see the red. So we sat under this hairdryer, called a “magneto-encephalograph,” which records very rapid responses in the brain, and asked the question “What’s happening in your brain when you see the conscious perception as compared to the nonconscious one?” You press a switch when you’re seeing one versus the other. (shows a diagram) This is what happens. In the conscious perception there is an explosion of simultaneously linked activity between many, many parts of the brain. I’m very close to this data because that’s me.

All of these green lines represent not actual nerve tracts, but correlated points of my brain that are connected to each other by nerve tracts, so that there is this unification.
So James’ notion of a construction by the brain is in fact accurate. One of the things that this tells you is that you are literally putting together all of the different aspects of the image that are perceived by your retina, and processed by your brain. In a very important sense, as Gerald Edelman, who has formulated many of these ideas, has put it: “Every perception is an act of creation.” In addition, everybody’s got a fingerprint of what their brain looks like. These are four people looking at exactly that same set of images. This is the way their brains respond to it when they see it. They were all conscious of it, they all said “Yes, those are red stripes. Yes, those are blue stripes.” Every brain sees it differently, and this is a function of the intrinsic individuality of every living thing in the biological world and especially of human brains. Because of our different heterogeneity based on genetic differences, experiential differences, and so on, we don’t all do it the same way, and language helps us to agree on what it is that we are seeing.

Similarly with music, where you are not dealing with space, but with time, if you ask the question, “How does the brain make sense out of musical tones?,” which in the western tradition, for western harmonies, you find that there is actually a coherent brain activity that we have in response to something like a Bach prelude which we do not have in response to a random sequence of notes. And if you look at the same kinds of interactivity in the brain you find that during this kind of perception you have again this linking of major areas across the brain. So, although in some sense it’s poetic to speak of a “right brain/left brain” difference. The fact of the matter is that things that are “right brain” are really happening everywhere and things that are “left brain” are really happening everywhere. There are certain aspects of it that may be more biased to one side versus another, but the brain is not highly localized in any sense at all. Everything that ever happens in your brain is happening as a unification of many, many, many areas at once.

How can one explain this? If it’s not a logic based system, what is the alternative? The alternative, the only other real way of parsing the world, is a selectional system. This goes back to Charles Darwin. What are the elements of a selectional system? You have to have a repertoire. In the case of Darwin’s species, it meant that you have lots of variant individuals in the population. In the case of Gerald Edelman’s notion of the brain, it means that you have lots of potential ensembles of neural circuits that can get linked up in different ways. You have to have a criterion. In the case of Darwin’s species, those that are fittest will have a better chance of surviving. In the case of Edelman’s notion of the brain, it is those particular ensembles of cells that are of value to the individual at that moment. Value also comes from partly our evolutionary heritage, and partly from our experiences through life. They will be selected and used. And finally, you have to have amplification. Which, in the case of Darwin, is one kind of individual will out reproduce others. Or it can be, in the case of the brain, that you have simply a strengthening of one kind of ensemble of cells versus another. And this the same diagram that Warren showed from one of Edelman’s books talking about different aspects of selection, part of which is developmental, as he said, where you start out with many more connections among brain cells than eventually end up in any given place. That’s one aspect that occurs during development, during embryonic period, but the more important one for what I’ve been discussing is what goes on during actual experience of the world, where you have this linking up of regions in this sort of reciprocal way. And this is a cartoon version of what I actually showed you in the brains of individuals.

So, all of these selectional systems are complex networks with extensive repertoires, and the repertoires are what make them work. So here is a graphic representation of the visual system of the brain.

There are many, many different areas connected in many, many different ways. So you can think of these, of a selectional event, when you are seeing something, as being a choosing (although there’s nobody in there actually doing the choosing), a selection from among these repertoires of sets which are activated and which then will give you the experience of seeing, and they get linked up with ones that are nonvisual as well, with the motor repertoires, and so on. One of the reasons that memory can be so unreliable is because all it has to give you is the feeling that you remember it, and you’ll be dead sure that that’s it, no matter how wrong you might be, and that’s because you are linking together these different kinds of ensembles of cells in the brain which will produce that collective activity, which is the experience of remembering. So, as Gerald Edelman has said, just as every perception is an act of creation, every memory is an act of imagination. Why? Because it’s not written somewhere in your brain, it’s constructed as you have the experience of it. Again, the individuality of perception.

So, this is the kind of way of thinking about it that we are trying to pursue at The Neurosciences Institute, which is trying to get to the bottom of what actually makes for this complex system that is a brain, which is ordered, but not fully ordered, but has a certain degree of heterogeneity to it, but is not completely random. And I think that I won’t try to go any further into it than that. But simply try to leave you with these concepts that the notion of how we can produce the sort of creative things that we do depends upon these repertoires of potential ways of doing things in the brain, where in fact from one day to the next or even one moment to the next, you are not even necessarily doing it exactly the same way, because there aren’t just dedicated railroad tracks in your brain, it’s an ensemble, and you can put together the same kind of activity and the same kind of perception using different ensembles of cells. And I think I’ll stop there.