What it Might "Feel" Like to be Connected to Devices That Will Expand or Enhance Human Function With Cyber Abilities

I've been a neuroscientist for almost 30 years, and I’ve concentrated my studies upon the structure and functions of the cerebral cortex because damage to this part of the brain causes the most severe effects on higher functions like language and memory.

You've probably seen many statistics about the brain or the cerebral cortex, but one of the key features of the brain that is often not appreciated is how important the connections between different parts of the brain are for the unique abilities of the human brain. The fibers that interconnect the neurons throughout the cerebral cortex are stained here in black.

In this parasagittal [1] section we’re looking at the brain from the side about half-way between the midline and ear. All this dark area contains the fibers that fill the core of the cerebral hemispheres. If you line up all these stained fibers end-to-end, they would extend more than a hundred thousand miles, more than four times around the earth’s equator, all packed into this small space. These fibers illustrate the true hard problem that challenges any attempt to construct a computational device with today’s technology that's anywhere near equivalent to the brain.

It is extremely unlikely that this amount of connectivity can be stuffed into such a small, mobile system. 


Image 10 - Cortical Connections

A great deal of neuroscience research over the past several decades, has compiled a rough wiring diagram of the many interconnection pathways in the cerebral cortex. This is a wiring diagram of just the visual part of the primate brain with the eyes down here at the bottom, and connections going up through pathways extending to many different parts of the cerebral cortex from the primary visual areas that receive the most direct inputs from the eyes, to the deepest areas many steps above the eyes where more abstract functions are processed at the top of the hierarchy, such as memory here in the hippocampus, or strategic planning processed here in the prefrontal cortex.

But many students of  the functional architecture of the brain fail to fully appreciate that each of these connections between cortical areas is Bidirectional in every case, connections that carry information from one area to another, are reciprocated by connections  carrying information in the opposite direction. The colliding influences of these reciprocal pathways continuously stir cortical activity with a mixture of bottom-up sensory information and top-down associations.


Image 11 - Neorolinteractivity

At no site throughout the cortex, indeed the nervous system, can activity be said to be purely sensory, independent of higher influences, even at the lowest level of sensory processing.

This means that whether the eyes are open or closed, the lowest levels of the system, the ‘primary’ sensory areas which receive the most direct inputs from the eyes, can be activated just as strongly by top down influences from the higher order areas of the cortex.

Studies of brain activation while subjects imagine that they are seeing a vivid visual scene with their eyes closed, find the same level of activation in primary sensory areas as when the eyes are open. In fact, in many cases it is not possible to distinguish the activation of visual cortex evoked by inputs from the eye and the  activation evoked by imagination with the eyes closed; these brain states of subjective experience are the same.

This research indicates there is a constant interaction between the bottom-up and top-down cortical pathways during conscious subjective experience. Although the technology that might provide definitive evidence that we create subjective experience interactively is still far beyond the limits of modern neuroscience; research has firmly established the existence of functional connections that would be ideally suited for this creative process.


Image 12 - Cortical Interactivity

Those of us who are interested in trying to understand this overwhelming complexity at a more detailed level have constructed computer simulations of large-scale cortical networks in an attempt to address these far-reaching questions regarding the neural basis of conscious experience that are beyond the reach of technologies that look more directly at the functioning brain. 

From the perspective of cortical NeuroInteractivity, it's important to simulate a completely interactive, closed system so that every path of information flow is reciprocated by a path coming back to close the interaction. Even the environment, the whole world, should be included in the form of a space for sensory-motor interactions, treated like any other area of interactions in the overall system.


Image 13 - Scanned Image

The following figures illustrate the basic characteristics of our simulated cortical dynamics. The model of a cortex is equivalent to what you might see in a developing fetus whose cortex becomes active quite early in the womb and develops in the absence of effective sensory feedback.

Under these isolated conditions the structured patterns of cortical activity coordinate the growth of connected networks that are self-organized even before the baby is born or the eyes are open. It is likely that the sort of dynamics and organization I’ll present in these simulations are present in the poorly developed cerebral cortex of the new born infant when it first emerges into the chaos of the outside world. 

What you're seeing in this figure is the evolution of activity patterns in one of our cortical simulations. Each of these sets of six square areas represents the instantaneous activity of 240,000 realistic neurons at sequential times in milliseconds shown below each panel over the course of the simulation. Each of the six cortical areas  represents 40,000 (200 X 200) neurons with the sensory/motor areas at the bottom and models of hippocampal and pre-frontal areas at the top. Adjacent areas are bidirectionally interconnected, and neurons are locally interconnected with nearby or more distant neighbors within each area. The interactions between each of these 240,000 simulated neurons are mediated by physiologically realistic ‘synaptic’ connections with 1,250 other neurons located in the same or adjacent areas for a total of 300 million connections in the overall network.

The realistic anatomical patterns of connection are based upon published tract-tracing studies conducted in our laboratory. Notice what's happening down here in the sensory/motor areas. These are the two parts of the brain that are most directly interconnected with the rest of the body. This is the sensory area here on the bottom left of each panel. Notice that the sensory/motor activity self-organizes into these focal expanding/collapsing ring patterns, gradually morphing and shifting position over time, sometimes splitting up into multiple foci [1], sometimes merging again into single foci at another location. This wandering activation process continues to evolve down here,

Image 14 - Scanned Image

where it splits into a triad of foci [2] like a little ‘Mickey Mouse’.  And down here into four foci, which split up into two.

This self-organizing process keeps changing on and on and on, in the absence of any external influences. While such methods reproduce the sort of the realistic dynamics observed in the living brain, you could probably look at this process for a lifetime and never recognize ‘consciousness’ or become aware that the system is ‘thinking’. Such higher functions can only develop over the course of interacting with the world, and only in this same context, can the meaningful significance of intelligent behavior be appreciated. 

A growing number of brave neuroscientists have embraced the full complexity of nonlinear dynamical brain activity in all its beauty, taking the bull by the horns in an attempt to identify the relationship between neural mechanisms and the dynamical mechanics responsible for the emergence of higher cortical functions such as consciousness, language, exploration and discovery. 


Image 15 - Insights

Given these simulated cortical dynamics were self-organized in the absence of structured sensory inputs, the potential significance of these simulated dynamics should be apparent in the behavioral dynamics of newborn infants whose cortical development in the womb, must have likewise self-organized in the absence of sensory feedback. The behavior of the newborn infant consists of an incessant series of spontaneous movements - flailing their limbs, nodding and shaking their heads, bending and twisting their torso, smacking their lips, contorting their face, and making every vocalization imaginable. This dance of the newborn resembles the dynamical behavior of our cortical simulation as it endlessly wanders from point-to-point exploring every possible cortical pattern.

The newborn infant finds itself suddenly immersed in a sea of meaningless sensations, most of which are the direct consequence of their constant movements; skin sensations caused by hand clapping and foot stomping; all sorts sounds emanating from their mouths; a visual world that jumps about with every movement of the head and eyes; and so on and on Out of this chaos, 


Image 16 - Developmental Breakthrough

babies must learn to see, to listen, to touch, and they begin by learning that the most predictable sensations are the most direct consequences of their own actions. For example, sensory feedback from the eyes evoke left-ward shifts of cortical visual activity as the immediate consequence of right-ward eye movements generated by persistent activity in the eye movement areas of motor cortex. The neural mechanisms responsible for learning strengthen the connections between neurons that are simultaneously active – ‘fire together, wire together’.

By synchronizing the cortical activity in sensory and motor areas, the consistent sensory consequences that immediately follow every movement we make drive these mechanisms to strengthen the interconnections between the neurons in sensory and motor areas simultaneously activated during each movement. The consistent effects of actions such as eye movements, which are repeated over and over again, stimulate the rapid growth of interconnected cortical networks that link together the motor neurons whose activity generates each movement with the sensory neurons that predict the consistent effects of the movement. The first such action-prediction networks to develop link together the most consistent patterns of cortical neurons in sensory and motor areas that are most frequently re-activated. By far the most consistent sensory consequences are self-generated, like the visual shifts caused by one’s own head and eye movements, or the sounds of one’s own vocalizations.

The extreme predictability of self- generated sensation ensures early development is dominated by the rapid formation of self-action - prediction networks. These ‘self’ networks provide a solid foundation to support the vast cortical structure of the conscious mind, reinforced by every sensation that re-affirms who we are.

These sensory self-predictions accompany every action we make, providing a frame of reference for the new sensory predictions we’ll associate with every action we take. This NeuroInteractive ‘self’-organizing process is the kernel that lifts consciousness up by its bootstraps. 


Image 17 - Developmental Breakthrough

Further development rapidly expands across Modalities** as the baby learns that the sight of its moving hand is another, somewhat less direct consequence of its self-intentions to move. Indeed, it's likely that early on in the development of self-prediction networks .


Image 18 - I wish

when a baby is crying very loudly like this, it's crying about the loud sound that it hears. But what dynamical mechanisms breaks this feedback cycle once the self-predictive network is firmly established?


Image 19 - Cauller & Conners

We’ve employed sensitive methods to study the cellular mechanisms responsible for cortical activation by microsurgical isolation of the top-down and bottom-up pathways in slices of living cortical tissue.

In this way, we have characterized the distinct influences of the top-down and bottom-up cortical inputs to individual cortical output neurons responsible for movements. It turns out that the top-down inputs are widely distributed across the surface of the cortex, generating an associative context of sustained excitation that primes specific patterns of cortical activity. In contrast, bottom-up sensory inputs evoke a very strong but brief flash of focused excitation, which is immediately followed by a much longer-lasting period of powerful inhibition that leaves behind a negative after-image of the sensory input pattern.


Image 20 - Imagined World

The pattern of cortical activity depends upon a competition between neurons within local regions of the cortex.

Within each ‘winner-take-all’ domain is thousands of neurons linked together in an interactive local network of pervasive mutual inhibition which suppresses the activity of the neurons within the domain surrounding the active ‘winners’ that dominate the competition at any given moment. Patterns of cortical activity evolve from one winner-take-all pattern to the next under the converging influences of the bottom-up sensory inputs and the top-down associative context.

By a process analogous to evolutionary natural selection, cortical competition selects the next set of winning neurons whose action-prediction network will generate the action most likely to confirm its predictions. The strong interconnections that bind together established action-prediction networks can sustain the activity of the network by mutual excitation. As the sensory component of the action-prediction network gradually transforms into the sensory pattern evoked by the consequences of the corresponding motor component, the inhibitory bottom-up after-effects evoked by sensory inputs more effectively suppress the sensory motor network and its corresponding action.

This bottom-up negative after-image persists during the next winner take-all competition and limits the set of networks that may gain an advantage from top-down priming. In this way, the stream of probing sensory-motor interaction that fill every moment of our conscious behavior is guided by the interplay between bottom-up and top-down cortical pathways as part of the same NeuroInteractive process responsible for the development of action prediction networks.

It’s as if the bottom-up sensory inputs are saying, ‘okay,  we can predict this action, now we’re ready to go on to the next action; we’ve already seen what’s over here, let’s go look over there’. As we identify and fully characterize the dynamical principles of this NeuroInteractive process, we can design artificial robotic systems based upon these same principles to further test our understanding of NeuroInteractivity.

Next Page

Footnotes

[1] Parasagittal - refers to a gait in which the legs are oriented entirely under the body, like columns. http://wiki.cotch.net/index.php/Parasagittal  March 9, 2007 12:22PM EST

[2] FOCI - a decision procedure for quantifier-free first-order formulas. It supports certain interpreted theories, such as equality, uninterpreted functions, linear arithmetic, and arrays.

http://www.kenmcmil.com/foci.html  March 9, 2007 12:46PM EST