Brain machine interfaces

The thought control phase feedback loop

I recently attended a talk on brain machine interfaces (BMI) given by José M. Carmena from Berkeley. Carmena described his current research into thought control of computers via brain implanted electrode arrays. The subjects were macaque monkeys, and the task was controlling a cursor, directing towards a target and “grasping” it. Successful execution of the task triggered a reward for  the monkey, juice.

Initially, the monkey was subjected to the task through motor manipulation of a control stick. This phase allowed the monkey to learn the task, as well as to identify motor neurons that could serve as stable output for the thought control phase. Additionally, a simple linear regression model was trained on those neurons in order to decode spike output into action commands for the cursor. In the later stage, the control stick was omitted, and the monkeys controlled the cursor directly through thought. After several days of learning the monkeys achieved a high success rate, even at the start of the trial. This suggests that single day learning was being consolidated into longer term motor memory, as occurs for normal every day motor tasks. The typical example for humans is riding a bicycle, a skill that once learned persists.

The brain’s natural plasticity is what allows it learn a direct a BMI in just days. Neural rewiring occurs in order to progressively adapt the input-output feedback loop. The monkey observes the cursor on the screen, this is the input, and through thought manipulates it, the output.

Of course, the task at hand was relatively simple, and performance was lower than through natural motor control of the stick. But still, it is remarkable that with a relatively crude interface (sampling a handful of neurons) and a simple regression model, the task is learned to high accuracy. It’s the brain that is doing most of the heavy lifting, suggesting that there is great potential for brain computer communication.

Performance increases and training time declines as motor memory consolidates

So what lies ahead? Firstly, better physical interfacing with the brain. Current approaches are limited to crude short lived electrode arrays that sample perhaps hundreds of neurons in a very invasive way. Secondly, better feedback loops. One of the reasons the monkey’s performance is higher in the control stick phase of the task is because it obtains very valuable feedback, not just from the cursor on the screen, but from its own body; the position and orientation of its arm/hand. All this feedback is missing in the thought control phase. So one way forward is to not just connect to neurons for output, but possibly to inject input as motor feedback.

But things don’t stop there. Injecting input into the brain need not only be a matter of feedback for control, but as a way to input sensory information in general. In one experiment, a rat was subject to artifcial stimuli corresponding to a virtual object, via directly stimulating neurons responsible for processing touch information from its whiskers (yes, its whiskers you read that right). So one could say that the rat “felt” the touch of a non existent object. Taking the idea even further, one could conceivably create entirely new senses, not linked to a specific part of the body, in order to sense surroundings in ways that do not correspond to anything that our bodies currently perceive.

This last line of thought is what I found to be most interesting, although I have to say it was not stated explicitly by Carmena in his talk, it’s speculation extrapolating the idea of virtual objects beyond current senses. But this is just one possibility. In general, brain computer communication opens up a lot of possibilities, many of which currently belong to the realm of science fiction. But as the speaker said, things that are being experimented today, like thought control, were the stuff of science fiction twenty years ago..

Art, regularity, novelty and dopamine

Dopamine (Wikipedia)

When writing up the post on regularity I was looking for an image that would match the content. As you can see if you go back to that entry the image shows a regular structure, a fragment of architecture. I was quite happy with it, but what turned out to be even more interesting was the article I lifted it from, which I did not look at until now.

It happens that in that post the author explores some of the same ideas I’ve considered when attempting to apply concepts from information theory[1] to yield an interpretation of artistic experience as a learning process. The notions of monotony and complexity are also given a similar (tentative) mathematical formalization in terms of algorithmic information theory, which is how we have just discussed regularity in the previous entry.

Of special interest is the hypothesis of how monotony (at the low complexity end of the spectrum, see below on AIC) is not only boring but in fact distressing, due to a mismatch between what our neural system is tuned to observe and what is actually observed

why is human neurological response actually negative? Some insight into the effect comes from the notion of Biophilia, which asserts that our evolution formed our neurological system within environments defined by a very high measure of a specific type of coherent complexity. That is, our neurological system was created (evolved) to respond directly and exquisitely to complex, fractal, hierarchical geometric environments. When placed in environments that have opposite geometrical features, therefore, we feel ill at ease.

In my brief essay on the matter I mentioned hierarchical nature of processing in the visual cortex, which is reflected in machine learning approaches using recurring neural networks such as HTM’s and deep learning. The hierarchical nature of the processing, as well as the dual top-down and bottom-up flow of information are well established in neuroscience. However, it remains to be seen if there is a neurological explanation for the observation that “we feel ill at ease” with, for example, monotonous geometry.

In any case, all this led me to to try to extend my information-theoretic picture of artistic expression and experience with a grounding in neurological processes. In particular, how does the brain deal with regularity and novelty as relates to learning and satisfaction/pleasure. Knowledge from neuroscience points to dopamine as both related to pleasure and reward and, critically, learning (temporal difference learning).

I’m still thinking many of these ideas through, but I’m going to try to summarize some of them in their crude form[6] so I can return to this as reference, please excuse the lack of polish.

Here are the main points:

* Artistic expression can be characterized in terms of its position in a spectrum whose extremes are the obvious/boring on one side and the unintelligible on the other. This spectrum corresponds to the algorithmic information content of the expression.

* Artistic experience is a learning process where regularities are extracted and predictions are made according to those regularities

* Novelty is the mismatch between prediction and observation, novelty is a departure from regularity

* Novelty, either as reward-prediction error, or as an intrinsic reward (indirect indicator of reward) motivating exploration[2], activates dopamine in the brain. In the latter case, this is related to the exploration vs exploitation aspect of reinforcement learning[3].

* Different dopamine response profiles yield different levels of sensation-seeking in individuals[4], which may partially account for different taste in terms of desired optimum points on the AIC spectrum

* The optimum balance between insufficient and excessive complexity in the AIC spectrum parallels the requirement of balance between task difficulty and skill in flow

* Artistic satisfaction/pleasure is an interplay between novelty and regularity[5], it may be possible to explain this within some model of reward mediated by dopamine[6]


Notes/References

[1] Perhaps the canonical example of this line of work is Jurgen Schmidhuber’s treatment

[2]  Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA [2006] , Pure novelty spurs the brain

[3] Exploration & Exploitation Balanced by Norepinephrine & Dopamine [2007]

[4] Midbrain Dopamine Receptor Availability Is Inversely Associated with Novelty-Seeking Traits in Humans [2008]

[5] Dopamine: generalization and bonuses [2002]

[6] In particular, it is unclear whether a dopaminergic model is applicable to standalone perception which is the case for artistic experience, since actions and explicit reward are absent, and the timescales may be too small to be compatible with such a model.

Hints of generality in machine learning

One of the themes in AI is the progressive replacement of hand crafted knowledge/programming with autonomous learning. Another theme is the progressive shift from narrow domain specific abilities to generally applicable performance. These themes are closely related: domain specific performance is usually achieved via encoding domain specific expert knowledge, provided by humans, into the AI itself. This encoding is fixed and static, and is potentially brittle if the agent is subjected to a domain outside its narrow region of applicability.

In this talk we see hints of generality in machine learning via the replacement of hand crafted, tuned features (as input to a later stage learning phase) with a learning phase that autonomously learns these features. Because domain specific features, for example for vision or speech recognition, are typically designed by subject matter experts, learning is constrained by that specific manual task. One cannot use vision features in an audio problem and so forth.

However, when machine learning is targeted at learning the features themselves (in an unsupervised scheme), the potential for generality becomes present. If features can be autonomously learned for various domains, the machine learning process becomes general in so far as the feature learning’s performance is comparable or superior to that using hand crafted knowledge. This is exactly what is demonstrated here.

And, to make it even more relevant in terms of current research avenues, this is related to findings in neuroscience that suggest the general applicability of some kinds of learning in the neocortex, where for example patients can learn to see with their auditory cortex or their somatosensory cortex (by rewiring the optic nerve). This suggests the possibility that there is a unique learning algorithm at work, at least in the learning of concepts at low levels of the hierarchy close to the perceptual level, which is exactly where features reside.