The brain-computer interface allows paralysed patients to play music just by thinking about it.ICCMR Research Team - University of PlymouthA pianist plays a series of notes, and the woman echoes them on a computerized music system. The woman then goes on to play a simple improvised melody over a looped backing track. It doesn't sound like much of a musical challenge — except that the woman is paralysed after a stroke, and can make only eye, facial and slight head movements. She is making the music purely by thinking.
This is a trial of a computer-music system that interacts directly with the user's brain, by picking up the tiny electrical impulses of neurons. The device, developed by composer and computer-music specialist Eduardo Miranda of the University of Plymouth, UK, working with computer scientists at the University of Essex, should eventually help people with severe physical disabilities, caused by brain or spinal-cord injuries, for example, to make music for recreational or therapeutic purposes. The findings are published online in the journal Music and Medicine1.
"This is an interesting avenue, and might be very useful for patients," says Rainer Goebel, a neuroscientist at Maastricht University in the Netherlands who works on brain-computer interfacing.
Therapeutic use
Evidence suggests that musical participation can be beneficial for people with neurodegenerative diseases such as dementia and Parkinson's disease. But people who have almost no muscle movement have generally been excluded from such benefits, and can enjoy music only through passive listening.
The development of brain–computer interfaces (BCIs) that enable users to control computer functions by mind alone offer new possibilities for such people (see Mental ping-pong could aid paraplegics). In general, these interfaces rely on the user's ability to learn how to self-induce particular mental states that can be detected by brain-scanning technologies.
Miranda and his colleagues have used one of the oldest of these systems: electroencephalography (EEG), in which electrodes on the skull pick up faint neural signals. The EEG signal can be processed quickly, allowing fast response times, and the instrument is cheaper and more portable than brain-scanning techniques such as magnetic resonance imaging and positron-emission tomography.
Previous efforts using BCIs have focused on moving computer screen icons such as cursors, but Miranda's team sought to achieve the much more complex task of enabling users to play and compose music. Miranda says that he first became aware of the then-emerging field of BCIs more than a decade ago while researching how to make music using brainwaves. "When I realized the potential of a musical BCI for the wellbeing of severely disabled people," he says, "I couldn't leave the idea alone. Now I can't separate this work from my activities as a composer."
The trick is to teach the user how to associate particular brain signals with specific tasks by presenting a repeating stimulus — auditory, visual or tactile — and getting the user to focus on it. This elicits a distinctive, detectable pattern in the EEG signal. Miranda and his colleagues show several flashing 'buttons' on a computer screen, which each trigger a musical event. The users push a button just by directing their attention to it.
For example, a button could be used to generate a melody from a preselected set of notes. The user can alter the intensity of the control signal – how 'hard' the button is pressed – by varying the intensity of attention, and the result is fed back to them visually as a change in the button's size. In this way, any one of several notes can be selected by mentally altering the intensity of pressing.
With a little practice, this allows users to create a melody as if they were selecting keys on a piano. And, as with learning an instrument, say the researchers, "the more one practices the better one becomes".
Back in control
The researchers trialled their system on a female patient who has locked-in syndrome, a form of almost total paralysis caused by brain lesions, at the Royal Hospital for Neuro-disability in London. During a two-hour session, she got the hang of the system and was eventually playing along with a backing track. She reported that "it was great to be in control again".
Goebel points out that the patients still need to be able to control their eye movements, which people with total locked-in syndrome cannot. In such partial cases, he says, "one can usually use gaze directly for controlling devices, instead of an EEG system". But Miranda points out that eye-gazing alone does not permit variations in the intensity of the signal. "Eye gazing is comparable to a mouse or joystick," he says. "Our system adds another dimension, which is the intensity of the choice. That's crucial for our musical system."
Miranda says that although increasing the complexity of the musical tasks is not a priority, music therapists have suggested it would be better if the system were more like a musical instrument — for instance, with an interface that looks like a piano keyboard. He admits that it is not easy to raise the number of buttons or keys beyond four, but is confident that "we will get there eventually".
"The flashing thing does not need to be on a computer screen," he says. It could, for example, be a physical electronic keyboard with light-emitting diodes on the keys. "You could play it by staring at the keys," he says.
-
References
- Miranda, E. R., Magee, W. L., Wilson, J. J., Eaton, J. & Palaniappan, R. Music and Medicine advance online publication doi:10.1177/1943862111399290 (2011).
.
Posts
