Neuroscientist Christian Mikutta sits at the bedside of an epileptic patient in a San Francisco hospital and plays musical chords for her. Some of the chord progressions are predictable, even for non-musicians. Others, however, are very improbable given the standard rules of musicality. Mikutta wants to understand how does a person’s brain react to an unexpected sound.
He doesn’t need to ask the patient, though. He is already receiving a record of her brain’s activity directly from the source – an array of electrodes lodged right inside her skull onto her gray matter.
Credit: Adeen Flinker, UC Berkeley
The reason the patient is in the hospital with electrodes on her brain has actually nothing to do with Mikutta’s research. She has a form of epilepsy that does not respond to medications. Her neurosurgeon decided that if he could figure out exactly which group of neurons in her head are the ones to go rogue and cause her seizures, he could remove them surgically and cure her epilepsy. A few days previously, the patient was admitted to the hospital and underwent the first of two surgical procedures: the doctor removed a part of her skull and placed a small plastic sheet packed with 64 electrodes in the location where he thought the seizures most likely originated from. Then he replaced the skull leaving the wires from the electrodes connected to a computer at the patient’s bedside – the same computer that collects data for Mikutta’s study.
The patient is to spend the next few days lying in the hospital waiting for a seizure to happen – a pivotal moment when her doctors will be able to pinpoint the culprit for her disease. They will then take her back to the operating room, reopen her head, cut out the aberrant neurons, remove the electrodes and close her skull.
It sounds really invasive, and it is. But for a person who has been plagued by seizures for most of her life, this kind of surgery might be a dream-come-true. About a thousand such procedures take place in the U.S. every year. The procedure results in seizure control in 70-90% of patients. It is increasingly used worldwide to treat uncontrolled seizure disorders.
To the delight of inquisitive neuroscientists, many patients volunteer to participate in scientific experiments aiming to better understand the workings of the mind while they are waiting for a seizure in the hospital bed. This is an ideal window of opportunity to peer into somebody’s brain. Quite literally.
“The research we do is an add-on to a standard-of-care practice,” says Robert Knight, Professor of psychology, neurology, and neuroscience at UC Berkeley’s Helen Wills Neuroscience Institute and world leader in this type of research method known as electrocorticography (ECoG).
Brian Pasley, a research fellow in Knight’s lab, became quite the media sensation a couple of years ago for his research with ECoG patients. Pasley managed to reconstruct words the patients heard just by looking at their brain activity. Media outlets around the world, from ABC News to The Sydney Morning Herald touted the results with titles such as “Telepathy machine reconstructs speech from brainwaves.”
Although not quite “mind-reading,” the computer algorithm Pasley and his colleagues developed could really help understand the needs of people who cannot speak due to neurological disorders such as stroke or Lou Gehrig’s disease. If you lost your ability to talk, it would be transformative to have a “speech prosthesis” – a device that can decipher your thoughts and translate them into signals for a computer to turn into speech.
In similar line of experiments, Jack Gallant, another scientist at the Helen Wills Neuroscience Institute, reconstructed images a person saw just by looking at her brain activity. Instead of the invasive ECoG method, however, Gallant used a standard non-invasive medical technique – functional Magnetic Resonance Imaging (fMRI).
“Actually, fMRI does not measure neural activity,” says Gallant. “It only records blood flow.” Localized increases in blood flow indicate heightened activity of the neurons nearby.
Gallant’s team played movies to people while the fMRI machine scanned their brains. fMRI machines can detect minuscule differences in the oxygen concentration in blood as it flows to different regions in the head. If certain locations in the brain were consuming more oxygen than usual, this meant that the image the person was seeing in this moment was activating that part of the cortex. The final step was to create a computer model, which could recreate the images from the brain activity alone.
The extraction of thoughts and sensory experiences from patterns of brain activity is called brain decoding. One goal of decoding is to make a detailed map of brain function.
“If you want to make a map of the brain, MRI is the best non-invasive method,” says Gallant. “It’s many times better than the next best technology.”
Gallant is certain that decoding will keep getting better as long as there is funding for neuroscience research. Two factors limit the accuracy of brain decoding: how well you can measure brain activity, and how well you can model the activity you measured. Gallant, Pasley and other scientists continue improving the mathematical models describing processes in the brain. “The real limitation is getting ever more detailed data on neural activity,” says Gallant.
The MRI magnet currently available at Berkeley is a standard 3-Tesla scanner. This model creates a magnetic force 60,000 times more powerful than the Earth’s magnetic field. Neuroscientists, however, are dreaming of using the next more powerful model, already commercially available. In three-dimensional imaging of the brain, a 3-Tesla scanner offers resolution the size of a rice grain (2 mm cubed). By contrast, a top-of-the-line 7-Tesla scanner could zoom into an eighth of that volume.
“The ultimate brain reconstruction task is to decode your internal speech, the little voice inside your head,” says Gallant. “That would be cool, although a bit scary too.” After all, internal speech conveys our most private thoughts. Decoding it would therefore open up private thought for external scrutiny with profound consequences for privacy.
“Like a lot of technology, it is both frightening and interesting,” says Gallant. “But it will happen, so it is important to prepare for this by beginning to address the ethical implications now.”
With major advances in neuroscience coming from Berkeley and other research centers, brain decoding will not remain in the realm of science fiction for much longer.
“The only reason why we can’t decode internal speech today is because our brain activity measurements are lousy,” says Gallant. “When they get better, it will be straightforward.”
