MIT Technology Review
Meta has developed a brain-reading system that can detect keystrokes from brain signals with 80% accuracy, but it’s far from becoming a consumer product. Using a large, expensive scanner, the research sheds light on how the brain processes language. While the technology isn’t practical yet, it could inform AI development.
Meta has an AI for brain typing, but it’s stuck in the lab
Author: Antonio Regalado
Back in 2017, Facebook unveiled plans for a brain-reading hat that you could use to text just by thinking. “We’re working on a system that will let you type straight from your brain,” CEO Mark Zuckerberg shared in a post that year.
Now the company, since renamed Meta, has actually done it. Except it weighs a half a ton, costs $2 million, and won’t ever leave the lab.
Still, it’s pretty cool that neuroscience and AI researchers working for Meta have managed to analyze people’s brains as they type and determine what keys they are pressing, just from their thoughts.
The research, described in two papers posted by the company (here and here), as well as a blog post, is particularly impressive because the thoughts of the subjects were measured from outside their skulls using a magnetic scanner, and then processed using a deep neural network.
“As we’ve seen time and again, deep neural networks can uncover remarkable insights when paired with robust data,” says Sumner Norman, founder of Forest Neurotech, who wasn’t involved in the research but credits Meta with going “to great lengths to collect high-quality data.”
According to Jean-Rémi King, leader of Meta’s “Brain & AI” research team, the system is able to determine what letter a skilled typist has pressed as much as 80% of the time, an accuracy high enough to reconstruct full sentences from the brain signals.
Facebook’s original quest for a consumer brain-reading cap or headband ran into technical obstacles, and after four years, the company scrapped the idea.
But Meta never stopped supporting basic research on neuroscience, something it now sees as an important pathway to more powerful AIs thatlearn and reason like humans. King says his group, based in Paris, is specifically tasked with figuring out “the principles of intelligence” from the human brain.
“Trying to understand the precise architecture or principles of the human brain could be a way to inform the development of machine intelligence,” says King. “That’s the path.”
The typing system is definitely not a commercial product, nor is it on the way to becoming one. The magnetoencephalography scanner used in the new research collects magnetic signals produced in the cortex as brain neurons fire. But it is large and expensive and needs to be operated in a shielded room, since Earth’s magnetic field is a trillion times stronger than the one in your brain.
Norman likens the device to “an MRI machine tipped on its side and suspended above the user’s head.”
What’s more, says King, the second a subject’s head moves, the signal is lost. “Our effort is not at all toward products,” he says. “In fact, my message is always to say I don’t think there is a path for products because it’s too difficult.”
The typing project was carried out with 35 volunteers at a research site in Spain, the Basque Center on Cognition, Brain, and Language. Each spent around 20 hours inside the scanner typing phrases like “el procesador ejecuta la instrucción” (the processor executes the instruction) while their brain signals were fed into a deep-learning system that Meta is calling Brain2Qwerty, in a reference to the layout of letters on a keyboard.
The job of that deep-learning system is to figure out which brain signals mean someone is typing an a, which mean z, and so on. Eventually, after it sees an individual volunteer type several thousand characters, the model can guess what key people were actually pressing on.
In the first preprint, Meta researchers report that the average error rate was about 32%—or nearly one out of three letters wrong. Still, according to Meta, its results are most accurate yet for brain typing using a full alphabet keyboard and signals collected outside the skull.
Research on brain reading has been advancing quickly, although the most effective approaches use electrodes implanted into the brain, or directly on its surface. These are known as “invasive” brain computer interfaces. Although they require brain surgery, they can very accurately gather electrical information from small groups of neurons.
In 2023, for instance, a person who lost her voice from ALSwas able to speak via brain-reading software connected to a voice synthesizer. Neuralink, founded by Elon Musk, is testing its own brain implant that gives paralyzed people control over a cursor.
Meta says its own efforts remain oriented toward basic research into the nature of intelligence.
And that is where the big magnetic scanner can help. Even though it isn’t practical for patients and doesn’t measure individual neurons, it is able to look at the whole brain, broadly, and all at once.
The Meta scientists say that in a second research effort, using the same typing data, they used this broader view to amass evidence that the brain produces language information in a top-down fashion, with an initial signal for a sentence kicking off separate signals for words, syllables, and finally typed letters.
“The core claim is that the brain structures language production hierarchically,” says Norman. That’s not a new idea, but Meta’s report highlights “how these different levels interact as a system,” says Norman.
Those types of insights could eventually shape the design of artificial-intelligence systems. Some of these, like chatbots, already rely extensively on language in order to process information and reason, just as people do.
“Language has become a foundation of AI,” King says. “So the computational principles that allow the brain, or any system, to acquire such ability is the key motivation behind this work.”
Correction: Meta posted two papers describing its brain-typing results on its website. An earlier version of this story incorrectly said they had been published at arXiv.org.
Credits: TCA, LLC.