Mind-reading brain implant converts thoughts to speech almost instantly: 'breakthrough'

A photo of researchers connecting a person's brain implant to a voice synthesizer computer.
The brain-computer interface can provide a synthetic voice to those unable to speak. (Image credit: Noah Berger)

A brain implant that uses artificial intelligence (AI) can almost instantaneously decode a person's thoughts and stream them through a speaker, new research shows. This is the first time researchers have achieved near-synchronous brain-to-voice streaming.

The experimental mind-reading technology is designed to give a synthetic voice to people with severe paralysis who cannot speak. It works by putting electrodes onto the brain's surface as part of an implant called a neuroprosthesis, which allows scientists to identify and interpret speech signals.

The brain-computer interface (BCI) uses AI to decode neural signals and can stream intended speech from the brain in close to real time, according to a statement released by the University of California (UC), Berkeley. The team previously unveiled an earlier version of the technology in 2023, but the new version is quicker and less robotic.

A streaming brain-to-voice neuroprosthesis to restore naturalistic communication - YouTube A streaming brain-to-voice neuroprosthesis to restore naturalistic communication - YouTube
Watch On

"Our streaming approach brings the same rapid speech decoding capacity of devices like Alexa and Siri to neuroprostheses," study co-principal investigator Gopala Anumanchipalli, an assistant professor of electrical engineering and computer sciences at UC Berkeley, said in the statement. "Using a similar type of algorithm, we found that we could decode neural data and, for the first time, enable near-synchronous voice streaming."

Anumanchipalli and his colleagues shared their findings in a study published Monday (March 31) in the journal Nature Neuroscience.

Related: AI analysis of 100 hours of real conversations — and the brain activity underpinning them — reveals how humans understand language

The first person to trial this technology, identified as Ann, suffered a stroke in 2005 that left her severely paralyzed and unable to speak. She has since allowed researchers to implant 253 electrodes onto her brain to monitor the part of our brains that controls speech — called the motor cortex — to help develop synthetic speech technologies.

"We are essentially intercepting signals where the thought is translated into articulation and in the middle of that motor control," study co-lead author Cheol Jun Cho, a doctoral student in electrical engineering and computer sciences at UC Berkeley, said in the statement. "So what we’re decoding is after a thought has happened, after we've decided what to say, after we’ve decided what words to use and how to move our vocal-tract muscles."

AI decodes data sampled by the implant to help convert neural activity into synthetic speech. The team trained their AI algorithm by having Ann silently attempt to speak sentences that appeared on a screen before her, and then by matching the neural activity to the words she wanted to say.

The system sampled brain signals every 80 milliseconds (0.08 seconds) and could detect words and convert them into speech with a delay of up to around 3 seconds, according to the study. That's a little slow compared to normal conversation, but faster than the previous version, which had a delay of about 8 seconds and could only process whole sentences.

The new system benefits from converting shorter windows of neural activity than the old one, so it can continuously process individual words rather than waiting for a finished sentence. The researchers say the new study is a step toward achieving more natural-sounding synthetic speech with BCIs.

"This proof-of-concept framework is quite a breakthrough," Cho said. "We are optimistic that we can now make advances at every level. On the engineering side, for example, we will continue to push the algorithm to see how we can generate speech better and faster."

Patrick Pester
Trending News Writer

Patrick Pester is the trending news writer at Live Science. His work has appeared on other science websites, such as BBC Science Focus and Scientific American. Patrick retrained as a journalist after spending his early career working in zoos and wildlife conservation. He was awarded the Master's Excellence Scholarship to study at Cardiff University where he completed a master's degree in international journalism. He also has a second master's degree in biodiversity, evolution and conservation in action from Middlesex University London. When he isn't writing news, Patrick investigates the sale of human remains.

You must confirm your public display name before commenting

Please logout and then login again, you will then be prompted to enter your display name.