Restoring Speech to Paralyzed Patients: The History of Speech BCI

Can paralyzed patients speak without opening their mouths, just by thinking?

Are invasive speech BCIs really that powerful?

On March 25, Neuralink released a new video showing exactly this process.

Of course, this result was not achieved in a single step. The training had three stages: first, the patient read the text on the screen aloud. Second, the patient silently mouthed the text on the screen. Third, the patient simply “thought” the words in their mind. That is what we just saw.

The model for each stage had to be trained before moving to the next one. If you think carefully about this process, there are three problems to solve here. First, what signal should be decoded? Second, what model should be used for decoding? Third, how do we prove the safety of the BCI?

In fact, the development of speech BCIs has unfolded in exactly these three stages: first, figuring out what signal to decode; then, figuring out what model to use; and now, reaching the stage of verifying long-term implantation safety.

Next, I’ll walk you through the history of speech BCIs. If you only care about recent progress, you can jump straight to Chapter 4.

Stage 1: 1999–2001, which signal to decode?

Let’s start with two papers from 1998 and 2001. At that time, a U.S. team unexpectedly discovered that high-gamma signals in the human brain were directly related to heard vowels and consonants. One of the researchers, Nathan Crone, later became a key figure in this field.

Honestly, this was absolutely one of the most important discoveries in the entire history of brain-computer interfaces.

What does the high-gamma signal mean? The signal recorded at each electrode site is itself a mixture of periodic and aperiodic components.

With a band-pass filter, we can separate it into different frequency bands. Researchers in the field conventionally divided it into seven bands. Any band above 80 Hz is what people in the field call the high-gamma signal.

Scalp EEG, also called non-invasive EEG, can cover only 0.1 Hz to 80 Hz. Only intracranial EEG, also called invasive EEG, can capture signals above 80 Hz.

This is because the human skull has very high electrical resistance, nearly the highest of any tissue in the body, so only low-frequency signals can pass through it.

That is why the relationship between high-gamma signals and language was discovered so late. Non-invasive EEG was invented in 1924; invasive EEG was developed in the 1950s by two surgeons; brain-computer interfaces were proposed in 1973. Yet the cornerstone of speech BCIs was not discovered until 1998.

Now that humans know what signal to decode, the next step is to study what model to use for decoding.

Stage 2: 2009–2020, which model for decoding?

Now the input is a high-gamma time series, and the output is text or speech. So how do we complete this process?

Because the input signal is produced by a complex system composed of many units, and machine learning and deep learning are especially good at handling such complex systems. Like many other fields, our field also experienced a shift from machine learning to deep learning. In our field, that shift happened around 2019.

Before 2019, papers all relied on manually extracting features and then adding an SVM classifier. They could only decode phonemes. After 2019, sentence decoding became possible. The most effective model was the LSTM.

Many people ask: why didn’t anyone try deep learning before 2019? You have to remember that AlexNet in 2012 was a hand-built CNN, and people even had to manually coordinate communication between two GPUs. Anyone without a computer science background simply couldn’t manage that. It was not until TensorFlow in 2015 and PyTorch in 2016 that people no longer had to hand-build models. With about ten lines of code, you could set up an RNN, and the barrier to entry dropped dramatically.

It should be pointed out that all of these studies were done in patients with short-term implants, usually people with brain tumors or epilepsy. But the patients who truly need speech BCIs are those with ALS or stroke.

Studies with short-term implants can tell us that LSTMs are effective, but they cannot tell us whether long-term implanted speech BCIs are safe.

Stage 3: 2021–2026, Safety of long term implantation

Finally, in 2021, the University of California, San Francisco (UCSF) produced the world’s first long-term implanted speech BCI that could decode sentences. This patient had muscular atrophy, and they asked the patient to read the text on the screen aloud. They achieved a 25% word error rate with a 50-word vocabulary.

Here’s a small tip: when judging whether a BCI paper is reliable, see whether the authors dare to release complete supplementary videos. That one trick filters out most of the fluff.

In 2023, UCSF added a digital avatar on top of this and expanded the vocabulary from 50 to 1,024, with almost no change in word error rate.

In the same year, on another electrode material, the Utah Array, the BrainGate team expanded the vocabulary from 50 to 125,000. Keep in mind that everyday English uses only about 10,000 common words, so 125,000 already includes many names and places. Their word error rate was 23.8%.

In 2024, the UCSF team built a bilingual BCI with a vocabulary of 104 and a word error rate of 25%.

In the same year, the BrainGate team made an even bigger breakthrough: on a 125,000-word vocabulary, they reduced the word error rate to 2.5%. A key reason was that their participant was in the early stage of ALS and in better condition.

In 2025, the UCSF team achieved brain-to-voice, allowing the patient to control pitch, loudness, and speaking rhythm, but the word error rate rose to 50%.

In the same year, BrainGate also achieved brain-to-voice.

Listening to all this, you may feel that it’s always just these two teams. And yes, before Neuralink joined, the only groups doing long-term implanted speech BCIs well were UCSF and BrainGate.

In 2026, Neuralink released a video, but they did not publish their vocabulary size or word error rate. Still, one major advance is clear: in the earlier papers, participants either read aloud or silently mouthed the words, whereas Neuralink reached pure imagined speech.

There are now 5 to 10 patients with long-term implants, and their very existence already demonstrates the long-term safety of speech BCIs. The reason is simple: if the devices were not safe, these patients would not have made it this far.

But at present, speech BCIs still have not entered major hospitals the way coronary bypass surgery has.

In my view, there are two key problems that must be solved to get there in the future. First, after many years of implantation, when BCI performance declines, how can patients continue to benefit? Second, patients differ in disease severity, so the help provided by a BCI also differs. How can we ensure that implantation makes life easier for every patient, not just some of them?

In essence, both problems come down to designing a fallback algorithm. The ceiling is already very high: earlier work has enabled communication close to normal conversation. But the floor of speech BCIs has not yet been built.

I see three ways to do this. First, give patients 1,000 English words or 1,000 Chinese characters, enough to handle communication about eating, drinking, toileting, and other daily needs. Second, there is no need to pursue flashy brain-to-audio systems; brain-to-text is enough, and TTS can handle the rest afterward. Third, we also do not need to rely on imagined speech. It is enough if the patient simply tries hard to speak.

After watching this, you may have two questions. Could this method expose thoughts the patient does not want to say out loud? And are there any public datasets that people can use for validation? A new video will address these two questions, and you are also welcome to leave your own questions in the comments.

This is all for today.