Back to Bigger Picture

BIGGER PICTURE Accessibility & independence

Why imagining a complicated letter beat pointing at a key

A paralysed hand cannot put ink on paper. But the brain activity associated with writing can remain distinguishable years later. Stanford used that surviving skill to create a text interface.

A tiny BrainGate electrode array and its fine wires in front of a US dime for scale.Original photograph
A BrainGate electrode array shown against a US dime. Arrays of this kind record neural signals used to decode attempted handwriting.BrainGate.orgEditorial reuse with attribution
Copying speed
90 characters/minute
Free composition
73.8 characters/minute
Study participant
One man with spinal cord injury

How the idea works

  1. Attempt a letter

    Handwriting intentions generate distinct neural sequences.

  2. Recognise its pattern

    A trained recurrent network estimates the intended character.

  3. Build the message

    Recognised characters appear as self-paced text.

A familiar skill in an unfamiliar interface

The participant, identified as T5, attempted to write letters as though holding a pen. Sensors in the hand-related motor cortex recorded neural activity while his actual hand remained paralysed. Researchers trained a recurrent neural network on these sequences. The computer learned patterns associated with intended characters, not the visual appearance of handwriting.

Sources: [2]

Complex motion can be easier to recognise

A letter contains a distinctive sequence of curves, starts and stops. The researchers found that these changing patterns were more separable than simple straight movements used to steer a cursor. Complexity supplied information. That helps explain why replacing a point-and-click keyboard with attempted handwriting could increase communication speed.

Sources: [1][2]

He answered questions, not just prompts

Copying unseen sentences reached 90 characters per minute. When T5 composed answers to open questions, the rate was 73.8 characters per minute. These are characters, not words. Free composition matters because a useful communication aid must carry what its user wants to say, not merely reproduce a displayed exercise.

Sources: [1][2]

The keyboard was still incomplete

The experiment did not yet offer ordinary editing and deletion. Its much-quoted accuracy above 99% included an autocorrect language model applied afterward, not live. Daily calibration and the implanted hardware also remained practical hurdles. The study demonstrated a valuable decoding principle, not a finished consumer writing device.

Sources: [2]

The original sources

  1. Stanford HAISoftware Turns ‘Mental Handwriting’ into On-screen Words
  2. Nature · author manuscriptHigh-performance brain-to-text communication via handwriting

What to explore next

He opened a cabinet with a robotic arm controlled by brain signalsA weather forecast learned from decades of weather
Explore all 99 applications