Study co-author Maitreyee Wairagkar, a neuroscientist at the University of California, Davis, and her colleagues trained deep-learning algorithms to capture the signals in his brain every 10 milliseconds. Their system decodes, in real time, the sounds the man attempts to produce rather than his intended words or the constituent phonemes — the subunits of speech that form spoken words.
This is a really cool approach. They’re not having to determine speech meaning, but instead picking up signals after the person’s brain has already done that part and is just trying to vocalize. I’m guessing they can capture nerve impulses that would be moving muscles in the face, mouth, lips, and possibly larynx and then using the AI to quickly determine which sounds that would produce in those few milliseconds those conditions exist. Then the machine to produces the sounds artificially. Because they’re able to do this so fast (in 10 milliseconds) it can get close to human body response and reproduction of the specific sounds.
This is nothing short of stunning, had no idea anyone was even close to this sort of interface. And it’s only an 8-bit input! Fuck me, I would have made a (totally ignorant) guess of at least a couple of thousand sensors.
Hoped for a video. :(
it’s 256 electrodes, yes, but the article doesn’t say whether those electrodes are simple digital signals or if each one has some analog range they resolve. Even if it’s 100% binary, the tresholding (what level of neural activity is considered a 1 or 0) could be adaptive.
This is amazing technology. I can’t imagine how it would feel to have your ability to speak and even sing back after losing it.
I also wonder whether this could let someone sing or speak outside their normal range. When I was young I was in the school choir - after puberty hit and my voice dropped, I still had the muscle memory for high notes that my vocal cords would not allow.
Ah! Didn’t consider the input could be analog and not 1 or 0. That opens up a wide range!
Coming soon to an interrogation room near you
This is exciting and terrifying. I am NOT looking forward to the future anymore.
Ok…that’s a bit scary for me.
