Story image

Facebook faces new frontiers with real time brain-to-text research

01 Aug 2019

Back in 2017 Facebook announced that it was going to team up with the University of California, San Francisco (UCSF) to develop a wearable device that lets people type words by simply imagining themselves speaking those words aloud. 

Two years later, the research is well underway. This week Facebook provided an update into how the research is going, and its potential application in augmented reality (AR) glasses. 

Facebook believes that AR glasses will help to realise a future that’s accessible, hands-free, and instant – regardless of distractions, geography, or disabilities.

Facebook’s research is still in the early stages. It takes the concept of brain-computer interface (BCI). In the past, the technology has allowed people to feed themselves, and to fly a jet simulator, says Facebook Reality Labs research director Mark Chevillet.

But there have been limitations to BCI, because all actions have required implanted electrodes. Chevillet says the long term goal is to turn BCI and its possibilities into the reality of a non-invasive and wearable device.

While voice assistants like Siri and Google Assistant are popular at home or in the office, how many people have used them in crowded rooms, or in a busy street? That’s where ‘thinking’ words comes into the equation.

Facebook Research Labs didn’t know if a completely silent speech interface was possible, let alone how it would work with the brain. Currently electrodes are still the go-to for making BCI work.

Chevillet and UCSF neurosurgeon Edward Chang shared similar goals for BCI technologies. They conducted their own studies into whether brain activity associated with speaking could be used to transcribe text on a computer screen.

The researchers sampled a group of volunteers who were already getting brain surgery treatment for epilepsy. Their algorithm was able to decode and transcribe a full set of spoken words and phrases in real time. The algorithm can only recognise a tiny proportion of words, but it’s a start.

Facebook and UCSF see the potential for infrared as a way of making non-invasive technology work.

“We don’t expect this system to solve the problem of input for AR anytime soon,” Facebook admits. 

“It’s currently bulky, slow, and unreliable. But the potential is significant, so we believe it’s worthwhile to keep improving this state-of-the-art technology over time. And while measuring oxygenation may never allow us to decode imagined sentences, being able to recognise even a handful of imagined commands, like ‘home,’ ‘select,’ and ‘delete,’ would provide entirely new ways of interacting with today's VR systems — and tomorrow's AR glasses.”

Optical technologies in smartphones and LiDAR can also help build better BCI devices that work more accurately than electrodes – perhaps they could be the answer, Facebook says.

There are also other issues that BCI technologies need to take into account.

“For example, how can we ensure the devices are safe and secure? Will they work for everyone, regardless of skin tone? And how do we help people manage their privacy and data in the way they want?” Facebook asks.

Chevillet says the researchers have already learned a lot, and the next step is to start talking with the community. BCI technology is still in its infancy, but it’s never too early to talk about ethics.

“We can’t anticipate or solve all of the ethical issues associated with this technology on our own,” he says.

“What we can do is recognise when the technology has advanced beyond what people know is possible, and make sure that information is delivered back to the community.”

“Neuroethical design is one of our program’s key pillars — we want to be transparent about what we’re working on so that people can tell us their concerns about this technology," Chevillet concludes.

Story image
06 Nov
Game review: Hideo Kojima presents Death Stranding
Death Stranding is a fun and unique game if you want to play something new and different. If you want all-out action, you may want to go out and play something else instead. It’s best to research the game as much as you can before you decide to buy it though.More
Story image
04 Nov
Diablo IV announced at BlizzCon 2019
An ancient evil threatens to engulf Sanctuary in darkness, and new heroes have been called upon to face challenges the world has never seen before.More
Story image
11 Nov
PwC report: Nextgens will lead family businesses into the digital age
Family businesses should look to next generation (nextgen) leaders if their company is to thrive in the digital age, but that can only happen with greater support and trust by those currently in charge.More
Story image
14 Nov
Hands-on review: OPPO ENCO Q1 Wireless Noise Cancelling Headphones
The tone and clarity is brilliant. I’m just waiting to see how they cope with Barry Gibb’s falsetto. The melody and lyrics are clear with rich tones. Oh no! I can feel my feet tapping and my arms want to disco! Help! More
Story image
18 Nov
Game review: Red Dead Redemption 2 (PC)
If you’ve the PC for it, the PC version of Red Dead Redemption 2 is a huge improvement over the console version, which is really saying something as the game looks fantastic on PS4 and Xbox One.More
Story image
11 Nov
Game review: Mario & Sonic at the Olympic Games Tokyo 2020
Since the release of Mario Party, Nintendo fans have been waiting for a new party game full of mini games they can play alone or with friends. Mario & Sonic at the Olympic Games Tokyo 2020 is exactly that.More