How one group is remaking sign language for the VR age

Jenny was taught to sign as a baby, but she only learned the basic stuff — enough to get by.

The Californian, who was born with auditory processing disorder and a right eardrum that doesn’t vibrate in response to sound, was raised with what community members call the auditory-oral method, meaning she learned how to speak out loud and read lips. Jenny went to school with both D/deaf and hearing children. (Jenny and others interviewed for this story asked that their full names not be used for privacy reasons.)

But as she got older, Jenny’s interest in sign language grew. She became more involved in her local D/deaf community when she reached middle school, and began to study the language independently.

(Capital-D “Deaf” is a term reserved for those who are prelingually Deaf, use signing as their first language, and grew up around Deaf culture, while “deaf” is used for people with severe hearing problems or later-life hearing loss who use signing as a second language. “D/deaf,” meanwhile, refers to both groups.)

Unfortunately, the local cohort of D/deaf people in Jenny’s area were largely elderly — and the generational gap was just too much. She was desperate to connect with likeminded people who shared her interests. So when she came across a YouTube video of D/deaf people signing on the VRChat platform in 2019, she saw it as an opportunity and got herself a headset.

“I was so flustered and so confused about how to use VR controllers to sign,” says Jenny. Then a friend introduced her to Ray, a Deaf VR user who was fluent in online sign language. “He sat with me [in VR], and we went through a bunch of signs together,” she says.

Ray welcomed her into his 30-person VRChat community, in which people, using digital avatars, communicated using an adapted version of American Sign Language. “It was such a beautiful experience, and everyone was so kind,” Jenny says. “They welcomed me into the friend group. They became my people.”

That bunch of friends has since evolved into an organization called Helping Hands, a 5,000-strong online community based on VRChat and Discord that aims to better connect D/deaf and hearing people. Together, the all-volunteer group — which includes everyone from first-language signers to hobbyists — has created a VR-friendly version of sign language and is teaching it to people from all over the world.

Today, Helping Hands offers classes for American, British, Korean, French, and German sign language, as well as opportunities for D/deaf and hearing people to socialize through VRChat. “We have parties, game nights, movie nights, hangouts. We have places for you to practice signs,” says Jenny, a key organizer and interpreter.

The community is home to both certified sign language translators and amateur volunteers who help translate for D/deaf members when needed. In addition, there are hearing people who join Helping Hands for a range of reasons — including a desire to work around speech disorders like selective mutism, which can make it difficult for some people to speak out loud — and the group stresses it is not a D/deaf-only organization.

“We’re trying to make VRChat accessible to everybody,” says Mike Culver, a 39-year-old hearing man from Michigan who was introduced to the D/deaf community through an IRL friend he met while studying computer science at college.

“Now, in VRChat, most people will be familiar when they see people signing, because they know about Helping Hands,” Culver adds. “It’s been a huge improvement from what I remember a couple of years ago.”

Changing lives

Although the community is small, it's been life-changing for many Deaf members, who often find themselves isolated in the physical world.

“In real life, I need to save up money and plan with other Deaf people to do live meetups, and then we separate out again,” says DM, a 25-year-old member of Helping Hands. DM, who is from Kentucky, lost his hearing as a baby as a result of a respiratory virus. “We want to do it more often, but it takes more time and energy — and we couldn’t do that during COVID.”

DM found out about VR’s D/deaf community when he stumbled across a video on YouTube, and spent years working and saving so he could build a computer, get the right setup, and join in. “We can interact and socialize with people from anywhere, without stressing about budget limitations and planning,” he says. “You can see each other just fine and make great memories by hanging out. I’m very thankful for that.”

Jenny says that the community has helped to connect both D/deaf and hearing people with sign language and Deaf culture. “I’ve met several D/deaf people who were raised hearing and don’t sign, or who lost their hearing at a later age, and they use our community to learn,” she says. (According to the National Institute of Deafness, more than 90 percent of Deaf children are born to hearing parents, which can further isolate them from the Deaf community.)

“A lot of Helping Hands students have become interpreting or Deaf studies majors at college,” Jenny adds. “We’re like a stepping stone for them.”

Still, the community has its limitations. “In real life, we can manipulate our hands in all these different ways for clarity. But in VRChat, we can only do what the controllers allow, and our hand doesn’t exactly match,” explains DM. The community has had to create their own solutions to sign effectively.

For instance, instead of crossed fingers, which aren’t possible in VR, Helping Hands signers twist their hands to imply that their fingers are crossed. (D/deaf users are also somewhat held back by VR’s lack of facial tracking, as facial expressions are vital to communication in sign language.)

“We’ve tried to make the adjustments as fair to everyone as possible,” DM adds, “as not everyone can afford index controllers,” which allow users to make basic hand gestures and can cost hundreds of dollars. Those without such devices are limited to seven different hand gestures in the VR world.

Culver’s hope is that VR tech becomes more accessible for D/deaf people everywhere, and that their needs are considered more frequently by developers. It’s already looking good. “We periodically get meeting invites or casual drop-ins with developers and designers, and they check in with us to see what we think of their products,” he says. “We’re making sure that accessibility is a frontline consideration for platforms and hardware systems by giving feedback.”

Like Culver, Jenny wants to bring sign language to as many people as possible. She’s currently touring the country as part of a documentary she’s featured in, We Met in Virtual Reality. Her wish is that the film will open up the world of VR sign language to more people.

“I hope that we will just grow and grow and grow, and get better at what we do,” she says excitedly. “I’m already seeing that happen every day. I hope it doesn’t stop anytime soon.”

VR Week 2022 is an in-depth look at the decades-long future of VR successes, failures, and innovations to come.

More VR Week 2022 stories:

• 'Zenith' is an early access take on VR 'World of Warcraft'
• VR needs to address the huge motion sick elephant in the room
• Hot job alert: metaverse architect
• The 9 best apps to get started working in VR
• Inside the volunteer 'police department' arresting people in VR
• There’s no VR internet in our future — at least not one we’d want
• The 11 best hardcore VR games you should play right now
• From ‘The Lawnmower Man’ to Meta: VR’s most important moments