BEYOND GRATEFUL

A life changing-experience brings Kayo Yin (SCS 2022) to CMU and drives her visionary research.

SHILO REA

At age six, Kayo Yin (SCS 2022) moved with her family from Japan to Paris, and she quickly began to learn the importance of language.

“My mother struggled to learn French,” Yin said. “It made everything from going to the grocery store and doctor visits to connecting with people challenging.”

Yin, now a master’s student in the Language Technologies Institute, eventually became fluent in Japanese, French, Mandarin and English, propelled by spending time in different places. She found learning local cultures considerably easier when she could communicate with people.

But Yin and her mother have the ability to hear, and they primarily use spoken languages. Learning to speak, write and read a new spoken language is becoming increasingly easy with resources such as automatic translators and language learning apps.

During her college years at Ecole Polytechnique in Paris, Yin volunteered at a homeless shelter where she met a deaf woman.

“Being deprived of language made her isolated,” Yin said, “Growing up, she didn’t have access classes for any signed language. She did learn to read and write in French, but that was not sufficient to allow her to fully express herself or make meaningful connections with people in her daily life.”

Yin started learning French sign language and pursued sign language translation as part of her undergraduate research. This life-changingexperience helped bring Yin to Carnegie Mellon and the LTI.

Her research on machine translation focuses on breaking down communication barriers between people, notably those in marginalized communities and those who use less common languages.

Natural language processing uses computers to understand what humans say and write, helping people communicate through tools such as machine translation, voice-controlled assistants and chatbots. But it primarily concentrates on spoken language, excluding the more than 200 signed languages around the world and the nearly 70 million people who rely on them to communicate.

“Siri is great for English, but not all languages. There isn’t a Duolingo -type app to learn a signed language. I want to widen access to language technologies for all communities,” Yin said. “We can use what we know already about spoken languages and build tools for a whole family of languages that are currently overlooked.”

Her vision has gained significant traction. Over the summer, Yin co-authored “Including Signed Languages in Natural Language Processing,” that won the Best Theme Paper at the 59th Annual Meeting of the Association for Computational Linguistics. The paper’s recognition has sparked discussions at CMU and other institutions to include signed languages in more research. Yin has been contacted by researchers from around the world about potential collaborations.

She is beyond grateful.

““It gives me tremendous motivation to see my work make a change to the real world and to real people,” Yin said. ““I’m fascinated by how signed languages work and how meaning is conveyed, and it is very exciting to be able to use that knowledge in my research.” ■