Robotic sign language interpreter for deaf Māori in development

New robotic interpreters built by experts at the University of Auckland will bring sign language underpinned by a Māori worldview to Turi Māori - Māori who are deaf. Dr Tauwehe Tamati, who is leading the project alongside Dr Ho Seok Ahn told Morning Report, the signs have been developed in partnership with Turi Māori, whānau, Iwi and Hapū. "I think that's why I got involved in the project in the first place. That would have been at least five years ago when I was approached by our colleagues. "I had to think through how to connect the expertise of my colleagues in robotics, megatronics and so on. How do you connect it with a culture, with a people? And particularly with the most vulnerable of our own people." The robotic interpreters will bring sign language underpinned by a Māori worldview to Turi Māori - Māori who are deaf. Tamati is bilingual in Te Reo Māori and English. She used both languages and a whānau-based home sign with her father who was hard of hearing for most of his life, becoming profoundly deaf in his final 20 years. "Our dad grew up in Te Ao Māori. He knew he was Māori and he was a hard worker at our pā. The cultural, social, and collective life on the marae continually reinforced his identity and sense of belonging. "Over time, however, he stopped going to the pā and didn't like relatives coming around. "We'd be making up excuses because he was deaf. We couldn't help him get through that. So when my engineering colleagues approached me about the project, I just thought of him and said, 'I'll do this for dad."' Researchers used Māori sign-language models who were filmed and photographed, to capture the different elements of communication, including mannerisms and lip movement. The model robots have been made at both adult and children's sizes. An advanced AI system called a weighted multimodal network was then used to develop the robots ability to interpret the signs. The University of Auckland says the robots are now built and functioning within its research lab. Models have been made at both adult and children's sizes. Work on the project began more than five years ago as part of the Science for Technological Innovation National Science Challenge, with additional financial support coming from the Vision Mātauranga Fund and the Endeavour Fund in 2024. "I think our project now has reached such a level of success that we do have all the people in the hapū excited and participating. "They don't need to hear us. They just look around and they know they can feel they are the reason why we're all helping," Tamati said.