UKZN graduate develops AI tool to translate South African Sign Language

Inspired by his experiences growing up with deaf parents, the technology aims to help deaf people communicate more independently. A UKZN graduate could help address communication barriers faced by people who are hard of hearing in South Africa. Growing up with deaf parents, Akhil Hansrajh saw first-hand the difficulties they experienced when accessing services at banks, government departments and municipal offices - often relying on family members to speak on their behalf. “So, what I've designed is a software that watches a person sign through a camera and says what they've signed out loud as they sign it.” Graduating with a Bsc Degree in Computer Engineering - Hansrajh from Pietermaritzburg - has developed a South African Sign Language-to-Speech Translation System as his final-year project. The 24-year-old says a production-ready version of the technology could allow people to communicate more independently, without requiring others to know South African Sign Language. “The camera tracks the position of the person's hands, arms, and face, and an AI model that I trained with my own recordings then looks at how these points move over time and decides which sign it is seeing. Once the system is confident, a text-to-speech voice says the letter, the number, or the word that was detected. “So, in theory, a deaf person can sign to a teller who's never learned sign language, and the teller hears these signs in English.” Hansrajh says a major challenge during the project was developing technology specifically for South African Sign Language. He explains that while the South African and American sign-language alphabets are broadly similar, significant differences emerge when moving beyond individual letters. “For instance, if you were to sign the word name in American Sign Language, in South African Sign Language, it actually refers to toilets. So, a model trained on American signs would confidently say the wrong word in a South African context. Additionally, the grammar is different between both languages. In South African Sign Language, you would sign ‘your name what?’, whereas in American Sign Language, you would sign ‘what is your name?’” ALSO READ: Ramaphosa extends Madlanga Commission deadline to January Hansraj says another limitation is that the system currently works in only one direction. While it can allow a deaf person to communicate with a hearing person- the hearing person's response would still need to be written down or understood through lip-reading. Hansraj says he has not yet actively pursued funding or formal partnerships to commercialise the technology- saying his immediate focus after graduation has been starting his career. However, he says he is open to discussions with banks, government departments, universities and organisations within the deaf community that may be interested in developing the project further. Looking ahead, Hansrajh plans to continue developing his sign language translation technology - with a focus on improving its ability to recognise words and generate more cohesive and fluent speech. He says for him, the ultimate measure of success would be a future where a deaf person can walk into a bank or municipal office independently and communicate without needing another person to interpret. Follow the ECR Newswatch WhatsApp channel here We are also on Facebook and X (formerly Twitter) Friday's cold southerlies and rain mean staying inside with coffee an… Nine women. Nine lives lost. And one difficult question for parents: how d…