To improve the quality of communication with deaf people, the developers of the SOL Surdo-Online service presented a new solution, where the translator is no longer a person, but artificial intelligence.
The service itself is a video conference between a person with a hearing impairment, a service provider and an intermediary - a Russian sign language translator. The developers want to equip as many public places in Russia and abroad as possible with their platform, so that deaf people are on an equal footing with others in receiving information and services. The service is already in use at some airports, pharmacy chains and diagnostic medical centers.
Using the service & quot; SOL Surdo-Online & quot; in pharmacy / SOL Surdo-Online
“There are approximately 200 thousand deaf and one million hard of hearing people living in Russia - sign language speakers. Their ability to communicate with the outside world is minimized, and the state provides only 40 hours of free sign language interpretation. Using the online platform that translators work on can improve translation efficiency by at least five times. This is how the idea of the Surdo-Online project was born, which we launched in September 2016. Now we have decided to improve the system by introducing artificial intelligence”, - explained the founder of the project Alexey Melnik.
Now the service works according to the following scheme: companies are provided with access to a cloud service, where they can contact sign language interpreters online in video mode. Deaf people communicate with them in sign language, after which the translators transfer the information to the service providers and vice versa. The service works through a regular Google Chrome browser. That is, any tablet or computer is connected to it, and infrastructure costs are reduced to almost zero.
Using the SOL Surdo-Online service
However, by the end of 2018, the developers plan to release a pilot prototype of the video platform, where artificial intelligence is used for gesture recognition. That is, the program will be able to independently communicate with deaf people without the help of translators. She will recognize gestures using a trained neural network. Thanks to numerous videoconferences with the participation of sign language speakers, a database of gestures is now being assembled - a dataset. Each of them needs at least 1000 entries from different people. And the total vocabulary of the Russian sign language is more than 10,000 gestures. The main difficulty lies in the volume of gestures - this is a 3D language that needs to be digitized in order for the translation process to become fully automated.
Using the SOL Surdo-Online service
Promotional video:
Now the founders of "Surdo-Online" are residents of the socially oriented accelerator Phil. Tech. On September 26-27, the team took part in the international EdCrunch conference, where the project was evaluated by foreign experts. According to Peter Childs, Director of the Innovation Design Engineering Program in the UK, SOL Surdo-Online is a realistic project that already has a working prototype of the device. The project still needs investment and advice, but it already demonstrates high potential even at the international level.