“Yandex Browser now” has a function to translate video content from Chinese, report Yandex representatives.
The neural network was trained not only to translate with subtitles, but to overlay voice acting in Russian when watching a video in Chinese. According to Yandex representatives, after adding the translation function from Chinese to Russian, it became possible to work with other languages of similar complexity - Japanese and Korean.
In the process of translating video content, the audio sequence is first converted into text, then the neural network gives the resulting text a more literary look, after which the resulting text is sent for translation into Russian. The voice acting of the resulting text with the help of a neural network is synchronized with the video sequence.
In order to “teach” the neural network Chinese, it was first forced to learn 6500 hieroglyphs, as well as to learn four tones, with the help of which the meaning of what was said in Chinese is determined.
Now “Yandex Browser”, when watching a YouTube video in Chinese, will offer to translate the audio sequence into Russian.
