How does the noise canceling function using AI be implemented in the video conferencing tool 'Google Meet'?
Local time On June 8, 2020, the 'noise canceling function' utilizing AI was implemented in the web browser version of the video conferencing tool '
How Google Meet's noise cancellation denoiser works | VentureBeat
https://venturebeat.com/2020/06/08/google-meet-noise-cancellation-ai-cloud-denoiser-g-suite/
The noise canceling function, which is newly installed in Google Meet, is a function that removes 'noise' such as the barks of dogs and the driving noise of vacuum cleaners from the calls of users who participate in video conferences. You can understand the power of this noise canceling function by watching the following movie.
Google Meet noise cancellation-YouTube
In the movie, it shows the state of talking on Google Meet with a man who says 'It's 1 minute before boarding an airplane'. Noises such as human voices are included in the voice, making it difficult to hear the male voice.
However, turning on the noise canceling function...
The noise disappears. It is possible to hear only the male voice.
This noise canceling function is implemented in the web browser version of Google Meet, but it will be enabled for each account one by one, so it is expected that it will take more than one month to be available for all accounts. The noise canceling feature will be implemented in Android Me and iOS Meet Google Meet, but it is not revealed when to start.
The noise canceling function is turned on by default in the web browser version of Google Meet that implements the noise canceling function. If you want to turn off the noise cancellation function, select 'Audio' tab from 'Settings' and tap the 'Noise cancellation' toggle button.
This noise cancellation function distinguishes 'noise' and 'non-noise' by AI. This AI is trained by supervised learning , and the development team advanced the learning of AI using a labeled dataset. According to Lachapelle, who led the development team, the development of the noise cancellation function started in the autumn of 2018, and it has been used in Google for over a year.
During the development, Lachapelle explained, 'We extracted audio from Google internal meetings and YouTube movies and used it for AI training.' In addition to actual meetings, experiments are also being conducted to 'discriminate between intentionally created noise', and sometimes engineers also brought in maracas, guitars, accordions, etc. to work.
It seems that there were various difficulties in developing a noise cancellation function, and the first problem was to decide 'which sound is noise', Lachapelle said. It's impossible for everyone to agree on which sounds are noise, but when training AI you have to decide 'what sounds are noise'. Mr. Lachapelle gave examples of noise such as 'slamming doors,' 'dog roaring,' 'musical instrument sound,' 'vacuum cleaner sound,' but on the other hand, 'child fighting' sounds. Says that it should not be canceled, so it does not treat it as noise unless it becomes an extreme sound. We will continue to adjust what sounds are noise, and we will continue to consider it over a long time, Lachapelle said.
Lachapelle has also released a movie demonstrating the noise cancellation function. You can understand how the noise cancellation function works in a real environment by watching the following movie.
Google Meet's noise cancellation feature in action-YouTube
In the movie, the rustling sound of Mr. Lachapelle touching the candy bag and...
The sound of clicking a knock-type pen
It emits 'unpleasant sounds during a call', such as the sound of a spoon hitting a cup.
However, when the noise canceling function is turned on, even if you touch the candy bag, you will hardly hear the rustling sound. However, when the noise cancel function is turned on, the sound seems to be slightly muffled.
To the level where you can hardly hear the clicking sound of the pen.
On the other hand, the sound of clapping seems to be erased.
Even if you decide which sound is actually noise, AI does not necessarily recognize that sound as noise. According to Lachapelle, it's difficult to cancel noise in a dog's voice roaring in a certain range. Human speech also has a fairly wide range, and it was difficult to identify the “range used in languages other than English” due to the bias in the data set used for AI training. However, based on the idea that 'it is important to support multiple languages', Lachapelle's team repeated the test, and it seems that the noise cancellation function has been made compatible with multiple languages.
The goal of this noise-canceling feature is 'to improve conversation,' says Lachapelle. ``Mobile users are usually in a very noisy environment,'' says Lachapelle, as the popularity of smartphones has made it easy to make voice and video calls with anyone, anytime, anywhere. We have set the noise cancellation feature for noisy environments so that we can do it.'
Google Meet voice is encrypted and transferred from the device to Google's data center, then re-encrypted and then transferred to another user's Google Meet. When performing noise canceling processing on this audio, there are two possible processing methods: 'processing on the cloud side' and 'processing on the application side'. Lachapelle said, 'The processing on the application side is a heavy burden on the hardware' 'Because of the recent advances in infrastructure, even if media operations are performed in the cloud, there is only a delay of about 20 milliseconds,' he said. I have taken the option of 'processing with.' In addition, it is said that voice data during a call on Google Meet can not be accessed from the outside for purposes other than noise cancellation processing, and it will not be used for AI training, Mr. Lachapelle says privacy Emphasizes that is maintained.
Due to the epidemic of the new coronavirus infection, the number of active users of video conferencing tools has increased 20-fold between December 2019 and April 2020. The video conferencing tool with the largest number of users at the time of article creation is 'Zoom', but Google has made Google Meet free on April 30, 2020, and is showing a stance against Zoom.
Related Posts:
in Web Service, Web Application, Video, Posted by darkhorse_log