Listening to Learn - Audio Recognition Improvements in Computer Systems

krnel (78)in #artificial-intelligence • 10 years ago (edited)

Does Siri not understand you and give you bad answers?

Does DragonNaturally Speaking not understand what you said and write something completely different?

Can these features be improved? Yes. We can get better cellphone, smarkphone and robot assistance.

How? By using the tested and working techniques of video learning, and using them for audio learning.

As of now machines have been learning through online videos, and can do great facial recognition and other video recognition tasks. The video learning is surpassing the audio learning because the online videos are often tagged to identify many features within the video. So video has had the advantage of learning through labeled videos.

Now objects are recognized. Taking the learning from the video, and going out to unlabeled videos, the machines are taught to associate the video objects with sounds in the video frames and link them together.

The team at MIT says they are transferring data from the domain of video to the domain of sound where they don't have as much data. The program is called SoundNet.

Their computer program was given over 2 million Flickr videos, totaling over 1 year of run time.

How did the machine learn the unlabeled sounds? It would associate the object in the video, movements, etc., with the sound that was produced at the same time. Babies were linked up to baby babble.

In a test to sort between the sounds of rain, sneezes, roosters and ticking clocks, the computer has a 92.2% accuracy rate, while humans perform at 95.7% accuracy. Some sounds are giving the SoundNet app trouble. Footsteps can be mistaken for door knocks. There is still room for imporvement and learning, and it is being done. But so far, they have had good results.

The study will be fully presented at Barcelona's Neural Information Processing Systems conference in December.

Advances in machine learning through audio can help human-computer interaction greatly. We use our eyes to look at the world around us. Our eyes are like the computers graphics capability. The next dimension for understanding reality is the audio realm.

Siri and other voice recognition software will be much more able to help us navigate and find information through computer technology. Currently, background noises prevent accurate interpretation of your own voice. This learning methodology can help the computer program not confuse our speech with the distracting noises of a clock, ambulance siren, dog barking, etc.

Another aspect of improvement would be home security. Earlier threat detection can be done with improved sound recognition, such as windows shattering, or a smoke alarm sound.

With better voice recognition, we could even talk to them in an interrogatory style to question and get answers from them. No, they wouldn't feel interrogated, but they would be able to understand our questions and the tone we are using. This could tie in well with Amazon's Echo and Google's Home assistant.