Like Magic, Get Audio From Video Only Recordings

This is pretty cool, though the fine print shows big limitations.

Watch the video:

The two big constraints are:

  • Massive processing power: "Processing each [15 second] video typically took 2 to 3 hours using MATLAB on a machine with two 3.46GHz processors and 32GB of RAM."
  • The frame rates of the videos averaged 2000fps - that's a lot...

Also, it appears that the pixel density of the object needs to be quite high as the FoV in the images is very tight around the object being monitored.

Surely there's espionage / military applications but general use is far off.

For more, see the project's home page and its research paper.

Login to read this IPVM discussion.
Why do I need to log in?
IPVM conducts unique testing and research funded by member's payments enabling us to offer the most independent, accurate and in-depth information.

**** ***** ** ****** ****. :)

*** ***** ***** ** *** ****** ******** ******* - ****'* a ***...

************** ***** *****-************ **** **** ********* ** **** ******...

** ***** ***********, *******, **** **** ** ******** ******* ******. Because ** *quirk ** *** ****** ** **** *******' *******, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second. While this audio reconstruction wasn't as faithful as it was with the high-speed camera, it may still be good enough to identify the gender of a speaker in a room; the number of speakers; and even, given accurate enough information about the acoustic properties of speakers' voices, their identities.

* ***** *** '*****' **** ** *** ******** ******* ******* of ****. ***** ** ** *** * ******** ** *** particular ****** ** ****, **** * *** ** ****** ******* is, ** ** ****** *********** ** * ***** ** ***** basis ******** ** ** * *** ********* ** *** * pixel *****!?!

***, **** ******* * ***** ***** ***** *** ****** ** an ***** ********* ****** ********* ** *** ******** *** * could *** **** ** ** ***** ******* ****.

**********, ***** ** * *** ******* ********* ** ***** ******** or **** **** ***** *** **** ******* ** ***** **** 30x *** ***** ****.

**** ** **** *******.**********:

*** *********** **** ******** * ********* ** *** ********* *** analyzing ************ *****. *** ****** ** * ******* ****** ******** of ** ***** ** **************—******** ** ****, **** ** ********* devices. ** ** ***** ***, **'* **** ********* ** ****** the ****** ******** ** **** ** ***** *** *** ************ ofone *** ** ************** ** * ****. Ordinarily, that's not a problem, but with fast-moving objects, it can lead to odd visual artifacts. An object—say, the rotor of a helicopter—may actually move detectably between the reading of one row and the reading of the next. For Davis and his colleagues, this bug is a feature. Slight distortions of the edges of objects in conventional video, though invisible to the naked eye, contain information about the objects' high-frequency vibration. And that information is enough to yield a murky but potentially useful audio signal.

*** ***** ********** ** ***** ****** (********* ********) ** ** high ** **** **, ***** ** *** *** **** *** mere *****(** **) *****. *** ***** * ***** = ***** hz, ***** **** ***** ******** *** *** ****** **** ** 8000hz, ** ******** ** ********** ******, ** ***** ********** ** ******* *************. ***** * **** off ** *** ***, *** ********... ** ** ******* ******* a ****** **** *** **** *** *** ****-*** ******, ***'** know **** **** *** ** **...

*** ** **** *******, **** **** **** *** ****** ** rolling, ******* *** ******* ***** ** ***. ;)

********! *** ****** *** *********.

* ***'* ******* **** ****'* **** *** ******* ***** ** a *** ******* ***.

**** ********* ** *******:

** *** *******, "*********," *** *******, *** ********* *** ***** ** * **** at *** *** ***** ***** ***, ****** ********* **** * psychiatric *********** ** *** ** ***** * ****** *****. ...******* hypothesizes **** ****a ******* ***** *** ******* **********, he might derive ****** from the clay that could provide clues. With a computer hook-up, they hear a high-pitched noise and something that sounds **** * **** — the mother's pet name for her son.

**** ** ****** ** **** *** ****-***-*** ******** ** *** sounds ***** ** *** **** *** *** ********** ***** ******* in *** ****, *** ****'* ***. ** ***** **** *** the ******* **** *****.

**** ***********. **** ***** ** ***** * ***. * *** this *** **** **** **** ** ********/********** *** *****

**** ** *** ***** ******** ** **** **** *** ** "The **** ******". **** **** ***** **** ********** ** *** out * ***** *****.