WOW! M.I.T. Researchers can recreate sound from objects in the room

a reply to: EvillerBob

Again this is IDIOTIC.

Where did I say only the sound of a human voice will be recreated?

Here you go again. Debating against a claim that I never made.

You said:

We are not saying that it is impossible to read the vibrations of the bag caused by sound in the room. We are saying that you will end up reading ALL the vibrations of the bag, which will be caused by a combination of all other sounds and sources of vibration in the room. If it is a perfectly quiet room with only one person talking, the process will work. The experiment proves that. If you have lots of noise, or lots of people talking, all of those will be causing the bag to vibrate.

I have said over and over again that the sounds from the room are recreated as a WHOLE!!! WHY ARE YOU GUYS SO DISHONEST???

You keep trying to debate against things that I never said!!

Of course other sounds will be recreated and I have said that over and over again.

PLEASE TRY TO DEBATE AGAINST THE THINGS i HAVE SAID!

a reply to: ChaosComplex

I'm pretty sure that that tech is what is being used.

The Michael Rubinstein, featured in the video (0:48), is the same Michael Rubinstein that co-authored 'The Visual Microphone' report.

Edit to add:

Yup, same guy, both the report and the video are cited on his bio page:
Michael (Miki) Rubinstein

originally posted by: neoholographic
a reply to: EvillerBob

PLEASE TRY TO DEBATE AGAINST THE THINGS i HAVE SAID!

A debate generally has merits on either side.

This isn't a debate. This is you being wrong, and us being polite enough to try and turn it into a learning experience rather than just laughing at you and walking away.

a reply to: ChaosComplex

Yes they directly reference this work

Four pictures showing small section of voice waveforms of two people in a room. Lower half grayed out to keep it simple.

Waveform of person A

Waveform of person B

Combined waveform of person A + B

No part of the crisp packet cant vibrate in two directions at the same time so the waveform produced by the crisp packet would be something similar to this

As can be seen, it does not resemble either of the two voices.
Two voices heard at the same time would defeat this device.

Is my understanding of this correct?

a reply to: VoidHawk

As can be seen, it does not resemble either of the two voices.
Two voices heard at the same time would defeat this device.

Is my understanding of this correct?

I think you are correct but I am not sure "defeat" is the right word. There are definite limitations on what it can do though. Its not really a "device" either more like a technique that is in its very early stages of development.

From their paper people.csail.mit.edu...

Discussion and Limitations
Information from Unintelligible Sound. Many of our examples
focus on the intelligibility of recovered sounds. However, there
are situations where unintelligible sound can still be informative.
For instance, identifying the number and gender of speakers in
a room can be useful in some surveillance scenarios even if intelligible
speech cannot be recovered. Figure 11 shows the results
of an experiment where we were able to detect the gender of
speakers from unintelligible speech using a standard pitch estimator
[De Cheveigne and Kawahara 2002]. On our project web page
we show another example where we recover music well enough for
some listeners to recognize the song, though the lyrics themselves
are unintelligible in the recovered sound

a reply to: ZetaRediculian

Agreed "defeat" maybe wrong
I only posted it as OP seemed unable to grasp the idea that the crisp packet can be influenced by more than one sound.
As you and many others have said, this demo was conducted under ideal conditions. In the real world it's seriously limited.

If they do ever manage to make this work reliably, just imagine combining it with tech that can see through walls, something similar to the scanners we sea in airports?

a reply to: neoholographic

Again, this has nothing to do with the video or the article.

you mean this article where they reference the paper and that you quoted over and over?

In the experiments reported in the Siggraph paper, the researchers also measured the mechanical properties of the objects they were filming and determined that the motions they were measuring were about a tenth of micrometer. That corresponds to five thousandths of a pixel in a close-up image, but from the change of a single pixel’s color value over time, it’s possible to infer motions smaller than a pixel.

and on the website people.csail.mit.edu...

This is our 2014 SIGGRAPH paper. We will be posting the paper, video, data, and some source code soon!

And the paper and the YouTube video that have exactly the same title and list of names?


The paper where this was taken from


Yep. totally unrelated.

a reply to: ZetaRediculian

Of course it's unrelated because there's no optical plate below the bag a chips that they video taped through soundproof glass, earplugs or the plants.

This was one experiment that was part of several experiments that took place.

Show me where they said contact vibrations have anything to do with the vibrations used to recreate sound.

Show me the optical plates in the other experiments.

Show me where they said contact vibrations are the vibrations used to recreate sound.

It's just more nonsense.

a reply to: neoholographic

Of course it's unrelated because there's no optical plate below the bag a chips that they video taped through soundproof glass, earplugs or the plants.

You mean the optical plate this plant is on? Looks very similar to the photo of their setup.


I suspect its behind SOUNDPROOF glass to eliminate noise and not to demonstrate that they don't need a microphone. That's my guess!

Earbuds produce their own sound? Do you think that's different in any way?

So the example of their controlled experimental setup is wrong and completely unrelated to anything? You Win!

I'm hoping the guy from MIT stops by and you tell him he's wrong! That would be really funny and I would have a new respect for you. I might start peppering the whole team with emails until someone shows up. You have to do it. Please! That would go down in history. Dude! you are the master of the troll! You will be legendary like bloodninja! HA HA! I love it!

a reply to: ZetaRediculian

What optical plate?

Are you dense?

There's no optical plate under the potato chips when they recreated sound through the soundproof glass. None with the plant. It's in a vase.

In the picture with the optical plate you can see the chips sitting on a big square optical plate.

There's also none with the ear plugs.

Again, you're not making any sense here.

a reply to: neoholographic

Who said anything about an optical plate?

Are you a silly poopy pants?

What are you talking about?

The paper that this whole discussion is based on is wrong except for the parts you are talking about.

Try debating something I said.

The paper and the video have NOTHING to do with each other.

I totally get you now!

We're like two trolls in a pod.

a reply to: VoidHawk

This is a good consideration.

I wonder if they tested two people talking at once as you suggest and also allot of background noise on top of that to see how clearly it would come out.

If not I hope they do and we would all be interested in hearing the results.

I imagine in order to take out all the back ground noise or second voice they would need to break everything down with the software but like you pointed out, would the bag of chips have captured both voices and all the other noises vibrations with each their own vibration pattern so it would be easily deciphered. Or does it become a jumbled mess ?

Maybe if they only look at one object in the room, however what if they compiled the data from more than one object and were still able to easily split and piece it all together b/c different objects picked up all the vibrations ? If the bag of chips missed person B's second sentence b/c of back ground noise or person A began talking, perhaps a different object in the room captured it ? The density and way that each object itself captures vibrations could vary, but now we are getting much more complex.

leolady

a reply to: leolady

My apologies to the people looking for real information and actual discussion.

I will try to make up for the silliness.

Here is a link to the project page people.csail.mit.edu... If you scroll down, you will see some samples of their results.

Here is another link with more people.csail.mit.edu...

And the Actual SIGGRAPH Paper people.csail.mit.edu...

I didn't see any experiments with multiple speakers. As far as looking at multiple objects, I believe the camera had to be focused and zoomed in on one object. So that the object took up the whole view. I suppose you could do it with a better camera or multiple cameras.

It looks like they did everything to eliminate noise like filming in soundproof rooms, indoors, isolating vibrations. All of their experiments are of recovering one sound in a controlled environment. That they got this to work at all is amazing.

They did mention in their paper that they possibly could decipher that multiple people were in the room in the cases of unintelligible sound.

Discussion and Limitations
Information from Unintelligible Sound. Many of our examples
focus on the intelligibility of recovered sounds. However, there
are situations where unintelligible sound can still be informative.
For instance, identifying the number and gender of speakers in
a room can be useful in some surveillance scenarios even if intelligible
speech cannot be recovered. Figure 11 shows the results
of an experiment where we were able to detect the gender of
speakers from unintelligible speech using a standard pitch estimator
[De Cheveigne and Kawahara 2002]. On our project web page
we show another example where we recover music well enough for
some listeners to recognize the song, though the lyrics themselves
are unintelligible in the recovered sound

Don't forget to check out the super sexy video!
people.csail.mit.edu...

originally posted by: leolady
a reply to: VoidHawk

I imagine in order to take out all the back ground noise or second voice they would need to break everything down with the software but like you pointed out, would the bag of chips have captured both voices and all the other noises vibrations with each their own vibration pattern so it would be easily deciphered. Or does it become a jumbled mess ?

The vibrations are not multi-layered, the vibration of the bag is the sum of all the different vibrations trying to affect it - though this sum may vary depending on what part of the bag you are looking at. If the bag was between two people, then each side of the bag would be more heavily influenced by the sound waves coming from the person closest. The problem is, for this technique to be effective, the bag needs to be able to easily transmit vibration, which means that it will also easily transmit the two conflicting vibrations from one side of the bag to the other, as the two sides are not isolated.

I think that the technology as it stands would not be capable of isolating two separate simultaneous elements of audio from the one bag, to a level that would allow intelligible speech to be recovered. Two bags, each in front of a different person with some distance between them? More possible, but each would still be affected by all of the sound in the room.

What would be interesting is a stack of papers - one office I visit occasionally has stacks of paperwork on the desks and people talk to each other over the top of the stacks. What you effectively have is a collection of light, responsive edges (the edges of the paper sticking out of the stack, facing the speaker, with each speaker facing a different side of the stack) but attached to a heavier mass (the main body of the pile) which might serve to act as the vibration-equivalent of a "heat-sink" for the higher voice frequencies. You might be able to pick out the individual audio from each speaker just by analysing each side of the stack separately, though remember that sound waves bounce (ie echo) so sound from one speaker will still have some impact on the other side. You might even be able to use the two sets of information to filter each other, improving the quality.

All supposition and guesswork, but I think that would be an interesting experiment - and it's also a common real-world scenario!

Which reminds me - should I tempt fate by asking OP if he knows why it's almost impossible to fully compensate for echo?

originally posted by: ZetaRediculian

Ah. Now I see why Neoholographic jumped down my throat when I used the term "unwanted vibrations" in this response to me:

originally posted by: neoholographic

What are unwanted vibrations? Show me in the article on this technology where they talked about unwanted vibrations.

Again, you don't understand what you're talking about.

I guess I show have used the term "undesired vibrations" instead, like the MIT did in the article you (ZetaRediculian) posted.


Again, I think this is a very clever thing these guys did, and I find it very interesting that they could achieve this. Howebver, as this technology stands now, there are certainly limitations form unwanted -- errrr, I men "undesired" -- vibrations. A chip bag or a plant are not efficient speaker/microphone diaphragms, and cannot produce a dynamic range of sounds, so other vibrations and other sounds would mask the "target" sound too easily.

If the team could find a method for cancelling out these undesired frequencies, then this would be more than just a very-cool-yet-controlled demonstration.

a reply to: Soylent Green Is People

Again, a total lack of understanding as to what's being said. You said:

Again, I think this is a very clever thing these guys did, and I find it very interesting that they could achieve this. Howebver, as this technology stands now, there are certainly limitations form unwanted -- errrr, I men "undesired" -- vibrations. A chip bag or a plant are not efficient speaker/microphone diaphragms, and cannot produce a dynamic range of sounds, so other vibrations and other sounds would mask the "target" sound too easily.

Who said there were limitations because of unwanted vibrations? They were talking about contact vibrations in one of the experiments they ran. Here's how they described it in the paper.

The setup for these experiments consisted of an object, a loudspeaker, and the camera, arranged as shown in Figure 4. The loudspeaker was always placed on its own stand separate from the surface holding the object in order to avoid contact vibrations.

Again, they're talking about CONTACT VIBRATIONS in this one experiment. This has nothing to do with unwanted vibrations masking anything.

If you want to talk about Contact Vibrations specifically then I agree. You can't have a back of chips on the same surface area as a speaker playing music.

The only time they talked about vibrations that could possibly mask the sound is through CONTACT VIBRATIONS. This has NOTHING to do with the vibrations that are less than 100th of a pixel used to recreate sound.

For instance if you had 2 bags of chips on the same surface area then you wouldn't have CONTACT VIBRATIONS.

This is one experiment where the Researchers didn't want contact vibrations between the chips and the speaker so they put them on separate surfaces.

originally posted by: neoholographic
a reply to: Soylent Green Is People

Again, a total lack of understanding as to what's being said. You said:

Again, I think this is a very clever thing these guys did, and I find it very interesting that they could achieve this. Howebver, as this technology stands now, there are certainly limitations form unwanted -- errrr, I men "undesired" -- vibrations. A chip bag or a plant are not efficient speaker/microphone diaphragms, and cannot produce a dynamic range of sounds, so other vibrations and other sounds would mask the "target" sound too easily.

Who said there were limitations because of unwanted vibrations? They were talking about contact vibrations in one of the experiments they ran. Here's how they described it in the paper.

The setup for these experiments consisted of an object, a loudspeaker, and the camera, arranged as shown in Figure 4. The loudspeaker was always placed on its own stand separate from the surface holding the object in order to avoid contact vibrations.

Again, they're talking about CONTACT VIBRATIONS in this one experiment. This has nothing to do with unwanted vibrations masking anything.

If you want to talk about Contact Vibrations specifically then I agree. You can't have a back of chips on the same surface area as a speaker playing music.

The only time they talked about vibrations that could possibly mask the sound is through CONTACT VIBRATIONS. This has NOTHING to do with the vibrations that are less than 100th of a pixel used to recreate sound.

For instance if you had 2 bags of chips on the same surface area then you wouldn't have CONTACT VIBRATIONS.

This is one experiment where the Researchers didn't want contact vibrations between the chips and the speaker so they put them on separate surfaces.

And once again you post information that shoots down your own argument.

In this case, Contact Vibrations would be considered undesired vibrations, hence the seperation of the speaker from the surface holding the object to be observed.

And just how would two bags of chips on a single surface eliminate contact vibrations?

a reply to: ChaosComplex

Nope,

so other vibrations and other sounds would mask the "target" sound too easily.

First, they never said it would mask the target sound to the point where it couldn't be recreated. They just wanted to avoid CONTACT VIBRATIONS in this one experiment.

So other sounds wouldn't mask the recreation of sound coming from the room. That's just nonsense.

If you have a speaker vibrating on the same surface area of a bag of chips then you can have contact vibrations.

So if your a criminal you would have to have speakers in contact with every surface area throughout the room. You would have to have speakers on walls, tables, ceilings, beams and more and still the article said nothing about these undesired vibrations stopping them from recreating the sounds from the room.

They just said they wanted to minimize undesired vibrations in this one experiment. They never said these vibrations would stop them from recreating sound.

originally posted by: neoholographic
They just said they wanted to minimize undesired vibrations in this one experiment. They never said these vibrations would stop them from recreating sound.

.....If the vibrations weren't detrimental to the experiment, why would they go to the trouble of eliminating them?

It's as if you are reading this research with zero prior knowledge of how sound vibrations effect physical bodies, and interpreting what you are reading in your own special way.

You should read up on how a microphone works, and pretend the bag of chips (or plant, earbuds, whatever object that is being observed) is the diaphragm. Once you understand that relationship of the object to air movements caused by sound waves, then you will see why everything you are saying at this point is...well, for lack of a better term, absolutely ridiculous.

So if your a criminal you would have to have speakers in contact with every surface area throughout the room. You would have to have speakers on walls, tables, ceilings, beams and more and still the article said nothing about these undesired vibrations stopping them from recreating the sounds from the room.

Wrong. All you would need is any source of sufficient background noise. Once the noise pollution reaches sufficient saturation, discerning individual wave forms will be next to impossible. It wouldn't matter if you had any amount of speakers between 1 and 134567876543456 and each one utilized an isolation pad. The speaker creates sound by disturbing the air. Disturb the air enough, and it's game over for this tech in it's current state.