What you see can be affected by what you hear, and vice-versa

We tend to think of vision and hearing as two separate senses.
But they are not as separate as we think, since one sense can affect the other.

What you see can be affected by what you hear as noted in this recent article:

Language can play a powerful role in what we see

“Studies like this are helping us show that language is a powerful tool for shaping perceptual systems, acting as a top-down signal to perceptual processes. In the case of vision, what we consciously perceive seems to be deeply shaped by our knowledge and expectations.”

And those expectations can be altered with a single word.

I've often wondered if what people see in UFO sightings is shaped by their expectations. Now researchers have shown that a single word can affect what we see.

"If language affects performance on a test like this, it indicates that language is influencing vision at a pretty early stage. It’s getting really deep into the visual system.”

This statement by professor Lupman reflects how deeply vision can be affected by a single word.

On the flip side, what you hear can be affected by what you see, as demonstrated in this video of the McGurk effect:

The effect shows that we can't help but integrate visual speech into what we 'hear'.

Note that even researchers who are fully aware of the sensory illusion are still affected by it. The sound doesn't change in reality, but the sound as perceived by humans does change based on visual cues.

Our senses are wonderful, but apparently they are not so separate as we might like to think. If a single word can affect what we see, and reading lips can affect what we hear, how much can our expectations affect what we see and hear?

Wow that vid really proves its point

I was aware that what we see and hear were affecting each other, i've seen it so often while watching people who are watching TV (try that some time) , but I think it goes further than just sight and sound, I think it affects the way we think as well. This is one of the reasons I wont have a tv in my home.

im hearing both at the same time.

im hearing fa and ba at the same time only slightly spaced apart

Very interesting! It is said to believe half of what you see and none of what you hear but this may be an exception haha.

N/A

I remember seeing a documentary, i think it was made by BBC television that had a similar theory about sight and taste. A group of people were gtiven a brown milk shake and asked what flavour it was. They all replied chocolate, when it was actually strawberry flavoured. Good OP.

reply to post by Arbitrageur


Interesting. I wonder why it doesn't affect the sense of smell? If a kitchen is filled with the smell of fish, but someone set it up so that I SEE someone pulling a tray of (cold) cookies from the oven, I'm pretty sure I'm going to smell the fish.

The McGurk effect seems to be telling me what I'm 'supposed' to hear...I heard the distinction between a 'b' and a 'v'...

I really hate being told what I can see or hear...and I really hate the implication that I should be told what I can see or hear...

Thats very tricky.

This is junk.

Now...what I glean from the 'research'...is something different.

Å99

reply to post by jiggerj

I suspect more research is needed to determine the extent to which other senses may be affected by language. As the OP source stated:

The study demonstrates a deeper connection between language and simple sensory perception than previously thought, and one that makes Prof Lupyan wonder about the extent of language’s power. The influence of language may extend to other senses as well.

So it says Prof Lupyan wonders about how it might affect other senses.

We seem to learn from an early age that taste and smell are strongly entangled senses, when we pinched our nose closed when eating something that we didn't like the taste of, so we wouldn't taste the bad taste so much.

I have seen that video before and this effect is one of the reasons I often say that I do not think "Seeing is believing"- as so many love to repeat.

No, our senses are not infallible, it is not because you saw it, heard it, smelled it, felt it, tasted it, that it really exists as you think it does!

reply to post by Bluesma

Very true. I'd like to see "I know what I saw, part 2: 'Maybe I don't' ", but I'm afraid that part ill never air because it would be too unpopular.

We like to have a much higher opinion of the accuracy of our senses than is justified by the evidence.

People don't like being told maybe they didn't really see what they thought they saw. Perhaps they did, but we can't really be sure if what we see can be altered by something like a single spoken word.

I suppose we already understood the fallibility of senses, as indicated by the common phrase "Pics or it didn't happen", but it's interesting to see this scientific research on more specifics about the way our senses can be altered or affected.

Originally posted by fenian8
I remember seeing a documentary, i think it was made by BBC television that had a similar theory about sight and taste. A group of people were gtiven a brown milk shake and asked what flavour it was. They all replied chocolate, when it was actually strawberry flavoured. Good OP.

I've done that experiment - there's nothing worse than going into the fridge at night, picking out a can of drink, and then thinking it tastes funny, then realizing it's a different brand to what you were expecting.

Though, there's a brand of alcohol free beer that is available locally and has has a can the same colour as bottles of shampoo. To me it tastes like shampoo

Great thread!

This demonstrates very well that what you perceive as reality is actually made up in your head. Raw sensory data undergoes significant processing before you become aware of it.

I guess that all the senses pass through this processing system before being presented the conscious mind.