PSYCH 221 Final Project

Audio-Optical Illusions

Patrick Schindler

Introduction

Methods

Results

Conclusions

References

Appendix I


 Methods


In order to investigate the effect of audio input on visual perception I created a great deal of illusions that I -thought- would easily illustrate the phenomenon. I did a great deal of reading, research, and trial and error in an attempt to both uncover new examples and correctly reconstruct some previously discovered examples.

Below are a few of the Audio-Optical Illusions that tended to work better than others.



The Bounce Illusion:

There is no Audio-Optical Illusion better documented than the “bounce” illusion. This is where two objects are traveling in a direction where their paths will cross. At the moment they cross the viewer will perceive one to two things: Either the objects will pass through each other and continue on their respective courses or they will appear to collide and bounce off of each other. Given no audio input most viewers will report that the objects appear to pass through each other. Given a short burst of sound right at the moment of impact most viewers will report that the objects appear to bounce off of each other.

This is a very, very interesting phenomenon. The fact that the viewer tends to perceive the objects passing through each other in silence seems intuitive. We have learned that the eye will seek out a third dimension whenever it can (this being the key to most optical illusions) but we have not talked much about the other aspects of three dimensions. Of course in a three dimensional space we are not limited to just vision but we also have sound, touch, smell, and taste. In this case our perception (for lack of a better term) reads into the expectation that two objects colliding should make a sound and changes our perception accordingly.

Amazing!

Now what about the other senses? Can you think of an experiment where the could be employed to extract the same kind of effect? That is something to think about. But back on track.

I have created two different versions of the bounce illusion. The first of which is pictured below:



Admittedly this is a crude illusion. The second version employs a three dimensional space with textured spears but the trouble is that timing is critical in a 3D space and if the graphics card shutters at all some times the spears will not reach the “meeting point” at precisely the right time which causes the illusion to lean in the direction of “bounce” instead of “pass”.

In the illusion above the series of green bars illuminate from the outside in at a rate set by the frequency dial. They meet precisely in the middle and it is at that moment that the illusion occurs. Although this illusion appears simple there was actually quite a bit of tuning behind the scene to get it right. For instance, to maintain the “realism” of the scene I had to delete one of the steps where the two bars meet. This is because I chose for there to be an even number of bars, had the number been uneven no correction would have been necessary. The reason for using an even number of bars was simple: 32 bits in an integer allowed me to use binary arithmetic to drive the signals.

On the screen are several controls. The large toggle switch (pictured in the down position) will silence the illusion for calibration. When the switch is in the up position the sound selected by the pull-down menu (currently showing “Tink”) is played precisely at the moment the two bars meet. There are multiple sounds that can be selected with titles like: “tink”, “swoosh”, “short swoosh”, “long swoosh”, etc. In the results section we will talk about how the type of sound played effects the way the viewer reacts to the illusion.

An important aspect of this illusion is the ability to trigger the sound early or late. In addition to the type of sound effecting the results of the illusion the timing is also critical. Generally (given a short “tink” sound) the user will stop experiencing the bounce illusion if the sound is played a few hundred milliseconds before or after the point at which the bars meet. In this illusion I allow the user to select how many bars (early or late) the sound will play. Again, look to the results section for more information on how the timing of the sound effects the illusion.

The Multi Bounce Illusion:

Similar to above, the Multi Bounce Illusion works on the same principle but this time I have introduced a 3 dimensional environment and multiple objects.



I created many versions of this illusion. Above is one version where the two white balls start in the upper corners and meet in the middle. Depending on how the viewer perceives the illusion each ball either continuous on its course to the opposite corner or it bounces creating a “V” shape and returning to the corner directly below its starting point. In this particular image the vector of the red ball can be controlled to cause it to interfere with the collision of the white balls. What do you think would happen if the red ball crossed the path (vertically from top to bottom) of the two white balls right at their intersection?

Controls are provided in this illusion to control the resolution of the balls (i.e. the frame rate), the speed the balls move, the sound (or lack there of) when they intersect, and the behavior or any additional objects introduced into the scene.

The trouble with a 3D rendering like this is getting the graphics card to render smoothly enough to fool the eye. If you use too high of a frame rate the processor bogs down and becomes jumpy. If you use too low of a frame rate the balls tend to look like they are jerking or stepping. It was very difficult to tune in the illusion and (at present) it is just tuned in for my particular graphics card.

The sky is the limit (or rather the graphics card) for the number of objects or the way they interact. I challenge the reader to think of interesting patterns that may illicit interesting responses. One idea is to have 6 balls oriented at the 6 points of a hexagram as pictured below:



The Blue points are where the balls would have the opportunity to either pass by or collide. Each ball would have two separate points in its trajectory where it could have a collision. You could imagine several very different patters emerging. If not sounds are played the balls will appear to circle their respective large triangles. If a sound is played at each intersection then the balls would appear to circle only the smaller outside triangles.



The Flying Ball Illusion:

In this illusion there is a single ball arching across a 3D plane from left to right. The ball spends some period of time off screen then repeats.



There are two ways that sound can be incorporated into this illusion. The first is in the form of a cannon blast moments before the ball enters the screen. When a viewer hears a cannon blast and then sees a round object arching across their field of vision they tend to associate the moving object with the cannon blast. A “splat” sound that plays just after the ball leaves the screen further enforces this correlation.

BUT WHY? Why should the viewer assume that there is any correlation between the cannon blast and the flying object?

The second way sound can augment the visual input is in the form of a Doppler effect sound that is played just after the ball leaves the screen and ends just before it comes back on. This sound is intended to lead the viewer to envision the ball circling around then where they have some small field of vision to see it pass. Although this effect is not necessarily all that convincing without stereo, it does bring out one interesting fact:

In the first Illusion the viewer “visualized” a series of objects starting from the left and ending on the right. In the second Illusion the viewer visualizes a continuous movement of one object around and around. This is a situation where we have used sound to change the users visual perception of whether they are viewing one object or many objects.



The Assumption Illusion:

In this illusion the intention is to lead the viewer into assuming they are going to see one thing but showing them another. The expectation is that viewers who are presented a distorted image (that is slowly coming into focus) with no sound will be quicker to identify the picture than viewers who are given the incorrect stimulus.



Example: Say that we wanted to slowly bring into focus a picture of a beautiful yellow speed boat sitting on the water. Before we started developing this picture what if we played the sound of a taxi cab blowing its horn. Do you think that the people who heard the car horn would take longer to recognize the picture as a boat?

The controls for this picture are fairly clear: You have a button that announces a “hint” as to what the picture may be and a slider that allows you to control the resolution of the picture.

One could argue that this is not an Audo-Optical Illusion. I argue that it is. The illusion is whatever misguided images the viewer projects onto the distorted image. It is highly unlikely that a viewer will assume that any given picture will turn out to be a taxicab if they are not mislead by the honking sound.



The Ventriloquism Illusion:

Everyone at some point has been fascinated by the tricks of a ventriloquist. Many people believe in “voice throwing” and the such. If you are one of those who still believes in magic, stop reading. (SPOILER) In actuality there is not such thing as “voice throwing”. In fact, the way a ventriloquist fools you into “hearing” the voice coming from the dummies mouth is by carefully moving the dummies mouth in perfect time with the sound. If the ventriloquist were to stop moving the dummies mouth or lose pace then you would instantly detect that the voice was coming from his/her throat. The reason is that humans are programmed to attach voices to mouths. If you see a mouth moving and you hear a voice you will involuntarily associate the voice with the mouth.



In this Illusion an audio track is started with a blank screen. In this case the voice is that of “Tricky Dick” in a debate against the fallen JFK (talk about optical illusions. . .). There is a single speaker places some distance away from the screen and the viewer is allowed to listen for some short period of time. As with the radio, viewers will tend to stare at the speaker as if it has moving lips. When the cover screen is removed and Richards face comes into view the viewer will instantly undergo the illusion that the voice now seems to emanate directly from the speakers mouth.

Normally this illusion would be considered the type where the visual system is tricking the auditory senses into placing the origin of a sound at an incorrect location. This is true, but that does not mean that a visual illusion has not occurred at the same time. To my mind the sound has tricked the viewer into perceiving that the moving mouth is emitting sound when in fact it is not. If there were no sound then the viewer would not assume that sound was coming from the mouth. With the sound track playing it is almost as if you can see the sounds being formed at the lips. This is why it is so disturbing to have a soundtrack that is off.

Perhaps the Ventriloquist is a better example. By throwing his/her voice the ventriloquist is creating the optical illusion that the Dummy is talking. I guess it is a matter of how you look at it. Ok, ok, I admit it. This is really more of a case where vision drives sound but it is so interesting, how could I pass it up?



The Experiment:

After you look at this stuff long enough(especially illusions that you have created yourself) you become incredibly desensitized and biased. It has come to the point where I can look at a “bouncing ball” illusion and see either case (the pass or the bounce) at will. For this reason it is important to bring in a fresh set of eyes. I collected as many reluctant people as I could round up (in this case the number was 7 including myself) and presented the above mentioned Audio-Optical Illusions (and a few others) while questioning them on what they saw and what they thought. For some experiments it was as simple as tuning the speed of the moving objects till they saw them “passing” then applying the different sounds to see if they perceived the “bounce”. In other instances (like the Assumption Illusion) it was much more subjective. I would issue the “hint” audio and coax the viewer into making guesses about what they were seeing. Admittedly this was not the most scientific of methods. Most users were jaded by the Internet and (knowing me) were steeling themselves for a big monster to pop out on the screen or some other startling event. This made it hard to collect good data but in general, the results were consistent.