Laboratory for Synthetic Perceptive, Emotive and Cognitive Systems
Institute of Audiovisual Studies, Universitat Pompeu Fabra
&
Division of Applied Acoustics, Chalmers University of Technology, Gothenburg, Sweden
published in Brain Research Reviews: doi:10.1016/j.brainresrev.2009.07.001
cited here with kind permission of Aleksander Väljamäe
Abstract
The aim of this paper is to provide a first review of studies related to auditorily-inducedself-motion
(vection). These studies have been scarce and scattered over the years and over
several research communities including clinical audiology, multisensory perception
of self motion and its neural correlates, ergonomics, and virtual reality. The
reviewed studies provide
evidence that auditorily-induced vection has behavioral, physiological and neural
correlates. Although the sound contribution to self-motion perception appears
to be weaker than the visual
modality, specific acoustic cues appear to be instrumental for a number of domains
including posture prosthesis, navigation in unusual gravitoinertial environments
(in the air, in space, or
underwater), non-visual navigation, and multisensory integration during self-motion.
A number of open research questions are highlighted opening avenue for more
active and systematic
studies in this area.
Keywords:
illusory self-motion, vection, spatial sound, cognitive acoustics,
multisensory
perception, virtual auditory displays, auditory motion
1 Introduction..........................................................................................................................4
2 Vection measures ..................................................................................................................4
3 Auditory motion perception.................................................................................................
6
4 Circular vection ....................................................................................................................7
4.1 Audiokinetic nystagmus in clinical audiology...............................................................
8
4.2 AKN and vection research .............................................................................................9
4.3 Postural responses to rotating auditory stimulation .....................................................10
4.4 Virtual reality and auditory presence...........................................................................11
15 Linear vection......................................................................................................................11
6 Self-motion and auditory localization...............................................................................12
7 Cross-modal interactions occurring with auditorily-induced vection ...........................13
7.1 Circular vection............................................................................................................14
17.2 Linear vection ..............................................................................................................14
7.3 Neural correlates for auditory-vestibular interaction...................................................15
8 General discussion ..............................................................................................................16
8.1 Methodology ................................................................................................................16
8.2 Spatial sound parameters .............................................................................................16
8.3 Multisensory interactions.............................................................................................18
1 Introduction
We often rely on our audition when evaluating objects motion, like, for example,
the velocity of
an approaching car. Can acoustic cues provide us with information about our
own motion, real or
illusory? While visually induced illusory self-motion (termed vection) has been
extensively
studied over more than a century (see Hettinger, 2002; Dichgans & Brandt,
1978 for reviews),
research on auditory vection cues has received only minor attention. This paper
intends to
provide a first integrative review on the relevant findings which have been
scattered over the
years and over several research areas including clinical audiology, multisensory
perception of
self-motion and its neural correlates, ergonomics, and virtual and augmented
reality. Although
auditory cues for self-motion perception tend to be weaker than for the visual
modality,
knowledge on this research topic may provide important contributions to posture
prosthesis,
navigation in unusual gravitoinertial environments (in the air, in space, or
underwater), non13
visual navigation, understanding of multisensory integration of self-motion
cues and auditory
localization during locomotion.
An illusory self-motion sensation may arise in many real life situations when
an observer
is exposed to a moving visual pattern occupying a large portion of her/his field
of view. For
example, the illusion can occur when seeing a departing train on a neighboring
track while
expecting your train to move. After some time, moving external stimuli distinctively
become
perceived as being stationary and the illusion of self-motion occurs. While
a strong vection
illusion can be induced by visual stimuli, other sensory inputs like auditory
and somatosensory
(vibration) cues can also produce this sensation (Dichgans & Brandt, 1978,
and references
therein). Similarly to visually-induced vection, auditorily-induced vection
(AIV) can be elicited
using real or virtual sound fields moving relative to the listener's point of
audition. The sensation
of vection in such acoustic simulations can be easily destroyed by other sensory
modality
providing information about a stable external environment, or a reference frame
(Lackner, 1977).
Therefore, studies on AIV require specific measures to assess this illusion,
such as blindfolding
and the plausibility of physical motion of the experimental setup. This might
explain the
controversial results from audiology research presented in section 4, since
self-motion illusion
was not the primary aim of these studies.
It is important to note that the vestibular organ provides information about
gravito
inertial body influences. Therefore, in situations of constant velocity self-rotation
or self
translation where no such vestibular cues are available, the surrounding environment
cues start to
play an important role in the evaluation of the ambiguous sensory information:
“I am moving”
vs. “Surround is moving” (Lackner, 1978). This clearly implies a
possibility for cognitive
influences on self-motion cues processing, where a conscious decision about
the ego-centric
reference frame has to be made (Zajonc, 1998). As recently hypothesized by Wright
and
colleagues (Wright, DiZio, & Lackner, 2006) different mechanisms may be
involved in illusory
self-motion perception, and the vection sensation vividness may be also modulated
by the
motion simulation context and physical interactivity. Knowledge on the non-perceptual
vection
cues becomes increasingly important since ecologically valid stimuli are used
in immersive
virtual environments (e.g., Durlach, Allen, Darken, Garnett, Loomis, & Templeman,
2000).
Therefore in this review we concentrate both on the perceptual and cognitive
contributions to
auditorily-induced vection.
The present review leaves out some research areas, which are
not related to the perception
of illusory self-motion and auditory motion cues. It has been shown that the
vestibular system
can be influenced by high-intensity and, typically, low-frequency sound (Parker,
Tubbs, &
Littlefield, 1978; Sheykholeslami & Kaga, 2002; see (Reschke & Parker,
1970) for a review on
the research prior to 1970). Another branch of research encompasses stationary
sound influences
on postural balance (see Bensel, Dzendolet, & Meiselman, 1968; Sakellari
& Soames, 1996 and
references therein). Results in this research topic are rather inconclusive
and dependant of the
methodology and stimuli used. Finally, the Tulio syndrome refers to the vestibular
activation via
sound which may occur during certain disorders affecting the middle and inner
ear (e.g., Ishizaki,
Pyykko, Aalto, & Starck, 1991).
2 Vection measures
Already at the end of the 19th century, in the earliest work on auditorily-1
induced vection Viktor
Urbantschitsch noted that the “self-motion sensation is so slight that
one should pay special
attention to notice it and a specific methodology should be used in the experiments”
(Urbantschitsch, 1897, p. 236). Since then, many vection measurement techniques
have been
applied; however, the sensation of self-motion still remains a complex, largely
subjective,
phenomenon to access (Hettinger, 2002).
In the case of a stimulus that produces a weak vection sensation, as is the
case for
auditory modality, instances of reported self-motion illusion per stimulus type
(vection reports or
binary vection) can be directly used for statistical analyses (e.g., Lackner,
1977; Väljamäe,
Larsson, Västfjäll, & Kleiner, 2005). Vection onset time (or vection
latency) is another measure
based on participant’s shift in phenomenological experience when a moving
external stimulus
suddenly appears stationary and an illusory self-motion phase occurs. It is
a common practice in
vection research to use subjective scales when accessing the strength of the
perceived self14
motion (vection intensity) or the compellingness (convincingness) of a body
displacement in a
particular direction (e.g., Ijsselsteijn, de Ridder, Freeman, Avons, & Bouwhuis,
2001; Wright et
al., 2006). Here one may draw an analogy with a force description where vection
intensity shows
its strength and compellingness the direction. While highly correlated in most
of the situations,
these two subjective scales can be also dissociated during simulations where
self-motion context
is created but no clear directional cues are present, like in works using vibrotactile
(Väljamäe,
Larsson, Västfjäll, & Kleiner, 2006) or vestibular stimulation
(Marme-Karelse & Bles, 1977).
The intensity of the vection sensation can be also accessed continuously throughout
a stimulus
presentation, for example, by using a joystick or a lever. From such continuous
recordings of
vection intensity, the duration of the sensation and vection onset times could
be also determined.
It should be noted that special care must be taken when designing questions
for accessing
subjective experience of self-motion since responses will depend on the reference
frame used by
participants. For example, illusory self-motion can be reported relative to
presented moving
stimulus or to the ground of a laboratory (see Stoffregen & Bardy, 2001,
p. 205-206 for a
discussion on subjective reports of perceived motion). It is also important
to combine vection
self-reports with more objective measures since an experimental setup can create
dissociations
between conscious awareness of the environmental situation and subconscious
motor control,
like in the case of a broken escalator phenomenon (Fukui, Kimura, Kadota, Shimojo,
& Gomi, 2009).
Apart from subjective ratings of vection sensation, a variety of psychophysical
measurement techniques have been adopted, including estimations of the magnitude
of the
perceived (but illusory) displacement, self-motion velocity or illusory body
tilt (Hettinger, 2002).
Several more indirect measures of vection have been also proposed. It has been
shown that the
threshold for object motion detection is higher when the vection sensation is
stronger (Probst,
Krafczyk, Brandt, & Wist, 1984). Similarly, the perception of visually-induced
self-motion has
also been shown to modulate time-to-collision judgments (Gray & Regan, 2000).
Alternatively, a
temporal order judgments (TOJ) task was recently used in combination with the
subjective
vection measures to access the effects of optokinetic stimulation (Teramoto,
Watanabe,
Umemura, Matsuoka, & Kita, 2004).
Behavioral measures are also often used for quantifying the vection sensation.
The
movement of the visual surrounding induces head and body displacement in the
same direction
as the moving stimulus (Dichgans & Brandt, 1978). The amplitude of the body
sway tends to
increase with the strength of the vection sensation (Hettinger, 2002 and references
therein).
Accessing the postural sway can be a complex and equipment-intensive procedure,
involving
various methodologies such as force platforms, body-mounted accelerometers,
or video methods.
However, several recent studies show that visually-induced postural reactions
are not rigid
responses to optokinetic stimulation only but are rather responses to both perceptual
and
cognitive self-motion cues (Bronstein & Buckwell, 1997; Guerraz, Thilo,
Bronstein, & Gresty,
2001). Therefore it is important to combine postural responses with the reports
of participants’
subjective sensation.
One of the physiological correlates of vection is nystagmus - involuntary eye
movements
which take place when longer (≥1 s) head movements occur and which are
a part of the
vestibulo-ocular reflex (VOR) (Cohen & Raphan, 2004). The nystagmoid eye
movements
contain two components – a slow, compensatory phase in the direction opposite
to head motion,
and a quick phase, which has a restorative function. A large body of 1 research
has been
investigating the optokinetic nystagmus (OKN) induced by rotating
visual stimuli (see Cohen &
Raphan, 2004 and references therein), but some works have also reported the
occurrence of
auditorily-induced, audiokinetic nystagmus (Dodge, 1923; Hennebert, 1960; Lackner,
1977;
Gekhman, 1991, see section 4). Nystagmus often accompanies the illusory self-motion;
however,
6vection can be experienced without nystagmoid eye movements (Brandt, Dichgans,
& Koenig,
1973) and vice versa, when a small visual field is presented. In addition, recent
studies have
demonstrated that imagining whole-body rotation can evoke nystagmus (Rodionov,
Zislin, &
Elidan, 2004). Thus, although nystagmus tends to occur during real or illusory
self-motion, it
might not categorize objectively the vection inducing power of external stimuli.
Brandt and
Dichgans (as cited in Dieterich, Bense, Stephan, Yousry, & Brandt, 2003)
suggested that the
afternystagmus, observable within some seconds after the vection inducing stimuli
offset, might
serve as a more definitive indicator of a strong self-motion sensation.
Other physiological measures of vection include electrodermal activity and
cardiovascular responses (e.g., Cheung, Hofer, Heskin, & Smith, 2004). However,
they
sometimes entail large individual differences that might be dependent upon individual
stereotypes and differences in the somatosensory and vestibular contributions
to autonomic
regulation (Aoki et al., 2000). A number of recent studies address the neural
correlates of vection
including EEG (Tokumaru, Kaida, Ashida, Yoneda, & Tatsuno, 1999), magnetoencephalography
(MEG) (Wiest, Amorim, Mayer, Schick, Deecke, & Lang, 2001; Nishiike, Nakagawa,
Tonoike,
Takeda, & Kubo, 2001), positron emission tomography (PET) (Brandt, Bartenstein,
Janek, &
Dieterich, 1998; Deutschländer, Bense, Stephan, Schwaiger, Dieterich, &
Brandt, 2004), and
functional magnetic resonance imaging (fMRI) (Stephan, Deutschländer, Nolte,
Schneider,
Wiesmann, Brandt, et al., 2005; Baumgartner, Valko, Esslen, & Jäncke,
2006). It has been
shown, for example, that optokinetic stimulation inhibits vestibular areas (Dieterich
et al., 2003
and references therein), or that similar brain areas are deactivated during
linear or roll vection
types (Deutschländer et al., 2004). However, this research just starts
to reveal the brain processes
underlying vection perception and its relation to motion processing mechanisms.
3 Auditory motion perception
How do humans form mental representations of the world on the basis of auditory
cues? In his
classical work, Bregman (1990) coined the term “auditory scene analysis”
referring to the way
we perceive the surrounding sound environment and create a corresponding mental
model.
However, it remains an open question how exactly this mental image is formed
based on acoustic
cues. Bregman argued that acoustic events are modeled as separate “auditory
streams” at the
high levels of the auditory system. Traditionally, the research on auditory
scene analysis has
been investigating bottom-up processes where low-level acoustic cues (intensity,
frequency, etc.)
form a unitary percept, i.e. a stream. However, auditory streams, or auditory
objects as termed
by Blauert (1997), can be associated with the physical objects producing these
sounds (“sound of
a bus”, “sounds like an engine”). Therefore, apart from low-level
physical cues the “auditory
streams” can be formed on the basis of high-level cognitive factors. This
highlights that our
everyday listening experience can also influence auditory scene perception.
Following Gibson’s
“ecological” approach, this research area has been recently addressed
by ecological
psychoacoustics (Gaver, 1993; Neuhoff, 2004).
Object motion perception definitely plays an important role in resolving conflicting
information about self- vs. surround motion. The mechanism of auditory motion
perception is a
complex phenomenon with many parameters and it remains as an active area of
research. Recent
brain imaging studies show that a specific “movement-sensitive”
area in auditory cortex is most
likely to exist (Warren, Zielinski, Green, Rauschecker, & Griffiths, 2002)
thus indicating
separate neural mechanisms for stationary and moving sounds localization.
Three main cues for discrimination of auditory motion are intensity, binaural
cues, and
the Doppler effect (Lufti & Wang, 1999). Intensity cues arise from the changes
in sound pressure
level emitted by a moving sound source. Binaural cues provide information about
the interaural
time and level differences (ITD and ILD) at listener’s ears. The Doppler
effect results in
perceived frequency shifts in the case of motion between a sound source and
a listener. Lufti and
7
Wang (1999) thoroughly examined these three main cues and showed 1 that for
sound object
velocities below 10 m/s, intensity and binaural cues were the most salient in
providing
information about distance traveled. The Doppler shifts were dominant for judgments
on sound
object velocity and acceleration. For higher velocities (50 m/s), the Doppler
shift tended to
dominate in all discrimination tasks. It is important to note, however, that
the cue dominance
depended not only on the task but also varied between tested individuals.
The intensity cue is known to be dominant for the perception of traveled distance
(Rosenblum, Carello, & Pastore, 1987). Recently, it was shown that continuous
intensity changes
can elicit an illusion of pitch shift that is roughly four times larger than
the actual frequency shift
caused by the Doppler effect (McBeath & Neuhoff, 2002). The authors suggested
that perception
of Doppler frequency shifts in everyday listening is almost entirely driven
by perceived intensity
changes.
The intensity cue dynamics provide several secondary cues contributing to the
auditory
motion perception. When a sound source passes a listener, the “point of
closest passage” is
clearly marked by the highest intensity peak (McBeath & Neuhoff, 2002).
Intensity can be also
used for a time-to-arrival estimation by tracking the intensity change rate,
termed as acoustic tau
(Lee, 1990; Shaw, McGowan, & Turvey, 1991). However, the acoustic tau and
the auditory
motion parallax (auditory equivalent to visual parallax) may have a minor impact
on the auditory
motion perception compared to stronger cues such as intensity and reverberation
(Speigle &
Loomis, 1993). Studies on sound intensity perception revealed another effect
related to the
perception of approaching or looming sound sources. In his recent study, Neuhoff
(2001) showed
that a continuous intensity increase results in a stronger perceived loudness
change compared to
the percept produced by the same amount of intensity fall. This asymmetry in
“looming” sound
perception also resulted in a difference in the reports about traveled distance
perception. Several
concurrent studies corroborated the fact that approaching sounds have perceptual
and behavioral
priority and that sound objects perceived as approaching have greater biological
salience than
receding ones since they might be of a potential threat (Hall & Moore, 2003;
Neuhoff, Planisek,
28 & Seifritz, 2009).
An alternative view on the auditory motion perception mechanism was suggested
by
Grantham (1986), whose “snapshot hypothesis” stated that listeners
base their judgment on the
total distance traveled by an object and not on the object velocity. Recent
findings by (Carlile &
Best, 2002) suggest that both direct perception of motion cues and displacement
detection, can
take place. The effects of high frequencies attenuation due to air absorption
(Begault, 1994) may
also play a role in the judgments of distance to a sound source. In addition,
distance perception
depends on the type of sound source and on the listener’s familiarity
to it (Blauert, 1997). For
instance, judgments of traveled distance have been found to be more accurate
for ecological
sounds and sounds that are within listeners’ reach (Rosenblum, Wuestefeld,
& Anderson, 1996).
Finally, the acoustic environment plays an important role in auditory distance
perception,
especially for indoor conditions where the ratio between direct and reflected
sound is known to
be one of the most salient cues (Bronkhorst & Houtgast, 1999). Research
shows that listeners can
use reflected sound for echolocation in navigation tasks (Stoffregen & Pittenger,
1995;
Rosenblum, Gordon, & Jarquin, 2000).
To summarize, the presented information shows that, apart from purely acoustic
cues,
the perception of auditory motion is influenced by the ecological context. It
supports the
suggestion by Popper and Fay (1997) that the main function of the auditory localization
mechanism may be to provide an input to the listener’s perceptual model
of the environment
rather than exact estimates of sound sources locations and trajectories.
4 Circular vection
Circular vection refers to the illusion of self-rotation about one or more of
the three body axes
(Dichgans & Brandt, 1978). Most of the research on visually-induced circular
vection has been
concentrated on illusory rotation about the upright body’s z axis, also
referred as “yaw”
(Hettinger, 2002), and a similar trend could be seen for circular AIV (see subsection
4.3 for
vertical, “pitch” rotating auditory stimuli). The earliest reports
on auditorily-induced circular
vection in the horizontal plane can be found in (Urbantschitsch, 1897) 1 and
1910 study by von
Stein (as cited in Mergner & Becker, 1990).
As mentioned in the previous section, rotating visual stimulus can evoke optokinetic
nystagmus, a phenomenon that was first described by Dodge (1903). He also conducted
a first
exploratory, but very insightful, study on nystagmoid eye-movements elicited
by rotating
acoustic fields (Dodge, 1923). In the first experiment of this study, a hand-driven
sound-cage
equipped with a vibrating telephone receiver or a small electric buzzer revolved
around a seated
participant whose eyes were closed. Several problems aroused from this study.
First, the
trajectory of the sound was often perceived as oscillating back and forth in
front of the listener.
In addition, a stationary camera for recording eye-movements created an acoustic
shadow that
provided a cue for sound cage motion (cf. audible facing angle in Neuhoff, 2003).
In spite of
these technical disturbances, occasional sensations of self-rotation “independent
of contact with
the table, headset, and seat” (Dodge, 1923, p. 112) were reported together
with some recordings
of the eye-movements. This initial observation led to the subsequent experiments,
with a
redesigned apparatus allowing the rotation of the sound-cage or the sitting
platform. The
possibility for physical rotation of the platform aimed at reducing the negative
influence of
participant’s “knowledge on the impossibility of rotation”
(Dodge, 1923, p. 113). The acoustic
shadow from the recording camera was also greatly reduced in the new setup.
These
modifications in the apparatus increased the number of illusory self-rotation
reports. In these new
experiments, small congruent/incongruent oscillations were applied to the sitting
platform while
the sound-cage revolved in one direction. Dodge hypothesized that if auditorily-induced
eye22
movement exists this might enhance or reduce the vestibular-ocular responses
produced by small
movements of the platform. The photograph records seemed to support these predictions;
however, the researcher noted that these results might have been accounted for
the visual
imagery triggered by sounds. Sounds and vibration produced by the platform motion
were also
identified as potential self-motion cues. Although Dodge concluded that the
“consciousness of
rotation at or near the threshold was undoubtedly aided by environmental sounds…”
and that “a
systematic quantitative study of these reinforcements and inhibitions would
be illuminating but
difficult” (Dodge, 1923, p. 116), these research topics remained untouched
for several decades.
4.1 Audiokinetic nystagmus in clinical audiology
During the 1960s several research groups studied the feasibility of using moving
auditory stimuli
for clinical audiology purposes. Hennebert first used the “audiokinetic
nystagmus” (AKN) term
in his work showing that saccades could be created when varying white noise
intensity at both
ears sinusoidally with a phase difference of 90 degrees (Hennebert, 1960). The
author claimed
that other types of moving stimuli, like music, could be used,and that AKN is
involuntary and
cannot be willfully suppressed. Unfortunately, the experimental conditions and
stimuli were not
clearly described in the manuscript. Several years later a more rigorous study
on the feasibility of
AKN was conducted (Weber & Milburn, 1967). The stimulus was a 1 kHz pulsating
tone that
alternated between the headphone channels. The experiment contained 2 sessions,
which differed
in that in the second part, naive participants were instructed to resist any
attempt of eye
movements. Post-experimental screening showed that stimuli did not appear as
rotating around
the head but was rather perceived as moving from ear to ear inside the head
or in an arc behind
it. Electrooculography (EOG) recordings were used to quantify the degree of
eye-movements
rather than to detect audiokinetic nystagmus patterns. The first part of the
experiment registered
significantly more eye-movements during stimuli presentation compared to the
silence periods.
However, restraining the eye movement led to a less than normal eye-activity,
thus contradicting
Hennebert’s claim about the involuntary nature of AKN.
Watson (1968) also studied the feasibility of audiokinetic nystagmus using a
16-
loudspeaker array for creating a rotating sound field. Although non-periodic
eye-movements
could be often seen in EOG records at the stimulus onset, they were not correlated
with sound
field parameters (different velocity and frequency range). Watson concluded
that the AKN
technique appeared to be unreliable for clinical audiology purposes. He also
pointed out that the
instructional set might possibly explain these negative results, since blindfolded
participants
were instructed to fix their gaze on a central point. McFarland and Weber (1969)
continued the
AKN research. They used white noise stimuli delivered in three ways: 1) activating
one of two
loudspeakers placed at either side of participants; 2) alternating 1 the sound
between two
loudspeakers at 2 s periods; and 3) moving a motor-driven loudspeaker at 900/s
angular velocity.
No AKN was registered, since the analysis of EOG records showed no correlation
between eye
movements and auditory stimuli motion parameters. In general, only physically
moving stimuli
resulted in a significant increase in the ongoing eye-movements compared to
the control
segments. Later studied conducted by Schaefer and colleagues et al. (Schaefer,
Süss, & Fiebig,
1981; Fiebig, Schaefer, & Süss, 1981), confirmed the negative results
from Watson (1968), and
McFarland and Weber (1969). Both voluntary and involuntary eye-movements were
induced by
different types of moving acoustic stimuli in 350 healthy test persons; however,
no clear
nystagmus patterns were observed (Schaefer et al., 1981). Two other research
groups studied the
AKN applicability for hearing impairment diagnosis and concluded that 1) this
method can
access only directional hearing deficits but not serve as a topical diagnosis
of central
disturbances (Ganz & Matten, 1969); and 2) that further studies are necessary
to clarify the
inconclusive results (Zalewski, 1976).
There are several factors which could explain the lack of support for AKN in
the
mentioned studies above. First, Hennebert and his successors did not link the
subjective
sensation of illusory motion to eye movement patterns. Therefore, the testing
procedures did not
take into account various factors that can potentially destroy the illusion
of self-motion (e.g., eye
gaze or participants’ mental set about the “impossibility of rotation”).
Second, in all the
experiments on AKN mentioned so far, a relatively short time of stimulus presentation
was used
(20-30 s, except 3 minutes in Watson, 1968). It is known that visually-induced
vection onset
time can vary between 1-15 seconds (Dichgans & Brandt, 1978). However, auditorily-induced
vection latency has shown to have a longer latency (5-30 seconds in Väljamäe,
Larsson,
Västfjäll, & Kleiner, 2009), suggesting that the above mentioned
stimuli might be too short for
inducing illusory self-motion and nystagmoid eye movements. Finally, as was
discussed in
section 3, nystagmoid eye movements do not necessarily correspond to the subject’s
sensation of
vection.
4.2 AKN and vection research
In the context of vection, audiokinetic nystagmus was again addressed at the
end of the 1970s in
different laboratories by Lackner (1977), and Marme-Karelse and Bles (1977).
In the study by
Marme-Karelse and Bles (1977) a moving acoustic field was created by a rotating
loudspeaker.
The stimulus contained clicks and started the rotation with an angular acceleration
of 0.30/s2, up
to a final angular velocity of 500/s. Five out of eight participants reported
circular vection;
however, no clear AKN was registered for this purely auditory condition. In
other conditions,
visual, vestibular, auditory-vestibular and visual-vestibular stimulations were
tested. The results
showed that 1) the auditory stimulation is much weaker than the optokinetic
one; 2) incongruent
auditory and vestibular rotational cues produced a directionally unstable vection
sensation,
suggesting their comparable vection-inducing power; 3) in the case of faster
loudspeaker
acceleration (100/s2), vestibular stimuli dominated the directionally incongruent
auditory cues.
The authors also noted that future studies should address such important simulation
factor as the
participants’ “mental set” about the believability of the
vection-inducing sound environment,
which should provide an ecologically valid spatial context (e.g., using naturalistic
sounds).
Lackner (1977) studied AKN using three angular velocities for rotating sound-fields:
300/s, 600/s and 900/s; and three stimuli types: pure tones of 100 Hz and 1000
Hz, and white
noise. The acoustic fields were 3 minutes long and were rendered either using
a 6-loudspeakers
array or headphone-based playback containing stereophonic recordings done at
the center of this
array. The external sound field stimulation using white noise led to the maximum
number of
vection reports (40-75% of the trials). The fastest, 900/s angular velocity
was significantly less
effective than the two lower angular velocities. Clear recordings of AKN were
obtained when the
compelling sensation of self-motion was perceived and lasted longer than 10
seconds. The AKN
recordings also showed good correlation with the velocity of the acoustic fields
– the average
numbers of fast (restorative) nystagmus components detected were 3.4, 6.7, and
9 for 300/s,
600/s, and 900/s rotations respectively. In addition, no auditorily-induced
vection was reported
when the experimental room was visible to participants.
Following studies by Hennebert (1960) and Lackner (1977), a very 1 systematic
study of
AKN was conducted by Gekhman (1991). He used headphone-based rendering of stimuli
containing clicks and presentation durations of 90-130 s long. The auditory
image motion at 45,
60, 90, 180, 281, and 450 0/s angular velocities was created using interaural
time differences
varying in the range of ± 2 s. Five participants took part in three sessions
with 1) no specific
instructions; 2) attention focused task (concentration on auditory stimuli);
and 3) divided
attention (arithmetic problem solving). Instances of nystagmoid, saccadic and
smooth tracking
eye-movements were identified during a specific pre-processing of EOG data (extrapolation
of
slow component of eye-movements). The analysis of AKN instances in the first
session revealed
their bell-shaped dependence on the auditory motion velocity, with a clear maximum
of
nystagmoid eye-movements at 600/s, thus, comparable to Lackner’s results
(1977). The same
bell-shaped pattern, but with a maximum for the 1800/s velocity, was observed
in the other two
sessions, where participant’s attention was manipulated. The authors suggested
that the lack of
decrease in AKN observed for the divided attention session might reflect the
training after the
preceding focused attention session. Nevertheless, Gekhman (1991) suggested
that attention
focus on the moving auditory stimuli might play a dominant role in the auditory-ocular
interaction. This claim is in line with more recent studies showing that attention
can modulate
the processing of visual (Yardley, Gardner, Lavie, & Gresty, 1999) or vestibular
(Rees, Frith, &
Lavie, 1997) motion cues.
4.3 Postural responses to rotating auditory stimulation
The studies with stationary acoustic fields usually show that listeners can
use sound information
for balance maintenance, especially if sufficient spatial cues are provided
(Petersen, Magnusson,
Johansson, Åkesson, & Fransson, 1995; Petersen, Magnusson, Johansson,
& Fransson, 1996;
Easton, Greene, DiZio, & Lackner, 1998). Sometimes, however, destabilizing
effects of sound
have been also reported (Raper & Soames, 1991; Sakellari & Soames, 1996).
Only a few studies
addressed the influence of rotating auditory fields on human postural balance.
In a pilot study described by Marme-Karelse and Bles (1977) a moving sound image
was
created using a binaural presentation of white noise. When participants perceived
the moving
sound image as passing in front of them, the stabilometer recorded their body
inclination
forwards. Another study used a 16-loudspeaker array to test the influence of
the auditory motion
velocity on body posture, in a dark or lit room (Neetz, Süss, & Schaeffer,
1980). Lateral body
sway was more pronounced in the darkness and was higher for a stimulus of 1800/s
angular
velocity compared to 720/s. Tanaka and colleagues (Tanaka, Kojima, Tokeda, Ino,
& Ifukube
2001) used rotating white noise stimuli delivered via headphones. Twelve healthy
participants
forming young (mean age 21.9) and elderly (mean age 68.9) groups were tested
for a number of
non-acoustic factors. Results showed that the lateral sway was significantly
increased by the
rotating acoustic field under the conditions of reduced tactile sensation (standing
on a soft vs.
rigid surface) or deprivation of visual cues (eyes closed vs. open). Moreover,
this influence was
stronger for the elderly participants.
In a more recent study, the binaural presentation of auditory motion of approximately
300/s velocity and 5 seconds duration showed to affect the position of participant’s
center of
gravity (Altman, Varyagina, Gurfinkel, & Levik, 2005). In a subsequent experiment,
the authors
compared three vection measures: joystick for indicating the perceived auditory
motion pattern
and velocity, postural responses, and EOG. The proprioceptokinetic illusion
of head rotation was
created by a horizontal sway of the platform (0.2 0/s velocity, ±7 degrees
maximum left/right
48 displacement) with the participant seated and with the head immobilized by
a special helmet.
49 The higher perceived velocity of the accompanying moving sound was associated
with increases
50 in both the illusion of the head rotation and in amplitude of the EOG; a
reverse pattern occurred
51 for low perceived velocity (Altman et al., 2005).
To the best of the author’s knowledge, the only laboratory setup that
simulated “pitch”
rotation using auditory motion in the median, also called mid-saggital, plane
(Agaeva &
Altman,2005; Agaeva, Altman, & Kirillova, 2006). In the first study, participants
stood on a
platform under an arch-shaped 53-loudspeaker array covering the upper 180 degrees
of the
median plane (Agaeva & Altman, 2005). Noise stimuli moving forwards or backwards
along the
arch with velocities of 38, 58 and 1150/s were used. The results showed that
the slower (and thus
longer) stimulus was most instrumental in destabilizing the participants’
posture. There was no
significant influence of the sounds direction on postural responses. A more
recent study used the
same methodology, but only stimuli moving from the front to the back (Agaeva
et al., 2006).
Apart from the dependence on velocity reported previously, the authors found
that the
participants tended to lean forward in response to the approaching stimuli.
Although the stimulus
and the body tilt directions usually coincide for visually-induced vection (e.g.,
Dichgans &
Brandt, 1978), recent research on vertical visual stimulation suggests more
complex postural
responses to upward optokinetic stimulation (dissociation between head and body
movements)
9 than to downward stimuli (Kobayashi, Fushiki, Asai, & Watanabe, 2005).
4.4 Virtual reality and auditory presence
Traditionally, vection studies have been conducted using artificial stimuli
(e.g., classic black14
and-white striped patterns) and only recently more naturalistic, ecological
stimuli have been
used. In the visual domain, research has shown that the environmental context
can indeed play a
modulating role in vection responses (Wright et al., 2006; Riecke, Schulte-Pelkum,
Avraamides,
von der Heyde, & Bülthoff, 2006). For example, in study by Riecke et
al. (2006) the effect of the
coherence of a visual scene layout on vection responses was investigated using
photorealistic
stimuli and mosaic-like scrambled versions. From the perspective of perceptual
(bottom-up)
processing, the scrambled versions of a scene had higher spatial-frequency content,
which is
known to increase the vection sensation (Dichgans & Brandt, 1978). However,
the results
showed that the globally consistent stimuli (unscrambled) resulted in stronger
vection responses,
thus demonstrating the importance of contextual (top-down) effects. In the auditory
domain, the
scrambling of the visual scene can correspond to noise or to naturalistic, everyday
sounds. Using
ecological stimuli also allows accessing the sense of presence (“being
there”) in the simulated
environment, which is believed to be crucial for human-centered evaluation of
multi-modal
virtual environments (Lombard & Ditton, 1997).
Larsson and colleagues (Larsson, Väljamäe, Västfjäll, &
Kleiner, 2004) created circular
vection using three types of auditory objects: artificial (e.g., pink noise),
“auditory landmarks” -
sound sources that can be recognized by listeners as fixed objects (e.g., “fountain”
sound), and
moveable objects (e.g., “bicycle” sound). As predicted, the rotating
acoustic fields containing
ecological sounds led to stronger vection and presence responses than the artificial
stimuli, with
the auditory landmarks type being the most instrumental. This result is in agreement
with the
observation made by Martens (2004), where the level of perceived linear vection
in a virtual
acoustic environment was dependent on the surrounding sound objects likelihood
to be in
motion. In addition, increase in the number of sound sources (from 1 to 3) or
in the rotation
velocity (200/s, 400/s, or 600/s) also increased the intensity ratings of illusory
self-motion. A later
study on circular AIV, contrasting only landmarks vs. moveable sound object
categories,
supported Larsson et al.’s hypothesis that ecological sounds may contribute
to a reference frame
in a virtual environment (Väljamäe et al., 2009). Remarkably, a similar
categorization of
auditory objects was proposed earlier in a practical guide for visually impaired
people (Hill &
Ponder, 1976), where various orientation techniques are described.
Using similar stimuli to the ones by Larsson et al. (2004), the effects of spatial
sound
fidelity on AIV were investigated (Väljamäe, Larsson, Västfjäll,
& Kleiner, 2004). Virtual
acoustic fields were synthesized using either generic or subjects’ own
pre-measured,
individualized Head-Related Transfer Functions (HRTFs). Post-experimental verbal
probing
revealed that using generic HRTFs resulted in a perceptual quality degradation
of the created
virtual acoustic spaces (e.g., distorted sound sources trajectories or in-head
localization).
However, this affected spatial presence ratings but not the vection responses.
In the light of this
result, the higher vection responses obtained by Lackner (1977) with a 6-loudspeaker
array as
compared to stereophonic recording might be more likely due to a higher spatial
resolution of the
rotating acoustic field rather than to the effect of auditory image externalization.
5 Linear vection
Compared to circular auditorily-induced vection research, there is much 1 less
work related to
linear vection. The first such study can be attributed to Soames and Raper (1992),
who evoked
very crude auditory motion sensation by using two loudspeakers placed at the
front/back or
left/right from the listener. The stimuli, containing a 250-Hz tone or speech,
alternated every 10 s
between the loudspeakers. The results showed more postural sway instances for
tone than for
speech, however, without any correlation to open/closed eyes condition. Another
study also
made use of static loudspeakers to simulate two very slowly moving pink noise
stimuli (0.13 or
0.07 m/s) for creating a forward/backward or left/right self-motion illusion
(Sakamoto, Osada,
Suzuki, & Gyoba, 2004). While no significant deviations in the postural
body sway were
10 registered, vection intensity ratings showed a significant asymmetry between
the
forward/backward illusory self-motion in favor of self-motion backwards. This
asymmetry might
be explained by different reaction to approaching versus receding sound fields
that were
simulated in the setup. As discussed in section 3, the perception of approaching
objects has been
shown to be more biological salient for both auditory-only (Hall & Moore,
2003) and auditory15
visual stimuli (Maier, Neuhoff, Logothetis, & Ghazanfar, 2004). In line
with this hypothesis,
approaching sounds have been shown to result in higher linear vection responses
(Väljamäe,
Larsson, Västfjäll, & Kleiner, 2005). In this study AIV was created
using virtual acoustic scenes
moving either with a constant velocity or accelerating, and containing two band-limited
pink
noise sources. Surprisingly, participants tended to assign a meaning to the
presented auditory
scene despite the use of band-passed pink noise as virtual sound objects (e.g.,
train, or metro, or
driving in a tunnel). In addition, the auditory scene context had an effect
on the percentage of
front-back and back-front reversals. Sound localization in binaural synthesis
systems has
previously been found to be strongly asymmetric in favor of sound appearance
behind a listener
(Begault, 1994). In the experiment by Väljamäe et al. (2005) this
asymmetry was not observed
and the auditory objects were perceived in front of the listener, favoring scenarios
eliciting self26
motion forwards.
It is known that seeing one’s own body, a self-avatar, in a virtual
environment increase
the sensation of presence (Slater & Usoh, 1994). Following this
“body-centered interaction”
paradigm, sounds representing one’s self-motion in virtual environment
may enhance the vection
sensation. In an experiment with linear AIV using ecological sounds, the addition
of an engine31
like sound to the moving auditory scene significantly increased vection responses
(Väljamäe,
Larsson, Västfjäll, & Kleiner, 2008). The experiment also accessed
mental motor imagery which
refers to user abilities to imagine dynamic processes, including self-motion
(e.g., Hall, Pongrac,
& Buckolz, 1985). It is common to separate motor imagery into visual motor
imagery (seeing
performing of the body movement) and kinesthetic motor imagery (imaging the
feeling that the
actual body-movement produces). The engine sound facilitation effect was significantly
correlated with participants’ kinesthetic but not visual or auditory imagery,
thus suggesting the
relation of a first person perspective in the perception of the self-motion
representation sounds.
To conclude, linear AIV might benefit from specific cues that are not available
for
circular vection simulations, for example, “the point of closest passage,”
the Doppler effect,
acoustic parallax (see section 3 and discussion section 8.2). It should be noted
that comparing the
results from experiments on linear (Väljamäe et al., 2005) and circular
(Larsson et al., 2004)
vection, it seems that the sensation of self-translation can be more easily
induced by moving
acoustic fields than the illusion of self-rotation.
6 Self-motion and auditory localization
It is often assumed that auditory localization solely depends on acoustic parameters.
However, a
number of studies show that the computation of auditory direction also involves
non-auditory
information from visual, vestibular, tactile, and proprioceptive sources which
provide cues about
the body posture (see Lackner, 1983 for a review). In the “audiogyral”
illusion, a perceived
sound source location is displaced during acceleration in the direction opposite
to the physical
self-rotation (Clark & Graybiel, 1949). In the “audiogravic”
illusion, the direction of the auditory
target shifts in the opposite direction to the displacement of the gravitiointertial
force resultant
(Graybiel & Niven, 1951; DiZio, Held, Lackner, Shinn-Cunningham, & Durlach,
2001). This
means that a sound object perceived at a median plane during the upright 1 body
position will be
displaced towards the downward ear during the tilt (Lewald & Karnath, 2002).
Pettorossi and colleagues (Pettorossi, Brosch, Panichi, Botti, Grassi, &
Troiani, 2005)
asked participants to trace a stationary acoustic target with a pointer while
being passively
rotated (yaw) on a platform from -40 to 40 degrees azimuth. Results showed that
the vestibular
information about the whole-body movement significantly improved the accuracy
of the
continuous trace of the acoustic target. Some experimental conditions contained
“asymmetric”
oscillations of the platform, during which the subjective sensation of body
rotation ceased after
some stimulation time. When no vection was perceived, the acoustic target pursuit
was much less
smooth and contained rapid corrections occurring when the discrepancy between
body
orientation and sound location became large. This highlights the influence of
the subjective
vection sensation on auditory localization performance. Remarkably, a recent
study on the spatial
hearing abilities of thirty-five visually-impaired children (most congenitally
blind) showed that
the sound localization cues arising from the self-motion may provide sufficient
information for
the developmental calibration of spatial hearing (Ashmead, Wall, Ebinger, Eaton,
Snook-Hill, &
Yang, 1998).
Early works provided contradictory information regarding the
influence of visually-induced
circular vection on sound localization. While Gemelli found the sound displacement
effect
dependency of perceived self-motion, Arnoult study did not confirm this result
(as cited in
Thurlow & Kerr, 1970). Two decades later, Thurlow and Kerr (1970) found
consistent sound
localization displacements in the direction of visual rotation and noted that
these effects were
mainly related to nystagmoid eye-movements. Recently, such dependences have
been studied for
thoroughly (Otake, Kashio, Sato, & Suzuki, 2006). Participants performed
an ITD and ILD
discrimination tests while seeing an optokinetic pattern rotating at 30 or 900/s
angular velocity.
Only the ITD discrimination was significantly altered by fast but not slow rotation
of visual
stimuli. The authors hypothesized that this might be reflecting the presence
of different
processing pathways for ITD and ILD information. Significantly, the effects
of illusory and real
body self-motion on ITD discriminations were also reported previously (Cullen,
Collins, Dobie,
& Rappold, 1992). In addition, the results from Otake et al. (2006) showed
a correlation between
nystagmoid eye movements and shifts in ITD reversals. More specifically, the
subjective
auditory median plane was shifted to the side opposite to the experienced vection
direction. This
coincides with results from Lewald and Karnath (2001), who also showed the intracranical
sound
localization shifts to the direction opposite to passive whole-body rotation
7 Cross-modal interactions occurring with auditorily-induced vection
During the past decade a substantial body of research has started to reveal
the rules according to
which the brain integrates sensory inputs from different modalities into unified
perceptual
representations and has confirmed that human perception is multisensory in its
nature (Ghazanfar
& Schroeder, 2006). Recent studies point out at least three important characteristics
in the
multisensory integration of dynamic stimuli (Soto-Faraco, Spence, Lloyd, &
Kingstone, 2004).
First, one modality tends to “capture” or dominate other modality
motion percepts (motion
direction, speed or trajectory) in the case of conflicting sensory information.
In such incongruent
situations, the visual modality seems to have a higher weight, in other words,
is more
“dominant” over other modalities (Soto-Faraco, Kingstone, &
Spence, 2003). Second, it seems
that the motion information (e.g., motion direction) stands over and above the
static cues (e.g.
spatial location of stimuli). Third, these interactions between dynamic audio-visual
information
can take place at early processing stages in the brain (e.g., Bremmer (2005);
see section 7.3 for
more details). Taken together, the research over the last years provides growing
evidence that
motion cues are processed across different sensory modalities.
This section will review the research that specifically addresses the multisensory
interaction effects during the perception of illusory self-motion. Some studies
mentioned earlier
sometimes used vestibular (Dodge, 1923; Marme-Karelse & Bles, 1977) or proprioceptive
(Altman et al., 2005) cues to enhance or inhibit the effects of auditorily-induced
vection. A
similar but more psychophysical methodology, referred as multi-modal nulling,
which
determines the “point of subjective equality" between two directionally
incongruent modalities,
has been used in non-auditory vection research (Prothero, 1998). Such methodology
serves to
measure the weight of stimuli from one sensory modality in terms of perceptual
changes
observed in a second sensory modality. However, in this section we will address
instances where
the combination of auditory and other sensory modality enhances the vection
sensation.
7.1 Circular vection
Similarly to Dodge (1923), directionally congruent/incongruent vestibular-auditory
cues for
circular vection were tested by Schinauer and colleagues (Schinauer, Hellmann,
& Höger, 1993).
The auditory stimuli used in this study contained a binaurally recorded “common
room”
containing 4 sound objects (stereophonic music, water splashes and a typewriter)
rotating around
the listener at 160/s angular velocity. Vection intensity ratings were significantly
higher for
directionally compatible stimuli than for non-moving, or moving in the opposite
direction,
acoustic stimuli. Participants also reported that their perceived vection velocity
during the
experiment was higher than the one in the post-experimental demonstration with
the blindfold
removed. The authors suggested that sound cues helped participants to “assess
their status of
perceived self-motion” (Schinauer et al., 1993, p. 381). More recently,
the cognitive influence of
auditory cues on the passive self-rotation perception has been investigated
(Jürgens, Nasios, &
Becker, 2003). In one of the self-motion extrapolation conditions, listeners
were trained to use an
acoustic cue (pitch increase at a rate of one octave per 90 degrees rotation)
as an indicator of
their angular displacement during passive rotation in darkness. The performance
for self-rotation
condition with this auditorily-guided imagery was worse than for the conditions
with optokinetic
input, but comparable with a pure vestibular stimulation, especially for lower
angular velocities
(150/s and 300/s). As the authors suggested, such results may show that cognitive
mechanisms
are an important component for navigation, especially, in the situation where
self-motion
perception has to be carried on in the absence of continuous visual and vestibular
motion cues.
A virtual reality study by Väljamäe et al. (2009) combined rotational
acoustic fields
containing ecological sounds (see section 4.4) with vibrotactile stimulation
delivered under
participants’ seat. While vibrations did not enhance significantly AIV
responses for all
experimental conditions, specific auditory-vibrotactile facilitation could be
observed for the
rotating acoustic field containing a single engine sound. Other auditory-vibrotactile
study used a
hammock chair with participants’ feet either on solid ground or suspended
(Riecke, Feuereissen,
& Rieser, 2008). Individualized binaural recordings of two sound sources
rotating synchronously
at 600/s angular velocity were used. For participants who experienced AIV (8
out of 16),
vibrations significantly increased vection responses; feet suspension also had
an enhancing effect
on AIV.
Research using moving auditory (16-loudspeaker setup) and visual stimuli showed
facilitation of visually-induced vection is some of the participants (Nakamura,
Ueki, Tanaka, &
Ifukube, 1997). Two recent experiments by Riecke and colleagues (Riecke, Väljamäe,
&
Schulte-Pelkum, 2009) investigated whether visually-induced circular vection
can be enhanced
by concurrently rotating auditory cues that match visual landmarks (e.g., a
fountain sound).
Participants sat behind a curved projection screen displaying rotating panoramic
renderings of a
market place. Apart from a no-sound condition, the headphone-based auditory
stimuli consisted
of mono sound, ambient sound, or low/high spatial resolution auralizations using
generic
HRTFs. While merely adding non-rotating (mono or ambient) sound showed no effects,
moving
sound stimuli facilitated both vection and presence in the virtual environment.
This spatialization
benefit was maximal for a medium (20°x15°) field-of-view (FOV), reduced
for a large FOV
(54°x45°) and unexpectedly absent for the smallest FOV (10°x7.5°).
Increasing auralization
spatial fidelity (from low, comparable to 5-channel home theatre systems, to
high, 5° resolution)
provided no further benefit. This ceiling effect might indicate the lower weight
of spatial
auditory cues in the process of dynamic multisensory integration.
7.2 Linear vection
Sakamoto and colleagues (Sakamoto, Suzuki, Suzuki, & Gyoba, 2008) used a
setup where
participants swung in the forward direction while listening to a binaurally
presented auditory
object moving either leftwards or rightwards from the center. The lateral direction
of the
perceived self-motion was assessed using a two-alternative-forced-choice (2AFC)
paradigm, and
cross-modal motion conditions were compared with stationary sound and silent
conditions. The
results showed that the moving auditory image significantly biased participants’
judgments about
the perceived vection direction. Unexpectedly, reported linear vection directions
coincided with
the stimulus motion direction, rather than being opposite to it. Same setup
was used to study the
effects of lateral visual motion presentation, and this time reported linear
vection direction was
opposite to the visual stimuli. Authors concluded that a virtual sound source
simulating auditory
motion acted as a target rather than vection inducting stimuli. It should be
noted that this study
used rather short stimuli presentation time, 20 s, which could restrain the
build up of AIV.
In another auditory-vestibular interaction experiment, several acceleration
velocities
were used in creating uni- and bimodal vection inducing stimulation (Kapralos,
Zikovitz, Jenkin,
& Harris, 2004). Auditory motion was created using a falling intensity sound
from two stationary
loudspeakers at participants’ sides. The estimations of the traveled distance
were slightly more
accurate for bimodal than for physical motion alone. As the authors pointed
out themselves, the
responses accuracy suggested that participants could use other vection cues
created by physical
motion (e.g., initial offset, “jerk” accompanying self-motion onset,
airflow and noise from the
platform physical motion).
Väljamäe et al. (2006) investigated the effects of additional vibrotactile
stimulation on
linear auditorily-induced vection (same experimental setup and auditory stimuli
was in Väljamäe
et al. (2008)). The auditory scenes contained moving sound fields (auditory
landmarks) and/or a
non-spatialized sound representing a sonic self-motion metaphor (an engine-like
sound). The
study showed that vibrotactile stimulation could significantly enhance the self-motion
and
presence responses. However, the vibrotactile stimulation effects varied depending
on the
auditory scene content. The cross-modal enhancement of AIV and presence ratings
was observed
when vibrotactile stimulation accompanied the engine sound. The starting of
the engine sound
was synchronized with the vibrations initial burst, leading to a recognizable
auditory-vibrotactile
stream. This, together with the results from the circular vection study by Väljamäe
et al. (2009),
suggests that the vibrotactile enhancement of self-motion simulation might be
strongest when
vibrations are consistent with cues from other sensory modalities. For example,
such
multisensory grouping can occur when simulating a ride on a pavement road where
vibrations
will coincide with a shaking visual scene and corresponding sound pattern (c.f.
discussion on
Gestalt principles by Bregman (1990)).
7.3 Neural correlates for auditory-vestibular interaction
Apart from the “audiogyral” and “audiogravic” illusions,
where the kinesthetic component plays
an important role in the interaction of audition and vestibular system, Lewald
and collaborators
provided compelling evidence for more direct links between these two modalities.
Auditory
vestibular interactions were observed during vibratory stimulation of neck muscles
(Lewald,
Karnath, & Ehrenstein, 1999), cold caloric stimulation of the semicircular-canal
system (Lewald
& Karnath, 2001), whole body tilt (Lewald & Karnath, 2002) and finally,
during focal
transcranial magnetic stimulation (rTMS) (Lewald, Wienemann, & Boroojerdi,
2004). The
authors propose that two types of interaction can occur depending on whether
afferent sensory
information is related to 1) head position relative to the body or 2) whole
body position relative
to the external environment (Lewald & Karnath, 2002). The authors also provide
evidence that
the posterior parietal cortex (PPC), which is also a part of the auditory “where”
stream, may
represent a neural substrate of the perceptual stability in spatial hearing.
Interestingly, other group of researchers highlights also the role of the PPC,
more
specifically, the ventral intraparietal area (VIP), in the multisensory encoding
of spatial and
motion information, as required for goal-directed movements in external space
(Bremmer,
2005). While other sensory modalities have been known to contribute to the cortical
system for
posture control (Guldin & Grüsser, 1998 and references therein), the
recent study specifically
showed, for the first time, that the VIP area exhibited high neuronal activity
in response to virtual
sound sources (Schlack, Sterbing, Hartung, Hoffmann, & Bremmer, 2005). In
addition, the
measurement of auditory-evoked magnetic fields in elderly patients showed that
auditory cortex,
parieto-insular vestibular cortex, and inferior parietal cortex might be involved
in the
maintenance of equilibrium (Oe, Kandori, Murakami, Miyashita, Tsukada, &
Naritomi, 2002).
These studies suggest that auditory cues might take an active role in supramodal
encoding of
self-motion.
8 General discussion
From the present review it is clear that moving acoustic fields can evoke the
sensation of vection
and its behavioral, physiological, and neural correlates. However, our knowledge
about the
perceptual and cognitive processes underlying auditorily-induced vection is
rather scarce and
thus more systematic studies are needed. Apart from being an interesting topic
for basic research
on auditory and multisensory perception, such knowledge may affect a number
of practical
applications: systems supporting human spatial orientation and performance in
weightless and
unusual gravitoinertial force environments (e.g., Lyons, Gillinghan, & Teas,
1990); systems for
navigation when visual cues are not available (e.g., Loomis, Golledge, &
Klatzky, 1998);
technologies used for vestibular system prosthesis (e.g., Chiari, Dozza, Cappello,
Horak,
Macellari, & Giansanti, 2005); virtual reality simulations of self-motion
and cybertherapy (e.g.,
Durlach et al., 2000); and artistic installations and performances which explicitly
make use of
spatial sound (e.g., Morita, & Tokuno, 2002). The following sections summarize
the previous
work and discuss the issues of experimental methodology, auditory parameters
and cross-modal
interactions which can be important for the future research.
8.1 Methodological issues
As could be seen from the methods reviewed in section 2, there is no single
and robust measure
of vection even in well-established visual research. As with other complex phenomena,
a
combination of different vection measurement techniques might bring better insights
to it.
Vection clearly has a different “vividness” on a subjective level
across individuals, but it is
largely unknown whether this sensation can be reflected in more objective measures.
Vection
onset time or indirect psychophysical methods (e.g., motion detection threshold
or time-to
contact) might help to quantify the vection sensation. More objective vection
measures would
also allow for a comparison between different vection studies within and across
different sensory
modalities. In addition, the multimodal nature of vection perception allows
for new experimental
designs. For example, since the threshold for object motion detection is found
to be higher when
a vection sensation is induced (Probst et al., 1984), it has been suggested
that a similar
methodology could be used with moving auditory objects (Wertheim, 1994). However,
one may
also study how visually-induced circular vection can influence the perception
of looming sounds
or modulate the changing-loudness aftereffect (c.f. Kitagawa & Ichihara,
2002).
The continuous response measurement of the “vection waveform” can
provide new
insights to multisensory self-motion perception. For example, after sustaining
visually-induced
vection for some time, vection enhancing auditory stimuli can be added to the
simulation. The
cross-modal facilitation of the vection sensation can be then compared to the
unisensory
baseline. Alternatively, one can reduce or completely remove visual cues and
see whether
auditory information can be used to maintain the visually-induced vection sensation.
Both
continuous subjective and physiological measures (e.g., skin conductance or
nystagmus) might
be incorporated in such “vection waveform” assessment.
8.2 Spatial sound parameters
Acoustic cues known to be salient for the perception of auditory motion, such
as sound intensity
and binaural cues, are also likely to be important for inducing vection. However,
it remains
largely an open question which physical and contextual parameters of spatial
sound can be most
instrumental for creating the sensation of vection. As discussed below, many
times these cues
may have both perceptual and cognitive impact on vection sensation.
The externalization of the auditory images may be one of AIV modulating factors
providing external reference frame as shown in studies by McFarland and Weber
(1969), and
Lackner (1977). Unfortunately, in these studies the loudspeaker-based presentation
was
contrasted with headphone rendered stereophonic sound which spectral cues were
not adapted to
listener’s pinna and head/torso. Therefore, it is difficult to say whether
the externalization per se
or the quality of the provided spatial cues influenced these authors’
results. Virtual acoustic
spaces based on binaural synthesis can provide a new tool for such investigations.
In this
technique, pre-measured HRTFs are convolved with a non-spatialized sound to
position it in the
virtual acoustic space (Kleiner, Dalenbäck, & Svensson, 1993). In binaural
sound, one of the
predictors for proper auditory image externalization is the match between a
listener’s own
HRTFs and the ones used for the synthesis (however, HRTFs individualization
not always lead
to the most natural sounding as shown by Usher and Martens, (2007)). In a recent
study
contrasting acoustic fields created using individualized vs. generic HRTF catalogues,
no
differences in vection responses were found (Väljamäe et al., 2004).
However, the
externalization influence on vection perception was not the primary aim of this
study and more
dedicated research is needed to resolve this question. Finally, the externalization
of the rendered
auditory scene is shown to significantly affect the subjective sensation of
presence, which is an
important factor for virtual reality applications (Hendrix & Barfield, 1996;
Väljamäe et al.,2004).
The quality of the spatial cues influence the perceived trajectories
of moving auditory
objects which, in turn, should modulate the vection inducing power of the moving
scene. A
typical resolution for the current HRTF catalogues used for binaural synthesis
is 50 in the
horizontal plane (Algazi, Duda, Thompson, & Avendano, 2001; Gardner &
Martin, 1995). It is
known that the human auditory system allows for as little as 1 degree spatial
resolution in the
horizontal plane and 4-9 degrees in the vertical plane depending on the stimulus
spectrum
(Blauert, 1997). These resolutions degrade as the angle of the sound incidence
moves away from
the front of the listener, somewhat resembling the peripheral vision properties.
Perrott (1984)
studied the perception of two concurrently active sound sources and found that
the concurrent
minimum audible angle (CMAA) is larger than the standard minimum audible angle
(MAA)
defined for a single sound source. In free field conditions, the CMAA will depend
on the spatial
position of both sources and the frequency and amplitude content of the signals
(Shinn-
Cunningham, Ihlefeld, Satyavarta, & Larsson, 2005; Best, van Schaik, &
Carlile, 2004). Finally,
the minimum audible movement angle (MAMA) is known to be higher than audible
angle for
stationary sources and it is dependent on a sound object velocity. More specifically,
the auditory
resolution of moving sound sources tends to follow a U-shaped function where
the perception
degrades for velocities outside the optical velocity range (Saberi & Perrott,
1990). More
dedicated studies should define the spatial sound resolution necessary for vection
inducing
acoustic environments since this would allow the optimization of the rendering
techniques and
technologies involved (see also next subsection on cross-modal effects on spatial
sound quality).
Another vection facilitation factor is a number of sound sources forming a moving
acoustic field (Larsson et al., 2004; Väljamäe et al., 2004). However,
this enhancement might be
linked either with the increased quality of the spatial auditory cues, or with
the number of
perceived auditory objects. To disambiguate this condition, one might use virtual
sounds
occupying pre-defined frequency bands. Psychoacoustics research suggests that
auditory
localization is based on a critical bandwidth model where ITD and ILD cues are
analyzed
separately within a set of frequency bands (Blauert, 1997). Such assumption
has been
successfully used for advanced spatial sound compression algorithms (Baumgarte
& Faller,
2003). If binaural cues are the primary course of results in Larsson et al.
(2004), then a similar
circular AIV should be created by a single source occupying three critical bands
or three sound
sources occupying one critical band each. In such comparison, one might also
investigate the
relative weight of ITD and ILD cues since two studies on visually-induced vection
showed the
impact on time but not on intensity discrimination tests (Otake et al., 2006;
Cullen et al., 1992).
In visual research, random-dot kinematograms are typically used as stimuli to
study the
properties of low-level cues in motion perception. Virtual acoustic displays
based on binaural
synthesis may be used for creating acoustic kinematograms containing a large
number of point
sound sources of controlled duration and spatial properties (e.g., rotating
rain drops). Finally, if
higher-level contextual information plays a role in the vection dependence on
the amount of
objects in the moving auditory scene, then the category of virtual sounds (“landmark”
vs.
“moveable” type) can also be an additional modulating factor.
The distance to perceived auditory sources can be another factor influencing
AIV.
Different distances to auditory objects in a moving acoustic field may provide
a more robust
reference frame representing an external environment. When several objects are
present within a
scene, the viewer’s motion changes their relative angular positions (visual
motion parallax) and
this can serve as a self-motion cue (Bronstein & Buckwell, 1997). Auditory
motion parallax (the
auditory equivalent to visual parallax) was found to have a minor impact on
the perception of
auditory distance, compared to stronger cues as, for example, sound intensity
(Speigle &
Loomis, 1993). However, this might be different in the case of auditorily-induced
vection
simulations. In addition, cognitive factors might be also involved, since auditory
distance
perception can depend on the sound type, for example, whisper versus loud voice
(Blauert,
1997). Auditory parallax might exhibit a strong vection inducing power if the
moving scene
would contain several “auditory landmarks” at various distances.
Finally, there might be a
difference in the vection-inducing power between virtual sounds moving in a
peripersonal area
vs. more distant space. Recent neuroscience research highlights the importance
of processing the
information regarding the peripersonal space, which defines a margin of safety
around the body
(see Graziano & Cooke, 2006 for a review). The perception of such personal
space is crucial for
avoiding obstacles during self-motion or coordinating a defensive behavior in
the case of
approaching objects.
All the reviewed studies on auditorily-induced vection used only one plane for
presenting the moving auditory objects. However, it might be that the presentation
of several
auditory objects in separate planes at different elevation angles could be more
effective than the
same number of objects in one plane. Then, the previously discussed question
of whether such
difference is due to enriched spatial cues or to more contextual aspects might
arise.
In addition, the “size” and directivity of a moving sound source
might be also a vection
contributing cue. Typically, vection inducing acoustic fields are created by
real or virtual sound
objects that are represented as point sources (e.g., Lackner, 1977; Väljamäe
et al., 2004).
However, this is a very crude approximation to real sonic environments, where
sound objects
provide percepts of the “apparent source width” (Begault, 1994)
and the “audible facing angle”
(Neuhoff, 2003). Recently, a technique for extending the apparent shape and
width of a single
virtual sound source has been described (Potard & Burnett, 2003). Perceived
as landmarks,
“enlarged” virtual sound objects may play an important role in creating
vection. At the same
time, the correspondence between the simulated width of the virtual source and
its ecological
content can be another research question to address (e.g., a single typewriter
sound creating a
sound wall rotating around the user). Remarkably, an acoustic shadow created
by an obstacle
between a sound source and a listener may also serve as an additional cue for
self-motion
perception (Dodge, 1923).
In specific situations, for example, when entering a tunnel,
the acoustic environment
provides a significant cue about the change in spatial location. There is some
evidence that
human may use “echolocation” for navigating in a reverberant environment
(Stoffregen &
Pittenger, 1995; Rosenblum et al., 2000). Study by Stoffregen and colleagues
(Stoffregen,
Villard, Kim, Ito, & Bardy, 2009) showed that listeners could detect linear
motion of a
surrounding room (a box on wheels without a floor surrounded a listener), even
when only
reflected sound cues were available. Simulating outdoor environments using acoustic
modeling
has shown to affect auditory presence ratings but not the vection sensation
(Larsson et al., 2004).
Since the perception of auditory distance indoors heavily depends on reverberation
parameters
(Bronkhorst & Houtgast, 1999), room acoustics cues might contribute to self-motion
perception
in specific environments.
Finally, similarly to visually-induced vection, increasing the velocity of a
moving
acoustic stimulus augments the circular vection sensation (Lackner, 1977; Neetz
et al., 1980;
Larsson et al., 2004). However, an optimal angular velocity range around 600/s
seems to exist for
AIV (Lackner, 1977; Gekhman, 1991). In the case of linear vection, the Doppler
effect might
play an important role for higher simulated velocities.
8.3 Multisensory interactions
A number of recent studies on the “vestibular cortex” areas in the
brain started to reveal its
multisensory nature, where not only vestibular signals, but also visual, tactile
and auditory self
motion cues converge (Bremmer, 2005 and references therein). However, the multisensory
vection research involving auditory cues is just emerging. A few studies available
suggest that
auditory cues may have a similar perceptual weight as vestibular cues when rotating
in dark at
low velocities (Marme-Karelse & Bles, 1977; Schinauer et al., 1993). More
systematic studies
should help to quantify the sound contribution to circular or linear vection
perception. A number
of auditory parameters listed in the previous section might interact with other
sensory modalities.
For example, it is an open question whether intracranially perceived (non-externalized)
auditory
motion might interact with optokinethic stimulation. If it does, will such interaction
be less
efficient than properly externalized rotating acoustic fields accompanying visual
motion? How
crucial can be the directional or velocity congruency between such auditory-visual
stimuli for
vection perception? Can non-spatialized sound with a rhythmic temporal structure
be used for
restoring (filling-in) degraded visual scene dynamics and thus indirectly enhance
vection (c.f.
Väljamäe & Soto-Faraco, 2008)? Such knowledge is not only important
for improving the design
of motion simulations, but also is contributing to a better understanding of
cross-modal
interactions in the perception of stimuli dynamics. If sound appears to have
a lower perceptual
weight compared to other modalities, this might be outweighed by its cognitive
contributions to
the vection sensation. The dependence of the self-motion illusion on the congruent
or conflicting
combinations of ecological auditory and visual objects (e.g., a fountain sound
matched to a bus
image) can be another interesting topic for future research. While keeping physical
parameters of
audio-visual landmarks intact, such cognitive conflicts might modulate the sensation
of illusory
self-motion.
Acknowledgements
This work was supported by European Community under the FET Presence Research
Initiative
project POEMS (Perceptually Oriented Ego-Motion Simulation), IST-2001-39223,
and the
Swedish Research Council. The author would like to thank Dr. Mendel Kleiner
and Dr. Shuichi
Sakamoto for translations of German and Japanese publications, and anonymous
reviewers for
their detailed comments and insightful ideas that helped to improve the manuscript.
References
Agaeva, M. Y., & Altman, Y. A. (2005). Effect of a sound stimulus on postural
reactions.
Human Physiology, 31(5), 511–514. Translated from Fiziologiya Cheloveka,
31(5), 21–25.
Agaeva, M. Y., Altman, Y. A., & Kirillova, I. Y. (2006). Effects of a sound
source moving in a
vertical plane on postural responses in humans. Neuroscience and Behavioral
Physiology,
36(7), 773-780.
Algazi, V. R., Duda, R. O., Thompson, D. M., & Avendano, C. (2001). The
CIPIC HRTF
database. In Proceedings of the IEEE Workshop on Applications of Signal Processing
to Audio
and Acoustics. New Platz, NY, USA, 2001 (pp. 99–102).
Altman, Y. A., Varyagina, O. V, Gurfinkel, V. S., & Levik, Y. S. (2005).
The Effects of moving
sound images on postural responses and the head rotation illusion in humans.
Neuroscience
and Behavioral Physiology, 35(1), 103-106. Translated from Ros. Fiziol. Zh.
im. I. M.
Sechenova, 89(6), 756–761.
Aoki, M., Thilo, K. V., Burchill, P., Golding, J.F., Gresty, M.A. (2000) Autonomic
response to
real versus illusory motion (vection), Clinical Autonomic Research, 10, 23-28.
Ashmead, D. H., Wall, R. S., Ebinger, K. A., Eaton S. B., Snook-Hill M. M. &
Yang X. (1998)
Spatial hearing in children with visual disabilities. Perception, 27, 105–122.
Baumgarte, F., & Faller, C. (2003). Binaural cue coding - Part I: Psychoacoustic
fundamentals
and design principles. IEEE Transactions on Speech and Audio Processing, 11(6),
509-519.
Baumgartner, T., Valko, L., Esslen, M., & Jäncke, L. (2006). Neural
correlate of spatial presence
in an arousing and noninteractive Virtual Reality: An EEG and psychophysiology
study.
Cyberpsychology and Behavior, 9(1), 30-45.
Begault, D. R. (1994). 3D sound for virtual reality and multimedia. New York:
Academic Press.
Bensel, C. K., Dzendolet, E., & Meiselman, J.L. (1968). Body sway during
long term standing
and as affected by pure tones. Perceptual Psychophysiology, 4, 171-174.
Best, V., van Schaik, A., & Carlile, S. (2004). Separation of concurrent
broadband sound sources
by human listeners. Journal of the Acoustical Society of America, 115, 324 –336.
Blauert, J. (1997). Spatial Hearing (Rev. ed.). Cambridge, MA: MIT Press.
Brandt, T., Bartenstein, P., Janek, A., & Dieterich, M. (1998). Reciprocal
inhibitory visual31
vestibular interaction: Visual motion stimulation deactivates the parieto-insular
vestibular
cortex. Brain 121, 1749–1758.
Brandt, T., Dichgans, J., & Koenig, E. (1973). Differential effects of central
versus peripheral
vision on egocentric and exocentric motion perception. Experimental Brain Research,
16, 476-
491.
Bregman, A. S. (1990). Auditory scene analysis: the perceptual organization
of sound.
Cambridge, MA: MIT Press.
Bremmer, F. (2005). Navigation in space: The role of the macaque ventral intraparietal
area.
Journal of Physiology, 566(1), 29–35.
Bronkhorst, W., & Houtgast, T. (1999). Auditory distance perception in rooms.
Nature, 397,
517-520.
Bronstein, A. M., & Buckwell, D. (1997). Automatic control of postural sway
by visual motion
parallax. Experimental Brain Research, 113, 243–248.
Carlile, S., & Best, C. (2002). Discrimination of sound velocity in human
listeners. Journal of
Acoustical Society of America, 111, 1026–1035.
Cheung, B., Hofer, K., Heskin, R., & Smith, A. (2004). Physiological and
behavioral responses
to an exposure of pitch illusion in the simulator. Aviation, Space, and Environmental
Medicine,
75(8), 657-665.
Chiari, L., Dozza, M., Cappello, A., Horak, F.B., Macellari, V. & Giansanti,
D. (2005) Audio50
biofeedback for balance improvement: An accelerometry-based system, IEEE Transactions
on
Biomeical Engineering, 52 (12), 2108–2111.
Clark, B., & Graybiel, A. (1949). The localization: The audiogyral illusion.
Journal of
Psychology, 28, 235–244.
Cohen, B., & Raphan, T. (2004). The Physiology of the Vestibuloocular Reflex
(VOR). Springer
Handbook of Auditory Research, 19, 235-285.
Cullen, J. K., Collins, M., Dobie, T. H., & Rappold, P. W. (1992). The effects
of perceived
motion on sound-source lateralization. Aviation, Space and Environmental Medicine,
63, 498-
504.
Deutschländer, A., Bense, S., Stephan, T., Schwaiger, M., Dieterich, M.,
& Brandt, T. (2004).
Rollvection versus linearvection: comparison of brain activations in PET. Human
Brain
Mapping, 21, 143–153.
Dichgans, J., & Brandt, T. (1978). Visual-vestibular interaction: Effects
on self-motion
perception and postural control. In R. Held, H.W. Leibowitz & H.L. Teuber
(Eds.) Handbook
of Sensory Physiology Vol. VIII: Perception (pp. 755-804). Berlin: Springer
Verlag.
Dieterich, M., Bense, S., Stephan, T., Yousry, T. A., & Brandt, T. (2003).
fMRI signal increases
and decreases in cortical areas during small-field optokinetic stimulation and
central fixation.
Experimental Brain Research, 148, 117–27.
DiZio, P., Held, R., Lackner, J. R., Shinn-Cunningham, B. & Durlach, N.
(2001) Gravitoinertial
force magnitude and direction influence head-centric auditory localization.
Journal of
Neurophysiology, 85, 2455-2460.
Dodge, R. (1903). Five types of eye movement in the horizontal meridian plane
of the field of
regard. American Journal of Physiology, 8, 307–329.
Dodge, R. (1923). Thresholds of rotation. Journal of Experimental Psychology,
6, 107-137.
Durlach, N., Allen, G., Darken, R., Garnett, R.L., Loomis, J., & Templeman,
J. (2000). Virtual
environments and the enhancement of spatial behavior: toward a comprehensive
research
agenda. Presence: Teleoperators & Virtual Envimnments, 9, 593–614.
Easton, D. R., Greene, A. J., DiZio, P., & Lackner, J. R. (1998). Auditory
cues for orientation
and postural control in sighted and congenitally blind people, Experimental
Brain Research,
118, 541–550.
Fiebig, E., Schaefer, K. P., & Süss, K. J. (1981) Form and accuracy
of voluntary Ocular tracking
movements in response to sinusoidally moving acoustic targets. Journal of Neurology,
226(2),
1432-1459.
Fukui, T., Kimura, T., Kadota, K., Shimojo, S., & Gomi, H. (2009). Odd Sensation
Induced by
Moving-Phantom which Triggers Subconscious Motor Program. PLoS ONE 4(6): e5782.
doi:10.1371/journal.pone.0005782
Ganz, H., & Mattern, B. (1969). Studies on audiokinetic eye movements. II.
Audiokinetics and
directional hearing tests. Zeitschrift für Laryngologie, Rhinologie, Otologie
und ihre
Grenzgebiete, 48(10), 763-80 (in German). [Untersuchungen über audiokinetische
augenbewegungen].
Gardner, W., & Martin, K. (1995). HRTF measurements of a KEMAR. Journal
of the Acoustical
Society of America, 97, 3907–3908.
Gaver, W. W. (1993). How do we hear in the world? Explorations in Ecological
Acoustics.
Ecological Psychology, 5, 285–313.
Gekhman, B. I. (1991). Audiokinetic nystagmus. Sensornye Sistemy, 5(2), 71–78
(in Russian).
Ghazanfar, A. A., & Schroeder, C. E. (2006). Is neocortex essentially multisensory?
Trends in
Cognitive Science, 10(6), 278-285.
Grantham, D. W. (1986). Detection and discrimination of simulated motion of
auditory targets in
the horizontal plane. Journal of the Acoustical Society of America, 79, 1939–1949.
Gray, R., & Regan, D. (2000). Simulated self-motion alters perceived time
to collision. Current
Biology, 10(10), 587–590.
Graybiel, A., & Niven, J. I. (1951). The effect of a change in direction
of resultant force on
sound localization: The audiogravic illusion. Journal of Experimental Psychology,
42, 227–230.
Graziano, M. S., & Cooke, D. F. (2006). Parieto-frontal interactions, personal
space, and
defensive behavior. Neuropsychologia, 44, 845–859.
Guerraz, M., Thilo, K. V., Bronstein, A. M., & Gresty, M. A. (2001). Influence
of action and
expectation on visual control of posture. Brain Research, Cognitive Brain Research,
11, 259–266.
Guldin, W. O., & Grüsser, O. J. (1998). Is there a vestibular cortex?
Trends in Neurosciences,
21, 254–259.
Hall, C. R., Pongrac, J., & Buckolz, E. (1985). The measurement of imagery
ability. Human
Movement Science, 4, 107–118.
Hall, D. A., & Moore, D. R. (2003). Auditory neuroscience: The salience
of looming sounds.
Current Biology, 13(3), R91-3.
Hendrix, C., & Barfield, W. (1996). The sense of presence within auditory
virtual environments.
Presence – Teleoperators and Virtual Environments, 5(3), 290-301.
Hennebert, P. E. (1960). Audiokinetic nystagmus. Acta Otolaryngolica, 51,412-415
(in French).
Hettinger, L. J. (2002). Illusory self-motion in virtual environments. In K.M.
Stanney (Ed.),
Handbook of Virtual Environments (pp. 471-492). NJ: Lawrence Erlbaum.
Hill, E. W., & Ponder, P. (1976). Orientation and mobility techniques: A
guide for the
practitioner. New York: American Foundation for the Blind.
Ijsselsteijn, W., de Ridder, H., Freeman, J., Avons, S. E., & Bouwhuis,
D. (2001). Effects of
stereoscopic presentation, image motion, and screen size on subjective and objective
corroborative measures of presence. Presence-Teleoperators and Virtual Environments,
10,
298-311.
Ishizaki, H., Pyykko, I., Aalto, H., & Starck, J. (1991). Tullio phenomenon
and postural stability:
Experimental study in normal and patients with vertigo. Annals of Otology, Rhinology
and
Laryngology, 100, 976-983.
Jürgens, R., Nasios, G., & Becker, W. (2003). Vestibular, optokinetic,
and cognitive contribution
to the guidance of passive self-rotation toward instructed targets. Experimental
Brain
Research, 151, 90–107.
Kapralos, B., Zikovitz, D., Jenkin, M. & Harris, L.R. (2004). Auditory cues
in the perception of
self-motion. In Proceedings of the 116th AES convention, Berlin, Germany, 2004,
Journal of
Audio Engineering Society (Abstracts), 52, 801-802
Stoffregen, T. A., Villard, S., Kim, C., Ito, K., & Bardy, B. G. (2009).
Coupling of head and
body movement with motion of the audible environment. J. of Experimental Psychology:
Human Perception & Performance, in press.
Kitagawa, N., & Ichihara, S. (2002). Hearing visual motion in depth. Nature,
416, 172-174.
Kleiner, M., Dalenbäck, B. I., & Svensson, P. (1993). Auralization:
An overview. Journal of
Audio Engineering Society, 41 (11), 861-875.
Kobayashi, K., Fushiki, H., Asai, M., & Watanabe, Y. (2005). Head and body
sway in response
to vertical visual stimulation. Acta Oto-Laryngologica, 125(8), 858-862.
Lackner, J. R. (1977). Induction of illusory self-rotation and nystagmus by
a rotating sound-field.
Aviation, Space and Environmental Medicine, 48(2), 129-131.
Lackner J. R. (1978). Some Mechanisms Underlying Sensory and Postural Stability
in Man. In
R. Held, H.W. Leibowitz, & H.L. Teuber (Eds.), Handbook of Sensory Physiology
Vol. VIII:
Perception (pp. 805-845). Berlin: Springer Verlag.
Lackner, J. R. (1983). The influence of posture on the spatial localization
of sound. Journal of
Audio Engineering Society, 31, 650-661.
Larsson, P., Västfjäll, D., & Kleiner, M. (2004). Perception of
self-motion and presence in
auditory virtual environments. In Proceedings of the Seventh Annual Workshop
of Presence,
Valencia, Spain, 2004 (pp. 252-258).
Lee, D. N. (1990). Getting around with light or sound. In R. Warren, & A.
H. Wertheim (Eds.),
Perception and Control of Self-motion: Resources for Ecological Psychology (pp.
487-505).
Hillsdale, NJ: Erlbaum.
Lewald, J., Karnath, H. O., & Ehrenstein, W. H. (1999). Neck-proprioceptive
influence on
auditory lateralization. Experimental Brain Research, 125, 389–396.
Lewald, J., & Karnath, H. O. (2001). Sound lateralization during passive
whole-body rotation.
European Journal of Neuroscience, 13, 2268-2272.
Lewald, J., & Karnath, H. O. (2002). The effect of whole-body tilt on sound
lateralization.
European Journal of Neuroscience, 16, 761-766.
Lewald, J., Wienemann, M., & Boroojerdi B. (2004) Shift in sound localization
induced by
rTMS of the posterior parietal lobe, Neuropsychologia, 42, 1598–1607
Lombard, M., & Ditton, T. (1997). At the heart of it all: The concept of
presence. Journal of
Computer-Mediated Communication, 3(2). Retrieved August 9, 2007 from
http://www.ascusc.org/jcmc/vol3/issue2/lombard.html.
Loomis, J. M., Golledge R,. G., & Klatzky, R. L. (1998). Navigation system
for the blind:
Auditory display modes and guidance. Presence: Teleoperators & Virtual Envimnments,
7,
193-203.
Lutfi, R. A., & Wang, W. (1999). Correlated analysis of acoustic cues for
the discrimination of
auditory motion. Journal of the Acoustical Society of America, 106(2), 919–928.
Lyons, T. J., Gillinghan, K. K. & Teas, D. C. (1990) The effect of acoustic
orientation cues on
instrumental performance in a fight simulator. Aviation, Space, and Environmental
Medicine,
61, 699-706, 1990.
Maier, J. X., Neuhoff, J. G., Logothetis, N. K., & Ghazanfar, A. A. (2004).
Multisensory
integration of looming signals by rhesus monkeys, Neuron, 43, 177-181.
Marme-Karelse, A. M., & Bles, W. (1977). Circular vection and human posture,
II. Does the
9 auditory system play a role? Agressologie, 18(6), 329-333.
Martens, W. L. (2004) The importance of perceived self-motion in experiencing
convincing
virtual acoustic rendering, Journal of Acoustic Society of America, 115, 2514.
McBeath M. K., & Neuhoff, J. G. (2002). The Doppler effect is not what you
think it is:
Dramatic pitch change due to dynamic intensity change. Psychonomic Bulletin
& Review, 9(2),
306-313.
McFarland, W. H., & Weber, B. A. (1969). An investigation of ocular response
to various forms
of sound field auditory stimulation. Journal of Auditory Research, 9(3), 236-239.
Mergner, T., & Becker, W. (1990). Perception of horizontal self-rotation:
Multisensory and
cognitive aspects. In R. Warren, & A. H. Wertheim (Eds.), Perception and
Control of Self
motion: Resources for Ecological Psychology (pp. 219-263). Hillsdale, NJ: Erlbaum.
Morita, S., & Tokuno, S. (2002). Visualizing Sound Flow on Orchestra Performance.
IEIC
Technical Report, 102(533), 25-32. (in Japanese).
Nakamura, H., Ueki, N., Tanaka, T., & Ifukube, T. (1997). Influence of light
or sound
stimulation on the perception of rotatory motion. Technical report of The Institute
of
Electronics, Information and Communication Engineers, HIP96, 46, 43-48. (In
Japanese).
Neetz, A., Süss, K. L, & Schaeffer, K. P. (1980). Influence of moving
acoustic sound sources on
body balance. Pfluegers Arch. Suppl., 384, R24.
Neuhoff, J. G. (2001). An adaptive bias in the perception of looming auditory
motion. Ecological
Psychology, 13(2), 87-110.
Neuhoff, J. G. (2003). Twist and shout: audible facing angles and dynamic rotation.
Ecological
Psychology, 15(4), 335–351.
Neuhoff, J. G. (2004). Ecological psychoacoustics: Introduction and history.
In J. G. Neuhoff
(Ed.) Ecological psychoacoustics (pp. 1-13). Amsterdam; Boston: Elsevier Academic
Press.
Neuhoff, J. G., Planisek, R., & Seifritz, E. (2009). Adaptive sex differences
in auditory motion
perception: looming sounds are special. Journal of Experimental Psychology:
Human
Perception and Performance, 35(1), 225–234.
Nishiike, S., Nakagawa, S., Tonoike, M., Takeda, N., & Kubo, T. (2001).
Information processing
of visually-induced apparent self motion in the cortex of humans: Analysis with
magnetoencephalography. Acta Oto-Laryngologica, 121, S545, 113 – 115.
Oe, H., Kandori, A., Murakami, M., Miyashita, K., Tsukada, K., & Naritomi,
H. (2002). Cortical
functional abnormality assessed by auditory-evoked magnetic fields and therapeutic
approach
in patients with chronic dizziness. Brain Research, 957(2), 373–381.
Otake, R., Kashio, A., Sato, T., & Suzuki, M., (2006). The effect of optokinetic
stimulation on
orientation of sound lateralization. Acta Oto-Laryngologica, 126, 718- 723
Parker, D. E., Tubbs, R. L., & Littlefield, V. M. (1978). Visual-field displacements
in human
beings evoked by acoustical transients. Journal of the Acoustical Society of
America, 63, 1912–1918.
Perrott, D. R. (1984). Concurrent minimum audible angle: a reexamination of
the concept of
auditory spatial acuity. Journal of the Acoustical Society of America, 75, 1201–1206.
Petersen, H., Magnusson, M., Johansson, R., Åkesson, M., & Fransson,
P.-A., (1995). Acoustic
cues and postural control. Scandinavian Journal of Rehabilitation Medicine,
27, 99-104
Petersen, H., Magnusson, M., Johansson, R., & Fransson, P.-A., (1996) Auditory
feedback
regulation of perturbed stance in stroke patients. Scandinavian Journal of Rehabilitation
Medicine, 28, 217-223
Pettorossi, V. E., Brosch, M., Panichi, R., Botti, F., Grassi, S., & Troiani,
D. (2005) Contribution
of self-motion perception to acoustic target localization. Acta Oto-Laryngologica,
125, 524-528.
Popper, A. N., & Fay, R. R. (1997). Evolution of the ear and hearing: Issues
and questions.
Brain, Behavior and Evolution, 50, 213-221.
Potard, G., & Burnett, I. (2003). A study on sound source apparent shape
and wideness. In
Proceedings of the 2003 International Conference on Auditory Display (ICAD’03),
Boston,
MA, USA, 2003 (pp. 25-28).
Probst, T., Krafczyk, S., Brandt, T., & Wist., E. (1984). Interaction between
perceived self7
motion and object motion impairs vehicle guidance. Science, 225, 536-538.
Prothero, J. D. (1998). The role of rest frames in vection, presence and motion
sickness.
(Doctoral dissertation, University of Washington, USA, 1998).
Raper, S.A., & Soames, R.W. (1991). The influence of stationary auditory
fields on postural
sway behaviour in man. European Journal of Applied Physiology, 63, 363–367.
Rees, G., Frith, C. D., & Lavie, N. (1997). Modulating irrelevant motion
perception by varying
attentional load in an unrelated task. Science, 278, 1616–1619.
Reschke, M. F., & Parker, D. E. (1970). Stimulation of the vestibular apparatus
in the guinea pig
by static pressure changes: Head and eye movements. Journal of the Acoustical
Society of
America, 48, 913–923.
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., &
Bülthoff, H. H.
(2006). Cognitive factors can influence self-Motion perception (vection) in
virtual reality.
ACM Transactions on Applied Perception, 3(3), 194-216.
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2008). Auditory self-motion
illusions ("circular
vection") can be facilitated by vibrations and the potential for actual
motion. Proceedings of
the 5th Symposium on Applied Perception in Graphics and Visualization (APGV
08), 147-
154. (Eds.) Creem-Regehr, S. H., K. Myszkowski, ACM Press, New York, NY, USA.
Riecke B. E., Väljamäe, A. & Schulte-Pelkum, J. (2009) Moving
sounds enhance the visually25
induced self-motion illusion (circular vection) in Virtual Reality. ACM Transactions
of
Applied Perception, 6(2).
Rodionov, V., Zislin, J., Elidan, J. (2004). Imagination of body rotation can
induce eye
movements. Acta Oto-Laryngologica, 124(6), 684-689.
Rosenblum, L. D., Carello, C. , & Pastore, R. E. (1987). Relative effectiveness
of three stimulus
variables for locating a moving sound source. Perception, 16, 175–186.
Rosenblum, L. D., Wuestefeld, A. P., & Anderson, K. L. (1996). Auditory
reachability: An
affordance approach to the perception of sound source distance. Ecological Psychology,
8(1),
1-24.
Rosenblum, L. D., Gordon, M. S., & Jarquin, L. (2000). Echolocating distance
by moving and
stationary listeners. Ecological Psychology, 12(3), 181-206.
Saberi, K., & Perrott, D. R. (1990). Minimum audible movement angles as
a function of sound
source trajectory. Journal of the Acoustical Society of America, 88, 2639–2644.
Sakamoto, S., Osada, Y., Suzuki, Y., & Gyoba, J. (2004). The effects of
linearly moving sound
images on self-motion perception. Acoustical Science & Technology, 25(1),
100-102.
Sakamoto, S., Suzuki, F., Suzuki, Y., & Gyoba, J. (2008). The effect of
linearly moving sound
image on perceived self-motion with vestibular information. Journal of Acoustic
Science and
Technology, 29(6), 391-393.
Sakellari, V., & Soames, R. W. (1996). Auditory and visual interactions
in postural stabilization.
Ergonomics, 39(4), 634-648.
Schaefer, K. P., Süss, K. J., & Fiebig, E. (1981). Acoustic-induced
eye movements. Annals of the
New York Academy of Sciences, 374, 674-688.
Schinauer, T., Hellmann, A., & Höger, R. (1993). Dynamic acoustical
stimulation affects self48
motion perception. In A. Schick (Ed.), Contributions to Psychological Acoustics.
Results of the
6th Oldenburg Symposium on Psychological Acoustics, 1993 (pp. 373-385). Oldenburg:
Bibliotheks- und Informationssystem der Carl von Ossietzky Universität
Oldenburg.
Schlack, A., Sterbing, S., Hartung, K., Hoffmann, K. P, & Bremmer, F. (2005).
Multisensory
space representations in the macaque ventral intraparietal area. Journal of
Neuroscience, 25,
4616–4625.
Shaw, B. K., McGowan, R. S., & Turvey, M.T. (1991). An acoustic variable
specifying time to
contact. Ecological Psychology, 3, 253-261.
Sheykholeslami, K., & Kaga, K. (2002). The otolithic organ as a receptor
of vestibular hearing
revealed by vestibular-evoked myogenic. Hearing Research, 165, 62-67.
Shinn-Cunningham, B. G., Ihlefeld, A., Satyavarta, E. L., & Larsson, E.
(2005). Bottom-up and
top-down influences on spatial unmasking. Acta Acustica united with Acustica,
91, 967-979.
Slater, M., & Usoh M. (1994). Body centred interaction in immersive Virtual
Environments. In
N. M. Thalmann, & D. Thalmann, (Eds.), Artificial Life and Virtual Reality,
(pp. 125-148).
John Wiley and Sons.
Soames, R. W., & Raper, S. A, (1992). The influence of moving auditory fields
on postural sway
behaviour in man. European Journal of Applied Physiology, 65, 241–245.
Soto-Faraco, S., Kingstone, A., & Spence, C. (2003). Multisensory contributions
to the
perception of motion. Neuropsychologia, 41(13):1847-62.
Soto-Faraco, S., Spence, C., Lloyd, D., & Kingstone, A. (2004). Moving multisensory
research
along: motion perception across sensory modalities. Current Directions in Psychological
Sciences, 13(1), 29-32.
Speigle, J. M., & Loomis, J. M. (1993). Auditory distance perception by
translating observers. In
Proceedings of the IEEE Symposium on Research Frontiers in Virtual Reality,
New York, 1993 (pp. 92-99).
Stephan, T., Deutschländer, A., Nolte, A., Schneider, E., Wiesmann, M.,
Brandt, T., et al. (2005).
Functional MRI of galvanic vestibular stimulation with alternating currents
at different
frequencies. Neuroimage, 26, 721–731.
Stoffregen, T. A., & Bardy, B. G. (2001). On specification and the senses.
Behavioral and Brain
Sciences, 24, 195-261.
Stoffregen, T. A., & Pittenger, J. B. (1995). Human echolocation as a basic
form of perception
and action. Ecological Psychology, 7, 181-216.
Tanaka, T., Kojima, S., Tokeda, H., Ino, S., & Ifukube, T. (2001). The influence
of moving
auditory stimuli on standing balance in healthy young adults and the elderly.
Ergonomics, 44
(15), 1403–1412.
Teramoto, W., Watanabe, H., Umemura, H., Matsuoka, K., & Kita, S. (2004).
Judgment biases
of temporal order during apparent self-motion. IEICE Transactions on Information
and
Systems, E87-D (6), 1466–1476.
Thurlow, W. R., Kerr, T. P. (1970). Effect of a moving visual environment on
localization of
sound. The American Journal of Psychology, 83, 112-118.
Tokumaru, O., Kaida, K., Ashida, H., Yoneda, I., & Tatsuno, I. (1999). EEG
topographical
analysis of spatial disorientation. Aviation, space, and environmental medicine,
70(3), 256- 263.
Usher, J., & Martens, W. L. (2007). Perceived naturalness of speech sounds
presented using
personalized versus non-personalized HRTFs. Proceedings of the 13th International
Conference on Auditory Display, Montréal, Canada, June 26-29.
Urbantschitsch, V. (1897). Über Störungen des Gleichgewichtes und
Scheinbewegungen [On
disturbances of the equilibrium and illusory motions]. Zeitschrift für
Ohrenheilkunde, 31, 234- 294.
Väljamäe, A., Larsson, P., Västfjäll, D., & Kleiner,
M. (2004). Auditory presence, individualized
head-related transfer functions, and illusory ego-motion in virtual environments.
In
Proceedings of the 7th Annual Workshop on Presence, Valencia, Spain, 2004 (pp.
141-147).
Väljamäe, A., Larsson, P., Västfjäll, D., & Kleiner,
M. (2005). Traveling without moving:
Auditory scene cues for translational self-motio. In Proceedings of the 11th
International
Conference on Auditory Display, ICAD’05, Limerick, Ireland, 2005.
Väljamäe, A., Larsson, P., Västfjäll, D., & Kleiner,
M. (2006). Vibrotactile enhancement of
auditorily-induced self-motion and presence. Journal of the Audio Engineering
Society,
54(10), 954-963.
Väljamäe, A., Larsson, P., Västfjäll, D., & Kleiner,
M. (2008) Sound representing self-motion in
virtual environments enhances linear vection. Presence: Teleoperators and Virtual
Environments, 17 (1), 43-56.
Väljamäe, A., & Soto-Faraco, S. (2008). Filling-in visual motion
with sounds. Acta
Psychologica, 129 (2), 249-254.
Väljamäe, A., Larsson P., Västfjäll D., & Kleiner, M.
(2009) Auditory landmarks enhance
circular vection in multimodal Virtual Reality, Journal of Acoustic Engineering
Society, 57(3), 111-120.
Warren, J. D., Zielinski, B. A., Green, G. R., Rauschecker, J. P., & Griffiths,
T. D. (2002).
Perception of sound source motion by the human brain. Neuron, 34, 139–148.
Watson, J. E. (1968). Evaluation of audiokinetique nystagmus as a test of auditory
sensitivity.
The Journal of Auditory Research, 8, 161-165.
Weber, B. A., & Millburn, W. O. (1967). The effects of a rhythmically moving
auditory stimulus
on eye movements in normal young adults. The Journal of Auditory Research, 7,
259-266.
Wertheim, A. H. (1994). Motion perception during self-motion: The direct versus
inferential
controversy revisited. Behavioral and Brain Sciences, 17(2), 293-355.
Wiest, G., Amorim, M. A., Mayer, D., Schick, S., Deecke, L., & Lang, W.
(2001). Cortical
responses to object-motion and visually-induced self-motion perception. Brain
Research,
Cognitive Brain Research, 12, 167–170.
Wright, W. G., DiZio, P., & Lackner, J. R. (2006). Perceived self-motion
in two visual contexts:
Dissociable mechanisms underlie perception. Journal of Vestibular Research,
16, 23-28.
Yardley, L., Gardner, M., Lavie, N., & Gresty, M. (1999). Attentional demands
of perception of
passive self-motion in darkness. Neuropsychologia, 37, 1293–1301.
Zalewski, P. (1976). Audiokinetic reactions. Otolaryngologia Polska, 30(3),
235-9 (in Polish).
Zajonc, R. B. (1998) Emotions. In D. T. Gilbert, S. T. Fiske, & G. Lindzey,
eds., Handbook of
social psychology, (pp. 591– 632), 4th Ed., Vol. 1. New York: Oxford University
Press.
___
Texte und Kommentare zum Labor 2009 werden hier veröffentlicht
Texts and commentaries on the 2009 lab and related research subjects will be published here.