|Sound localization behavior of early-blind and normal-sighted listeners|
| Sound localization in azimuth and elevation was measured
by head pointing. The auditory target was a broad-band 'buzzer', that was partly masked by broadband
Gaussian white noise of 60 dBA in the background. The signal-to-noise ratio (SNR) was systematically
varied between 0 dB (equally loud) and -21 dB (buzzer almost inaudible re. background).
(A,B) show the raw regression results (gain and correlation) per subject; open symbols: azimuth, closed symbols: elevation. Circles: sighted. Triangles: blind.
Panel (C) shows the results after normalization of the regression parameters, in order to get rid of the variability in baseline behavior (and allowing for averaging across subjects).
The blind and sighted were indistinguishable when comparing their azimuth localization (C, open symbols). However, in elevation the blind were clearly impaired when compared with the sighted (closed symbols).
Grey area within the two elevation curves corresponds to a difference of at least four standard deviations. At the extremes (no background noise, and at -21 dB S/N ratio) the two groups were the same. For the intermediate S/N ratios, however, the blind clearly performed much worse. As soon as background noise sets in, the blind are almost incapable of localizing elevation.
We conclude that elevation localization in the frontal hemifield requires training by vision. However, an intact visual system is not mandatory for learning to localize stimulus azimuth.
|Adaptation of sound localization in response to compressed vision|
|(A,C) Responses (averaged across subjects) across the
two-dimensional stimulus-response field were computed by the so-called 'local gain change', which
is the difference between the locally determined slopes of the stimulus-response relation after adapting to,
and before putting on, the minifying glasses. Blue colors indicate a local decrease of the
slope, while dark red would indicate an increase. Note that the local gain has only decreased in
the frontal field of view, roughly corresponding to the view offered by the lenses. These data show
for the first time that the human auditory localization system is trained by the visual system.|
(B,D) After taking off the glasses, localization returns to normal. Here, the difference was computed between the adapted state and the re-adapted state. Note that the increase in gain now extends to the entire visual field (probably because with the glasses off the field of view is much larger!).
|(Data are from the two different localization paradigms: retinal-based pointing of the laser while fixating straight-ahead (bottom), vs. eye pointing by foveating the laser pointer (top). Results were quite similar for both experiments, supporting the idea that adaptation was indeed taking place within the auditory localization system, rather than in the visual, or oculomotor systems).|
|Plasticity in the human sound localization system|
|Summary of the experiments with one of our
subjects. Each matrix respresents the data of a complete localization experiment. The thin white
matrix corresponds to the average locations of the targets. The blue and cyan dots are individual responses
for downward and upward directed target locations, respectively. The filled, interconnected yellow
dots are the average responses for targets in the same spatial region. Perfect localization requires
the yellow dots to overly the target matrix.
Upper left: localization on day 0, prior to insertion of the binaural molds, is good. Although the listener makes some systematic errors, the structure of the target distribution is well captured by the responses.
Clockwise: Localization results on successive days, starting at day 1, up to day 24. Azimuth responses were always accurate, but elevation responses were severely impaired. However, the participant gradually relearns to localize also stimulus elevation.
|Dynamic ensemble coding model of the Superior Colliculus|
|(A) Ensemble-coding model of the saccadic system, driven by 136 actual cell recordings. (B-D) Green traces are the predicted, reconstructed saccades of the model; blue traces are the measured responses. Note that the site of activation (red activity profiles in (B)) stays at a fixed location in the SC motor map. Model saccades are straight (C) and have normal velocity profiles (D); the model predicts the nonlinear main sequence quite well (E).|
(B,C) Results of a simulation with the model before (blue) and after (green) a small
localized lesion in the SC (represented by the hole in (A)). Note that the saccade vectors
are directed away from the lesion ((B); at the circle). Thus, saccades directed to a
location corresponding with the center of the lesion have a normal amplitude and direction; saccades
to a target closer than the lesion center are too small, but saccades to targets beyond the lesion
center are too large. Also the saccade directions point away from the lesion.|
(C) All saccades in and near the lesioned area also have substantially lower peak velocities.
These results correspond nicely to the experimental data from Lee, Rohrer and Sparks, published in Nature in 1988. Yet, our model does not employ a vector averaging scheme, but linear vector addition of individual cell contributions, in combination with a fixed number-of-spikes criterion (represented by the 'gate' in (A), which closes the omnipausse switch as soon as the total number of spikes exceeds the threshold).
|Model of auditory-evoked orienting|
(a) Scheme underlying the auditory-evoked orienting response of eyes and head. A broad-band sound (flat spectrum, left) is filtered by the two ears (through the HRTFs, left (L) and right (R)), whereafter the tonotopic neural stages in the brainstem project to monaural and binaural cells in the left and right Inferior Colliculus (IC). Cells respond monotonically to level (absolute level for monaural cells, level difference (ILD) for binaural cells) within their frequency tuning, and are modulated by eye position. The IC transmits its output to the deep layers of the Superior Colliculus (SC), in which a gaussian population of recruited cells encodes the gaze motor error signal. Inset left: vectorial scheme showing the required transformation of head-centered acoustic input (H) into eye-centered gaze motor error (M) by incorporating eye position (E).
|(b) Properties of an IC cell from the model. Best frequency
is 7 kHz. The cell responds to sound level (abcissa) in a monotonic way, and on top of that, on
changes in eye position (cf. left vs. right, different eye positions, in deg).
(c) The model consists of a neural network implementation of the scheme in (a). The IC contains 72 cells with randomly selected tuning parameters, like in (b). Output of the model is the localized Gaussian activity pattern in the motor SC. Training was done with the Widrow-Hoff delta rule. The figure shows a simulation of the model's output according to the inset in (a). Error is about 27 spikes (absolute error across the SC map, 10x10 matrix); correlation between model output and required output is 0.96, which is a typical result.
|Influence of sound duration and ~intensity on human sound localization|
|Influence of sound duration and
sound level on the stimulus-response relation (here quantified by the slope ('gain') of the
linear regression lines) of sound-localization responses for subject FF. Stimuli were broad-band
noise, either from one of 16 different durations/levels, all presented randomly interleaved
in one experimental session (about 950 trials).
Azimuth response components of the head movements (open circles) are very robust, as regression lines are stable for nearly all stimulus conditions (only at the weakest and shortest stimulus, there is more variability in the responses).
Elevation responses (filled triangles) are systematically affected by both sound duration (rows) and sound level (columns). For all sound levels, the elevation gain increases with sound duration. For the shortest durations, however, the slope is also clearly affected by sound level. Thus, both acoustic factors contribute to the elevation response.
|Saccades in a noisy multisensory environment
(A) Saccade scan patterns to a visual-only (V), auditory-only (A) and audiovisual (AV) target, which was hidden in a complex audio-visual scene (open dots: green LEDs; peripheral stars: background speakers). The V- and A-scans consist of multiple saccades before finally landing on the target, while the AV response reaches the target with one saccade. The auditory target was a broad-band 'buzzer', with an intensity 21 dB below the 60 dB background noise.
(B) Cumulative distributions of the percentage of responses that reached the target, as function of time after stimulus onset (data from subject BC). Clearly, the AV saccades in which the stimuli were presented simultaneously (AV), or the visual stimulus led auditory by 100 ms (V100A), were more successful than responses to unimodal stimuli, or to the condition in which the visual stimulus lagged the auditory by 100 ms (A100V) (from: VO&M-2004)
|Influence of static head position on pure-tone localization.
Eye position responses relative to the head are shown as function of head orientation relative to space (or body) for four different sound frequencies (1.0-7.5 kHz). Measured slopes are all negative, but most differ from -1.0 (which would mean full compensation for head orientation, i.e. target would be encoded in world coordinates). Note clear dependence of the slope on sound frequency: localization of the 2000 Hz tone is nearly independent of head orientation, suggesting that it remains in a head-centered code; the 1000 Hz tone responses follow a slope very close to -1.0. Most tones are represented neither in head-centered nor in world coordinates. Responses to Gaussian white noise follow a slope very close to -1.0 (not shown).
| Dynamic eye-head coordination|
Prediction of three different models to explain double-step behavior for rapid eye-head gaze shifts. The graphs show the predicted horizontal and vertical displacements of the eye-in-space ('gaze') for the second gaze shift in response to a sequence of two brief visual stimuli, plotted against the actually measured gaze shift.
The visual-predictive model assumes that the gaze-control system accounts for the initial retinal error of the first visual stimulus, in order to allow for a prior, preprogrammed, estimate of the upcoming gaze shift to that target. The motor-predictive model is assumed to use an actual motor command ('efference copy') of the upcoming gaze shift, whereas the dynamic feedback model takes the actual gaze shift following target presentation into account. In the dynamic double-step, these three models can be dissociated. It is clear from this figure that the predictions for the dynamic feedback model correlate best with the actually measured gaze shifts.
(A) data from paradigm in which the second target was triggered by the head movement; (B) second target was triggered by the eye movement.
|Two-axis vestibular primate chair
Chair for making whole-body rotations around two independent (hrizontal and vertical) axes. The axes are controlled by Harmonic Drive systems.