Jenkin, H. L., Dyde, R. T., Jenkin, M. R., Howard, I. P. and Harris, L. R. Relative role of visual and non-visual cues in judging the direction of 'Up': Experiments in the York tilted room facility, J. Vestib. Res., 13: 287-293, 2003.
Perceiving a direction as "up" is fundamental to human performance and perception. Astronauts in microgravity frequently experience reorientation illusions in which they, or their world, appear to flip and "up" becomes arbitrarily redefined. This paper assesses the relative importance of visual cues in determining the perceived up direction. In the absence of information about the origin of illumination, people interpret surface structure by assuming that the direction of illumination is from above. Here we exploit this phenomenon to measure the influence of head and body orientation, gravity and visual cues on the perceived up direction. Fifteen subjects judged the shape of shaded circles presented in various orientations. The circles were shaded in such a way that when the shading was compatible with light coming from above, the circle appeared as a convex hemisphere. Therefore, by finding which shaded circle appeared most convex, we can deduce the direction regarded as "up". The different cues contributing to this percept were separated by varying both the orientation of the subject and the surrounding room relative to gravity. The relative significance of each cue may be of use in spacecraft interior design to help reduce the incidence of visual reorientation illusions.
Jaekl, P., Jenkin, M. and Harris, L. R., Perceptual stability during active head movements orthogonal to gravity, J. Vestib. Res., 13: 265-271, 2003.
We measured how much the visual world could be moved during various head rotations and translations and still be perceived as visually stable. Using this as a monitor of how well subjects know about their own movement, we compared performance in different directions relative to gravity. For head rotations, we compared the range of visual motion judged compatible with a stable environment while rotating around an axis orthogonal to gravity (where rotation created a rotating gravity vector across the otolith macula), with judgements made when rotation was around an earth-vertical axis. For translations, we compared the corresponding range of visual motion when translation was parallel to gravity (when imposed accelerations added to or subtracted from gravity), with translations orthogonal to gravity. Ten subjects wore a head-mounted display and made active head movements at 0.5 Hz that were monitored by a low-latency mechanical tracker. Subjects adjusted the ratio between head and image motion until the display appeared perceptually stable. For neither rotation nor translation were there any differences in judgements of perceptual stability that depended on the direction of the movement with respect to the direction of gravity.
Jenkin, H. L., Dyde, R. T., Jenkin, M. R., and Harris, L. R., In Virtual Reality, which way is up? Proc. ICAT, Tokyo, Japan, 2003.
Virtual reality is often used to simulate environments in which the direction of up is not aligned with the normal direction of gravity or the body. How effective are these environments in terms of generating a compelling illusion of different up directions? Here we examine this question by asking: "In virtual reality, which way is up". Using an immersive projective display, subjects sat in a virtual room that could be rolled about the line of sight. Subjects indicted their perceived direction of up by adjusting the orientation of a shaded disk until it appeared maximally convex. This orientation depends upon the perceived direction of the illumination which thus indirectly indicates the perceived direction of up. Their judgements indicate that for physically upright subjects the visual display is an important factor in the perceived up direction. However this technique is limited to roll rotations away from the gravity direction in the range +/- 35 degrees.
Kapralos, B., Jenkin, M. R. M., Milios, E., Auditory perception and spatial (3D) auditory systems, York University Technical Report CS-2003-07, Department of Computer Science, 2003.
In order to enable the user of a virtual reality system to be fully immersed in the virtual environment, the user must be presented with believable sensory input. Although the majority of virtual environments place the emphasis on visual cues, replicating the complex interactions of sound within an environment will benefit the level of immersion and hence the user's sense of presence. Three dimensional (spatial) sound systems allow a listener to perceive the position of sound sources, and the effect of the interaction of sound sources with the acoustic structure of the environment. This paper reviews the relevant biological and technical literature relevant to the generation of accurate acoustic displays for virtual environments, beginning with an introduction to the process of auditory perception in humans. This paper then critically examines common methods and techniques that have been used in the past as well as methods and techniques which are currently being used to generate spatial sound. In the process of doing so, the limitations, drawbacks, advantages and disadvantages associated with these techniques are also presented.
Ripsman, A., Jenkin, M. R. M., and Jasiobedzki, P., Specular surface recovery via coded target stereopsis. York University Technical Report CS-2003-03, Department of Computer Science, 2003.Abstract A key challenge facing space-based computer vision systems is the presence of highly specular surfaces. Such surfaces present a challenge to convention vision systems due to specular reflections, which may mask the true location of objects and lead to incorrect measurements. The problem is complicated by the fact that the use of high-powered illuminants, such as laser beams, can be problematic since the instruments inside space structures are often sensitive to various forms of radiation.This article presents a new approach to reconstructing the local surface structure of highly specular objects. The system utilizes a commercial trinocular stereo vision system and a coded low-power two-dimensional illuminant to recover surface structure. The local surface structure of an object is obtained by projecting light patterns onto the object's surface and inferring surface structure by examining the interaction of the illuminant and the surface. The system recovers both specuar and diffuse surface regions. Experimental results demonstrate the effectiveness of the approach.
Kapralos, B., Barth, A., Ma, J., and Jenkin, M. A system for synchronous distance learning, Prov. VI 2003, Halifax, NS, 2003.Abstract Providing an effective mechanism for an instructor to interact with one or more remote classrooms is a complex problem in synchronous distance learning. Although a human operator at the remote classroom could be used to direct a video camera and hence provide feedback to the instructor as to the state of the class and to coordinate the instructor's attention to the class as questions are raised, more automated techniques are desirable. Here we describe a novel audiovisual sensor to support distance learning. This sensor provides the instructor with a panoramic view of the class and can automatically attend to students who signal their intent to interact with the instructor either by raising their hand or by asking a question vocally. This paper describes the sensor and the user interface that is provided to the instructor so (s)he can interact with the students in the remote class.
Hogue, A., Robinson, M., Jenkin, M. R., and Allison, R. S., A vision-based tracking system for fully immersive displays, Proc. International Immersive Projection Technologies Workshop/Eurographics Workshop on Virtual Environments 2003, Zurich, 2003.Abstract Six-sided fully immersive projective displays present complex and novel problems for tracking systems. Existing tracking technologies typically require tracking equipment that is placed in locations or attached to the user in a way that is suitable for typical displays of five or less walls but which would interfere with the immersive experience within a fully enclosed display. This paper presents a novel vision-based tracking technology for fully-immersive projective displays. The technology relies on the operator wearing a set of laser diodes arranged in a specific configuration and then visually tracking the projection of these lasers on the external walls of the display outside of the user's view. This approach places minimal hardware on the user and no visible tracking equipment is placed within the immersive environment. This paper describes the basic visual tracking system including the hardware and software infrastructure.
Jaekl, P. M., Jenkin, M. R., Dyde, R. T. and Harris, L. R., Perceptual stability during active and passive head translation: variations with direction. [Abstract]. J. of Vision, 3: 492a, 2003. http://journalofvision.org/3/9/492.Abstract The world in which we live normally appears perceptually stable as we move around during both active movements and when moving passively, as for example when traveling as a passenger. To achieve such stability, the visual motion experienced has to be compatible with that expected to be associated with the self motion. This in turn requires knowledge about the self motion. During active head movement, information is derived from an efference copy of motor commands as well as vestibular and other proprioceptive sources. Efference copy cues are not available during passive motion. Are there differences between the perception of visual stability under active and passive motion? In previous studies we measured the amount of visual motion needed for the world to appear stable during active head motion and found that ~1.5 times more visual movement was required than was geometrically necessary. Naso-occipital motion was closer to veridical than other directions (Jaekl et al., 2002, J. Vis. 2(7), 508a). The present study measured the visual motion needed to appear stable during passive motion. Subjects were translated passively at 0.5 Hz while sitting on a cart manually pulled against a powerful spring. Subject motion was naso-occipital, inter-aural or in between. Subjects wore a head-mounted display and were positioned in the centre of a virtual spherical world, radius 1m, which was updated in response to head movement monitored by a mechanical tracker. Subjects varied the ratio between head and image motion until the display appeared perceptually stable. During passive translation, even more visual motion was required to ensure the appearance of moving within an earth stable environment than when motion was active, often needing as much as twice the geometrically required value. Passive naso-occipital motion was again closer to requiring the geometrically correct amount. The experiments help establish the role of efference copy in the perception of self motion.
Jenkin, H. L., Dyde, R. T., Zacher, J. E., Jenkin, M. R. and Harris, L. R., Multi-sensory contributions to the perception of up: Evidence from illumination judgements. J. of Vision, 3: 638, 2003. http://journalofvision.org/3/9/638.Abstract There are many definitions of 'up'. Body orientation, gravity and vision each provide their own estimate but they are normally combined into a single percept. Often the cues coincide, as when standing in a well-lit environment. But what happens when they disagree? Does one dominate? Or do they all contribute to an average? We examined the contribution of body orientation, gravity and visual cues on 'up' perception when these cues were not in agreement. The perception of 3D shape from 2D shading served as an indirect measure of the perception of 'up', as light is normally assumed to come from above in the absence of illumination cues. Observers were (i) sitting upright in an upright room, (ii) lying on their side in an upright room, (iii) sitting upright in a room tilted 90, or (iv) lying in a tilted room. Stimuli were shown on a grey laptop screen arranged with the keyboard in the normal configuration relative to the body and that was surrounded by a clearly visible room. Each stimulus was a 2D disc shaded from black to white. Each trial started with the disc's shading axis randomly aligned. Observers rotated the disc until it appeared 'most convex'. The pattern of responses indicated that the perceived direction of 'up' is influenced by the direction of gravity, the orientation of the body and the orientation of the visual frame. The judgements were modelled as a weighted sum of vectors corresponding to the orientations of the body, gravity and the surrounding visual polarity. These data illustrate how the brain can resolve a common dilemma: how to deal with many sources providing normally redundant information about a single parameter. Knowing the relative weighting of these factors may be helpful in predicting performance on other related tasks, such as balancing, orienting or navigating in normal or unusual environments.
Kapralos, B., Jenkin, M. R. M., and Milios, E., Audio-visual localization of multiple speakers in a video teleconferencing setting, Int. J. Imaging Systems and Technology, 13:95-105, 2003.Abstract Attending to multiple speakers in a video teleconferencing setting is a complex task. From a visual point of view, multiple speakers can occur at different locations and present radically different appearances. From an audio point of view, multiple speakers may be speaking at the same time, and background noise may make it difficult to localize sound sources without some a priori estimate of the sound source locations. This paper presents a novel sensor and corresponding sensing algorithms to address the task of attending, simultaneously, to multiple speakers for video teleconferencing. A panoramic visual sensor is used to capture a 360 o view of the speakers in the environment and from this view potential speakers are identified via a color histogram approach. A directional audio system based on beamforming is then used to confirm potential speakers and attend to them. Experimental evaluation of the sensor and its algorithms are presented including sample performance of the entire system in a teleconferencing setting.
