Relating visual and pictorial space: Binocular disparity for distance, motion parallax for direction

ABSTRACT Interacting with people and three-dimensional objects depicted on a screen is perceptually different from interacting with them in real life. This difference resides in their corresponding perceptual spaces: The former involves pictorial space, and the latter, visual space. Studies have examined the perceptual geometry of pictorial or visual space, but rarely their connection. The current study connected the two spaces using a pointing task and investigated how binocular disparity and motion parallax affect this connection. In a virtual environment, a pointing virtual character was displayed within a frame and the participants rotated him to point at targets in visual space. What binocular disparity and motion parallax specified was independently manipulated, either the two-dimensional surface or its depicted three-dimensional content. In Experiment 1, we changed the virtual character's distance to the screen and found that binocular disparity determines the distance relationship between visual and pictorial space, but also introduces a relief depth expansion of the perceived virtual character. In Experiment 2, we changed the participants' viewing angle relative to the screen and found that motion parallax determines the directional relationship between visual and pictorial space. We discuss the theoretical and practical implications of our results in the context of video-mediated telecommunication.