NAVIGATION AND ILLUMINATION CONTROL FOR IMAGE-BASED VR
Résumé
Simulation sickness in virtual reality applications is usually due to the non-realtime display of
virtual environment. Realtime display of arbitrarily complex scene is a hard problem in traditional
geometry-based computer graphics. Image-based modeling and rendering (IBMR) provides an
alternative approach whose rendering time complexity is independent of scene complexity.
However, due to the lack of geometrical information, capabilities that are obvious to geometry-based
virtual reality become difficult problems in image-based virtual reality. In this paper we discuss how
two fundamental capabilities, navigation and illumination control, can be achieved in image-based
virtual reality applications.
To navigate an image-based scene, we need to know where pixels should be moved to and how
to solve their visibility. While correspondences or optical flows can answer the former question, the
visibility is more difficult to answer, as depth information may not be available. Deriving from
epipolar geometry, we propose a triangle-based visibility-ordering algorithm, which can correctly
resolve the occlusion without depth information. To control the illumination, we propose a new
image representation that not just allows navigation but also re-rendering under various
illuminations. By treating each image pixel as an ordinary surface element, we measure the apparent
BRDF of each pixel from reference images. By manipulating these apparent pixel BRDFs, we are
able to re-render (change the illumination of the scene in an image) without any geometry
information. Even shadows can be correctly re-rendered.