Immersive video is a modality that allows rendering any viewpoint to a scene (like in 3D graphics), while still being a video file format (a bit beyond 2D), hence it is often referred to as 2.5D video made for immersive virtual reality applications using real/natural content. This technology requires that the scene be captured from (half) a dozen of viewpoints, both in colour (RGB) and depth (D), under well-calibrated conditions. Our team has been active in this field for more than five years, actively contributing with some software tools to the upcoming MPEG Immersive Video (MIV) worldwide standard.
The thesis explores the potential of using low-cost RGBD cameras for 2.5D immersive video. Depth images should be of high quality, and it is not obvious that RGBD cameras reach the required quality level. Indeed, so far, we have used high-end depth estimation software to guarantee a high level of depth quality, which devices like Kinect cannot reach. Nevertheless, new depth sensing devices based on lidar and/or on-board AI-based stereo matching might represent viable alternatives, possibly at the cost of putting more of them around the scene, with additional filtering for achieving high depth coherence over the views.
The thesis student will get familiar with existing software tools, testing various RGBD devices for immersive video, and proposing tuned filtering algorithms targeting high-quality results.

Prerequisites: good C++ programming skills. Familiarity with using camera devices is a plus (many capturing sessions for data collection), as well as GPU programming basics, but neither of both are strictly required.

Keywords: depth estimation/sensing/processing

Promotor: Gauthier Lafruit

Contact: gauthier.lafruit@ulb.ac.be (or soon gauthier.lafruit@ulb.be)

Support: Yupeng Xie