Selected Publications

To help bridge the gap between internet vision-style problems and the goal of vision for embodied perception we instantiate a large-scale navigation task – Embodied Question Answering [1] in photo-realistic environments (Matterport 3D). We thoroughly study navigation policies that utilize 3D point clouds, RGB images, or their combination. Our analysis of these models reveals several key findings. We find that two seemingly naive navigation baselines, forward-only and random, are strong navigators and challenging to outperform, due to the specific choice of the evaluation setting presented by [1]. We find a novel loss-weighting scheme we call Inflection Weighting to be important when training recurrent models for navigation with behavior cloning and are able to out perform the baselines with this technique. We find that point clouds provide a richer signal than RGB images for learning obstacle avoidance, motivating the use (and continued study) of 3D deep learning models for embodied navigation.
CVPR, Oral,2019

We present Habitat, a new platform for the development of embodied artificial intelligence (AI). Training robots in the real world is slow, dangerous, expensive, and not easily reproducible. We aim to support a complementary approach – training embodied AI agents (virtual robots) in photorealistic 3D simulation and transferring the learned skills to reality.
We also describe the Habitat Challenge, an autonomous navigation challenge that aims to benchmark and accelerate progress in embodied AI.
arXiv, 2019

Recent Publications

. Embodied Question Answering in Photorealistic Environments with Point Cloud Perception. CVPR, Oral, 2019.

Preprint

. Habitat: A Platform for Embodied AI Research. arXiv, 2019.

Preprint Project

. Building Scale RGBD Alignment. CVPR, 2017.

Project

. (Cross-)Browser Machine Fingerprinting. NDSS, 2017.

Project

Projects