Inter and Intra Image Relationships in 3D Space

Abstract

Relationships are at the core of understanding images. A lot of progress in computer vision is due to the ability of convolutional neural networks to model relationships between pixels in images across our massive data-sets. Convolutional neural networks bake in relationships between pixels both within images and across image collections to perform better on tasks like image segmentation, object detection, etc.. Relationship can exists in the pixel, concept and object space which is a huge spectrum. We show that different kinds of relationships are relevant for different tasks. This thesis explores various other ways to model "inter and intra image relationships'' to help with the tasks of semantic correspondences, and 3D reconstruction.

In this thesis we explore the following two tasks a) semantic correspondence between images from the same category of objects where we show how relationships can be inherently learned and how geometric constraints helps us learn global meaningful relationships and b) 3D reconstruction from a single image where we explicitly factor relationships in 3D to help with the task of 3D reconstruction to improve the overall performance across multiple datasets.