Title: A Computational Framework for Norm-Aware Reasoning in Autonomous Systems   Abstract: Autonomous agents are increasingly deployed in complex social environments where they not only have to reason about their domain goals but also about norms that can impose constraints on task performance. Integrating task planning with norm aware reasoning is a challenging problem due to [...]

Title: Learning Transferable Cooperative Behavior in Multi-Agent Teams   Abstract: We study the emergence of cooperative behavior and communication protocols in multi-agent teams, for collaboratively accomplishing tasks like coverage control and formation control for swarms. Using graph neural networks to model inter-agent communications, we present state-of-the-art results in a fully decentralized execution framework which assumes [...]

Title: Decentralized Multiple Mobile Depots Route Planning for Replenishing Persistent Surveillance Robots   Abstract: Persistent surveillance of a target space using multiple robots has numerous applications. The continuous operation in these applications is challenged by the limited onboard battery capacity of the persistent robots. We consider the problem for replenishing persistent robots using mobile depots, [...]

Title: Generating 3D Human Animations from Single Monocular Images   Abstract: Endowing AI systems with the ability to formulate a three-dimensional understanding of human appearance from a single RGB image is an important component technology for applications such as person re-identification, biometrics, virtual reality and augmented reality. However, jointly inferring the texture map and 3D [...]

Title: Model-free Sensorless Manipulation   Abstract: This thesis is a study of 2D manipulation without sensing and planning, by exploring the effects of unplanned randomized action sequences on 2D object pose uncertainty. Our approach uses sensorless reorienting of an object to achieve a determined pose, regardless of the initial pose. Without using sensors and models [...]

Title: OpenPose: Whole-Body Pose Estimation   Abstract: We present the first single-network approach for 2D whole-body (body, face, hand, and foot) pose estimation, capable of detecting an arbitrary number of people from in-the-wild images. Our method maintains constant real-time performance regardless of the number of people in the image. This network is trained in a [...]

Title: Monocular Total Capture: Pose Face, Body, and Hands in the Wild   Abstract: We present the first method to capture the 3D total motion of a target person from a monocular view input. Given an image or a monocular video, our method reconstructs the motion from body, face, and fingers represented by a 3D deformable [...]

Title: 3D Shape Completion and Canonical Pose Estimation with Structured Neural Networks   Abstract: 3D point cloud is an efficient and flexible representation of 3D structures and the raw output of many 3D sensors. Recently, neural networks operating on point clouds have shown superior performance on various 3D understanding tasks, thanks to their power to [...]

Title: Leveraging Simulation for Computer Vision   Abstract: A large amount of labeled data is required to train deep neural networks. The process of data annotation on such a large scale is expensive and time-consuming. A promising alternative in this regard is to use simulation to generate labeled synthetic data. However, a network trained solely [...]

Title: Visual Imitation Learning for Robot Manipulation   Abstract:   Imitation learning has been successfully applied to solve a variety of tasks in complex domains where an explicit reward function is not available. However, most imitation learning methods require access to the robot's actions during demonstration. This stands in a stark contrast to how we [...]

Title: Highly Miniaturized Robots for Inspection of Small Nuclear Piping   Abstract: Bomb making in the 20th century resulted in the creation of massive facilities to produce Uranium. As part of a multi-billion-dollar agenda, the measurement of radioactivity is required for the safe disposal of residual Uranium in piping. Manual techniques have proven too approximate, [...]

Title: Topological Path Planning for Mobile Robot Applications   Abstract: Many path planning problems in mobile robot applications can be solved more efficiently in the topological space. By using the language of topology, the richer spatial information failed to captured by graph/grid-based map representations can be explicitly expressed and exploited. With that, it is possible [...]

Title: Deep Reinforcement Learning with Prior Knowledge   Abstract: Deep reinforcement learning has been applied to many domains from computer games, natural language processing, recommendation systems to robotics.  While model-free reinforcement learning algorithms are promising approaches to learning policies without knowledge of the system dynamics, they usually require much more data. In this thesis, we [...]

Title: 3D Object Detection from CT Scans using a Slice-and-fuse Approach   Abstract: Automatic object detection in 3D X-ray Computed Tomography imagery has recently gained research attention due to its promising applications in aviation baggage screening. The huge dimension of an individual 3D scan, however, poses formidable computational challenges when coupled with deep 3D convolutional [...]

Title: Interpretable Intuitive Physics Model   Abstract: Humans have a remarkable ability to use physical commonsense and predict the effect of collisions. But do they understand the underlying factors? Can they predict if the underlying factors have changed? Interestingly, in most cases humans can predict the effects of similar collisions with different conditions such as changes [...]

Title: Learning for Registration in 2D and 3D   Abstract: We explore the application of deep learning to 2D (image) and 3D (point cloud) registration, especially in scenarios where traditional methods can fail.   In the 2D case, we apply a recently-proposed learning method to the problem of aligning outdoor imagery taken across different seasons or [...]

Title: Inverse Reinforcement Learning for Autonomous Ground Navigation Using Aerial and Satellite Observation Data   Abstract: Inverse Reinforcement Learning(IRL) is a supervised learning paradigm where a learner observes expert demonstrations to learn the hidden cost function to mimic the expert's behavior. Eliminating the need for elaborate feature engineering, deep IRL approaches have been gaining interests [...]

Title: Detecting objects in videos under occlusion   Abstract: While object detection in videos has received a lot of attention in the past few years, most existing methods in this domain do not target detecting objects when they are occluded. However, being able to detect or track an object of interest through occlusion has been [...]

Title: Detailed Image Captioning with Hierarchical Attention   Abstract:  Automatic image description is the task of generating a natural sentence which reflects the visual content of an image. A lot of deep learning architectures have been explored in the past few years. While researchers have made great improvement on generating syntactically correct sentences by learning from [...]

Title:  Inching for Planetary Rovers   Abstract: . Inching, also called push-rolling, is method of moving for vehicles that can change the position of their wheels relative to their body.  Like an inchworm, it is possible to hold some wheels stationary while advancing the others and the body.  In this research a test apparatus capable [...]

Title: Improving Scalability in Multi-Robot Systems with Abstraction and Specialization   Abstract: Planning and controlling multi-robot systems is challenging due to high dimensionality. When the number of robots increases in the system, the complexity of computation grows exponentially. In this thesis, we examine the scalability problem in planning and control of the multi-robot system. We propose [...]

Title: A Generative Approach for Socially Compliant Navigation   Abstract: Robots navigating in human crowds need to optimize their paths not only for the efficiency of their tasks performance but also for the compliance to social norms. One of the key challenges in this context is due to the lack of suitable metrics for evaluating and [...]

Title: 3D Face Geometry Capture Using Monocular Video   Abstract: Accurate reconstruction of facial geometry has been one of the oldest tasks in computer vision. Despite being a long-studied problem, many modern methods fail to reconstruct realistic looking faces or rely on highly constrained environments for capture. High fidelity face reconstructions have so far been [...]

Title: Unsupervised Learning for 3D Reconstruction and Blocks World Representation Abstract: Recovering the dense 3D structure of a scene from its images has been a long-standing goal in computer vision. Recent years have seen attempts of encoding richer priors into the geometry-based pipelines with the introduction of learning based methods. We argue that the form of 3D [...]

Title: A structured model for action detection   Abstract:  A dominant paradigm for learning-based approaches in computer vision is training generic models, such as ResNet for image recognition, or I3D for video understanding, on large datasets and allowing them to discover the optimal representation for the problem at hand. While this is an obviously attractive approach, [...]

Title: Canonical Surface Mapping via Geometric Cycle Consistency   Abstract: We explore the task of Canonical Surface Mapping (CSM).  Specifically, given an image, we learn to map pixels on the object to their corresponding locations on an abstract 3D model of the category. But how do we learn such a mapping? A supervised approach would [...]

Title: Leveraging Structure for Generalization and Prediction in Visual System. Abstract: Our surrounding world is highly structured. Humans have a great capacity of understanding and leveraging those structures to generalize to novel scenarios and to predict the future. The thesis studies how computer vision systems benefit from the similar process -- leveraging inherent structures in [...]

Title: Self-Supervised Robot Learning   Abstract: Supervised learning has been used in robotics to solve various tasks like navigation, fine manipulation, etc.  While it has shown a promising result, in most cases the supervision comes from the human agent.  However, relying on human is a huge bottleneck to scale up these approaches.  In this thesis, [...]

Title: A fast algorithm for consecutive ones with applications in item labeling   Abstract: We develop and analyze a general problem in a crowd-sourced setting: users pick a label from a set of candidates for a set of items; the problem is to determine the most likely label for each item, as well as a ranking [...]

Title: Real-Time Collision Forecasting from Monocular Video   Abstract:  We explore the possibility of using a single monocular camera to forecast the time to collision between a suitcase-shaped robot being pushed by its user and other nearby pedestrians. We develop a purely image-based deep learning approach that directly estimates the time to collision without the need [...]

*LOCATION CHANGE TO NEWELL SIMON HALL ROOM 1305* Title: Computer Vision for Live Map Updates Abstract: Real-time traffic monitoring has had widespread success via crowd-sourced GPS data.  While drivers benefit from this low-level, low-latency map information, any high-level traffic data such as road closures and accidents currently have very high latency since such systems rely solely [...]

Title: Stereo Visual-Inertial-LiDAR Simultaneous Localization and Mapping   Abstract: Simultaneous Localization and Mapping (SLAM) is a fundamental task to mobile and aerial robotics. The goal of SLAM is to utilize onboard sensors for estimating the robot’s trajectory while reconstructing the surrounding environment (map) in real-time. The algorithm should also be able to perform loop closure, such [...]

Title: Robot Learning for Manipulation of Granular Materials   Abstract: Granular materials are ubiquitous in household and industrial manipulation tasks, but their dynamics are difficult to model analytically or through simulation. During manipulation, they provide rich multimodal sensory feedback. We present a robotic system we constructed for investigating manipulation of granular materials.   We present [...]

Title: Volumetric Correspondence Networks for Stereo Matching and Optical Flow Abstract: Many classic tasks in vision, such as the estimation of optical flow or stereo disparities, can be cast as dense correspondence matching. Well-known techniques for doing so make use of a cost volume, which is typically a 3D/4D tensor of match costs between all [...]

Title: Surfel-based RGB-D Reconstruction and SLAM with Global and Local Consistency   Abstract: Achieving high surface reconstruction accuracy in dense mapping has been a desirable target for both robotics and vision communities. In the robotics literature, simultaneous localization and mapping (SLAM) systems use depth-enabled cameras to reconstruct a dense map of the environment. They leverage [...]

Title: Precision UAV Landing in Unstructured Environments Abstract: The autonomous landing of a drone is an important part of autonomous flight. One way to have a high certainty of safety in landing is to return the drone to the same location it took-off from. Current implementations of the return-to-home functionality fall short when relying solely [...]

Title: Joint Surface Reconstruction from Monocular Vision and LiDAR   Abstract: In recent years, dense reconstruction gains popularity because of its broad applications in inspection, mapping, and planning. Cameras or LiDARs are generally deployed for 3D dense reconstruction. However, current reconstruction pipelines based on cameras or LiDARs have significant limitations in achieving an accurate and [...]