MSR Thesis Defense

Title: Scaling up Camera Calibration and Amodal 3D Object Reconstruction for Smart Cities Abstract: Smart cities integrate thousands of outdoor cameras to enhance urban infrastructure, but their automated analysis potential remains untapped due to various challenges. Firstly, the lack of accurate camera calibration information, such as its intrinsics parameters and external orientation, restricts the measurement [...]

Title: Surface Ripples: Analyzing Transient Vibrations on Object's Surfaces Abstract: The subtle vibrations on an object's surface contain information about its physical properties and interaction with the environment.  Prior works imaged surface vibration to recover the object's material properties via modal analysis, which discards the transient vibrations propagating immediately after the object is disturbed. In this [...]

Title: Learned Two-Plane Perspective Prior based Image Resampling for Efficient Object Detection Abstract:    Real-time efficient perception is critical for autonomous navigation and city scale sensing. Orthogonal to architectural improvements, streaming perception approaches have exploited adaptive sampling improving real-time detection performance. In this work, we propose a learnable geometry-guided prior that incorporates rough geometry of the [...]

Title: Using Drones and Remote Sensing to Understand Forests with Limited Labeled Data Abstract: Drones and remote sensing can provide observations of forests at scale, but this raw data needs to be interpreted to further scientific understanding and inform effective management decisions. This thesis studies two problems under the realistic constraint of limited domain-specific training [...]

Title: Lights, Camera, Render: Neural Fields for Structured Lighting Abstract: 3D scene reconstruction from 2D image supervision alone is an under-constrained problem. Recent neural rendering frameworks have made great strides in learning 3D scene representations to enable novel view synthesis, but they struggle to reconstruct geometry of low-texture regions or from sparse views. The prevalence of active [...]

Title: Learning to See in the Dark and Beyond Abstract: Robotic Perception in diverse domains such as low-light scenarios remains a challenge, even upon the employment of new sensing modalities like thermal imaging and specialized night-vision sensors. This is largely due to the high difficulty in obtaining labeled data for certain tasks. In this work, [...]

Title: Self-Supervised Costmap Learning for Off-Road Vehicle Traversability Abstract: Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to build an accurate physics model, or create informative labels to learn a model in a supervised manner, for these interactions. We propose a method [...]

Title: On Generalization and Benchmarking on Physical Robots   Abstract: Robotics research has seen significant advancements; however, the field remains predominantly demo-driven, making direct comparisons between methods difficult without replicating them on individual setups. While many simulation benchmarks exist, they usually feature contrived datasets and do not accurately reflect real-world performance. In my thesis, we [...]

Title: Towards Real-time Controllable Neural Face Avatars Abstract: Neural Radiance Fields (NeRF) are compelling techniques for modeling dynamic 3D scenes from 2D image collections. These volumetric representations would be well suited for synthesizing novel facial expressions but for three problems. First, deformable NeRFs are object agnostic and model holistic movement of the scene: they can [...]

Title: Design and Integration of Semantic Mapping System for Forest Fire Mitigation Abstract: Remote sensing technologies can provide an automated approach to monitor and analyze conditions in the forest environment over a period of time for forest maintenance and wildfire mitigation efforts. In particular, unmanned aerial vehicles (UAVs) are a promising remote sensing modality since they [...]

Title: Vision Model Diagnosis: A Generative Perspective Abstract: In the evolving landscape of computer vision, deep learning has emerged as a transformative force, enhancing a myriad of societal facets. The real-world deployment of such a deep vision model requires a reliable evaluation, particularly when the model can have different sensitivities across various semantic attributes and concepts. [...]

Title: Learning Task Preferences from Real-World Data Abstract:  In order to provide personalized assistance that is capable of adapting to the needs of unique individuals, it is necessary to understand peoples’ preferences for different tasks. Robot assistance often assumes a static model of the individual, while in the real world, people have different capabilities and needs [...]

Title:  Fast Staircase Detection and Estimation with Multi-View Merging for Multi-Robot Systems Abstract: When robotic systems are deployed in the real world, they demand advanced mobility capabilities to operate in complex, three-dimensional environments designed for human use, e.g., multi-level buildings. Staircases have been an integral part of facilitating vertical movement in these three-dimensional environments. This work [...]

Title: LightSpeed: Light and Fast Neural Light Fields on Mobile Devices Abstract: Real-time novel-view image synthesis on mobile devices is prohibitive due to limited on-device computational power and storage. Using volumetric rendering methods, such as NeRF and its derivatives, on mobile devices is not suitable due to the high computational cost of volumetric rendering. On the [...]

Title: Multi-agent Multi-objective Ergodic Search Abstract:  In order to find points of interest in a given domain, many planners use a priori information to guide the search to expedite the detection of targets. We present an approach to direct multiple agents (MA) to search a given domain subject to multiple objectives (MO), each characterized by its own information [...]

Title: Stochastic Optimization for Autonomous Navigation, Leveraging Parallel Computation   Abstract: Stochastic Optimal Control (SOC) is a framework that allows disturbances and uncertainty in system models to be accounted for in its optimization framework. Despite accounting for this uncertainty, many first and second order methods for solving SOC problems are subject to local minima and are [...]

Title: Learning Legged Robot Agility: Sim-to-Real and Beyond Abstract: Legged robotics has seen significant advancements in both manipulation and locomotion. However, there remain significant gaps compared to their biological counterparts, particularly in energy efficiency, natural motion, and the capacity for agile skills. This thesis primarily focuses on two aspects: the unified control of legged manipulators [...]

Title: Multi-Robot Control using Control Barrier Functions: Theory and Application Abstract: Control Barrier Functions (CBFs) have emerged as a powerful theoretical tool for designing controllers with provable safety guarantees. This work presents a novel methodology that leverages CBFs to synthesize controllers for multi-robot coordination. Two multi-agent use cases are explored, i.e., a) Non-Cooperative Herding and [...]