PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning Universal Humanoid Control

GHC 4405

Abstract: Since infancy, humans acquire motor skills, behavioral priors, and objectives by learning from their caregivers. Similarly, as we create humanoids in our own image, we aspire for them to learn from us and develop universal physical and cognitive capabilities that are comparable to, or even surpass, our own. In this thesis, we explore how [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Generative Robotics: Self-Supervised Learning for Human-Robot Collaborative Creation

NSH 4305

Abstract: While Generative AI has shown breakthroughs in recent years in generating new digital contents such as images or 3D models from high-level goal inputs like text, Robotics technologies have not, instead focusing on low-level goal inputs. We propose Generative Robotics, as a new field of robotics which combines the high-level goal input abilities of [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

3D Video Models through Point Tracking, Reconstructing and Forecasting

NSH 3305

Abstract: 3D scene understanding from 2D video is essential for enabling advanced applications such as autonomous driving, robotics, virtual reality, and augmented reality. These fields rely on accurate 3D spatial awareness and dynamic interaction modeling to navigate complex environments, manipulate objects, and provide immersive experiences. Unlike 2D, 3D training data is much less abundant, which [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Towards a Robot Generalist through In-Context Learning and Abstractions

NSH 1305

Abstract: The goal of this thesis is to discover AI processes that enhance cross-domain and cross-task generalization in intelligent robot agents. Unlike the dominant approach in contemporary robot learning, which pursues generalization primarily through scaling laws (increasing data and model size), we focus on identifying the best abstractions and representations in both perception and policy [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Vision-based Human Motion Modeling and Analysis

NSH 4305

Abstract: Modern computer vision has achieved remarkable success in tasks such as detecting, segmenting, and estimating the pose of humans in images and videos, reaching or even surpassing human-level performance. However, they still face significant challenges in predicting and analyzing future human motion. This thesis explores how vision-based solutions can enhance the fidelity and accuracy [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Physical Process-Informed Mapping for Robotic Exploration

NSH 4305

Abstract: Mobile robots used for information gathering tasks rely on dense, predictive mapping of large-scale regions to determine where to take measurements. Current approaches to mapping commonly rely on Gaussian process regression to spatially correlate data, extrapolate from sparse samples, and estimate uncertainty. However, these approaches do not incorporate meaningful information about physical processes that [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Learning to create 3D content

NSH 4305

Abstract: With the popularity of Virtual Reality (VR), Augmented Reality (AR), and other 3D applications, developing methods that let everyday users capture and create their own 3D content has become increasingly essential. Current 3D creation pipelines often require either tedious manual effort or specialized setups with densely captured views. Additionally, many resulting 3D models are [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Sensorimotor-Aligned Design for Pareto-Efficient Haptic Immersion in Extended Reality

GHC 4405

Abstract: A new category of computing devices is emerging: augmented and virtual reality headsets, collectively referred to as extended reality (XR). These devices can alter, augment, or even replace our reality. While these headsets have made impressive strides in audio-visual immersion over the past half-century, XR interactions remain almost completely absent of appropriately expressive tactile [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Evaluating and Improving Vision-Language Models Beyond Scaling Laws

GHC 6501

Abstract: In this talk, we present our work on advancing Vision-Language Models (VLMs) beyond scaling laws through improved evaluation and (post-)training strategies. Our contributions include VQAScore, a state-of-the-art alignment metric for text-to-visual generation. We show how VQAScore improves visual generation under real-world user prompts in GenAI-Bench. Additionally, we explore training methods that leverage the language [...]

PhD Thesis Proposal
PhD Student
Robotics Institute,
Carnegie Mellon University

Getting Optimization layers to play well with Deep Networks: Numerical methods and Architectures

NSH 4305

Abstract: Many real-world challenges, from robotic control to resource management, can be effectively formulated as optimization problems. Recent advancements have focused on incorporating these optimization problems as layers within deep learning pipelines, enabling the explicit inclusion of auxiliary constraints or cost functions, which is crucial for applications such as enforcing physical laws, ensuring safety constraints, [...]