This thesis aims to provide a framework for combining complementary tools that enable robots to manipulate objects in the world using diverse forms of knowledge. We consider heterogeneous types of knowledge, such as physics-based models, learned dynamics models, and model-free skills learned from human demonstrations. Each form of knowledge comes with its own assumptions about the world and the resources required. We approach the problem of using knowledge for decision-making primarily through a model-based planning lens, where robots use predictive models to select actions that achieve desired outcomes.

While predictive models are widely used for task and motion planning, there are often situations where their predictions deviate from real-world behavior, potentially causing task failure or even damage. Despite the limitations of all models, this thesis argues that models remain useful, even when highly inaccurate, if their limitations are properly accounted for. We consider various forms of model inaccuracy, including structural limitations of what the model can represent. Within this framework, we seek to expand robot capabilities by learning how models differ from reality using real-world data. Specifically, we learn and use regions of state-action space, called \textit{model preconditions} where the model reliably represents real-world dynamics. Quantifying model preconditions enables efficient capability expansion when acquiring knowledge to solve new tasks. Since knowledge acquisition through new data, such as human demonstrations or trying potentially unsafe actions in the real world can be costly, efficiency is crucial.

By defining model preconditions using predictive accuracy, we show that with very little data, task reliability can be significantly improved. By learning a model deviation estimator (MDE), we can constrain planning to regions where the model is accurate, thereby increasing the reliability of plans. Although the MDE formulation of a model precondition can be learned from pre-existing data, we present a method for actively learning a model precondition.  The thesis then introduces algorithms that use model preconditions to expand the robot’s capabilities. One method enables planning with multiple inaccurate models of varying forms and computational requirements, leading to more efficient planning and reliable execution while preserving theoretical guarantees of optimality. We also address situations where available models are insufficient for solving the problem. First, we demonstrate a method for integrating model-free skills into model-based planning with low data requirements.  Second, we propose a dynamics adaptation algorithm that efficiently selects and updates data for learned dynamics models, expanding capabilities to more complex dynamics.