School of Computer Science Part of Two New NSF AI Institutes

CMU Contributing to Innovations in Caregiving, Agriculture

Carnegie Mellon University’s School of Computer Science will contribute fundamental and cutting-edge research to a government-led push to bring about life-changing advances through artificial intelligence.

The U.S. National Science Foundation today announced a $220 million investment in 11 new Artificial Intelligence Research Institutes. School of Computer Science (SCS) researchers will participate in two of the new institutes.

Reid Simmons, a research professor in the Robotics Institute (RI) and the Computer Science Department (CSD), will lead CMU’s involvement in the AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups (AI-CARING), which seeks to develop AI systems for caregiving environments. George Kantor, a research professor in RI, will lead the university’s work in the AI Institute for Resilient Agriculture (AIIRA), which is focused on AI and robotics in agriculture.

Each institute will receive $20 million over five years. More than a third of that funding, about $7 to $8 million from each institute, will support research efforts at CMU that will involve dozens of SCS faculty, students and staff. Researchers from the Dietrich College of Humanities and Social Sciences and the Tepper School of Business will also be involved.

CMU’s College of Engineering is part of a third institute, the AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE) [LINK TO CIT PRESS RELEASE]. The institute will focus on designing wireless edge networks that are highly efficient, reliable, robust and secure. The CMU team includes Gauri Joshi, an assistant professor in the Electrical and Computer Engineering Department, as head of the university’s efforts; and Ameet Talwalkar, an assistant professor in the Machine Learning Department. Additionally, in 2020 the NSF named CMU a planning institute for developing a National Artificial Intelligence Research Institute in Physics and awarded the university a $500,000 grant.

The wide reach of these institutes illustrates the role AI will play in the next wave of transformational innovation and as a tool for improving our daily lives. CMU’s inclusion in these federal efforts to expand the use of AI underscores the university’s depth and breadth of expertise in the field, spanning deep learning, robotics, interaction, ethics and engineering. The university is one of the birthplaces of AI and will continue to push its boundaries through these new initiatives.

AI-CARING will be led by the Georgia Institute of Technology [LINK TO GEORGIA TECH RELEASE] with partial funding from Amazon and Google. The institute will initially focus on assisting in the care of the elderly by seeking to understand and manage the interaction between humans and AI agents. Researchers will develop methods to teach AI systems to learn a person’s needs, preferences and caregiving network, and enable the systems to adapt as those change over time. The tasks will involve not only coordinating a variety of AI agents — like voice-controlled assistants, smart devices or chatbots — but also managing human caregivers ranging from medical and assistance providers to family.

“What we’re trying to do is collaborate. We’re not replacing caregivers; we’re augmenting their capabilities,” Simmons said. “The AI has to adapt to the individual and even more so, it has to adapt to the network the individual is in. We’re intending that the AI is going to be assisting people over a long time, and while there will be plenty of time for the system to learn, there will also be plenty to learn.”

Simmons and his CMU team will bring core AI expertise to the project, as well as years of research and work in multi-agent cooperation, human-computer interaction, AI education and outreach, and ethics. Communication and education are large components of the project, as families, caregivers and those being cared for must understand how and why the AI makes its decisions. The project also emphasizes training the next generation of AI researchers and workers.

The CMU team consists of about a dozen faculty members spanning RI, CSD, the Human-Computer Interaction Institute, Dietrich and Tepper. The team includes Henny Admoni, who researches nonverbal human-robot interaction; Jodi Forlizzi and her work in human-AI interaction; Aaron Steinfeld, who works on assistive technology; Dave Touretzky and his work in teaching AI to K-12 students; Alex London, a professor of ethics and philosophy in Dietrich; and Tepper’s Anita Woolley, who studies human-AI teaming and collaboration.

The institute’s CMU ties even extend to its director, Sonia Chernova. She heads the Robot Autonomy and Interactive Learning Lab at Georgia Tech and earned her Ph.D. in computer science from CMU.

AIIRA, pronounced “eye-rah,” will be based at Iowa State University [LINK TO IOWA STATE PRESS RELEASE] and is funded by the U.S. Department of Agriculture’s National Institute of Food and Agriculture. The institute aims to use AI to develop digital twins to model plants at an unprecedented scale and with unparalleled precision. The digital modeling will allow for robust simulation and prediction. With the digital twins, for example, scientists will be able to simulate and predict how a crop will perform in climate scenarios that do not yet exist.

CMU brings its strength in AI, deep learning and robotics to the project. To build the digital twins, scientists will need large amounts of data about plants, soil and other aspects of the environment. The CMU team will design robot systems to gather this data and provide the AI and deep learning tools necessary to model it.

“Imagine a robotic manipulator pushing on a branch. We can record how much force was used to make the branch move and how much the branch moved. Fusing that with other data will help us create a predictive model,” Kantor said. “This is going to be a big opportunity to bring intelligent manipulation into the field.”

Seven faculty members from RI and the Machine Learning Department make up CMU’s contribution to the institute. The challenge for the CMU team is, in part, designing the never-before-used robotic tools that will be required in the field. A robotic manipulator that will interact with crops must be gentle to avoid damaging the plants, so CMU’s Nancy Pollard is designing soft manipulators for the project. She will use AI and video capture techniques from her Graphics Lab to build a hand optimized for the task, whether grabbing a leaf or pulling on a stem. Other CMU team members include Oliver Kroemer, who is designing techniques that will allow a robot to insert sensors into a plant’s stem and the ground; and Katia Sycara, who studies multi-agent systems and human-robot interaction. Other members of the team will tackle novel imaging techniques, machine learning for decision support and field robotics.

While the initial work will focus on using robotics and AI to gather and process data, the tools and techniques developed for those tasks could translate to other work on the farm, such as pollinating flowers, which is done by hand in some instances.

“There are people out there in the field with microscope goggles and tweezers peeling the pollen off one flower and transferring it to another. That takes a person a very long time,” Kantor said. “To me, the most exciting part is moving into this sort of manipulation.”