ABSTRACT:

FRIDA, a reference to the vibrant painter Frida Kahlo, stands for a Framework and Robotics Initiative for Developing Arts to promote human creativity. FRIDA supports intuitive ways for people to collaboratively create artworks including natural language, images, and sounds. Because FRIDA is for real-world arts, our work is uniquely different from digital art tools such as DALL-E or Midjourney that can only produce digital contents. The technical challenges that we address in the FRIDA project are largely due to the embodiment into a robotic platform and the real interactions with a physical environment. For instance, given a user’s high-level goal described in natural language, FRIDA’s robot arm uses a brush and acrylic paint to paint on a real canvas, taking one brush stroke at a time.

Through art, FRIDA explores fundamental robotics challenges including semantic planning, reasoning about noisy action spaces, simulation to reality gaps, and deformable manipulation. With FRIDA, we aim to connect the success of generative AI back to our physical world to promote traditional, real-world art with tangible mediums. In this talk, I will first share my vision and the lessons learned on this experimental, interdisciplinary project.

In the second part, we will discuss the evaluation challenge more broadly. Evaluation is a key part of scientific research and technological development where researchers strive to define standard evaluation metrics and methodologies. As the field of Artificial Intelligence (AI) is gradually pervading those areas that have been previously considered solely human abilities, we are challenged with the need for an innovative evaluation paradigm for measuring AI with respect to human values such as creativity, social compliance, and diversity. The Turing test, introduced in 1950, was devised to evaluate an AI system whether its behavior is indistinguishable from that of a human’s. I would like to invite the audience to think about how we can assess success in developing sophisticated AI beyond the Turing test. I will share preliminary results on how humans evaluate performance in two different problem domains: aviation and visual art.

BIO:

Jean Oh is an Associate Research Professor at the Robotics Institute at Carnegie Mellon University. Her general research areas reside at the intersection of vision, language, and planning in robotics. Jean’s current research is focused on developing high-level intelligence for robots in the domains of autonomous navigation and creativity. Her research group, Bot Intelligence Group (BIG), includes students from diverse disciplines including Robotics, Language Technologies, Computer Science, Machine Learning, and Mechanical Engineering. Her team has won the Best Student Paper Award at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’22) and two Best Paper Awards in Cognitive Robotics at IEEE International Conference on Robotics and Automation (ICRA) in 2015 and 2018. Her works on AI Pilot and FRIDA robot painting have been featured in various media including Aviation Today, CNET, Science Daily, The Telegraph, and CBS Pittsburgh.  Jean received Ph.D. in Language and Information Technologies at Carnegie Mellon University, M.S. in Computer Science at Columbia University, and B.S. in Biotechnology at Yonsei University in South Korea.