Event Location: GHC 4301

Abstract: Still pictures, animations and videos are used by artists to tell stories visually. Computer graphics algorithms create visual stories too, either automatically, or by assisting artists. Why is it so hard to build algorithms that can manipulate the content created by a visual artist? A primary reason is that artists introspect about where a viewer will look at and how their attention will flow across the scene, but algorithms do not have a similarly sophisticated understanding of the viewer.

Our key insight is that computer graphics algorithms should be designed to take into account how viewer attention is allocated. We first show that designing optimization terms based on viewers’ attentional priorities allows the algorithm to handle artistic license in the input data, such as geometric inconsistencies in hand-drawn shapes. We then show that measurements of viewer attention enables algorithms to infer high-level information about a scene, for example, the object of storytelling interest in every frame of a video.

All the presented algorithms retarget or augment the traditional form of a visual art. Traditional art includes artwork such as printed comics, i.e., pictures that were created before computers became mainstream, and artwork that can still be created in the way it was done before computers, for example, hand-drawn animation and live action films. Connecting traditional art with computational algorithms allows us to leverage the unique strengths on either side. We demonstrate these ideas on three applications: retargeting and augmenting hand animations, augmenting comic book pictures with two-dimensional camera moves, and retargeting widescreen films to a smaller screen.

Committee:Jessica Hodgins, Chair

Yaser Sheikh

Nancy Pollard

Adam Finkelstein, Princeton University