Event Location: NSH 1507
Bio: Phillip Isola is a PhD student in Brain & Cognitive Sciences at MIT, under the supervision of Ted Adelson. He studies a mix of human and computer vision.

Abstract: No pixel is an island. It is connected to the rest of the visual visual by a web of similarities, associations, and other relationships. In this talk, I will describe how we can learn some of these relationships and use them to uncover visual organization at two ends of the spectrum: grouping pixels into object segments and parsing image collections into states and transformations. How can a visual system learn to group pixels into objects? We have developed an algorithm that learns to associate visual features by identifying suspicious coincidences in their cooccurrence statistics. Consider a zebra. Black-next-to-white occurs suspiciously often, hinting that these colors have a common cause. We can model this suspicion using pointwise mutual information (PMI). If the PMI between two colors is high, then the colors probably belong to the same object. Grouping visual features with high PMI reveals object segments, matching state-of-the-art performance on segmentation and boundary detection. Statistical association is just one of many possible visual relationships. What about the relationship between a ripe and unripe apple? Here the relationship is not just association or similarity, it also has semantics, namely ripening. Natural objects can be related by a wide variety of transformations like this, e.g. bending, folding, aging. I will conclude the talk by describing my ongoing work on understanding this richer class of visual relationships, and how they can be used to organize entire visual collections.