Abstract: Nowadays splashy applications heavily depend on meticulously annotated datasets, data-driven and learning-based methods, among which pixel labeling plays an important role yet often lacks interpretability. In this talk, I will discuss how we deal with pixels with better interpretability.

Firstly, I’ll introduce the pixel embedding framework that allows for clustering pixels into discrete groups of interest. As an application, we train such a model for instance segmentation in an end-to-end manner, unlike multi-stage non-differentiable methods (e.g., Mask RCNN). Secondly, I’ll present our Predictive Filter Flow (PFF) framework for unsupervised learning. The PFF learns to output per-pixel filters that warp one image towards another. As applications, we use the PFF output to reconstruct images (e.g., deblur) and propagate object mask for tracking, enabling analysis of how each pixel is evolving, thus gaining better interpretability.

Bio: Shu Kong is 5th-year PhD candidate at UCI, working in the Computational Vision Group with Prof. Charless Fowlkes. His research is motivated by a desire to create intelligent systems that benefit human life, primarily through visual signals and interaction between human and machines. His methodology can be summarized as “data-driven approach to vision problems through learning“, concerning understanding pixels from low to high level. In the meantime, he actively utilizes his algorithms to high-throughput microscopy analysis, spanning biology, neuronscience and phytology.

Homepage: https://www.ics.uci.edu/~skong2/