Abstract:
In the evolving landscape of computer vision, deep learning has emerged as a transformative force, enhancing a myriad of societal facets. The real-world deployment of such a deep vision model requires a reliable evaluation, particularly when the model can have different sensitivities across various semantic attributes and concepts. This forms the nucleus of Vision Model Diagnosis (VMD), which aims to provide insights into the analysis and evaluation of the trustworthiness of deep vision models (e.g., robustness, interpretability, and fairness). VMD tries to visualize the deep vision model’s sensitivity, interpret its performance across attributes, and fix potential vulnerabilities/biases. The main goal of this talk is to demonstrate VMD from a generative perspective: how generative models can help diagnose a vision model’s decision fairness and its robust behavior under various semantic conditions. The use of generative models with different paradigms, including Conditional VAE, CLIP-guided StyleGAN, and Latent Diffusion, can provide rich semantic analysis for a vision model’s sensitivities across attributes and visualize where the model fails. We hope that this talk can provide valuable insights into how a diagnostic process should be constructed to address issues of model trustworthiness and alignments.