Understanding Machine Vision through Human Vision

Abstract

Recent success in machine vision has been largely driven by advanced computer vision methods, most commonly known as deep learning based methods. While we have seen tremendous performance improvements in machine visual tasks, such as object categorization and segmentation, there remain two major issues in deep learning. Firstly, deep networks have been largely unable to adapt to novel yet similar datasets unseen in training time. More specifically, deep neural networks lack robustness. Secondly, there is still a lack of clear understanding at the inner mechanisms that drive deep learning. More specifically, deep neural networks lack interpretability. However, notice that both issues are not found in biological visual systems. Noisy, spotty images do not confuse our perception of image contents; we can explain our categorization of ‘dog’ via attribute descriptions. Despite the success of biological vision, current training in computer vision has been largely driven by computational programs with limited consideration of biological systems. In this thesis, we propose to address both robustness and interpretability through the incorporation of understandings in biological visual systems. To do so, we tackle one of the reasons for few incorporations of biological systems in computational advances - the lack of a publicly available, suitable large, and diverse neural dataset. We collect a novel, large-scale functional magnetic resonance imaging (fMRI) dataset in order to gather sufficient data on biological image perception. We show that our dataset contains brain activations that positively correlate to presented visual images.