Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing

Abstract

Existing GAN inversion and editing methods are well suited for only a target
images that contain aligned objects with a clean background, such as portraits and animal faces, but often struggle for more difficult categories with
complex scene layouts and object occlusions, such as cars, animals, and outdoor images. We propose a new method to invert and edit such complex
images in the latent space of GANs, such as StyleGAN2. Our key idea is to
explore inversion with a collection of layers, spatially adapting the inversion
process to the difficulty of the image. We learn to predict the “invertibility”
of different image segments and project each segment into a latent layer.
Easier regions can be inverted into an earlier layer in the generator’s latent
space, while more challenging regions can be inverted into a later feature
space. Experiments show that our method obtains better inversion results
compared to the recent approaches on complex categories, while maintaining downstream editability.