Abstract

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 → Cityscapes and SYNTHIA → Cityscapes.

Framework

Experiment

Conclusion

In this paper, we reveal that adversarial alignment based segmentation DA might be dominated by head classes and fail to capture the adaptability of different categories evenly. To address this issue, we proposed a novel framework that iteratively exploits our improved I-FGSPM to extrapolate the perturbed features towards more domain-invariant regions and defenses against them via an adversarial training procedure. The virtues of our method lie in not only the adaptability of model but that it circumvents the intervention among different categories. Extensive experiments have verified that our approach significantly outperforms the state-of-the-arts, especially for the hard tail classes.