Abstract

Detecting out-of-distribution (OOD) inputs is critical for safely deploying deep neural networks in the open world. Recent distance-based contrastive learning methods demonstrated their effectiveness by learning improved feature representations in the embedding space. However, those methods might lead to an incomplete and ambiguous representation of a class, thereby resulting in the loss of intra-class semantic information. In this work, we propose a novel diversified multi-prototype contrastive learning framework, which preserves the semantic knowledge within each class’s embedding space by introducing multiple fine-grained prototypes for each class. This preserves intrinsic features within the in-distribution data, promoting discrimination against OOD samples. We also devise an activation constraints technique to mitigate the impact of extreme activation values on other dimensions and facilitate the computation of distance-based scores. Extensive experiments on several benchmarks show that our proposed method is effective and beneficial for OOD detection, outperforming previous state-of-the-art methods.

Framework

Experiment

Conclusion

In this paper, we propose a multi-prototype contrastive learning framework to further enhance the quality of embeddings. We preserve the semantic knowledge within each class’s embedding space by introducing a multiple prototype contrastive learning module. In particular, this module involves encouraging samples to maintain proximity to their respective coarse-grained prototypes, simultaneously aligning them with the corresponding fine-grained prototypes. This strategy ensures the preservation of inherent characteristics within the ID data, consequently enhancing the model’s capacity to distinguish OOD samples. Moreover, we introduce an activation constraints technique, which operates by constraining the features of training samples through the removal of exceedingly high and low activations. This procedure effectively mitigates the impact of extreme activation values and encourages the representations of knowledge contained within other activation values. Extensive experiments show our method can significantly improve the performance of OOD detection by effectively improving the quality of feature space embeddings.