Abstract

Self-supervised learning (especially contrastive learning) has attracted great interest due to its huge potential in learning discriminative representations in an unsupervised manner. Despite the acknowledged successes, existing contrastive learning methods suffer from very low learning efficiency, e.g., taking about ten times more training epochs than supervised learning for comparable recognition accuracy. In this paper, we reveal two contradictory phenomena in contrastive learning that we call under-clustering and over-clustering problems, which are major obstacles to learning efficiency. Under-clustering means that the model cannot efficiently learn to discover the dissimilarity between inter-class samples when the negative sample pairs for contrastive learning are insufficient to differentiate all the actual object classes. Over-clustering implies that the model cannot efficiently learn features from excessive negative sample pairs, forcing the model to over-cluster samples of the same actual classes into different clusters. To simultaneously overcome these two problems, we propose a novel self-supervised learning framework using a truncated triplet loss. Precisely, we employ a triplet loss tending to maximize the relative distance between the positive pair and negative pairs to address the under-clustering problem; and we construct the negative pair by selecting a negative sample deputy from all negative samples to avoid the over-clustering problem, guaranteed by the Bernoulli Distribution model. We extensively evaluate our framework in several large-scale benchmarks (e.g., ImageNet, SYSU30k, and COCO). The results demonstrate our model’s superiority (e.g., the learning efficiency) over the latest state-of-the-art methods by a clear margin. 

Framework

Experiment

Conclusion

Although SSL has shown promising results on ImageNet, its learning efficiency is still low. We attribute the inherited drawback of contrastive learning to under-clustering and over-clustering. To overcome these two problems, we propose a novel SSL framework using a truncated triplet loss. We employ triplet loss containing rich negative sample information to address the under-clustering problem. We trim negative samples to prevent the over-clustering problem, guaranteed by the Bernoulli Distribution model. Our method significantly improves the learning efficiency of SSL, leading to a state-of-the-art performance in several large-scale benchmarks and varieties of downstream tasks.