在《Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks》论文中解释了伪标签学习为何有效,它的有效性可以在两个方面进行考虑,原文如下:Low-Density Separation between Classes“The goal of semi-supervised learning is to improve generalization performance using unlabeled data. The cluster assumption states that the decision boundary should lie in low-density regions to improve generalization performance (Chapelle et al., 2005). Recently proposed methods of training neural networks using manifold learning such as Semi-Supervised Embedding and Manifold Tangent Classifier utilize this assumption. Semi-Supervised Embedding (Weston et al., 2008) uses embedding-based regularizer to improve the generalization performance of deep neural networks. Because neighbors of a data sample have similar activations with the sample by embedding based penalty term, it’s more likely that data samples in a high-density region have the same label. Manifold Tangent Classifier (Rifai et al., 2011b) encourages the network output to be insensitive to variations in the directions of low-dimensional manifold. So the same purpose is achieved.”Entropy Regularization“Entropy Regularization (Grandvalet et al., 2006) is a means to benefit from unlabeled data in the framework of maximum a posteriori estimation. This scheme favors low density separation between classes without any modeling of the density by minimizing the conditional entropy of class probabilities for unlabeled data.”作者考虑的两个点: