Semi-supervised Hopfield model: Theoretical and Numerical results

In the daily practice of Machine Learning, fully labeled datasets are a luxury: labels demand expensive and time-consuming human annotation, whereas raw, unlabeled data can be harvested automatically and in bulk. Semi-supervised learning, where the network jointly exploits the few labeled and the many unlabeled examples at its disposal, is the standard answer to this asymmetry, yet a statistical mechanical theory of semi-supervised Hebbian learning is still lacking. In this paper we fill this gap for the Hopfield network: we prescribe a synaptic coupling given by the convex combination, weighted by a mixing parameter λ∈[0,1], of the supervised and unsupervised Hebbian kernels built from the same archetypes, and we solve for the emergent computational capabilities of the resulting network. A signal-to-noise analysis yields the one-step Mattis magnetization and the learning threshold, i.e., the minimum dataset size for stable retrieval, the latter being obtained in closed-form for a balanced dataset. Using Guerra's interpolation, we then derive the replica-symmetric quenched pressure in the high-storage regime, treating the correlated disorder generated by the supervised and unsupervised channels through a particular eigen-channel decomposition. The resulting phase diagram shows that a mixed strategy outperforms both pure protocols. Finally, we prove that the quenched pressure is convex in λ, so thermodynamics cannot select an interior mixture: λ is therefore a learning hyperparameter. All the analytical findings are successfully checked against extensive Monte Carlo simulations.