In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington-Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line.

About the de Almeida–Thouless line in neural networks

Linda Albanese
;
Andrea Alessandrelli;Adriano Barra
2024-01-01

Abstract

In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington-Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line.
File in questo prodotto:
File Dimensione Formato  
About the de Almeida–Thouless line in neural networks.pdf

accesso aperto

Tipologia: Versione editoriale
Licenza: Creative commons
Dimensione 1.51 MB
Formato Adobe PDF
1.51 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11587/508230
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact