Statistical Mechanics of Learning

Participating group members:	Ralf Eichhorn, Peter Reimann
Main external cooperation partners:	Chris Van den Broeck, Géza Györgyi, Michael Biehl

Order parameter function x(q) of a model neuron beyond its storage capacity (continuous replica symmetry broken phase) for different inverse temperatures beta.

General context: neural networks, modeling of learning and data-analysis by means of methods from classical statistics and statistical mechanics, especially replica-techniques. In particular: new concepts and optimization of so-called unsupervised learning. Unified description of memorization and generalization problems. Theoretical limits for learning (Bayes limit) and their practical implementation as learning algorithms. Generalization for neural network setups of replica-symmetry-breaking methods, originally introduced by G. Parisi in the context of spin glass theory.

Main publications:

P. Reimann and C. Van den Broeck
Learning by Examples from a Non-Uniform Distribution
Phys. Rev. E 53, 3989 (1996)

C. Van den Broeck and P. Reimann
Unsupervised Learning by Examples: On-line versus Off-line
Phys. Rev. Lett. 76, 2188 (1996)

G. Györgyi and P. Reimann
Parisi Phase in a Neuron
Phys. Rev. Lett. 79, 2746 (1997)

M. Copelli, R. Eichhorn, O. Kinouchi, M. Biehl, R. Simonetti, P. Riegler and N. Caticha
Noise robustness in multilayer neural networks
Europhys. Lett. 37, 427 (1997)

P. Reimann
Unsupervised Learning of Distributions
Europhys. Lett. 40, 251 (1997)

G. Györgyi and P. Reimann
Beyond Storage Capacity in a Single Model Neuron: Continuous Replica Symmetry Breaking
J. Stat. Phys. 101, 679 (2000)

Last update on August 4, 2004.