Artificial Neural Networks/Hebbian Learning
Hebbian Learning
editHebbian learning is one of the oldest learning algorithms, and is based in large part on the dynamics of biological systems. A synapse between two neurons is strengthened when the neurons on either side of the synapse (input and output) have highly correlated outputs. In essence, when an input neuron fires, if it frequently leads to the firing of the output neuron, the synapse is strengthened. Following the analogy to an artificial system, the tap weight is increased with high correlation between two sequential neurons.
Mathematical Formulation
editMathematically, we can describe Hebbian learning as:
![{\displaystyle w_{ij}[n+1]=w_{ij}[n]+\eta x_{i}[n]x_{j}[n]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/251c256db555ab331594cfa7caf8660c17e74470)
Here, η is a learning rate coefficient, and and are, respectively, the outputs of the ith and jth elements at time step n.
![{\displaystyle x_{i}[n]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/2ae3a986c99fd36b6b09e7f31f46ab8369086f2f)
![{\displaystyle x_{j}[n]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7fcf8cd6500d65efb41e041f6e814a5e1515dc07)
Plausibility
editThe Hebbian learning algorithm is performed locally, and doesn’t take into account the overall system input-output characteristic. This makes it a plausible theory for biological learning methods, and also makes Hebbian learning processes ideal in VLSI hardware implementations where local signals are easier to obtain.