Artificial Neural Systems

In artificial neural networks [Bis95,MMR97], the input passes through a connected network of simple processing units called neurons to the output. Often, each neuron represents a function $\Phi : \mathbb{R}^d \rightarrow \mathbb{R}$ which maps several inputs to a single output by a weighted summation and non-linear transformation step. In competitive networks (such as Hamming network, MaxNet) a neuron has excitory effect on itself and inhibitory effects on its neighbors. For example, the max-function and $k$-means clustering can be expressed as a winner-take-all network. An adaptive approach that allows for a variable number of clusters and is controlled by a vigilance parameter is Adaptive Resonance Theory (ART) [CG88]. The most popular networks for clustering are topologically organized networks. The Self-Organizing (feature) Map (SOM) was proposed by Kohonen [Koh95] and fits a predefined network structure to the data in a topology preserving fashion by updating the winner and its adjacent neighbors.