Artificial neural networks

Definition

Interconnected artificial computational units use their connection structure and parameters to represent transformations from inputs to outputs.

The 1943 McCulloch–Pitts model showed that neuron-like units could perform logical computation. Later trainable networks added parameter learning; early logical models and modern deep networks should not be conflated as the same mature architecture.

Network architecture and learning algorithms are different dimensions. Perceptrons, convolutional networks, and Transformers follow different architectural approaches, while backpropagation is a commonly used method for computing gradients.

Principles and impact

Problems solved

Represented relatively complex functions and patterns through combinations of computational units.

Principles

Computational units pass information through connections, and the network structure and parameters jointly determine the output.

Improvements

Not recorded

Significance

Provided a basic modeling framework for multilayer representations and modern neural learning.

History and applications

People

Warren S. McCulloch、Walter Pitts

Organizations

Not recorded

Events

Not recorded

Applications

Pattern recognition、Function approximation

Limitations

A model’s expressive capacity, the results attainable through training, and its actual generalization performance are not the same thing.

Upstream and downstream relations

Upstream

←BackpropagationusesError backpropagation computes training gradients for differentiable multilayer networks; network architectures and training methods are modeled separately.

Sources: Learning Representations by Back-propagating Errors

Downstream

→Deep learningapplied inDeep learning learns representations using multilayer neural architectures; this does not imply that all neural networks fall within deep learning.

Sources: A Fast Learning Algorithm for Deep Belief Nets ImageNet Classification with Deep Convolutional Neural Networks

→Machine learningapplied inArtificial neural networks with trainable parameters can serve as learning models; early fixed logical networks did not automatically have the ability to learn.

Sources: A Logical Calculus of the Ideas Immanent in Nervous Activity Learning Representations by Back-propagating Errors

→Convolutional neural networkshas specialized formConvolutional neural networks use local connectivity and weight sharing to construct a specific neural network architecture.

Sources: Gradient-based Learning Applied to Document Recognition

→Perceptronhas specialized formThe perceptron is a specific class of neural computation model; this entry focuses on its single-layer classification form.

Sources: The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain Learning Representations by Back-propagating Errors

→Transformerhas specialized formThe Transformer is a class of neural network architectures that uses attention as the central mechanism for organizing information exchange.

Sources: Attention Is All You Need

Definition

Principles and impact

Problems solved

Principles

Improvements

Significance

History and applications

People

Organizations

Events

Applications

Limitations

Upstream and downstream relations

Upstream

Downstream

Explore this technology's connections