Skip to content

Questions about Feedforward neural network

Short answers, pulled from the story.

What is a feedforward neural network and how does it work?

A feedforward neural network is an artificial neural network in which information flows in a single direction, from inputs through weighted connections to outputs, with no feedback loops. Learning occurs by adjusting connection weights after each data point is processed, using backpropagation to minimize the difference between the network's output and the desired result.

Who invented the feedforward neural network?

The simplest feedforward network, a single weight layer with linear activation trained by least squares, was created by Legendre around 1805 and Gauss around 1795 for predicting planetary movement. Warren McCulloch and Walter Pitts proposed the binary artificial neuron in 1943, and Frank Rosenblatt proposed the multilayered perceptron in 1958.

Who invented backpropagation for neural networks?

Seppo Linnainmaa published the modern form of backpropagation in his 1970 master thesis. Paul Werbos applied it to neural networks in 1982. David E. Rumelhart and colleagues popularized the algorithm in 1986 but did not cite the earlier work.

What was the first working deep learning algorithm?

Alexey Grigorevich Ivakhnenko and Valentin Lapa published the Group Method of Data Handling in 1965, identified as the first working deep learning algorithm. It trained arbitrarily deep neural networks layer by layer through regression analysis and was used to train an eight-layer neural network in 1971.

What is the difference between a feedforward neural network and a recurrent neural network?

A feedforward neural network passes information in one direction only, from inputs to outputs, with no loops. A recurrent neural network contains loops that allow information from later processing stages to feed back to earlier stages. Feedforward architecture is required for backpropagation because feedback loops create an infinite regress that cannot be differentiated.

What is a multilayer perceptron and how does it differ from a simple perceptron?

A multilayer perceptron is a feedforward neural network with at least three layers, often using nonlinear activation functions, capable of classifying data that is not linearly separable. A simple perceptron uses a linear threshold function and can only separate linearly separable classes; it is trained by the delta rule, a basic form of gradient descent.