Questions about Convolutional neural network
Short answers, pulled from the story.
What is a convolutional neural network and how does it work?
A convolutional neural network (CNN) is a type of feedforward neural network that learns features by optimizing small filters, or kernels, that slide across input data to produce feature maps. Pooling layers then reduce the spatial dimensions of those maps, and fully connected layers perform final classification. Shared weights across the filters dramatically reduce the number of parameters compared to traditional fully connected networks.
Who invented the convolutional neural network?
The foundational architecture traces to Kunihiko Fukushima, who introduced the neocognitron in 1980 with its shared-weights convolutional layers and downsampling layers. Yann LeCun and colleagues in 1989 made training fully automatic by applying backpropagation to learn kernel coefficients directly from images, and formalized this in the seven-layer LeNet-5 network published in 1995.
What was the biological inspiration for convolutional neural networks?
CNNs were inspired by work Hubel and Wiesel conducted in the 1950s and 1960s on the cat visual cortex. Their 1968 paper identified simple cells, which respond to edges of specific orientations within a small receptive field, and complex cells, which have larger receptive fields and are insensitive to exact edge position. Fukushima used this cascading model as the direct blueprint for the neocognitron.
When did convolutional neural networks become practical with GPU acceleration?
The first GPU implementation of a CNN was described in 2006 by K. Chellapilla and colleagues, running four times faster than a CPU equivalent. In 2011, Dan Ciresan and colleagues at IDSIA achieved a 60-times speedup over CPU training and won an image recognition contest with superhuman performance. AlexNet's win at the ImageNet Large Scale Visual Recognition Challenge in 2012 is widely cited as the catalytic event for the modern AI boom.
What are the main applications of convolutional neural networks?
CNNs are used in image and video recognition, medical image analysis, natural language processing, drug discovery, financial time series analysis, brain-computer interfaces, recommender systems, and game playing. In 2015, Atomwise's AtomNet used CNNs for structure-based drug design, predicting candidate treatments for Ebola and multiple sclerosis.
How did AlphaGo use convolutional neural networks to beat human Go players?
AlphaGo used two CNNs driving a Monte Carlo tree search: a policy network to select candidate moves and a value network to evaluate board positions. Before AlphaGo, a 12-layer CNN trained on professional human games had already correctly predicted the professional move in 55% of positions, matching the accuracy of a 6-dan human player, and beat the traditional program GNU Go in 97% of games.