Short answers, pulled from the story.
Supervised learning is a machine learning paradigm in which an algorithm learns to map input data to a specific output by training on labeled input-output pairs. The label "supervised" refers to the role of a teacher who provides correct answers during training. It is commonly used for classification tasks, such as spam detection, and regression tasks, such as predicting house prices.
Bias refers to systematic prediction errors that occur regardless of which training set the algorithm uses, while variance refers to how much predictions fluctuate across different training sets. Prediction error is related to the sum of bias and variance, creating a tradeoff: more flexible algorithms reduce bias but increase variance, while rigid algorithms do the opposite.
Overfitting occurs when a model fits training data so closely that it fails to generalize to new examples. It can be caused by noisy training labels or by attempting to learn a function that is too complex for the model. Prevention techniques include early stopping, removing noisy training examples, and structural risk minimization, which adds a regularization penalty such as the L2 norm to the optimization.
The most widely used supervised learning algorithms include support-vector machines, linear regression, logistic regression, naive Bayes, linear discriminant analysis, decision trees, the k-nearest neighbors algorithm, neural networks such as the multilayer perceptron, and similarity learning. No single algorithm performs best on all problems, a result formalized by the No free lunch theorem.
Empirical risk minimization seeks the function that minimizes average loss over the training set, which can lead to overfitting when the hypothesis space is large. Structural risk minimization adds a regularization penalty to the optimization, preferring simpler functions and reducing variance. The penalty has a Bayesian interpretation as a prior probability over candidate functions.
Supervised learning is applied across a wide range of domains including spam detection, speech recognition, handwriting recognition, optical character recognition, object recognition in computer vision, bioinformatics, cheminformatics, information retrieval, landform classification using satellite imagery, and spend classification in procurement processes.