FaceNet

• 1. Introduction

  ›

  FaceNet encodes a human face as exactly 128 numbers. Those numbers place the face at a specific coordinate in a high-dimensional mathematical space. Two photographs of the same person will land close together in that space. Two photographs of different people will sit far apart. Florian Schroff, Dmitry Kalenichenko, and James Philbin, all affiliated with Google, built the system. They presented it for the first time at the 2015 IEEE Conference on Computer Vision and Pattern Recognition. On the Labeled Faces in the Wild dataset, under the unrestricted with labeled outside data protocol, FaceNet reached 99.63% accuracy. That was the highest score on record for that benchmark. How a network learns to place faces at such precise coordinates is the central puzzle. What kind of training and what kind of architecture make this possible forms the rest of the story.

  1

• 2. Learning from Batches

  ›

  For each round of training, researchers assembled batches of roughly 1,800 images. Every person represented in a batch appeared in exactly 40 images. The remaining slots held randomly selected images from other identities. This mix gave the network its training signal.

  The system was optimized using stochastic gradient descent with standard backpropagation, combined with the AdaGrad algorithm. The initial learning rate was set at 0.05. Researchers lowered it as training neared completion and the model took its final shape.

  What drove all these updates toward their goal was a cost function the researchers called the triplet loss.

  1

• 3. The Triplet Loss

  ›

  The researchers identified the triplet loss as a key innovation of FaceNet. Alongside it, they introduced a new online method for mining the training triplets. That mining approach was as central to the system as the loss function itself.

  The broader impact reached beyond facial recognition entirely. The triplet loss has since become central to a variety of one-shot learning problems. These are challenges where a system must perform accurately from very limited training examples.

  The convolutional neural network that implemented this training strategy was built in two distinct versions, called NN1 and NN2.

  1

• 4. Two Architectures

  ›

  The practical difference between NN1 and NN2 lay in their parameter counts and computational requirements. The NN1 model carries 140 million parameters in total and requires roughly 1.6 billion floating point operations to run. It begins with a convolution layer, conv1, that processes input images at 220 by 220 pixels with three color channels. A sequence of convolution, pooling, and normalization layers follows, compressing the spatial dimensions and building up increasingly abstract features. The final layers are fully connected, with two maxout layers preceding the network's output and a final L2 normalization step.

  FaceNet's accuracy on the YouTube Faces DB reached 95.12%.

  1