Distilling Facial Knowledge With Teacher-Tasks:
Semantic-Segmentation-Features for Pose-Invariant Face-Recognition

Face-recognition (FR) is becoming the go-to authentication technology for access control and verification applications. Its popularity starts with evolution of smart phones, where over 100 million devices offer it as seamless-unlock method [16]. This has led other industries to follow suit, where commercial real-estate [13], aviation [18] and banking [6] now use FR as a means to differentiate customer experience. This is made possible by advances in deep-learning [21]; state-of-the-art models can now discern 1 cooperative face from over 50,000 [4]. Having robust tolerance to pose-variations, however, is still a challenge [28].

Pose-variations are facial rotations over yaw and pitch. These change the relative-position of key-points (e.g., nose, eyes) and introduce variance within identity classes. As such, FR algorithms can struggle to discern the same person rotating from different people [28]. In particular when applying stringent industry false-acceptance-rate thresholds [4], variations in pose often result in false-rejections [28].

Current state-of-the-art methods rely on alignment techniques and/or sophisticated loss-functions to address pose-variability. Through alignment pre-processing, algorithms can project a cooperative-face for identification [10]. Alternatively, contrastive loss-functions (such as triplet) implicitly address pose-variations through relative class-distance [21]. While notable advances, even best-in-class algorithms struggle to achieve 100% accuracy on competition data-sets [8].

This paper presents the Seg-Distilled-ID network. This is a new approach to knowledge-distillation, using a teacher-task in lieu of a teacher-network. The Seg-Distilled-ID network is first jointly trained on both identification and (teacher) semantic-segmentation tasks, where the teacher-task is then removed. This “distills” the semantic-structures as context for precise identification (see Fig. 1). Recognition accuracy is benchmarked against three state-of-the-art encoders on the Mut1ny commercial face-segmentation data-set [15] (11,830 images selected from 67 subjects, varied over pose and lighting). The proposed Seg-Distilled-ID network achieves best-in-class accuracy using 2.4M inference parameters. These results demonstrate distilling semantic-segmentation features can efficiently address face-recognition pose-invariance.

In summary, this paper makes the following contributions:

• Novel knowledge-distillation method via teacher-tasks.

• Best-in-class ID accuracy with efficient parameter space (99.9%, Mut1ny faces [15]).

Face-recognition (FR) research starts in the 1970s as a template-matching problem. This is pioneered by the discovery that statistical-distributions (e.g., Eigenfaces [25]) are generally robust. This is expanded upon by using hand-crafted features to describe distinguishable features [14]. These features, however, are insufficient at large-scales. This ultimately transitions to deep-learning (DL) solutions (starting with DeepFace [24]). Today, state-of-the-art DL networks emphasize a combination pose-alignment pre-processing [10] (which may include 3-D projection [7, 29]) and contrastive loss-functions [21].

Despite all these advances, pose-variations are still a challenge [28]. To address this, a new trend is to apply multi-task-learning (MTL) for sharing context. This is first exemplified by Ranjan et al, who combine landmarks and pose tasks on a face-detection network (HyperFace) to improve reliability [17]. Others have now recently applied this approach to identification. Yin and Xiaoming have a pose estimation task connected to the identification features [27]. Wang et al alternatively use semantic descriptors of sub-structures: size of eyes, nose and cheeks [26]. Both approaches show consistent (small) improvements on FR competition data-sets.

This research differentiates on these findings by using a knowledge-distillation approach with a precise-descriptor: semantic-segmentation. The aforementioned approaches share common features with the identification task. In this case, the network is jointly trained on facial-structure with identification, then “distills” the teacher-task. The “distilled” semantic-features enable the encoder to generate precise features, enabling efficient pose-invariance recognition.

This research proposes a novel application of Multi-Task-Learning (MTL) to improve face-recognition pose-robustness. Facial semantic-segmentation is “distilled” by using a “teacher-task.” By encoding relative facial-structure, the loss function can better discern inter versus intra class variations.

This experiment evaluates the identification accuracy when introducing significant pose-variations. The purpose is to demonstrate the utility of distilling face-segmentation as contextual features. The Seg-Distilled-ID network is validated against identification networks using MobileNetV2 [20] without segmentation-context and three state-of-the-art network encoders.

Table 1 shows the performance evaluation results. As generally expected, having a stronger encoder correlates with better ID classification. All networks but MobileNetV2-ID train to a validation accuracy of at least 95% (training data not shown for space). This understandable from the encoder architectures. For example, InceptionV3-ID network has a relatively-high parameter-count with factorized-convolutions [23] and trains robustly. Conversely, the MobileNetV2-ID stops early and clearly over-fits due to its efficient design.

This performance disparity exemplifies the benefits of distilling semantic-segmentation features. Despite MobileNetV2-ID over-fitting, the Seg-Distilled-ID has the highest accuracy score evaluated. This is achieved while retaining the MobileNet architecture’s efficiency (approximately one-tenth of the VGG and Inception network parameters). The parenthesis indicates that 2.4M parameters are used for inference and 6.5M are used for jointly training with the teacher-task.

The parameter efficiency is explainable by using the semantic-segmentation knowledge to select optimal features. Top-tier encoders use large parameter-spaces to implicitly infer context, enabling them to perceive information the base MobileNetV2-ID cannot. This methodology instead explicitly provides context through the facial-structure teacher-task. Fig. 3 shows the semantic-segmentation masks; one can infer how the features that encode facial-structure variation across pose enable precise identification across pose. This feature robustness enables the Seg-Distilled-ID to efficiently achieve best-in-class performance.

Note that generalized pose robustness is very much novel. Others demonstrate re-aligning the face in 3-D space can improve identification robustness (such as LDF-Net [7] and GridFace [29]). These methods are effective but degrade as yaw and pitch increase. It is hypothesized the 3-D alignment algorithms synthetically inferring the obscured facial features cascades bias from the projector. The Seg-Distilled-ID avoids this bias by learning facial-structures in a one-shot approach.

This paper presents the Seg-Distilled-ID network to address pose-invariance face-recognition. This is a novel application of knowledge-distillation, where an ID-task is jointly-trained with a “distilled” teacher semantic-segmentation-task. Benchmarking with state-of-the-art encoders ResNet-101 [5], VGG-19 [22] and InceptionV3 [23] shows the proposed Seg-Distilled-ID network achieves best-in-class performance using minimal parameters (MobileNetV2 [20] encoder).

Next steps include larger-scale evaluation with varied context-encoding methods. The Mut1ny data-set [15] has only 67 subjects in the synthetic-face repository at this time; hence, the planned next step is attempt transfer-learning these features onto Labelled Faces in the Wild [8]. To accommodate the increase in ID classes, a comparison of U-Net [19] and DeepLabV3 [2] designs will be done with various task-architectures. Benchmarking will also include face-alignment pre-processing and contrastive loss-functions.

The authors would like to give special thanks to Ford Motor Company for funding this research via University Alliance Grant. This in particular includes Ford engineers Justin Miller and Jon Diedrich for their continued support.