[

Early works usually use the metric-based methods. The common goal of these methods is to learn a distance metric that distinguishes the intra-class and inter-class. These common methods include the contrastive loss and triple losses. The contrastive losses (18; 19; 20) are those that use paired samples (positive or negative) to train a network to predict whether they belong to the same class. And the triplet loss (14; 39; 40; xiang2022learning) consider that the distance between the query and positive samples is greater than that between the query and negative samples by a given distance margin. Since contrastive and triplet loss often lead to slow convergence, N-pair loss (32) is designed, which improves the training convergence by considering the distance between the query samples and other multi negative samples jointly at each update. But these sample-to-samples comparisons often suffer from the explosion of sample space. Thus, some sample-to-class schemes, e.g., center loss (15) and variants (21), are proposed to solve the problem of explosive growth in computation, they learn a center for each category and require the same category features to be closed to its center. But these kind methods fail to address the problem of open set in face recognition.

Compared to the deep metric learning methods, the angular-margin-based loss (13; 11; 42) often remodel the last fully connected layer of classification network to different form. It explicitly adds discriminative constraints to the target logit and makes the same category feature space more compact, it is more efficient and stable, L-softmax (42) constructed a large-margin softmax loss and employed a piecewise function to guarantee the monotonicity of the cosine function. A-softmax (13) normalized the weight matrix $w$, and learned face features on a hypersphere manifold. In fact, L-softmax and A-softmax are difficult to train due to the complex design of the angular margin. To relieve this dilemma, some researchers (10; 11) try to simplify this question. They set the bias term to zero and normalize the weight and embedding feature. The representation of the last fully connected layer is reduced to vector inner product form. And Cosface (10) introduced an additive cosine margin. Arcface (11) introduced an additive angular margin. More recently, Adaptiveface (37) and Fariloss (38) introduce adaptive margin strategy by adjusting the strength of the supervision process during training to address the problem of unbalanced data.

Compared with existing methods, our proposed SubFace strategy is different from them in the following aspect: (1) Our proposed SubFeature Strategy tries to improve the overall performance of features through mining the discriminative power of local features, supporting it to be more discriminative, while previous researches (such as Arcface and Cosface) mainly lay emphasis on global feature to learn a discriminative model. (2) The proposed framework has much fewer parameters and can be embedded into existing face recognition model seamlessly, which make it more flexible and adaptable in real-world scenarios. (3) To the best of our knowledge, we are among the first attempt to introduce feature approximation mechanism on face recognition task.

Usually a training system consists of three parts: training data, feature extraction network, and loss function. And a face training system can be expressed as $\left\{\right\{x_i,y_i\},f,F_{loss}\}$ that contains $N$ number identities, each training sample $x_i$ corresponds to a label $y_i$, $f(\cdot )$ is the feature extractor, $f\left(x_i\right)\in R^d$ represents the embedded feature of i-th face sample. $F_{loss}$ denotes the loss function during training, which can be represented as:

The loss function provides a distance measurement, under which similar targets are as close and non similar targets are far away. Intuitively, human face has obvious nonlinear characteristics, adopting new loss functions in deep neural networks is effective approach to make features more discriminative. However, in addition to the non-linear discriminative embedding, there is clearly locally distinguishable embedding for face recognition. For example, we can distinguish and recognize pedestrians by discriminative parts with attention mechanism (xiang2020multi). Inspired by this, we take a big step forward to explore the identification ability of local features during the training process, which can significantly boost the performance in a large degree for face recognition task.

In this section, we present the approximation strategy to the face recognition training task. Firstly, we illustrate the feature approximation mechanism in Section 3.2.1, and then in Section 3.2.2, we give more detailed information about the subfeature normalization strategy.

We employ three widely used datasets CASIA (24), A MS1M-RetinaFace (54), MS1MV2 (11) as our training datasets. The MS1MV2 contains 5.8M images and 85K identities. The MS1M-RetinaFace dataset contains 5.1M images of 93 K identities, which is a refined dataset of MS1M (53). And the CASIA dataset contains 10,577 identities and 0.5 M images. Note that all face images are resized to 112 $\times$ 112.

We employ the widely used CNN architecture MobilefaceNet (13), ResNet-50, ResNet-100 (1; 2) as the backbone and set the embedding feature dimmension to 512. And during testing, two margin-based loss functions (ArcFace and CosFace) are used to make detailed comparison. We set the scale s to 64, the angular margin m to 0.5 when using ArcFace and the cosine margin m to 0.4 of CosFace. The mini-batch size is set to 512. The SGD momentum is set to 0.9 and the weight decay is empirically set to 5e-4. The learning rate starts from 0.1 and is divided by 10 at 36k, 52k iterations for CASIA, and the total training process is finished at 65k iterations. For MS1MV2, the learning ratio is divided at 100k, 160k iterations, the training process is finished at 180k iterations. For MS1M-RetinaFace, we divided the learning rate at 100k, 160k, 210k iterations and finished at 250k iterations. All experiments are conducted on a server equipped with two Nvidia A100 GPUs on Pytorch (paszke2019pytorch) framework.

We employ the ResNet-50 as the embedding network and the CASIA as training dataset to train ArcFace using our strategy, and we compare the verification result under different sampling ratio in the range of [0-1], as shown in Fig. 3. The experiment shows that the performances of all sample ratio do no change a lot except when sampling ratio is low.

For example, when the sampling rate is 0.1, it leads to a decrease in the recognition accuracy. More because the representation dimension of the feature is obviously insufficient. In this experiment, it means that only 51-dimensional features are used to represent the face during the training. And when the sampling ratio is greater than 0.4, our approach has little negative impacts on model performance, the performances on the three validation sets are more just fluctuating. And in order to make a fair comparison, we empirically set the sampling ratio at 0.7 in the follow experiments.

To further validate the effectiveness of the our proposed method, we perform several ablation studies on the individual component of our proposed SubFace method.

Effects of feature approximation: In this experiment, we test our approximation strategy on the validation dataset LFW, CALFW, CPLFW respectively with backbone ResNet-50, and we choose the ArcFace as the loss function. As illustrated in Fig. 4, we give the feature distance distribution of face pairs. The distance distribution maps of face pairs at feature sampling rate 0.5 and 1.0 are given²²2The feature sampling rate of 1.0 represents the original training strategy.. The Euclidean distances between positive pairs

are skewed to the smaller side compared with the situation at sampling ratio of 1.0. And there is no difference of the negative sample pair under the two cases in terms of the distance distribution. Our random strategy makes the embedding features more compact, as the discriminative power of features is more balanced across the entire dimension.

Effects of subfeature normalization: In this experiments we verify the convergence of our approximation strategy by observing the volatility of the vector angle. We denote ${\theta }_j$ as the angle between the embedding feature $f_i$ and the center $w_j$, ${\theta }_j^{{}^{\prime }}$ as the angle between the subfeature $f_{(i,r)}$ and the subcenter $w_{(j,r)}$. In this experiment, we adopt the average cosine distance defined in 46 and judge the convergence by the variation of cosine.

As illustrated in Fig. 5, the change of sampling ratio does not affect the final convergence, and the arbitrary $r\ast d$ dimensional subspace can approximate the performance of full-dimensional features. During the same time, we also observe that the change of sampling rate can affect the volatility of the average cosine distance. To be more specific, when the sampling rate is smaller, the deviation is larger at the initial stage of training and the volatility of diffs is greater at later stage.

Effects of lightweight model: Lightweight models, which are benefited from smaller network structures, fewer model parameters, as well as superior performance, are now widely used in the application of embedded devices, computer vision, etc. In this section, we choose MobilefaceNet as the backbone network since MobilefaceNet is more balanced between computation, parameters and model performance. The parameter amount of Mobilefacenet is 0.99M, and the calculation amount is 439.8M (FLOPs), and it performs well on relevant data. We train ArcFace on CAISA and MS1M-RetinaFace respectively. According to the results in Table 1, our approximation strategy shows the performance superiority on lightweight models. For example, it can achieve the best accuracy of 99.58% on LFW and 96.22% on Age-DB30 respectively when trained on MS1M-RetinaFace dataset. And when using CASIA dataset, our trained model also shows a competitive performance, outperforming the previous work (7) and surpassing the original result by 0.08% on AgeDB-30 dataset.

Effects of small-scale & large-scale trainset: To validate the effects of small-scale & large-scale trainset, we adopt the two training set CASIA and MS1MV2 to test the performance of our sampling method. We treat the CASIA as small-scaled trainset and the MS1MV2 as large-scaled trainset. In order to make a fair comparison and reduce the volatility of the test results themselves, we choose ResNet-100 as the backbone, and adopt the arcface loss and cosface loss for optimization process.

According to the experimental results in Table 2, there are obvious performance gaps that the results on the big model (ResNet-100) are generally better than the results of the small model (ResNet-50). The results on the big-scale dataset are better than those on the small-scale dataset using the same model. And ArcFace generally outperforms CosFace under the same conditions using our strategy. For example, when using ResNet-100, ArcFace with our approximation strategy can achieve the best verification performance on AgeDB-30 dataset (e.g. 98.23%), surpassing the results (46) by 0.03% with ArcFace(0.5) (46) when trained on MS1MV2. However, the performance difference between our SubFace strategy and ArcFace (46) is very slight on LFW and CFP-FP dataset respectively.

In this section, we evaluate the model performance with different settings on the face verification datasets such as LFW (26), CFP-FP and agedb-30 (28). During the same time, we also give the performance of our methods on large-pose and large-age datasets CPLFW (29), CALFW (30), and the large-scale image datasets megaface (31), IJB-B (23), and IJB-C (35) respectively.

Evaluation on LFW, CALFW and CPLFW datasets: LFW dataset is one of the widely used benchmarks for unconstrained face verification on images and videos, it contains 6,000 comparison pairs, with 3,000 positive pairs and 3,000 negative pairs. CPLFW and CALFW datasets are recently introduced which show higher pose and age variations with same identities from LFW. According to the results from the Table 3, ArcFace with our strategy can achieve the best performance of 99.85% and 93.48% on LFW and CPLFW respectively. When compared with the original ArcFace (46), we have improved performance on all the three sets. Even with the results (the row 5 in Table 3) on a larger training sets Glint360k (46), the performance is surpassed by +0.02% and +0.09% on LFW and CALFW datasets respectively.

Evaluation on IJB-B and IJB-C datasets: The IJB-B dataset contains 1,845 subjects with 21.8 k still images and 55 k frames from 7,011 videos. The IJB-C contains 3,531 subjects with 31.3 k still images and 117.5 k video frames. And there are 10,270 genuine matches and 8m imposter matches in the IJB-B verification protocol. The IJB-C verification protocol provides a total 19,557 genuine matchs and 15,639K impostor matches.

On the IJB-B and IJB-C datasets, we employ the MS1M-retinaFace and MS1MV2 dataset as the training data and the ResNet-100 as the embedding network, the sampling ratio is choosen at 0.7 for the fair comparison with the most recent methods. As shown in Table 4, our method further improves the performace the TAR (@FAR=1e-4) to 96.85% and 95.47% on IJB-C and IJB-B respectively. And when using the same training dataset MS1MV2, the accuracy difference between our method and ArcFace (11) is 0.81% (94.2% vs 95.01%) and 0.78% (95.6% vs 96.38%) on IJB-B and IJB-C respectively, which surpasses the performance in CurricularFace (52) by a clear margin.

Evaluation on MegaFace dataset: Finally, we evaluate the performance on the MegaFace Challenge. The evaluation protocol of MegaFace includes gallery and probe sets. The gallery set contains 1M images of 690k different individuals and the probe set contains 100 k photos of 530 unique individuals from facescrub (36). As there are some noises in the original Megaface, we adopt the refined megaface dataset (11) to make a fair comparison. Table 5 shows the results obtained by ResNet-100 and the previous works for both identification and verification tasks on this dataset. From the result we can see that ResNet-100 with our strategy can achieve state-of-the-art results with respect to other recent models under verification scenarios.

In this work, we propose a face training method based on local feature approximation, which has achieved competitive results on several benchmark datasets. On the one hand, compared with existing training methods, we mainly use the local feature for training, and we also verify the convergence and feature compactness of this method in Section 4.4. Unfortunately, the sampling dimension corresponding to the subfeature is consistent with all data in the same batch, which may not be an optimal choice in same cases. For example, the distinctive local area of the face may be different between profile and frontal, or the face under the condition of illumination and non-illumination. Therefore, there is still room for further optimization in the selection of our random sampling method.

On the other hand, we perform in-depth empirical analysis of our method, and it is surprising to observe that the training optimization is difficult to reach a comprehensive optimization state in all performance indicators, which leads to a slightly performance degradation on some benchmark datasets. For example, as shown in Table 2, the verification performances of our method on LFW and CFP-FP datasets are 99.83% and 98.50% respectively. While the best results of previous methods on LFW and CFP-FP datasets are 99.83% and 98.51% individually. The reasons for this phenomenon may come from the following aspects:

First, the sampling factor in our experiment is set to 0.7, maybe this optimal parameter should be different in various experiments.

Second, according to the Table 5, the performance gap between our method and existing methods in some datasets (MegaFace dataset) is relatively small, where the fluctuation of this difference can be completely compensated through code, training strategy and data enhancement, etc.

Third, the main purpose of our optimization method is to promote the compactness of positive sample features, so as to improve the performance of face classification. In the comparative study in Table 2, the performance difference here may be mainly resulted from the difference of the processing method on some hard samples. And these challenges warrant further research and consideration when deploying the face recognition model in real scenarios.