尽管在视觉情感识别方面取得了很多进展，但研究人员已经意识到，现代深层网络倾向于利用数据集特征来学习输入和目标之间的虚假统计关联。这种数据集特性通常被视为数据集偏差，这会损害这些识别系统的稳健性和泛化性能。在这项工作中，我们从因果推理的角度审查了这个问题，在该因果推理的角度将这种数据集特征称为混杂因素，它误导了系统以学习虚假相关性。为了减轻数据集偏见带来的负面影响，我们提出了一种新颖的介入情绪识别网络（IERN）来实现后门调整，这是因果推断中一种基本的解污染技术。具体而言，IERN首先将与数据集相关的上下文功能从实际的情感功能中解散，前者形成混杂因素。然后，情感功能将被迫在被送入分类器之前平均看到每个混杂层。一系列设计的测试验证了IERN的功效，并且对三个情感基准的实验表明，IERN的表现优于最先进的方法，以实现无偏见的视觉情感识别。代码可从https://github.com/donydchen/causal_emotion获得

Machine learning models rely on various assumptions to attain high accuracy. One of the preliminary assumptions of these models is the independent and identical distribution, which suggests that the train and test data are sampled from the same distribution. However, this assumption seldom holds in the real world due to distribution shifts. As a result models that rely on this assumption exhibit poor generalization capabilities. Over the recent years, dedicated efforts have been made to improve the generalization capabilities of these models collectively known as -- \textit{domain generalization methods}. The primary idea behind these methods is to identify stable features or mechanisms that remain invariant across the different distributions. Many generalization approaches employ causal theories to describe invariance since causality and invariance are inextricably intertwined. However, current surveys deal with the causality-aware domain generalization methods on a very high-level. Furthermore, we argue that it is possible to categorize the methods based on how causality is leveraged in that method and in which part of the model pipeline is it used. To this end, we categorize the causal domain generalization methods into three categories, namely, (i) Invariance via Causal Data Augmentation methods which are applied during the data pre-processing stage, (ii) Invariance via Causal representation learning methods that are utilized during the representation learning stage, and (iii) Invariance via Transferring Causal mechanisms methods that are applied during the classification stage of the pipeline. Furthermore, this survey includes in-depth insights into benchmark datasets and code repositories for domain generalization methods. We conclude the survey with insights and discussions on future directions.

The task of Few-shot learning (FSL) aims to transfer the knowledge learned from base categories with sufficient labelled data to novel categories with scarce known information. It is currently an important research question and has great practical values in the real-world applications. Despite extensive previous efforts are made on few-shot learning tasks, we emphasize that most existing methods did not take into account the distributional shift caused by sample selection bias in the FSL scenario. Such a selection bias can induce spurious correlation between the semantic causal features, that are causally and semantically related to the class label, and the other non-causal features. Critically, the former ones should be invariant across changes in distributions, highly related to the classes of interest, and thus well generalizable to novel classes, while the latter ones are not stable to changes in the distribution. To resolve this problem, we propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency by replacing the patch-level information and supervision of the query images with random gallery images from different classes from the query ones. We theoretically show that such an augmentation mechanism, different from existing ones, is able to identify the causal features. To further make these features to be discriminative enough for classification, we propose Correlation-guided Reconstruction (CGR) and Hardness-Aware module for instance discrimination and easier discrimination between similar classes. Moreover, such a framework can be adapted to the unsupervised FSL scenario.

Understanding the facial expressions of our interlocutor is important to enrich the communication and to give it a depth that goes beyond the explicitly expressed. In fact, studying one's facial expression gives insight into their hidden emotion state. However, even as humans, and despite our empathy and familiarity with the human emotional experience, we are only able to guess what the other might be feeling. In the fields of artificial intelligence and computer vision, Facial Emotion Recognition (FER) is a topic that is still in full growth mostly with the advancement of deep learning approaches and the improvement of data collection. The main purpose of this paper is to compare the performance of three state-of-the-art networks, each having their own approach to improve on FER tasks, on three FER datasets. The first and second sections respectively describe the three datasets and the three studied network architectures designed for an FER task. The experimental protocol, the results and their interpretation are outlined in the remaining sections.

Deep models for facial expression recognition achieve high performance by training on large-scale labeled data. However, publicly available datasets contain uncertain facial expressions caused by ambiguous annotations or confusing emotions, which could severely decline the robustness. Previous studies usually follow the bias elimination method in general tasks without considering the uncertainty problem from the perspective of different corresponding sources. In this paper, we propose a novel method of multi-task assisted correction in addressing uncertain facial expression recognition called MTAC. Specifically, a confidence estimation block and a weighted regularization module are applied to highlight solid samples and suppress uncertain samples in every batch. In addition, two auxiliary tasks, i.e., action unit detection and valence-arousal measurement, are introduced to learn semantic distributions from a data-driven AU graph and mitigate category imbalance based on latent dependencies between discrete and continuous emotions, respectively. Moreover, a re-labeling strategy guided by feature-level similarity constraint further generates new labels for identified uncertain samples to promote model learning. The proposed method can flexibly combine with existing frameworks in a fully-supervised or weakly-supervised manner. Experiments on RAF-DB, AffectNet, and AffWild2 datasets demonstrate that the MTAC obtains substantial improvements over baselines when facing synthetic and real uncertainties and outperforms the state-of-the-art methods.

更广泛的人重新识别（Reid）在最近的计算机视觉社区中引起了不断的关注。在这项工作中，我们在身份标签，特定特定因素（衣服/鞋子颜色等）和域特定因素（背景，观点等）之间构建结构因果模型。根据因果分析，我们提出了一种新颖的域不变表示，以获得概括的人重新识别（DIR-REID）框架。具体而言，我们首先建议解散特定于特定的和域特定的特征空间，我们提出了一种有效的算法实现，用于后台调整，基本上是朝向SCM的因果干预。已经进行了广泛的实验，表明Dir-Reid在大规模域泛化Reid基准上表现出最先进的方法。

常规的去命名方法依赖于所有样品都是独立且分布相同的假设，因此最终的分类器虽然受到噪声的干扰，但仍然可以轻松地将噪声识别为训练分布的异常值。但是，在不可避免地长尾巴的大规模数据中，该假设是不现实的。这种不平衡的训练数据使分类器对尾巴类别的歧视性较小，而尾巴类别的差异化现在变成了“硬”的噪声 - 它们几乎与干净的尾巴样品一样离群值。我们将这一新挑战介绍为嘈杂的长尾分类（NLT）。毫不奇怪，我们发现大多数拖延方法无法识别出硬噪声，从而导致三个提出的NLT基准测试的性能大幅下降：Imagenet-NLT，Animal10-NLT和Food101-NLT。为此，我们设计了一个迭代嘈杂的学习框架，称为“难以容易”（H2E）。我们的引导理念是首先学习一个分类器作为噪声标识符不变的类和上下文分布变化，从而将“硬”噪声减少到“ Easy”的噪声，其删除进一步改善了不变性。实验结果表明，我们的H2E胜过最先进的方法及其在长尾设置上的消融，同时在传统平衡设置上保持稳定的性能。数据集和代码可从https://github.com/yxymessi/h2e-framework获得

As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on reweighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the "bad" while keep the "good". We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks 1 : Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.

尽管在过去的几年中取得了重大进展，但歧义仍然是面部表情识别（FER）的关键挑战。它可能导致嘈杂和不一致的注释，这阻碍了现实世界中深度学习模型的性能。在本文中，我们提出了一种新的不确定性标签分布学习方法，以提高深层模型的鲁棒性，以防止不确定性和歧义。我们利用价值空间中的邻里信息来适应培训训练样本的情绪分布。我们还考虑提供的标签将其纳入标签分布时的不确定性。我们的方法可以轻松地集成到深层网络中，以获得更多的培训监督并提高识别准确性。在各种嘈杂和模棱两可的环境下，在几个数据集上进行了密集的实验表明，我们的方法取得了竞争成果，并且超出了最新的最新方法。我们的代码和模型可在https://github.com/minhnhatvt/label-distribution-learning-fer-tf上找到。

人类的情感认可是人工智能的积极研究领域，在过去几年中取得了实质性的进展。许多最近的作品主要关注面部区域以推断人类的情感，而周围的上下文信息没有有效地利用。在本文中，我们提出了一种新的深网络，有效地识别使用新的全球局部注意机制的人类情绪。我们的网络旨在独立地从两个面部和上下文区域提取特征，然后使用注意模块一起学习它们。以这种方式，面部和上下文信息都用于推断人类的情绪，从而增强分类器的歧视。密集实验表明，我们的方法超越了最近的最先进的方法，最近的情感数据集是公平的保证金。定性地，我们的全球局部注意力模块可以提取比以前的方法更有意义的注意图。我们网络的源代码和培训模型可在https://github.com/minhnhatvt/glamor-net上获得

面部表达识别（FER）是一个具有挑战性的问题，因为表达成分始终与其他无关的因素（例如身份和头部姿势）纠缠在一起。在这项工作中，我们提出了一个身份，并构成了分离的面部表达识别（IPD-fer）模型，以了解更多的判别特征表示。我们认为整体面部表征是身份，姿势和表达的组合。这三个组件用不同的编码器编码。对于身份编码器，在培训期间使用和固定了一个经过良好训练的面部识别模型，这可以减轻对先前工作中对特定表达训练数据的限制，并使野外数据集的分离可行。同时，用相应的标签优化了姿势和表达编码器。结合身份和姿势特征，解码器应生成输入个体的中性面。添加表达功能时，应重建输入图像。通过比较同一个体的合成中性图像和表达图像之间的差异，表达成分与身份和姿势进一步分离。实验结果验证了我们方法对实验室控制和野外数据库的有效性，并实现了最新的识别性能。

分布式概括（OOD）都是关于对环境变化的学习不变性。如果每个类中的上下文分布均匀分布，则OOD将是微不足道的，因为由于基本原则，可以轻松地删除上下文：类是上下文不变的。但是，收集这种平衡的数据集是不切实际的。学习不平衡的数据使模型偏见对上下文，从而伤害了OOD。因此，OOD的关键是上下文平衡。我们认为，在先前工作中广泛采用的假设，可以直接从偏见的类预测中注释或估算上下文偏差，从而使上下文不完整甚至不正确。相比之下，我们指出了上述原则的另一面：上下文对于类也不变，这激励我们将类（已经被标记为已标记的）视为不同环境以解决上下文偏见（没有上下文标签）。我们通过最大程度地减少阶级样本相似性的对比损失，同时确保这种相似性在所有类别中不变，从而实现这一想法。在具有各种上下文偏见和域间隙的基准测试中，我们表明，配备了我们上下文估计的简单基于重新加权的分类器实现了最新的性能。我们在https://github.com/simpleshinobu/irmcon上提供了附录中的理论理由和代码。

学习强大的表示是图形神经网络（GNN）的一个中心主题。它需要从输入图中炼制关键信息，而不是琐碎的模式，以丰富表示。为此，图表注意力和汇集方法占上风。他们主要遵循“学会参加”的范式。它最大限度地提高了上述子图和地面真理标签之间的相互信息。然而，这种训练范例易于捕获微级子图和标签之间的虚假相关性。这种杂散的相关性对分布（ID）测试评估有益，但在分布外（OOD）测试数据中引起差的概括。在这项工作中，我们从因果角度重新审视GNN建模。在我们的因果假设之上，琐碎的信息是关键信息和标签之间的混淆，它在它们之间打开了一个后门路径，使它们保持虚拟相关。因此，我们提出了一个新的解压缩训练范式（DTP），更好地减轻了批评信息的混淆效果并锁存，以提高表示和泛化能力。具体而言，我们采用注意模块解开关键的子图和微不足道的子图。然后我们使每个关键的子图相当与不同的琐碎子图相互作用，以实现稳定的预测。它允许GNN捕获一个更可靠的子图，其与标签的关系跨越不同的分布。我们对综合和现实世界数据集进行了广泛的实验，以证明有效性。

多任务学习是基于深度学习的面部表情识别任务的有效学习策略。但是，当在不同任务之间传输信息时，大多数现有方法都考虑了特征选择，这可能在培训多任务网络时可能导致任务干扰。为了解决这个问题，我们提出了一种新颖的选择性特征共享方法，并建立一个用于面部表情识别和面部表达合成的多任务网络。该方法可以有效地转移不同任务之间的有益特征，同时过滤无用和有害信息。此外，我们采用了面部表情综合任务来扩大并平衡训练数据集以进一步提高所提出的方法的泛化能力。实验结果表明，该方法在那些常用的面部表情识别基准上实现了最先进的性能，这使其成为现实世界面部表情识别问题的潜在解决方案。

Recent years witnessed the breakthrough of face recognition with deep convolutional neural networks. Dozens of papers in the field of FR are published every year. Some of them were applied in the industrial community and played an important role in human life such as device unlock, mobile payment, and so on. This paper provides an introduction to face recognition, including its history, pipeline, algorithms based on conventional manually designed features or deep learning, mainstream training, evaluation datasets, and related applications. We have analyzed and compared state-of-the-art works as many as possible, and also carefully designed a set of experiments to find the effect of backbone size and data distribution. This survey is a material of the tutorial named The Practical Face Recognition Technology in the Industrial World in the FG2023.

了解因果关系有助于构建干预措施，以实现特定的目标并在干预下实现预测。随着学习因果关系的越来越重要，因果发现任务已经从使用传统方法推断出潜在的因果结构从观察数据到深度学习涉及的模式识别领域。大量数据的快速积累促进了具有出色可扩展性的因果搜索方法的出现。因果发现方法的现有摘要主要集中在基于约束，分数和FCM的传统方法上，缺乏针对基于深度学习的方法的完美分类和阐述，还缺乏一些考虑和探索因果关系的角度来探索因果发现方法范式。因此，我们根据变量范式将可能的因果发现任务分为三种类型，并分别给出三个任务的定义，定义和实例化每个任务的相关数据集以及同时构建的最终因果模型，然后审查不同任务的主要因果发现方法。最后，我们从不同角度提出了一些路线图，以解决因果发现领域的当前研究差距，并指出未来的研究方向。

我们专注于视觉接地管道语言与位置之间的混淆偏见，在那里我们发现偏差是主要的视觉推理瓶颈。例如，接地过程通常是一种琐碎的语言 - 位置关联，没有视觉推理，例如，将任何包含绵羊的语言查询接地到近中心区域，由于绵羊在图像中心的地面真实位置存在地面真相位置。首先，我们将视觉接地管道框架框成了因果图，其显示图像，查询，目标位置和底层混淆之间的因果关系。通过因果图，我们知道如何打破接地瓶颈：Deconfounded视觉接地。其次，为了解决混乱的挑战，即一般而言，我们提出了一种呼吁呼吁：引用表达式解构器（红色），以消除混淆偏差。第三，我们实施红色作为一种简单的语言关注，可以以任何接地方法应用。在流行的基准测试中，红色通过显着的边缘改善了各种最先进的接地方法。代码将很快提供：https://github.com/jianqiangh/deconfounded_vg。

现有的长尾分类（LT）方法仅着眼于解决阶级的失衡，即头部类别的样本多于尾巴类，但忽略了属性的不平衡。实际上，即使班级平衡，由于各种属性，每个类中的样本仍然可能会长时间尾。请注意，后者在根本上比前者更加普遍和具有挑战性，因为属性不仅是大多数数据集的隐含，而且在组合上也具有复杂性，因此平衡的昂贵。因此，我们引入了一个新的研究问题：广义的长尾分类（GLT），共同考虑两种失衡。通过“广义”，我们的意思是，GLT方法自然应该解决传统的LT，但反之亦然。毫不奇怪，我们发现大多数class LT方法在我们提出的两个基准中退化：Imagenet-GLT和Mscoco-GLT。我们认为这是因为他们过分强调了班级分布的调整，同时忽略了学习属性不变的功能。为此，我们提出了一种不变特征学习（IFL）方法，作为GLT的第一个强基线。 IFL首先从不完美的预测中发现具有不同类内分布的环境，然后在其中学习不变的功能。有希望的是，作为改进的功能主链，IFL提高了所有LT阵容：一个/两阶段的重新平衡，增强和合奏。代码和基准可在GitHub上获得：https：//github.com/kaihuatang/generalized-long-tailed-benchmarks.pytorch

Out-of-distribution (OOD) generalisation aims to build a model that can well generalise its learnt knowledge from source domains to an unseen target domain. However, current image classification models often perform poorly in the OOD setting due to statistically spurious correlations learning from model training. From causality-based perspective, we formulate the data generation process in OOD image classification using a causal graph. On this graph, we show that prediction P(Y|X) of a label Y given an image X in statistical learning is formed by both causal effect P(Y|do(X)) and spurious effects caused by confounding features (e.g., background). Since the spurious features are domain-variant, the prediction P(Y|X) becomes unstable on unseen domains. In this paper, we propose to mitigate the spurious effect of confounders using front-door adjustment. In our method, the mediator variable is hypothesized as semantic features that are essential to determine a label for an image. Inspired by capability of style transfer in image generation, we interpret the combination of the mediator variable with different generated images in the front-door formula and propose novel algorithms to estimate it. Extensive experimental results on widely used benchmark datasets verify the effectiveness of our method.

识别面部视频的连续情绪和动作单元（AU）强度需要对表达动态的空间和时间理解。现有作品主要依赖2D面的外观来提取这种动态。这项工作着重于基于参数3D面向形状模型的有希望的替代方案，该模型解散了不同的变异因素，包括表达诱导的形状变化。我们旨在了解与最先进的2D外观模型相比，在估计价值和AU强度方面表现性3D面部形状如何。我们基准了四个最近的3D面对准模型：Expnet，3DDFA-V2，DECA和EMOCA。在价值估计中，3D面模型的表达特征始终超过以前的作品，并在SEWA和AVEC 2019 CES CORPORA上的平均一致性相关性分别为.739和.574。我们还研究了BP4D和DISFA数据集的AU强度估计的3D面形状如何执行，并报告说3D脸部功能在AUS 4、6、10、12和25中与2D外观特征相当，但没有整个集合。 aus。为了理解这种差异，我们在价值和AUS之间进行了对应分析，该分析指出，准确的价值预测可能仅需要少数AU的知识。