Figure 1: FaceForensics++ is a dataset of facial forgeries that enables researchers to train deep-learning-based approaches in a supervised fashion. The dataset contains manipulations created with four state-of-the-art methods, namely, Face2Face, FaceSwap, DeepFakes, and NeuralTextures.
translated by 谷歌翻译
深度学习已成功地用于解决从大数据分析到计算机视觉和人级控制的各种复杂问题。但是,还采用了深度学习进步来创建可能构成隐私,民主和国家安全威胁的软件。最近出现的那些深度学习驱动的应用程序之一是Deepfake。 DeepFake算法可以创建人类无法将它们与真实图像区分开的假图像和视频。因此,可以自动检测和评估数字视觉媒体完整性的技术的建议是必不可少的。本文介绍了一项用于创造深击的算法的调查,更重要的是,提出的方法旨在检测迄今为止文献中的深击。我们对与Deepfake技术有关的挑战,研究趋势和方向进行了广泛的讨论。通过回顾深层味和最先进的深层检测方法的背景,本研究提供了深入的深层技术的概述,并促进了新的,更强大的方法的发展,以应对日益挑战性的深击。
translated by 谷歌翻译
Online media data, in the forms of images and videos, are becoming mainstream communication channels. However, recent advances in deep learning, particularly deep generative models, open the doors for producing perceptually convincing images and videos at a low cost, which not only poses a serious threat to the trustworthiness of digital information but also has severe societal implications. This motivates a growing interest of research in media tampering detection, i.e., using deep learning techniques to examine whether media data have been maliciously manipulated. Depending on the content of the targeted images, media forgery could be divided into image tampering and Deepfake techniques. The former typically moves or erases the visual elements in ordinary images, while the latter manipulates the expressions and even the identity of human faces. Accordingly, the means of defense include image tampering detection and Deepfake detection, which share a wide variety of properties. In this paper, we provide a comprehensive review of the current media tampering detection approaches, and discuss the challenges and trends in this field for future research.
translated by 谷歌翻译
InputOutput Input Output Fig. 1. Unlike current face reenactment approaches that only modify the expression of a target actor in a video, our novel deep video portrait approach enables full control over the target by transferring the rigid head pose, facial expression and eye motion with a high level of photorealism.We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipulations of facial expressions only, we are the irst to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor. The core of our approach is a generative neural network with a novel space-time architecture. The network takes as input synthetic renderings of a parametric face model, based on which it predicts photo-realistic video frames for a given target actor. The realism in this rendering-to-video transfer is achieved by careful adversarial training, and as a result, we can create modiied target videos that mimic the behavior of the synthetically-created input. In order to enable source-to-target video re-animation, we render a synthetic target video with the reconstructed head animation parameters from a source video, and feed it into the trained network ś thus taking full control of the
translated by 谷歌翻译
现在,合成视觉媒体发电和操纵的加速增长已经达到了引起重大关注并对社会造成巨大恐吓的地步。当务之急需要自动检测网络涉及虚假数字内容,并避免危险人造信息的传播以应对这种威胁。在本文中,我们利用和比较了两种手工制作的功能(Sift和Hog)以及两种深层特征(Xpection和CNN+RNN),以进行深层捕获检测任务。当训练集和测试集之间存在不匹配时,我们还会检查这些功能的性能。评估是对著名的FaceForensics ++数据集进行的,该数据集包含四个子数据集,深盘,face2face,faceswap和neuralTextures。最好的结果来自Xception,当训练和测试集都来自同一子数据库时,精度可能会超过99 \%。相比之下,当训练集不匹配测试集时,结果急剧下降。这种现象揭示了创建通用深击检测系统的挑战。
translated by 谷歌翻译
编辑和操纵视频中的面部特征是一种有趣而重要的研究领域,具有夸张的应用,从电影生产和视觉效果到视频游戏和虚拟助手的现实头像。据我们所知,本文提出了第一种在视频中进行了对面部表情的照相型式操纵的方法。我们的方法支持基于神经渲染和基于3D的面部表情建模的语义视频操纵。我们专注于通过改变和控制面部表情来互动操纵视频,实现有前途的光电温度效果。该方法基于用于3D面部形状和活动的脱屑表示和估计,为用户提供对输入视频中的面部表情的直观且易于使用的控制。我们还介绍了一个用户友好的交互式AI工具,该工具处理有关输入视频的特定部分的所需情绪操纵的人类可读的语义标签,并合成光电环境拟人的操纵视频。我们通过将情绪标签映射到价值(VA)值来实现这一点,又通过特别设计和训练的表达式解码器网络映射到解开的3D面部表达式。本文提出了详细的定性和定量实验,展示了我们系统的有效性以及它实现的有希望的结果。可以在补充材料中找到其他结果和视频(https://github.com/girish-03/deepsemmanipulation)。
translated by 谷歌翻译
近年来,视觉伪造达到了人类无法识别欺诈的复杂程度,这对信息安全构成了重大威胁。出现了广泛的恶意申请,例如名人的假新闻,诽谤或勒索,政治战中的政治家冒充,以及谣言的传播吸引观点。结果,已经提出了一种富有的视觉验证技术,以试图阻止这种危险的趋势。在本文中,我们使用全面的和经验方法,提供了一种基准,可以对视觉伪造和视觉取证进行深入的洞察。更具体地,我们开发一个独立的框架,整合最先进的假冒生成器和探测器,并使用各种标准来测量这些技术的性能。我们还对基准测试结果进行了详尽的分析,确定了在措施与对策之间永无止境的战争中的比较参考的方法的特征。
translated by 谷歌翻译
深度神经网络在人类分析中已经普遍存在,增强了应用的性能,例如生物识别识别,动作识别以及人重新识别。但是,此类网络的性能通过可用的培训数据缩放。在人类分析中,对大规模数据集的需求构成了严重的挑战,因为数据收集乏味,廉价,昂贵,并且必须遵守数据保护法。当前的研究研究了\ textit {合成数据}的生成,作为在现场收集真实数据的有效且具有隐私性的替代方案。这项调查介绍了基本定义和方法,在生成和采用合成数据进行人类分析时必不可少。我们进行了一项调查,总结了当前的最新方法以及使用合成数据的主要好处。我们还提供了公开可用的合成数据集和生成模型的概述。最后,我们讨论了该领域的局限性以及开放研究问题。这项调查旨在为人类分析领域的研究人员和从业人员提供。
translated by 谷歌翻译
Deep learning has enabled realistic face manipulation (i.e., deepfake), which poses significant concerns over the integrity of the media in circulation. Most existing deep learning techniques for deepfake detection can achieve promising performance in the intra-dataset evaluation setting (i.e., training and testing on the same dataset), but are unable to perform satisfactorily in the inter-dataset evaluation setting (i.e., training on one dataset and testing on another). Most of the previous methods use the backbone network to extract global features for making predictions and only employ binary supervision (i.e., indicating whether the training instances are fake or authentic) to train the network. Classification merely based on the learning of global features leads often leads to weak generalizability to unseen manipulation methods. In addition, the reconstruction task can improve the learned representations. In this paper, we introduce a novel approach for deepfake detection, which considers the reconstruction and classification tasks simultaneously to address these problems. This method shares the information learned by one task with the other, which focuses on a different aspect other existing works rarely consider and hence boosts the overall performance. In particular, we design a two-branch Convolutional AutoEncoder (CAE), in which the Convolutional Encoder used to compress the feature map into the latent representation is shared by both branches. Then the latent representation of the input data is fed to a simple classifier and the unsupervised reconstruction component simultaneously. Our network is trained end-to-end. Experiments demonstrate that our method achieves state-of-the-art performance on three commonly-used datasets, particularly in the cross-dataset evaluation setting.
translated by 谷歌翻译
在本文中,我们介绍了一种新颖的深入学习方法,用于“野外”视频中演员的情绪状态的光学逼真操纵。所提出的方法基于输入场景中的演员的参数3D面表示,其提供来自头部姿势和面部表达的面部身份的可靠性解剖。然后,它使用新的深度域翻译框架,以符合他们的动态,以一致而合理的方式改变面部表情。最后,改变改变的面部表情用于基于特别设计的神经面渲染器光实际地操纵输入场景中的面部区域。据我们所知,我们的方法是第一个能够通过唯一用作操纵情绪的语义标记来控制演员的面部表情,同时保持与语音相关的唇部运动。我们进行广泛的定性和定量评估和比较,展示了我们的方法的有效性以及我们获得的特别有希望的结果。我们的方法为神经渲染技术的有用应用开辟了一种新的可能性,从电影后生产和视频游戏到照片逼真的情感化身。
translated by 谷歌翻译
在本文中,我们介绍了一条神经渲染管道,用于将一个人在源视频中的面部表情,头部姿势和身体运动转移到目标视频中的另一个人。我们将方法应用于手语视频的具有挑战性的案例:给定手语用户的源视频,我们可以忠实地传输执行的手册(例如握手,棕榈方向,运动,位置)和非手术(例如,眼睛凝视,凝视,面部表情,头部移动)以照片真实的方式标志着目标视频。为了有效捕获上述提示,这些线索对于手语交流至关重要,我们以最近引入的最健壮和最可靠的深度学习方法的有效组合来建立。使用3D感知表示,将身体部位的估计运动组合并重新定位到目标签名者。然后将它们作为我们的视频渲染网络的条件输入,从而生成时间一致和照片现实的视频。我们进行了详细的定性和定量评估和比较,这些评估和比较证明了我们的方法的有效性及其对现有方法的优势。我们的方法产生了前所未有的现实主义的有希望的结果,可用于手语匿名。此外,它很容易适用于重新制定其他类型的全身活动(舞蹈,表演,锻炼等)以及手语生产系统的合成模块。
translated by 谷歌翻译
本文介绍了我们关于使用时间图像进行深泡探测的结果和发现。我们通过使用这些面部地标上的像素值构造图像(称为时间图像),模拟了在给定视频跨帧的468个面部标志物横跨给定视频框架中的临时关系。CNN能够识别给定图像的像素之间存在的空间关系。研究了10种不同的成像网模型。
translated by 谷歌翻译
随着过去五年的快速发展,面部身份验证已成为最普遍的生物识别方法。得益于高准确的识别性能和用户友好的用法,自动面部识别(AFR)已爆炸成多次实用的应用程序,而不是设备解锁,签到和经济支付。尽管面部身份验证取得了巨大的成功,但各种面部表现攻击(FPA),例如印刷攻击,重播攻击和3D面具攻击,但仍引起了不信任的问题。除了身体上的攻击外,面部视频/图像很容易受到恶意黑客发起的各种数字攻击技术的影响,从而对整个公众造成了潜在的威胁。由于无限制地访问了巨大的数字面部图像/视频,并披露了互联网上流通的易于使用的面部操纵工具,因此没有任何先前专业技能的非专家攻击者能够轻松创建精致的假面,从而导致许多危险的应用程序例如财务欺诈,模仿和身份盗用。这项调查旨在通过提供对现有文献的彻底分析并突出需要进一步关注的问题来建立面部取证的完整性。在本文中,我们首先全面调查了物理和数字面部攻击类型和数据集。然后,我们回顾了现有的反攻击方法的最新和最先进的进度,并突出显示其当前限制。此外,我们概述了面对法医社区中现有和即将面临的挑战的未来研究指示。最后,已经讨论了联合物理和数字面部攻击检​​测的必要性,这在先前的调查中从未进行过研究。
translated by 谷歌翻译
我们提出了自由式 - 人体神经通话的头部合成系统。我们表明,具有稀疏3D面部标志的建模面孔足以实现最先进的生成性能,而无需依赖诸如3D可变形模型之类的强统计学先验。除了3D姿势和面部表情外,我们的方法还能够将目光从驾驶演员转移到源身份。我们的完整管道由三个组件组成:一个规范的3D密钥估计器,可回归3D姿势和与表达相关的变形,凝视估计网络和建立在Headgan架构上的生成器。我们进一步实验发电机的扩展,以使用注意机制可容纳几次学习,以防万一可用多个源图像。与最新的重演和运动转移模型相比,我们的系统实现了更高的照片真实性与优越的身份保护,同时提供明确的注视控制。
translated by 谷歌翻译
Figure 1: We present an image synthesis approach that learns object-speci c neural textures which can be interpreted by a neural renderer. Our approach can be trained end-to-end with real data, allowing us to re-synthesize novel views of static objects, edit scenes, as well as re-render dynamic animated surfaces.
translated by 谷歌翻译
照片逼真的面部视频肖像重演益处虚拟生产和众多VR / AR经验。由于肖像应该保持高现实主义和与目标环境的一致性,任务仍然具有挑战性。在本文中,我们介绍了一种可靠的神经视频肖像,同步的致密和再生方案,其将头部姿势和面部表达从源actor传送到具有任意新的背景和照明条件的目标演员的肖像视频。我们的方法结合了4D反射场学习,基于模型的面部性能捕获和目标感知神经渲染。具体地,我们采用渲染到视频翻译网络首先从混合面部性能捕获结果中合成高质量的OLAT镜片和alpha锍。然后,我们设计了一个语义感知的面部归一化方案,以实现可靠的显式控制以及多帧多任务学习策略,以同时编码内容,分割和时间信息以获得高质量的反射场推断。在培训之后,我们的方法进一步实现了目标表演者的照片现实和可控的视频肖像编辑。通过将相同的混合面部捕获和归一化方案应用于源视频输入,可以获得可靠的面部姿势和表达编辑,而我们的显式alpha和Olat输出使高质量的依据和背景编辑能够实现。凭借实现同步致密和再生的能力,我们能够改善各种虚拟生产和视频重写应用程序的现实主义。
translated by 谷歌翻译
The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of (i) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and (ii) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given.
translated by 谷歌翻译
强大的深度学习技术的发展为社会和个人带来了一些负面影响。一个这样的问题是假媒体的出现。为了解决这个问题,我们组织了可信赖的媒体挑战(TMC)来探讨人工智能(AI)如何利用如何打击假媒体。我们与挑战一起发布了一个挑战数据集,由4,380张假和2,563个真实视频组成。所有这些视频都伴随着Audios,采用不同的视频和/或音频操作方法来生产不同类型的假媒体。数据集中的视频具有各种持续时间,背景,照明,最小分辨率为360p,并且可能包含模拟传输误差和不良压缩的扰动。我们还开展了用户学习,以展示所作数据集的质量。结果表明,我们的数据集具有有希望的质量,可以在许多情况下欺骗人类参与者。
translated by 谷歌翻译
Video synthesis methods rapidly improved in recent years, allowing easy creation of synthetic humans. This poses a problem, especially in the era of social media, as synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. Thus, there is a pressing need for accurate and robust deepfake detection methods, that can detect forgery techniques not seen during training. In this work, we explore whether this can be done by leveraging a multi-modal, out-of-domain backbone trained in a self-supervised manner, adapted to the video deepfake domain. We propose FakeOut; a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. We demonstrate the efficacy and robustness of FakeOut in detecting various types of deepfakes, especially manipulations which were not seen during training. Our method achieves state-of-the-art results in cross-manipulation and cross-dataset generalization. This study shows that, perhaps surprisingly, training on out-of-domain videos (i.e., videos with no speaking humans), can lead to better deepfake detection systems. Code is available on GitHub.
translated by 谷歌翻译
最近,面部生物识别是对传统认证系统的方便替代的巨大关注。因此,检测恶意尝试已经发现具有重要意义,导致面部抗欺骗〜(FAS),即面部呈现攻击检测。与手工制作的功能相反,深度特色学习和技术已经承诺急剧增加FAS系统的准确性,解决了实现这种系统的真实应用的关键挑战。因此,处理更广泛的发展以及准确的模型的新研究区越来越多地引起了研究界和行业的关注。在本文中,我们为自2017年以来对与基于深度特征的FAS方法相关的文献综合调查。在这一主题上阐明,基于各种特征和学习方法的语义分类。此外,我们以时间顺序排列,其进化进展和评估标准(数据集内集和数据集互联集合中集)覆盖了FAS的主要公共数据集。最后,我们讨论了开放的研究挑战和未来方向。
translated by 谷歌翻译