光保护综合技术的快速进展达到了真实和操纵图像之间的边界开始模糊的临界点。最近,一个由Mega-Scale Deep Face Forgery DataSet,由290万个图像组成和221,247个视频的伪造网络已被释放。它是迄今为止的数据规模,操纵(7个图像级别方法,8个视频级别方法),扰动(36个独立和更混合的扰动)和注释(630万个分类标签,290万操纵区域注释和221,247个时间伪造段标签)。本文报告了Forgerynet-Face Forgery Analysis挑战2021的方法和结果,它采用了伪造的基准。模型评估在私人测试集上执行离线。共有186名参加比赛的参与者,11名队伍提交了有效的提交。我们将分析排名排名的解决方案,并展示一些关于未来工作方向的讨论。
translated by 谷歌翻译
The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems. To accelerate the research on AI security, the Artificial Intelligence Security Competition (AISC) was organized by the Zhongguancun Laboratory, China Industrial Control Systems Cyber Emergency Response Team, Institute for Artificial Intelligence, Tsinghua University, and RealAI as part of the Zhongguancun International Frontier Technology Innovation Competition (https://www.zgc-aisc.com/en). The competition consists of three tracks, including Deepfake Security Competition, Autonomous Driving Security Competition, and Face Recognition Security Competition. This report will introduce the competition rules of these three tracks and the solutions of top-ranking teams in each track.
translated by 谷歌翻译
在本文中,我们提出了与IEEE计算机协会在CVPR 2022上同时与IEEE计算机协会研讨会同时举行的多手术检测挑战。我们的多手术检测挑战旨在检测自动图像操作,包括但不限于图像编辑,图像合成,图像合成,图像,图像,图像,图像合成,图像,图像编辑一代,图像Photoshop等。我们的挑战吸引了来自世界各地的674支团队,约有2000个有效的结果提交数量。我们邀请了前十支球队为挑战提供解决方案,其中三支球队在大结局中获得了奖项。在本文中,我们介绍了前三名团队的解决方案,以增强图像伪造检测领域的研究工作。
translated by 谷歌翻译
面部演示攻击检测(PAD)由于欺骗欺骗性被广泛认可的脆弱性而受到越来越长。在2011年,2013年,2017年,2019年,2020年和2021年与主要生物识别和计算机视觉会议结合的八个国际竞赛中,在八个国际竞赛中评估了一系列国际竞争中的八种国际竞争中的艺术状态。研究界。在本章中,我们介绍了2019年的五个最新竞赛的设计和结果直到2021年。前两项挑战旨在评估近红外(NIR)和深度方式的多模态设置中面板的有效性。彩色相机数据,而最新的三个竞争专注于评估在传统彩色图像和视频上运行的面部垫算法的域和攻击型泛化能力。我们还讨论了从竞争中吸取的经验教训以及领域的未来挑战。
translated by 谷歌翻译
AI的最新进展,尤其是深度学习,导致创建新的现实合成媒体(视频,图像和音频)以及对现有媒体的操纵的创建显着增加,这导致了新术语的创建。 'deepfake'。基于英语和中文中的研究文献和资源,本文对Deepfake进行了全面的概述,涵盖了这一新兴概念的多个重要方面,包括1)不同的定义,2)常用的性能指标和标准以及3)与DeepFake相关的数据集,挑战,比赛和基准。此外,该论文还报告了2020年和2021年发表的12条与DeepFake相关的调查论文的元评估,不仅关注上述方面,而且集中在对关键挑战和建议的分析上。我们认为,就涵盖的各个方面而言,本文是对深层的最全面评论,也是第一个涵盖英语和中国文学和资源的文章。
translated by 谷歌翻译
Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of <instrument, verb, target> combination delivers comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms by competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.
translated by 谷歌翻译
面部伪造技术的最新进展几乎可以产生视觉上无法追踪的深冰录视频,这些视频可以通过恶意意图来利用。结果,研究人员致力于深泡检测。先前的研究已经确定了局部低级提示和时间信息在追求跨层次方法中概括的重要性,但是,它们仍然遭受鲁棒性问题的影响。在这项工作中,我们提出了基于本地和时间感知的变压器的DeepFake检测(LTTD)框架,该框架采用了局部到全球学习协议,特别关注本地序列中有价值的时间信息。具体而言,我们提出了一个局部序列变压器(LST),该局部序列变压器(LST)对限制空间区域的序列进行了时间一致性,其中低级信息通过学习的3D滤波器的浅层层增强。基于局部时间嵌入,我们然后以全球对比的方式实现最终分类。对流行数据集进行的广泛实验验证了我们的方法有效地发现了本地伪造线索并实现最先进的表现。
translated by 谷歌翻译
The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.
translated by 谷歌翻译
In this paper, we introduce MINTIME, a video deepfake detection approach that captures spatial and temporal anomalies and handles instances of multiple people in the same video and variations in face sizes. Previous approaches disregard such information either by using simple a-posteriori aggregation schemes, i.e., average or max operation, or using only one identity for the inference, i.e., the largest one. On the contrary, the proposed approach builds on a Spatio-Temporal TimeSformer combined with a Convolutional Neural Network backbone to capture spatio-temporal anomalies from the face sequences of multiple identities depicted in a video. This is achieved through an Identity-aware Attention mechanism that attends to each face sequence independently based on a masking operation and facilitates video-level aggregation. In addition, two novel embeddings are employed: (i) the Temporal Coherent Positional Embedding that encodes each face sequence's temporal information and (ii) the Size Embedding that encodes the size of the faces as a ratio to the video frame size. These extensions allow our system to adapt particularly well in the wild by learning how to aggregate information of multiple identities, which is usually disregarded by other methods in the literature. It achieves state-of-the-art results on the ForgeryNet dataset with an improvement of up to 14% AUC in videos containing multiple people and demonstrates ample generalization capabilities in cross-forgery and cross-dataset settings. The code is publicly available at https://github.com/davide-coccomini/MINTIME-Multi-Identity-size-iNvariant-TIMEsformer-for-Video-Deepfake-Detection.
translated by 谷歌翻译
最近,由于社交媒体数字取证中的安全性和隐私问题,DeepFake引起了广泛的公众关注。随着互联网上广泛传播的深层视频变得越来越现实,传统的检测技术未能区分真实和假货。大多数现有的深度学习方法主要集中于使用卷积神经网络作为骨干的局部特征和面部图像中的关系。但是,本地特征和关系不足以用于模型培训,无法学习足够的一般信息以进行深层检测。因此,现有的DeepFake检测方法已达到瓶颈,以进一步改善检测性能。为了解决这个问题,我们提出了一个深度卷积变压器,以在本地和全球范围内纳入决定性图像。具体而言,我们应用卷积池和重新注意事项来丰富提取的特征并增强功效。此外,我们在模型训练中采用了几乎没有讨论的图像关键框架来改进性能,并可视化由视频压缩引起的密钥和正常图像帧之间的特征数量差距。我们最终通过在几个DeepFake基准数据集上进行了广泛的实验来说明可传递性。所提出的解决方案在内部和跨数据库实验上始终优于几个最先进的基线。
translated by 谷歌翻译
本文介绍了我们DFGC 2022竞赛的摘要报告。深层味道正在迅速发展,现实的面部折叠变得越来越欺骗性和难以检测。相反,检测深击的方法也正在改善。 Deepfake创作者和防守者之间有两党的比赛。这项竞赛提供了一个通用平台,用于基准在DeepFake创建和检测方法中当前最新的游戏之间的游戏。这场比赛要回答的主要研究问题是彼此竞争时两个对手的现状。这是去年DFGC 2021之后的第二版,具有新的,更多样化的视频数据集,更现实的游戏设置以及更合理的评估指标。通过这项竞争,我们旨在激发研究思想,以建立对深层威胁的更好的防御能力。我们还发布了我们的参与者和我们自己的DFGC 2022数据集,以丰富研究社区的DeepFake数据资源(https://github.com/nice-x/dfgc-2022)。
translated by 谷歌翻译
尽管深度学习已被广​​泛用于视频分析,例如视频分类和动作检测,但与体育视频的快速移动主题进行密集的动作检测仍然具有挑战性。在这项工作中,我们发布了另一个体育视频数据集$ \ textbf {p $^2 $ a} $ for $ \ usewessline {p} $ \ in $ \ usepline {p} $ ong- $ \ $ \ usepline {a} $ ction ction ction检测,由2,721个视频片段组成,这些视频片段从世界乒乓球锦标赛和奥林匹克运动会的专业乒乓球比赛的广播视频中收集。我们与一批乒乓球专业人士和裁判员合作,以获取出现在数据集中的每个乒乓球动作,并提出两组动作检测问题 - 行动定位和行动识别。我们使用$ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ fextbf {p $^2 $^2 $^2 $ a^2 $^2 $ a^2 $^2 $ a^2 $ a^2 $^$^2 $ a^2 $^2 $ a^2 $^2 $ a^2 $^2 $ a^2 $^2 $^2 $ a^2 $^2 $ a^2 $^2 $^2 $^2 $^2 $^2 $^2 $ a在各种设置下,这两个问题的$} $。这些模型只能在AR-AN曲线下实现48%的面积,以进行本地化,而识别次数为82%,因为Ping-Pong的动作密集具有快速移动的主题,但广播视频仅为25 fps。结果证实,$ \ textbf {p $^2 $ a} $仍然是一项具有挑战性的任务,可以用作视频中动作检测的基准。
translated by 谷歌翻译
乳腺癌是女性最常见的恶性肿瘤,每年负责超过50万人死亡。因此,早期和准确的诊断至关重要。人类专业知识是诊断和正确分类乳腺癌并定义适当的治疗,这取决于评价不同生物标志物如跨膜蛋白受体HER2的表达。该评估需要几个步骤,包括免疫组织化学或原位杂交等特殊技术,以评估HER2状态。通过降低诊断中的步骤和人类偏差的次数的目标,赫洛挑战是组织的,作为第16届欧洲数字病理大会的并行事件,旨在自动化仅基于苏木精和曙红染色的HER2地位的评估侵袭性乳腺癌的组织样本。评估HER2状态的方法是在全球21个团队中提出的,并通过一些提议的方法实现了潜在的观点,以推进最先进的。
translated by 谷歌翻译
随着生成模型的快速发展,基于AI的面部操纵技术,称为DeepFakes,已经变得越来越真实。这种脸部伪造的方法可以攻击任何目标,这对个人隐私和财产安全构成了新的威胁。此外,滥用合成视频在许多领域都显示出潜在的危险,例如身份骚扰,色情和新闻谣言。受到生理信号中的空间相干性和时间一致性在所生物的内容中被破坏的事实,我们试图找到可以区分真实视频和合成视频的不一致模式,从面部像素的变化是与生理信息高度相关的。我们的方法首先将多个高斯级别的eulerian视频放大倍数(EVM)应用于原始视频,以扩大面部血容量的变化引起的生理变化,然后将原始视频和放大的视频转换为多尺度欧拉宽度的空间 - 时间地图(MemstMap),其可以代表不同八度的时变的生理增强序列。然后,这些地图以列为单位重新装入帧修补程序,并发送到视觉变压器以学习帧级别的时空描述符。最后,我们整理了嵌入功能并输出判断视频是真实还是假的概率。我们在面部框架++和DeepFake检测数据集上验证了我们的方法。结果表明,我们的模型在伪造检测中实现了出色的性能,并在交叉数据域中显示出出色的泛化能力。
translated by 谷歌翻译
Temporal action detection (TAD) is extensively studied in the video understanding community by generally following the object detection pipeline in images. However, complex designs are not uncommon in TAD, such as two-stream feature extraction, multi-stage training, complex temporal modeling, and global context fusion. In this paper, we do not aim to introduce any novel technique for TAD. Instead, we study a simple, straightforward, yet must-known baseline given the current status of complex design and low detection efficiency in TAD. In our simple baseline (termed BasicTAD), we decompose the TAD pipeline into several essential components: data sampling, backbone design, neck construction, and detection head. We extensively investigate the existing techniques in each component for this baseline, and more importantly, perform end-to-end training over the entire pipeline thanks to the simplicity of design. As a result, this simple BasicTAD yields an astounding and real-time RGB-Only baseline very close to the state-of-the-art methods with two-stream inputs. In addition, we further improve the BasicTAD by preserving more temporal and spatial information in network representation (termed as PlusTAD). Empirical results demonstrate that our PlusTAD is very efficient and significantly outperforms the previous methods on the datasets of THUMOS14 and FineAction. Meanwhile, we also perform in-depth visualization and error analysis on our proposed method and try to provide more insights on the TAD problem. Our approach can serve as a strong baseline for future TAD research. The code and model will be released at https://github.com/MCG-NJU/BasicTAD.
translated by 谷歌翻译
随着面部伪造技术的快速发展,DeepFake视频在数字媒体上引起了广泛的关注。肇事者大量利用这些视频来传播虚假信息并发表误导性陈述。大多数现有的DeepFake检测方法主要集中于纹理特征,纹理特征可能会受到外部波动(例如照明和噪声)的影响。此外,基于面部地标的检测方法对外部变量更强大,但缺乏足够的细节。因此,如何在空间,时间和频域中有效地挖掘独特的特征,并将其与面部地标融合以进行伪造视频检测仍然是一个悬而未决的问题。为此,我们提出了一个基于多种模式的信息和面部地标的几何特征,提出了地标增强的多模式图神经网络(LEM-GNN)。具体而言,在框架级别上,我们设计了一种融合机制来挖掘空间和频域元素的联合表示,同时引入几何面部特征以增强模型的鲁棒性。在视频级别,我们首先将视频中的每个帧视为图中的节点,然后将时间信息编码到图表的边缘。然后,通过应用图形神经网络(GNN)的消息传递机制,将有效合并多模式特征,以获得视频伪造的全面表示。广泛的实验表明,我们的方法始终优于广泛使用的基准上的最先进(SOTA)。
translated by 谷歌翻译
Face manipulation technology is advancing very rapidly, and new methods are being proposed day by day. The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. Our key insight is that each person has specific biometric characteristics that a synthetic generator cannot likely reproduce. Accordingly, we extract high-level audio-visual biometric features which characterize the identity of a person, and use them to create a person-of-interest (POI) deepfake detector. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. As a result, when the video and/or audio of a person is manipulated, its representation in the embedding space becomes inconsistent with the real identity, allowing reliable detection. Training is carried out exclusively on real talking-face videos, thus the detector does not depend on any specific manipulation method and yields the highest generalization ability. In addition, our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos by building only on high-level semantic features. Experiments on a wide variety of datasets confirm that our method ensures a SOTA performance, with an average improvement in terms of AUC of around 3%, 10%, and 4% for high-quality, low quality, and attacked videos, respectively. https://github.com/grip-unina/poi-forensics
translated by 谷歌翻译
X-ray imaging technology has been used for decades in clinical tasks to reveal the internal condition of different organs, and in recent years, it has become more common in other areas such as industry, security, and geography. The recent development of computer vision and machine learning techniques has also made it easier to automatically process X-ray images and several machine learning-based object (anomaly) detection, classification, and segmentation methods have been recently employed in X-ray image analysis. Due to the high potential of deep learning in related image processing applications, it has been used in most of the studies. This survey reviews the recent research on using computer vision and machine learning for X-ray analysis in industrial production and security applications and covers the applications, techniques, evaluation metrics, datasets, and performance comparison of those techniques on publicly available datasets. We also highlight some drawbacks in the published research and give recommendations for future research in computer vision-based X-ray analysis.
translated by 谷歌翻译
胎儿镜检查激光​​光凝是一种广泛采用的方法,用于治疗双胞胎输血综合征(TTTS)。该过程涉及光凝病理吻合术以调节双胞胎之间的血液交换。由于观点有限,胎儿镜的可操作性差,可见性差和照明的可变性,因此该程序尤其具有挑战性。这些挑战可能导致手术时间增加和消融不完全。计算机辅助干预措施(CAI)可以通过识别场景中的关键结构并通过视频马赛克来扩展胎儿镜观景领域,从而为外科医生提供决策支持和背景意识。由于缺乏设计,开发和测试CAI算法的高质量数据,该领域的研究受到了阻碍。通过作为MICCAI2021内窥镜视觉挑战组织的胎儿镜胎盘胎盘分割和注册(FETREG2021)挑战,我们发布了第一个Largescale Multencentre TTTS数据集,用于开发广义和可靠的语义分割和视频摩擦质量algorithms。对于这一挑战,我们发布了一个2060张图像的数据集,该数据集是从18个体内TTTS胎儿镜检查程序和18个简短视频剪辑的船只,工具,胎儿和背景类别的像素通道。七个团队参与了这一挑战,他们的模型性能在一个看不见的测试数据集中评估了658个从6个胎儿镜程序和6个短剪辑的图像的图像。这项挑战为创建通用解决方案提供了用于胎儿镜面场景的理解和摩西式解决方案的机会。在本文中,我们介绍了FETREG2021挑战的发现,以及报告TTTS胎儿镜检查中CAI的详细文献综述。通过这一挑战,它的分析和多中心胎儿镜数据的发布,我们为该领域的未来研究提供了基准。
translated by 谷歌翻译
尽管令人鼓舞的是深泡检测的进展,但由于训练过程中探索的伪造线索有限,对未见伪造类型的概括仍然是一个重大挑战。相比之下,我们注意到Deepfake中的一种常见现象:虚假的视频创建不可避免地破坏了原始视频中的统计规律性。受到这一观察的启发,我们建议通过区分实际视频中没有出现的“规律性中断”来增强深层检测的概括。具体而言,通过仔细检查空间和时间属性,我们建议通过伪捕获生成器破坏真实的视频,并创建各种伪造视频以供培训。这种做法使我们能够在不使用虚假视频的情况下实现深泡沫检测,并以简单有效的方式提高概括能力。为了共同捕获空间和时间上的破坏,我们提出了一个时空增强块,以了解我们自我创建的视频之间的规律性破坏。通过全面的实验,我们的方法在几个数据集上表现出色。
translated by 谷歌翻译