我们提出Simprov-可扩展的图像出处框架，将查询图像匹配回到可信的原始数据库，并在查询上确定可能的操作。 Simprov由三个阶段组成：检索Top-K最相似图像的可扩展搜索阶段；一个重新排列和近乎解复的检测阶段，用于识别候选人之间的原件；最后，在查询中定位区域的操纵检测和可视化阶段可能被操纵与原始区域不同。 Simprov对在线再分配过程中通常发生的良性图像转换非常强大，例如由于噪声和重新压缩降解而引起的工件，以及由于图像填充，翘曲，尺寸和形状的变化而引起的过度转换。通过对比较器体系结构中可区分的翘曲模块的端到端训练，可以实现对实地转换的鲁棒性。我们证明了对1亿张图像的数据集的有效检索和操纵检测。

生成对抗网络（GAN）的快速进步为图像归因提出了新的挑战；检测图像是否是合成的，如果是的，则确定哪些GAN体系结构创建了它。独特的是，我们为该任务提供了一个解决方案，能够1）将图像与其语义内容匹配； 2）随着图像在线重新共享时，通常会遇到良性转换（质量，分辨率，形状等）的良好转化。为了使我们的研究形式化，收集了一个具有挑战性的基准归因88，以实现强大而实用的图像归因。然后，我们提出了基于表示混合和新颖损失的GAN指纹技术Repmix。我们验证了其追踪gan生成图像的出处的能力，不变到图像的语义内容以及对扰动的鲁棒性。我们表明，我们的方法从现有的GAN指纹识别方面有显着提高，既有语义泛化和稳健性。数据和代码可从https://github.com/tubui/image_attribution获得。

地理定位的概念是指确定地球上的某些“实体”的位置的过程，通常使用全球定位系统（GPS）坐标。感兴趣的实体可以是图像，图像序列，视频，卫星图像，甚至图像中可见的物体。由于GPS标记媒体的大规模数据集由于智能手机和互联网而迅速变得可用，而深入学习已经上升以提高机器学习模型的性能能力，因此由于其显着影响而出现了视觉和对象地理定位的领域广泛的应用，如增强现实，机器人，自驾驶车辆，道路维护和3D重建。本文提供了对涉及图像的地理定位的全面调查，其涉及从捕获图像（图像地理定位）或图像内的地理定位对象（对象地理定位）的地理定位的综合调查。我们将提供深入的研究，包括流行算法的摘要，对所提出的数据集的描述以及性能结果的分析来说明每个字段的当前状态。

近年来，已经产生了大量的视觉内容，并从许多领域共享，例如社交媒体平台，医学成像和机器人。这种丰富的内容创建和共享引入了新的挑战，特别是在寻找类似内容内容的图像检索（CBIR）-A的数据库中，即长期建立的研究区域，其中需要改进的效率和准确性来实时检索。人工智能在CBIR中取得了进展，并大大促进了实例搜索过程。在本调查中，我们审查了最近基于深度学习算法和技术开发的实例检索工作，通过深网络架构类型，深度功能，功能嵌入方法以及网络微调策略组织了调查。我们的调查考虑了各种各样的最新方法，在那里，我们识别里程碑工作，揭示各种方法之间的联系，并呈现常用的基准，评估结果，共同挑战，并提出未来的未来方向。

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.

In this paper our objectives are, first, networks that can embed audio and visual inputs into a common space that is suitable for cross-modal retrieval; and second, a network that can localize the object that sounds in an image, given the audio signal. We achieve both these objectives by training from unlabelled video using only audio-visual correspondence (AVC) as the objective function. This is a form of crossmodal self-supervision from video. To this end, we design new network architectures that can be trained for cross-modal retrieval and localizing the sound source in an image, by using the AVC task. We make the following contributions: (i) show that audio and visual embeddings can be learnt that enable both within-mode (e.g. audio-to-audio) and between-mode retrieval; (ii) explore various architectures for the AVC task, including those for the visual stream that ingest a single image, or multiple images, or a single image and multi-frame optical flow; (iii) show that the semantic object that sounds within an image can be localized (using only the sound, no motion or flow information); and (iv) give a cautionary tale on how to avoid undesirable shortcuts in the data preparation.

很少有细粒度的分类和人搜索作为独特的任务和文学作品，已经分别对待了它们。但是，仔细观察揭示了重要的相似之处：这两个任务的目标类别只能由特定的对象细节歧视；相关模型应概括为新类别，而在培训期间看不到。我们提出了一个适用于这两个任务的新型统一查询引导网络（QGN）。QGN由一个查询引导的暹罗引文和兴奋子网组成，该子网还重新进行了所有网络层的查询和画廊功能，一个查询实习的区域建议特定于特定于特定的本地化以及查询指导的相似性子网络子网本网络用于公制学习。QGN在最近的一些少数细颗粒数据集上有所改善，在幼崽上的其他技术优于大幅度。QGN还对人搜索Cuhk-Sysu和PRW数据集进行了竞争性执行，我们在其中进行了深入的分析。

Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.

可解释的人工智能（XAI）的新兴领域旨在为当今强大但不透明的深度学习模型带来透明度。尽管本地XAI方法以归因图的形式解释了个体预测，从而确定了重要特征的发生位置（但没有提供有关其代表的信息），但全局解释技术可视化模型通常学会的编码的概念。因此，两种方法仅提供部分见解，并留下将模型推理解释的负担。只有少数当代技术旨在将本地和全球XAI背后的原则结合起来，以获取更多信息的解释。但是，这些方法通常仅限于特定的模型体系结构，或对培训制度或数据和标签可用性施加其他要求，这实际上使事后应用程序成为任意预训练的模型。在这项工作中，我们介绍了概念相关性传播方法（CRP）方法，该方法结合了XAI的本地和全球观点，因此允许回答“何处”和“ where”和“什么”问题，而没有其他约束。我们进一步介绍了相关性最大化的原则，以根据模型对模型的有用性找到代表性的示例。因此，我们提高了对激活最大化及其局限性的共同实践的依赖。我们证明了我们方法在各种环境中的能力，展示了概念相关性传播和相关性最大化导致了更加可解释的解释，并通过概念图表，概念组成分析和概念集合和概念子区和概念子区和概念子集和定量研究对模型的表示和推理提供了深刻的见解。它们在细粒度决策中的作用。

X-ray imaging technology has been used for decades in clinical tasks to reveal the internal condition of different organs, and in recent years, it has become more common in other areas such as industry, security, and geography. The recent development of computer vision and machine learning techniques has also made it easier to automatically process X-ray images and several machine learning-based object (anomaly) detection, classification, and segmentation methods have been recently employed in X-ray image analysis. Due to the high potential of deep learning in related image processing applications, it has been used in most of the studies. This survey reviews the recent research on using computer vision and machine learning for X-ray analysis in industrial production and security applications and covers the applications, techniques, evaluation metrics, datasets, and performance comparison of those techniques on publicly available datasets. We also highlight some drawbacks in the published research and give recommendations for future research in computer vision-based X-ray analysis.

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

实例级图像检索（IIR）或简单的实例检索，涉及在数据集中查找包含查询实例（例如对象）的数据集中所有图像的问题。本文首次尝试使用基于实例歧视的对比学习（CL）解决此问题。尽管CL在许多计算机视觉任务中表现出令人印象深刻的性能，但在IIR领域也从未找到过类似的成功。在这项工作中，我们通过探索从预先训练和微调的CL模型中得出判别表示的能力来解决此问题。首先，我们通过比较预先训练的深度神经网络（DNN）分类器与CL模型学到的功能相比，研究了IIR转移学习的功效。这些发现启发了我们提出了一种新的培训策略，该策略通过使用平均精度（AP）损失以及微调方法来学习针对IIR量身定制的对比功能表示形式，从而优化CL以学习为导向IIR的功能。我们的经验评估表明，从挑战性的牛津和巴黎数据集中的预先培训的DNN分类器中学到的现成的特征上的表现显着提高。

We propose bilinear models, a recognition architecture that consists of two feature extractors whose outputs are multiplied using outer product at each location of the image and pooled to obtain an image descriptor. This architecture can model local pairwise feature interactions in a translationally invariant manner which is particularly useful for fine-grained categorization. It also generalizes various orderless texture descriptors such as the Fisher vector, VLAD and O2P. We present experiments with bilinear models where the feature extractors are based on convolutional neural networks. The bilinear form simplifies gradient computation and allows end-to-end training of both networks using image labels only. Using networks initialized from the ImageNet dataset followed by domain specific fine-tuning we obtain 84.1% accuracy of the CUB-200-2011 dataset requiring only category labels at training time. We present experiments and visualizations that analyze the effects of fine-tuning and the choice two networks on the speed and accuracy of the models. Results show that the architecture compares favorably to the existing state of the art on a number of fine-grained datasets while being substantially simpler and easier to train. Moreover, our most accurate model is fairly efficient running at 8 frames/sec on a NVIDIA Tesla K40 GPU.

即使在几个例子中，人类能够学会识别新物品。相比之下，培训基于深度学习的对象探测器需要大量的注释数据。为避免需求获取和注释这些大量数据，但很少拍摄的对象检测旨在从目标域中的新类别的少数对象实例中学习。在本调查中，我们在几次拍摄对象检测中概述了本领域的状态。我们根据培训方案和建筑布局分类方法。对于每种类型的方法，我们描述了一般的实现以及提高新型类别性能的概念。在适当的情况下，我们在这些概念上给出短暂的外卖，以突出最好的想法。最终，我们介绍了常用的数据集及其评估协议，并分析了报告的基准结果。因此，我们强调了评估中的共同挑战，并确定了这种新兴对象检测领域中最有前景的电流趋势。

Many visualization techniques have been created to help explain the behavior of convolutional neural networks (CNNs), but they largely consist of static diagrams that convey limited information. Interactive visualizations can provide more rich insights and allow users to more easily explore a model's behavior; however, they are typically not easily reusable and are specific to a particular model. We introduce Visual Feature Search, a novel interactive visualization that is generalizable to any CNN and can easily be incorporated into a researcher's workflow. Our tool allows a user to highlight an image region and search for images from a given dataset with the most similar CNN features. It supports searching through large image datasets with an efficient cache-based search implementation. We demonstrate how our tool elucidates different aspects of model behavior by performing experiments on supervised, self-supervised, and human-edited CNNs. We also release a portable Python library and several IPython notebooks to enable researchers to easily use our tool in their own experiments. Our code can be found at https://github.com/lookingglasslab/VisualFeatureSearch.

In this paper we present TruFor, a forensic framework that can be applied to a large variety of image manipulation methods, from classic cheapfakes to more recent manipulations based on deep learning. We rely on the extraction of both high-level and low-level traces through a transformer-based fusion architecture that combines the RGB image and a learned noise-sensitive fingerprint. The latter learns to embed the artifacts related to the camera internal and external processing by training only on real data in a self-supervised manner. Forgeries are detected as deviations from the expected regular pattern that characterizes each pristine image. Looking for anomalies makes the approach able to robustly detect a variety of local manipulations, ensuring generalization. In addition to a pixel-level localization map and a whole-image integrity score, our approach outputs a reliability map that highlights areas where localization predictions may be error-prone. This is particularly important in forensic applications in order to reduce false alarms and allow for a large scale analysis. Extensive experiments on several datasets show that our method is able to reliably detect and localize both cheapfakes and deepfakes manipulations outperforming state-of-the-art works. Code will be publicly available at https://grip-unina.github.io/TruFor/

基于文本的视频细分旨在通过用文本查询指定演员及其表演动作来细分视频序列中的演员。由于\ emph {emph {语义不对称}的问题，以前的方法无法根据演员及其动作以细粒度的方式将视频内容与文本查询对齐。 \ emph {语义不对称}意味着在多模式融合过程中包含不同量的语义信息。为了减轻这个问题，我们提出了一个新颖的演员和动作模块化网络，该网络将演员及其动作分别定位在两个单独的模块中。具体来说，我们首先从视频和文本查询中学习与参与者相关的内容，然后以对称方式匹配它们以定位目标管。目标管包含所需的参与者和动作，然后将其送入完全卷积的网络，以预测演员的分割掩模。我们的方法还建立了对象的关联，使其与所提出的时间建议聚合机制交叉多个框架。这使我们的方法能够有效地细分视频并保持预测的时间一致性。整个模型允许联合学习参与者的匹配和细分，并在A2D句子和J-HMDB句子数据集上实现单帧细分和完整视频细分的最新性能。

我们在没有监督的情况下解决了学习对象探测器的问题。与弱监督的对象检测不同，我们不假设图像级类标签。取而代之的是，我们使用音频组件来“教”对象检测器，从视听数据中提取监督信号。尽管此问题与声音源本地化有关，但它更难，因为检测器必须按类型对对象进行分类，列举对象的每个实例，并且即使对象保持沉默，也可以这样做。我们通过首先设计一个自制的框架来解决这个问题，该框架具有一个对比目标，该目标共同学会了分类和本地化对象。然后，在不使用任何监督的情况下，我们只需使用这些自我监督的标签和盒子来训练基于图像的对象检测器。因此，对于对象检测和声音源定位的任务，我们优于先前的无监督和弱监督的检测器。我们还表明，我们可以将该探测器与每个伪级标签的标签保持一致，并展示我们的方法如何学习检测超出仪器（例如飞机和猫）的通用对象。

本文提出了一种深度学习方法，用于在历史文档的数字收集中进行图像检索和图案斑点。首先，区域建议算法检测文档页面图像中的对象候选。接下来，考虑了两个不同的变体，这些模型用于特征提取，这些变体提供了实用值或二进制代码表示。最后，通过计算给定输入查询的特征相似性来对候选图像进行排名。一项强大的实验协议评估了DOCEXPLORE图像数据库上的每个表示方案（实用值和二进制代码）的建议方法。实验结果表明，所提出的深层模型与历史文档图像的最新图像检索方法相比，使用相同的技术用于模式斑点，优于2.56个百分点。此外，与基于实价表示的相关作品相比，提议的方法还将搜索时间缩短了200倍，并且存储的成本高达6,000倍。

机器学习模型通常会遇到与训练分布不同的样本。无法识别分布（OOD）样本，因此将该样本分配给课堂标签会显着损害模​​型的可靠性。由于其对在开放世界中的安全部署模型的重要性，该问题引起了重大关注。由于对所有可能的未知分布进行建模的棘手性，检测OOD样品是具有挑战性的。迄今为止，一些研究领域解决了检测陌生样本的问题，包括异常检测，新颖性检测，一级学习，开放式识别识别和分布外检测。尽管有相似和共同的概念，但分别分布，开放式检测和异常检测已被独立研究。因此，这些研究途径尚未交叉授粉，创造了研究障碍。尽管某些调查打算概述这些方法，但它们似乎仅关注特定领域，而无需检查不同领域之间的关系。这项调查旨在在确定其共同点的同时，对各个领域的众多著名作品进行跨域和全面的审查。研究人员可以从不同领域的研究进展概述中受益，并协同发展未来的方法。此外，据我们所知，虽然进行异常检测或单级学习进行了调查，但没有关于分布外检测的全面或最新的调查，我们的调查可广泛涵盖。最后，有了统一的跨域视角，我们讨论并阐明了未来的研究线，打算将这些领域更加紧密地融为一体。