在这项工作中，我们提出了一个具有结构性图形的新型不确定性感知对象检测框架，其中节点和边缘分别用对象及其空间语义相似性表示。具体而言，我们旨在考虑对象之间的关系，以有效地将它们背景化。为了实现这一目标，我们首先检测对象，然后测量其语义和空间距离以构建对象图，然后由图形神经网络（GNN）表示，用于完善对象的视觉CNN特征。但是，精炼CNN功能和每个对象的检测结果效率低下，可能不需要，因为其中包括不确定性低的正确预测。因此，我们建议通过将表示形式从某些对象（源）转移到有向图上的不确定对象（目标）来处理不确定的对象，而且还仅在对象上改善CNN功能，因为对象被认为是不确定的，其代表性输出来自GNN。此外，我们通过在不确定的物体上给予更大的权重来计算训练损失，以专注于改善不确定的对象预测，同时保持某些对象的高性能。我们将模型称为对象检测（UAGDET）的不确定性感知图网络。然后，我们在实验中验证了我们的大规模空中图像数据集，即DOTA，该数据集由大量对象组成，这些对象在图像中具有很小至大的对象，在该图像上，我们的对象可以改善现有对象检测网络的性能。

We propose a novel scene graph generation model called Graph R-CNN, that is both effective and efficient at detecting objects and their relations in images. Our model contains a Relation Proposal Network (RePN) that efficiently deals with the quadratic number of potential relations between objects in an image. We also propose an attentional Graph Convolutional Network (aGCN) that effectively captures contextual information between objects and relations. Finally, we introduce a new evaluation metric that is more holistic and realistic than existing metrics. We report state-of-the-art performance on scene graph generation as evaluated using both existing and our proposed metrics.

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

The goal of this paper is to detect objects by exploiting their interrelationships. Rather than relying on predefined and labeled graph structures, we infer a graph prior from object co-occurrence statistics. The key idea of our paper is to model object relations as a function of initial class predictions and co-occurrence priors to generate a graph representation of an image for improved classification and bounding box regression. We additionally learn the object-relation joint distribution via energy based modeling. Sampling from this distribution generates a refined graph representation of the image which in turn produces improved detection performance. Experiments on the Visual Genome and MS-COCO datasets demonstrate our method is detector agnostic, end-to-end trainable, and especially beneficial for rare object classes. What is more, we establish a consistent improvement over object detectors like DETR and Faster-RCNN, as well as state-of-the-art methods modeling object interrelationships.

场景图是一个场景的结构化表示，可以清楚地表达场景中对象之间的对象，属性和关系。随着计算机视觉技术继续发展，只需检测和识别图像中的对象，人们不再满足。相反，人们期待着对视觉场景更高的理解和推理。例如，给定图像，我们希望不仅检测和识别图像中的对象，还要知道对象之间的关系（视觉关系检测），并基于图像内容生成文本描述（图像标题）。或者，我们可能希望机器告诉我们图像中的小女孩正在做什么（视觉问题应答（VQA）），甚至从图像中移除狗并找到类似的图像（图像编辑和检索）等。这些任务需要更高水平的图像视觉任务的理解和推理。场景图只是场景理解的强大工具。因此，场景图引起了大量研究人员的注意力，相关的研究往往是跨模型，复杂，快速发展的。然而，目前没有对场景图的相对系统的调查。为此，本调查对现行场景图研究进行了全面调查。更具体地说，我们首先总结了场景图的一般定义，随后对场景图（SGG）和SGG的发电方法进行了全面和系统的讨论，借助于先验知识。然后，我们调查了场景图的主要应用，并汇总了最常用的数据集。最后，我们对场景图的未来发展提供了一些见解。我们相信这将是未来研究场景图的一个非常有帮助的基础。

在对象检测中，广泛采用了非最大抑制（NMS）方法以删除检测到的密集盒的水平重复，以生成最终的对象实例。但是，由于密集检测框的质量降低，而不是对上下文信息的明确探索，因此通过简单的交叉联盟（IOU）指标的现有NMS方法往往在多面向和长尺寸的对象检测方面表现不佳。通过重复删除与常规NMS方法区分，我们提出了一个新的图形融合网络，称为GFNET，用于多个方向的对象检测。我们的GFNET是可扩展的和适应性熔断的密集检测框，可检测更准确和整体的多个方向对象实例。具体而言，我们首先采用一种局部意识的聚类算法将密集检测框分组为不同的簇。我们将为属于一个集群的检测框构建一个实例子图。然后，我们通过图形卷积网络（GCN）提出一个基于图的融合网络，以学习推理并融合用于生成最终实例框的检测框。在公共可用多面向文本数据集（包括MSRA-TD500，ICDAR2015，ICDAR2017-MLT）和多方向对象数据集（DOTA）上进行广泛实验。

深度学习技术导致了通用对象检测领域的显着突破，近年来产生了很多场景理解的任务。由于其强大的语义表示和应用于场景理解，场景图一直是研究的焦点。场景图生成（SGG）是指自动将图像映射到语义结构场景图中的任务，这需要正确标记检测到的对象及其关系。虽然这是一项具有挑战性的任务，但社区已经提出了许多SGG方法并取得了良好的效果。在本文中，我们对深度学习技术带来了近期成就的全面调查。我们审查了138个代表作品，涵盖了不同的输入方式，并系统地将现有的基于图像的SGG方法从特征提取和融合的角度进行了综述。我们试图通过全面的方式对现有的视觉关系检测方法进行连接和系统化现有的视觉关系检测方法，概述和解释SGG的机制和策略。最后，我们通过深入讨论当前存在的问题和未来的研究方向来完成这项调查。本调查将帮助读者更好地了解当前的研究状况和想法。

几何深度学习最近对包括文档分析在内的广泛的机器学习领域引起了极大的兴趣。图形神经网络（GNN）的应用在各种与文档有关的任务中变得至关重要，因为它们可以揭示重要的结构模式，这是关键信息提取过程的基础。文献中的先前作品提出了任务驱动的模型，并且没有考虑到图形的全部功能。我们建议Doc2Graph是一种基于GNN模型的任务无关文档理解框架，以解决给定不同类型文档的不同任务。我们在两个具有挑战性的数据集上评估了我们的方法，以在形式理解，发票布局分析和表检测中进行关键信息提取。我们的代码可以在https://github.com/andreagemelli/doc2graph上自由访问。

Single-frame InfraRed Small Target (SIRST) detection has been a challenging task due to a lack of inherent characteristics, imprecise bounding box regression, a scarcity of real-world datasets, and sensitive localization evaluation. In this paper, we propose a comprehensive solution to these challenges. First, we find that the existing anchor-free label assignment method is prone to mislabeling small targets as background, leading to their omission by detectors. To overcome this issue, we propose an all-scale pseudo-box-based label assignment scheme that relaxes the constraints on scale and decouples the spatial assignment from the size of the ground-truth target. Second, motivated by the structured prior of feature pyramids, we introduce the one-stage cascade refinement network (OSCAR), which uses the high-level head as soft proposals for the low-level refinement head. This allows OSCAR to process the same target in a cascade coarse-to-fine manner. Finally, we present a new research benchmark for infrared small target detection, consisting of the SIRST-V2 dataset of real-world, high-resolution single-frame targets, the normalized contrast evaluation metric, and the DeepInfrared toolkit for detection. We conduct extensive ablation studies to evaluate the components of OSCAR and compare its performance to state-of-the-art model-driven and data-driven methods on the SIRST-V2 benchmark. Our results demonstrate that a top-down cascade refinement framework can improve the accuracy of infrared small target detection without sacrificing efficiency. The DeepInfrared toolkit, dataset, and trained models are available at https://github.com/YimianDai/open-deepinfrared to advance further research in this field.

在计算机视觉中长期以来一直研究了时间行动定位。现有的最先进的动作定位方法将每个视频划分为多个动作单位（即，在一级方法中的两级方法和段中的提案），然后单独地对每个视频进行操作，而不明确利用他们在学习期间的关系。在本文中，我们声称，动作单位之间的关系在行动定位中发挥着重要作用，并且更强大的动作探测器不仅应捕获每个动作单元的本地内容，还应允许更广泛的视野与相关的上下文它。为此，我们提出了一般图表卷积模块（GCM），可以轻松插入现有的动作本地化方法，包括两阶段和单级范式。具体而言，我们首先构造一个图形，其中每个动作单元被表示为节点，并且两个动作单元之间作为边缘之间的关系。在这里，我们使用两种类型的关系，一个类型的关系，用于捕获不同动作单位之间的时间连接，另一类是用于表征其语义关系的另一个关系。特别是对于两级方法中的时间连接，我们进一步探索了两种不同的边缘，一个连接重叠动作单元和连接周围但脱节的单元的另一个。在我们构建的图表上，我们将图形卷积网络（GCNS）应用于模拟不同动作单位之间的关系，这能够了解更有信息的表示来增强动作本地化。实验结果表明，我们的GCM始终如一地提高了现有行动定位方法的性能，包括两阶段方法（例如，CBR和R-C3D）和一级方法（例如，D-SSAD），验证我们的一般性和有效性GCM。

在本文中，我们考虑一种用于图像的不同数据格式：矢量图形。与广泛用于图像识别的光栅图形相比，由于文档中的基元的分析表示，矢量图形可以向上或向下缩放或向下扩展到任何分辨率而不进行别名或信息丢失的分辨率。此外，向量图形能够提供有关低级别元素组如何一起形成高级形状或结构的额外结构信息。图形矢量的这些优点尚未完全利用现有方法。要探索此数据格式，我们针对基本识别任务：对象本地化和分类。我们提出了一个有效的无CNN的管道，不会将图形呈现为像素（即光栅化），并将向量图形的文本文档作为输入，称为Yolat（您只查看文本）。 Yolat构建多图来模拟矢量图形中的结构和空间信息，并提出了双流图形神经网络来检测图表中的对象。我们的实验表明，通过直接在向量图形上运行，在平均精度和效率方面，Yolat Out-ut-Proped基于的物体检测基线。

少量对象检测（FSOD）旨在使用少数示例来检测从未见过的对象。通过学习如何在查询图像和少量拍摄类示例之间进行匹配，因此可以通过学习如何匹配来实现最近的改进，使得学习模型可以概括为几滴新颖的类。然而，目前，大多数基于元学习的方法分别在查询图像区域（通常是提议）和新颖类之间执行成对匹配，因此无法考虑它们之间的多个关系。在本文中，我们使用异构图卷积网络提出了一种新颖的FSOD模型。通过具有三种不同类型的边缘的所有提议和类节点之间的有效消息，我们可以获得每个类的上下文感知提案功能和查询 - 自适应，多包子增强型原型表示，这可能有助于促进成对匹配和改进的最终决赛FSOD精度。广泛的实验结果表明，我们所提出的模型表示为QA的Qa-Netwet，优于不同拍摄和评估指标下的Pascal VOC和MSCOCO FSOD基准测试的当前最先进的方法。

Assessing the critical view of safety in laparoscopic cholecystectomy requires accurate identification and localization of key anatomical structures, reasoning about their geometric relationships to one another, and determining the quality of their exposure. In this work, we propose to capture each of these aspects by modeling the surgical scene with a disentangled latent scene graph representation, which we can then process using a graph neural network. Unlike previous approaches using graph representations, we explicitly encode in our graphs semantic information such as object locations and shapes, class probabilities and visual features. We also incorporate an auxiliary image reconstruction objective to help train the latent graph representations. We demonstrate the value of these components through comprehensive ablation studies and achieve state-of-the-art results for critical view of safety prediction across multiple experimental settings.

我们介绍了一种名为RobustAbnet的新表检测和结构识别方法，以检测表的边界并从异质文档图像中重建每个表的细胞结构。为了进行表检测，我们建议将Cornernet用作新的区域建议网络来生成更高质量的表建议，以更快的R-CNN，这显着提高了更快的R-CNN的定位准确性以进行表检测。因此，我们的表检测方法仅使用轻巧的RESNET-18骨干网络，在三个公共表检测基准（即CTDAR TRACKA，PUBLAYNET和IIIT-AR-13K）上实现最新性能。此外，我们提出了一种新的基于分裂和合并的表结构识别方法，其中提出了一个新型的基于CNN的新空间CNN分离线预测模块将每个检测到的表分为单元格，并且基于网格CNN的CNN合并模块是应用用于恢复生成细胞。由于空间CNN模块可以有效地在整个表图像上传播上下文信息，因此我们的表结构识别器可以坚固地识别具有较大的空白空间和几何扭曲（甚至弯曲）表的表。得益于这两种技术，我们的表结构识别方法在包括SCITSR，PubTabnet和CTDAR TrackB2-Modern在内的三个公共基准上实现了最先进的性能。此外，我们进一步证明了我们方法在识别具有复杂结构，大空间以及几何扭曲甚至弯曲形状的表上的表格上的优势。

零拍摄对象检测（ZSD），将传统检测模型扩展到检测来自Unseen类别的对象的任务，已成为计算机视觉中的新挑战。大多数现有方法通过严格的映射传输策略来解决ZSD任务，这可能导致次优ZSD结果：1）这些模型的学习过程忽略了可用的看不见的类信息，因此可以轻松地偏向所看到的类别; 2）原始视觉特征空间并不合适，缺乏歧视信息。为解决这些问题，我们开发了一种用于ZSD的新型语义引导的对比网络，命名为Contrastzsd，一种检测框架首先将对比学习机制带入零拍摄检测的领域。特别地，对比度包括两个语义导向的对比学学习子网，其分别与区域类别和区域区域对之间形成对比。成对对比度任务利用从地面真理标签和预定义的类相似性分布派生的附加监督信号。在那些明确的语义监督的指导下，模型可以了解更多关于看不见的类别的知识，以避免看到概念的偏见问题，同时优化视觉功能的数据结构，以更好地辨别更好的视觉语义对齐。广泛的实验是在ZSD，即Pascal VOC和MS Coco的两个流行基准上进行的。结果表明，我们的方法优于ZSD和广义ZSD任务的先前最先进的。

Recent aerial object detection models rely on a large amount of labeled training data, which requires unaffordable manual labeling costs in large aerial scenes with dense objects. Active learning is effective in reducing the data labeling cost by selectively querying the informative and representative unlabelled samples. However, existing active learning methods are mainly with class-balanced setting and image-based querying for generic object detection tasks, which are less applicable to aerial object detection scenario due to the long-tailed class distribution and dense small objects in aerial scenes. In this paper, we propose a novel active learning method for cost-effective aerial object detection. Specifically, both object-level and image-level informativeness are considered in the object selection to refrain from redundant and myopic querying. Besides, an easy-to-use class-balancing criterion is incorporated to favor the minority objects to alleviate the long-tailed class distribution problem in model training. To fully utilize the queried information, we further devise a training loss to mine the latent knowledge in the undiscovered image regions. Extensive experiments are conducted on the DOTA-v1.0 and DOTA-v2.0 benchmarks to validate the effectiveness of the proposed method. The results show that it can save more than 75% of the labeling cost to reach the same performance compared to the baselines and state-of-the-art active object detection methods. Code is available at https://github.com/ZJW700/MUS-CDB

标记数据通常昂贵且耗时，特别是对于诸如对象检测和实例分割之类的任务，这需要对图像的密集标签进行密集的标签。虽然几张拍摄对象检测是关于培训小说中的模型（看不见的）对象类具有很少的数据，但它仍然需要在许多标记的基础（见）类的课程上进行训练。另一方面，自我监督的方法旨在从未标记数据学习的学习表示，该数据转移到诸如物体检测的下游任务。结合几次射击和自我监督的物体检测是一个有前途的研究方向。在本调查中，我们审查并表征了几次射击和自我监督对象检测的最新方法。然后，我们给我们的主要外卖，并讨论未来的研究方向。https://gabrielhuang.github.io/fsod-survey/的项目页面

遥感图像中的实例分段的任务，旨在在实例级别执行对象的每像素标记，对于各种民用应用非常重要。尽管以前的成功，但大多数现有的实例分割方法设计用于自然图像时，可以在直接应用于顶视图遥感图像时遇到清晰的性能下降。通过仔细分析，我们观察到由于严重的规模变化，低对比度和聚类分布，挑战主要来自歧视性对象特征。为了解决这些问题，提出了一种新颖的上下文聚合网络（CATNET）来改善特征提取过程。所提出的模型利用了三个轻量级的即插即用模块，即密度特征金字塔网络（Densfpn），空间上下文金字塔（SCP）和兴趣提取器（Hroie）的分层区域，以聚合在功能，空间和的全局视觉上下文实例域分别。 DenseFPN是一种多尺度特征传播模块，通过采用级别的残差连接，交叉级密度连接和具有重新加权策略来建立更灵活的信息流。利用注意力机制，SCP进一步通过将全局空间上下文聚合到当地区域来增强特征。对于每个实例，Hroie自适应地为不同的下游任务生成ROI功能。我们对挑战ISAID，DIOR，NWPU VHR-10和HRSID数据集进行了广泛的评估。评估结果表明，所提出的方法优于具有类似的计算成本的最先进。代码可在https://github.com/yeliudev/catnet上获得。

Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image. We propose a novel endto-end model that generates such structured scene representation from an input image. The model solves the scene graph inference problem using standard RNNs and learns to iteratively improves its predictions via message passing. Our joint inference model can take advantage of contextual cues to make better predictions on objects and their relationships. The experiments show that our model significantly outperforms previous methods for generating scene graphs using Visual Genome dataset and inferring support relations with NYU Depth v2 dataset.

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.