卷积神经网络的理论表明，移位阶段性的特性，即移动输入会导致同样移动的输出。但是，实际上，情况并非总是如此。这为场景文本检测带来了一个很好的问题，对于文本检测，一致的空间响应至关重要，无论文本在场景中的位置如何。使用简单的合成实验，我们证明了最先进的完全卷积文本检测器的固有移位方差。此外，使用相同的实验设置，我们展示了较小的体系结构变化如何导致改善的移位等效性和较小的检测器输出变化。我们使用文本检测网络上的现实世界培训时间表来验证合成结果。为了量化转移变异性的量，我们提出了一个基于公认的文本检测基准测试的度量。虽然提出的架构更改无法完全恢复移位均衡性，但添加平滑过滤器可以显着提高公共文本数据集的变化一致性。考虑到小移位的潜在影响，我们建议通过本工作中描述的指标扩展常用的文本检测指标，以便能够量化文本检测器的一致性。

Modern convolutional networks are not shiftinvariant, as small input shifts or translations can cause drastic changes in the output. Commonly used downsampling methods, such as max-pooling, strided-convolution, and averagepooling, ignore the sampling theorem. The wellknown signal processing fix is anti-aliasing by low-pass filtering before downsampling. However, simply inserting this module into deep networks degrades performance; as a result, it is seldomly used today. We show that when integrated correctly, it is compatible with existing architectural components, such as max-pooling and strided-convolution. We observe increased accuracy in ImageNet classification, across several commonly-used architectures, such as ResNet, DenseNet, and MobileNet, indicating effective regularization. Furthermore, we observe better generalization, in terms of stability and robustness to input corruptions. Our results demonstrate that this classical signal processing technique has been undeservingly overlooked in modern deep networks.

由于下采样层的存在，卷积神经网络缺乏转变标准。在图像分类中，最近提出了自适应多相下采样（APS-D）以使CNN完全换档不变。但是，在用于图像重建任务的网络中，它不能自动恢复转移标准规范。我们通过提出自适应多相上升采样（APS-U），传统上采样的非线性扩展来解决该问题，该传统上采样的非线性扩展，其允许CNNS与对称编码器 - 解码器架构（例如U-Net）进行CNN，以表现出完美的换档设备。利用MRI和CT重建实验，我们表明，网络包含APS-D / U层的网络展示了本领域的状态性能，而不会牺牲图像重建质量。此外，与数据增强和抗锯齿等先前的方法不同，从APS-D / U获得的标准规范中的增益也扩展到训练分布外的图像。

We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation. * This work was done during an internship at NVIDIA. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

现有的感知相似性指标假定图像及其参考非常适合。结果，这些指标通常对人眼无法察觉的小对齐误差敏感。本文研究了微小的未对准的影响，特别是输入图像和参考图像之间对现有指标的微小变化，并因此发展了耐转移的相似性度量。本文以LPIP为基础，这是一种广泛使用的知觉相似性度量，并探索了建筑设计注意事项，以使其可抵抗不可察觉的未对准。具体而言，我们研究了广泛的神经网络元素，例如抗异化滤波，汇总，跨性别，填充和跳过连接，并讨论它们在制作强大度量方面的作用。根据我们的研究，我们开发了一种新的基于神经网络的知觉相似性度量。我们的实验表明，我们的指标宽容不可察觉的转变，同时与人类的相似性判断一致。

Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typically sought through data augmentation, but patch-wise equivariance is more difficult. We present Harmonic Networks or H-Nets, a CNN exhibiting equivariance to patch-wise translation and 360-rotation. We achieve this by replacing regular CNN filters with circular harmonics, returning a maximal response and orientation for every receptive field patch.H-Nets use a rich, parameter-efficient and fixed computational complexity representation, and we show that deep feature maps within the network encode complicated rotational invariants. We demonstrate that our layers are general enough to be used in conjunction with the latest architectures and techniques, such as deep supervision and batch normalization. We also achieve state-of-the-art classification on rotated-MNIST, and competitive results on other benchmark challenges.

In this paper we introduce a new method for text detection in natural images. The method comprises two contributions: First, a fast and scalable engine to generate synthetic images of text in clutter. This engine overlays synthetic text to existing background images in a natural way, accounting for the local 3D scene geometry. Second, we use the synthetic images to train a Fully-Convolutional Regression Network (FCRN) which efficiently performs text detection and bounding-box regression at all locations and multiple scales in an image. We discuss the relation of FCRN to the recently-introduced YOLO detector, as well as other end-toend object detection systems based on deep learning. The resulting detection network significantly out performs current methods for text detection in natural images, achieving an F-measure of 84.2% on the standard ICDAR 2013 benchmark. Furthermore, it can process 15 images per second on a GPU.

基于2D图像的3D对象的推理由于从不同方向查看对象引起的外观差异很大，因此具有挑战性。理想情况下，我们的模型将是对物体姿势变化的不变或等效的。不幸的是，对于2D图像输入，这通常是不可能的，因为我们没有一个先验模型，即在平面外对象旋转下如何改变图像。唯一的$ \ mathrm {so}（3）$ - 当前存在的模型需要点云输入而不是2D图像。在本文中，我们提出了一种基于Icosahedral群卷积的新型模型体系结构，即通过将输入图像投影到iCosahedron上，以$ \ mathrm {so（3）} $中的理由。由于此投影，该模型大致与$ \ mathrm {so}（3）$中的旋转大致相当。我们将此模型应用于对象构成估计任务，并发现它的表现优于合理的基准。

In recent years, object detection has achieved a very large performance improvement, but the detection result of small objects is still not very satisfactory. This work proposes a strategy based on feature fusion and dilated convolution that employs dilated convolution to broaden the receptive field of feature maps at various scales in order to address this issue. On the one hand, it can improve the detection accuracy of larger objects. On the other hand, it provides more contextual information for small objects, which is beneficial to improving the detection accuracy of small objects. The shallow semantic information of small objects is obtained by filtering out the noise in the feature map, and the feature information of more small objects is preserved by using multi-scale fusion feature module and attention mechanism. The fusion of these shallow feature information and deep semantic information can generate richer feature maps for small object detection. Experiments show that this method can have higher accuracy than the traditional YOLOv3 network in the detection of small objects and occluded objects. In addition, we achieve 32.8\% Mean Average Precision on the detection of small objects on MS COCO2017 test set. For 640*640 input, this method has 88.76\% mAP on the PASCAL VOC2012 dataset.

We present an integrated framework for using Convolutional Networks for classification, localization and detection. We show how a multiscale and sliding window approach can be efficiently implemented within a ConvNet. We also introduce a novel deep learning approach to localization by learning to predict object boundaries. Bounding boxes are then accumulated rather than suppressed in order to increase detection confidence. We show that different tasks can be learned simultaneously using a single shared network. This integrated framework is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013) and obtained very competitive results for the detection and classifications tasks. In post-competition work, we establish a new state of the art for the detection task. Finally, we release a feature extractor from our best model called OverFeat.

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or 'atrous convolution', as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed "DeepLab" system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7% mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

空中图像中的物体比典型图像中的刻度和方向具有更大的变化，因此检测更加困难。卷积神经网络使用各种频率和方向特定的内核来识别受不同变换的对象;这些需要许多参数。采样等式网络可以根据对象的转换从输入特征映射调整采样，允许内核在不同的变换下提取对象的特征。这样做需要较少的参数，并且使网络更适合于代表可变形对象，如空中图像中的那些。然而，像可变形卷积网络一样的方法只能在某些情况下提供采样设备，因为用于采样的位置。我们提出了采样的等式自我关注网络，其认为自我关注限制在本地图像补丁中，因为用掩模而不是位置的卷积采样，以及设计变换嵌入模块，以进一步提高等值的采样能力。我们还使用新颖的随机标准化模块来应付由于空中图像数据有限的原因。我们表明，我们的型号（i）提供了比现有方法更好的采样量规范，而无需额外监督，（ii）提供对Imagenet的改进分类，并且（iii）在没有增加的情况下实现最先进的结果计算。

卷积神经网络（CNN）在翻译下是固有的等分反，但是，它们没有等效的嵌入机制来处理其他变换，例如旋转和规模变化。存在几种方法，使CNN通过设计在其他转换组下变得等效。其中，可操纵的CNN特别有效。然而，这些方法需要将滤波器重新设计标准网络，筛选涉及复杂的分析功能的预定义基的组合。我们通过实验证明，在选择的基础上的这些限制可能导致模型权重，这对主要深度学习任务进行了次优（例如，分类）。此外，这种硬烘焙的显式配方使得难以设计包括异质特征组的复合网络。为了规避此类问题，我们提出了隐含的等级网络（IEN），其通过优化与标准损耗术语相结合的多目标损耗函数来诱导标准CNN模型的不同层的等级。通过在ROT-MNIST上的VGG和RESNET模型的实验，ROT-TINIMAGENET，SCALE-MNIST和STL-10数据集上，我们表明IEN，即使是简单的配方，也要优于可操纵网络。此外，IEN促进了非均相过滤器组的构建，允许CNNS中的通道数量减少超过30％，同时保持与基线的表现。 IEN的功效进一步验证了视觉对象跟踪的难题。我们表明IEN优于最先进的旋转等级跟踪方法，同时提供更快的推理速度。

Self-attention has the promise of improving computer vision systems due to parameter-independent scaling of receptive fields and content-dependent interactions, in contrast to parameter-dependent scaling and content-independent interactions of convolutions. Self-attention models have recently been shown to have encouraging improvements on accuracy-parameter trade-offs compared to baseline convolutional models such as ResNet-50. In this work, we aim to develop self-attention models that can outperform not just the canonical baseline models, but even the high-performing convolutional models. We propose two extensions to selfattention that, in conjunction with a more efficient implementation of self-attention, improve the speed, memory usage, and accuracy of these models. We leverage these improvements to develop a new self-attention model family, HaloNets, which reach state-of-the-art accuracies on the parameterlimited setting of the ImageNet classification benchmark. In preliminary transfer learning experiments, we find that HaloNet models outperform much larger models and have better inference performance. On harder tasks such as object detection and instance segmentation, our simple local self-attention and convolutional hybrids show improvements over very strong baselines. These results mark another step in demonstrating the efficacy of self-attention models on settings traditionally dominated by convolutional models.

我们研究复杂的缩放作为一种自然的对称性和复杂的测量和表示独特的对称性。深度复杂网络（DCN）将实值的代数扩展到复杂域，而不会解决复杂值缩放。超现实占据复杂数字的限制性歧管视图，采用距离度量来实现复杂的缩放不变性，同时丢失丰富的复合值。我们分析了复杂的缩放，作为共同领域的转换和设计新颖的具有这种特殊转换的不变神经网络层。我们还提出了RGB图像的新型复合值表示，其中复值缩放表示色调偏移或跨色通道的相关变化。在MSTAR，CIFAR10，CIFAR100和SVHN上基准测试，我们的共同域对称（CDS）分类器提供更高的准确性，更好的泛化，对共同域变换的鲁棒性，以及比DCN和超现实的更低模型偏差和方差，具有较少的参数。

最近，场景文本检测是一个具有挑战性的任务。具有任意形状或大宽高比的文本通常很难检测。以前的基于分段的方法可以更准确地描述曲线文本，但遭受过分分割和文本粘附。在本文中，我们提出了基于关注的特征分解 - 改变 - 用于场景文本检测，它利用上下文信息和低级功能来增强基于分段的文本检测器的性能。在特征融合的阶段，我们引入交叉级注意模块来通过添加融合多缩放功能的注意机制来丰富文本的上下文信息。在概率图生成的阶段，提出了一种特征分解 - 重建模块来缓解大宽高比文本的过分分割问题，其根据其频率特性分解文本特征，然后通过添加低级特征来重建它。实验已经在两个公共基准数据集中进行，结果表明，我们的提出方法实现了最先进的性能。

最新的2D图像压缩方案依赖于卷积神经网络（CNN）的力量。尽管CNN为2D图像压缩提供了有希望的观点，但将此类模型扩展到全向图像并不简单。首先，全向图像具有特定的空间和统计特性，这些特性无法通过当前CNN模型完全捕获。其次，在球体上，基本的数学操作组成了CNN体系结构，例如翻译和采样。在本文中，我们研究了全向图像的表示模型的学习，并建议使用球体的HealPix均匀采样的属性来重新定义用于全向图像的深度学习模型中使用的数学工具。特别是，我们：i）提出了在球体上进行新的卷积操作的定义，以保持经典2D卷积的高表现力和低复杂性； ii）适应标准的CNN技术，例如步幅，迭代聚集和像素改组到球形结构域；然后iii）将我们的新框架应用于全向图像压缩的任务。我们的实验表明，与应用于等应角图像的类似学习模型相比，我们提出的球形溶液可带来更好的压缩增益，可以节省比特率的13.7％。同样，与基于图形卷积网络的学习模型相比，我们的解决方案支持更具表现力的过滤器，这些过滤器可以保留高频并提供压缩图像的更好的感知质量。这样的结果证明了拟议框架的效率，该框架为其他全向视觉任务任务打开了新的研究场所，以在球体歧管上有效实施。

由于字体，大小，颜色和方向的各种文本变化，任意形状的场景文本检测是一项具有挑战性的任务。大多数现有基于回归的方法求助于回归文本区域的口罩或轮廓点以建模文本实例。但是，回归完整的口罩需要高训练的复杂性，并且轮廓点不足以捕获高度弯曲的文本的细节。为了解决上述限制，我们提出了一个名为TextDCT的新颖的轻巧锚文本检测框架，该框架采用离散的余弦变换（DCT）将文本掩码编码为紧凑型向量。此外，考虑到金字塔层中训练样本不平衡的数量，我们仅采用单层头来进行自上而下的预测。为了建模单层头部的多尺度文本，我们通过将缩水文本区域视为正样本，并通过融合来介绍一个新颖的积极抽样策略，并通过融合来设计特征意识模块（FAM），以实现空间意识和规模的意识丰富的上下文信息并关注更重要的功能。此外，我们提出了一种分割的非量最大抑制（S-NMS）方法，该方法可以过滤低质量的掩模回归。在四个具有挑战性的数据集上进行了广泛的实验，这表明我们的TextDCT在准确性和效率上都获得了竞争性能。具体而言，TextDCT分别以每秒17.2帧（FPS）和F-measure的F-MEASIE达到85.1，而CTW1500和Total-Text数据集的F-Measure 84.9分别为15.1 fps。

Single-frame InfraRed Small Target (SIRST) detection has been a challenging task due to a lack of inherent characteristics, imprecise bounding box regression, a scarcity of real-world datasets, and sensitive localization evaluation. In this paper, we propose a comprehensive solution to these challenges. First, we find that the existing anchor-free label assignment method is prone to mislabeling small targets as background, leading to their omission by detectors. To overcome this issue, we propose an all-scale pseudo-box-based label assignment scheme that relaxes the constraints on scale and decouples the spatial assignment from the size of the ground-truth target. Second, motivated by the structured prior of feature pyramids, we introduce the one-stage cascade refinement network (OSCAR), which uses the high-level head as soft proposals for the low-level refinement head. This allows OSCAR to process the same target in a cascade coarse-to-fine manner. Finally, we present a new research benchmark for infrared small target detection, consisting of the SIRST-V2 dataset of real-world, high-resolution single-frame targets, the normalized contrast evaluation metric, and the DeepInfrared toolkit for detection. We conduct extensive ablation studies to evaluate the components of OSCAR and compare its performance to state-of-the-art model-driven and data-driven methods on the SIRST-V2 benchmark. Our results demonstrate that a top-down cascade refinement framework can improve the accuracy of infrared small target detection without sacrificing efficiency. The DeepInfrared toolkit, dataset, and trained models are available at https://github.com/YimianDai/open-deepinfrared to advance further research in this field.

The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be extended beyond global symmetries to local gauge transformations. This enables the development of a very general class of convolutional neural networks on manifolds that depend only on the intrinsic geometry, and which includes many popular methods from equivariant and geometric deep learning.We implement gauge equivariant CNNs for signals defined on the surface of the icosahedron, which provides a reasonable approximation of the sphere. By choosing to work with this very regular manifold, we are able to implement the gauge equivariant convolution using a single conv2d call, making it a highly scalable and practical alternative to Spherical CNNs. Using this method, we demonstrate substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns.