法医车牌识别（FLPR）仍然是在法律环境（例如刑事调查）中的公开挑战，在刑事调查中，不可读取的车牌（LPS）需要从高度压缩和/或低分辨率录像（例如监视摄像机）中解密。在这项工作中，我们提出了一个侧面信息变压器体系结构，该结构嵌入了输入压缩级别的知识，以改善在强压缩下的识别。我们在低质量的现实世界数据集上显示了变压器对车牌识别（LPR）的有效性。我们还提供了一个合成数据集，其中包括强烈退化，难以辨认的LP图像并分析嵌入知识对其的影响。该网络的表现优于现有的FLPR方法和标准最先进的图像识别模型，同时需要更少的参数。对于最严重的降级图像，我们可以将识别提高多达8.9％。

免费可用且易于使用的音频编辑工具使执行音频剪接变得直接。可以通过结合同一人的各种语音样本来说服伪造。在考虑错误信息时，在公共部门都很重要，并且在法律背景下以验证证据的完整性很重要。不幸的是，用于音频剪接的大多数现有检测算法都使用手工制作的功能并做出特定的假设。但是，刑事调查人员经常面临来自未知特征不明的来源的音频样本，这增加了对更普遍适用的方法的需求。通过这项工作，我们的目标是朝着不受限制的音频剪接检测迈出第一步，以满足这一需求。我们以可能掩盖剪接的后处理操作的形式模拟各种攻击方案。我们提出了一个用于剪接检测和定位的变压器序列到序列（SEQ2SEQ）网络。我们的广泛评估表明，所提出的方法的表现优于现有的剪接检测方法[3，10]以及通用网络效率网络[28]和regnet [25]。

While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train. 1

变压器注意机制中的设计选择，包括弱电感偏置和二次计算复杂性，限制了其用于建模长序列的应用。在本文中，我们介绍了一个简单的，理论上的，单头的门控注意机制，配备了（指数）移动平均线，以将局部依赖性的电感偏置纳入位置 - 敏锐的注意机制中。我们进一步提出了一个具有线性时间和空间复杂性的大型变体，但通过将整个序列分为固定长度的多个块，仅产生最小的质量损失。对广泛的序列建模基准测试的广泛实验，包括远距离竞技场，神经机器翻译，自动回归语言建模以及图像和语音分类，表明，巨人比其他序列模型取得了重大改进，包括变种物的变体和最新的变体模型状态空间模型。

人类活动识别是计算机视觉中的新出现和重要领域，旨在确定个体或个体正在执行的活动。该领域的应用包括从体育中生成重点视频到智能监视和手势识别。大多数活动识别系统依赖于卷积神经网络（CNN）的组合来从数据和复发性神经网络（RNN）中进行特征提取来确定数据的时间依赖性。本文提出并设计了两个用于人类活动识别的变压器神经网络：一个经常性变压器（RET），这是一个专门的神经网络，用于对数据序列进行预测，以及视觉变压器（VIT），一种用于提取显着的变压器的变压器（VIT）图像的特征，以提高活动识别的速度和可扩展性。我们在速度和准确性方面提供了对拟议的变压器神经网络与现代CNN和基于RNN的人类活动识别模型的广泛比较。

Leveraging the advances of natural language processing, most recent scene text recognizers adopt an encoder-decoder architecture where text images are first converted to representative features and then a sequence of characters via `sequential decoding'. However, scene text images suffer from rich noises of different sources such as complex background and geometric distortions which often confuse the decoder and lead to incorrect alignment of visual features at noisy decoding time steps. This paper presents I2C2W, a novel scene text recognition technique that is tolerant to geometric and photometric degradation by decomposing scene text recognition into two inter-connected tasks. The first task focuses on image-to-character (I2C) mapping which detects a set of character candidates from images based on different alignments of visual features in an non-sequential way. The second task tackles character-to-word (C2W) mapping which recognizes scene text by decoding words from the detected character candidates. The direct learning from character semantics (instead of noisy image features) corrects falsely detected character candidates effectively which improves the final text recognition accuracy greatly. Extensive experiments over nine public datasets show that the proposed I2C2W outperforms the state-of-the-art by large margins for challenging scene text datasets with various curvature and perspective distortions. It also achieves very competitive recognition performance over multiple normal scene text datasets.

由于深度学习的进步和数据集的增加，自动许可证板识别（ALPR）系统对来自多个区域的牌照（LPS）的表现显着。对深度ALPR系统的评估通常在每个数据集内完成;因此，如果这种结果是泛化能力的可靠指标，则是可疑的。在本文中，我们提出了一种传统分配的与休假 - 单数据集实验设置，以统一地评估12个光学字符识别（OCR）模型的交叉数据集泛化，其在九个公共数据集上应用于LP识别，具有良好的品种在若干方面（例如，获取设置，图像分辨率和LP布局）。我们还介绍了一个用于端到端ALPR的公共数据集，这是第一个包含带有Mercosur LP的车辆的图像和摩托车图像数量最多的图像。实验结果揭示了传统分离协议的局限性，用于评估ALPR上下文中的方法，因为在训练和测试休假时，大多数数据集在大多数数据集中的性能显着下降。

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. * Equal contribution. Listing order is random. Jakob proposed replacing RNNs with self-attention and started the effort to evaluate this idea. Ashish, with Illia, designed and implemented the first Transformer models and has been crucially involved in every aspect of this work. Noam proposed scaled dot-product attention, multi-head attention and the parameter-free position representation and became the other person involved in nearly every detail. Niki designed, implemented, tuned and evaluated countless model variants in our original codebase and tensor2tensor. Llion also experimented with novel model variants, was responsible for our initial codebase, and efficient inference and visualizations. Lukasz and Aidan spent countless long days designing various parts of and implementing tensor2tensor, replacing our earlier codebase, greatly improving results and massively accelerating our research.† Work performed while at Google Brain.‡ Work performed while at Google Research.

我们提出了基于神经网络的手写文本识别（HTR）模型体系结构，可以训练，以识别手写或印刷文本的完整页面，而无需图像分割。它基于图像到序列体系结构，它可以提取图像中存在的文本，然后正确地序列，而无需对文本和非文本的方向，布局和大小施加任何约束。此外，还可以训练它以生成与格式，布局和内容相关的辅助标记。我们使用角色级别的词汇，从而实现任何主题的语言和术语。该模型在IAM数据集中在段落级别识别中实现了新的艺术品。当对现实世界手写的免费表测试答案进行评估时 - 与弯曲和倾斜的线条，图纸，表，数学，化学和其他符号进行评估时，它的性能要比所有市售的HTR Cloud API都要好。它作为商业Web应用程序的一部分部署在生产中。

变压器最近在计算机视觉中获得了越来越高的关注。然而，现有研究大多用于特征表示学习的变压器，例如，用于图像分类和密集预测，变压器的普遍性是未知的。在这项工作中，我们进一步调查了对图像匹配和度量学习的应用变压器的可能性。我们发现视觉变压器（VIT）和带解码器的Vanilla变压器由于它们缺乏图像与图像而受到图像匹配。因此，我们进一步设计了两种天真的解决方案，即vit的查询画廊串联，并在香草变压器中的Query-Gallery横向关注。后者提高了性能，但它仍然有限。这意味着变压器中的注意机制主要用于全局特征聚合，这不是自然适用于图像匹配。因此，我们提出了一种新的简化解码器，它可以使用SoftMax加权丢弃全部注意力实现，只能保持查询关键相似性计算。此外，还应用全局最大池和多层的Perceptron（MLP）头来解码匹配结果。这样，简化的解码器在计算上更有效，而同时对图像匹配更有效。所谓的方法称为传输函数，在概括的人重新识别中实现最先进的性能，在几个流行的数据集中分别在Rank-1中的性能增长高达6.1％和5.7％。代码可在https://github.com/shengcailiao/qaconv获得。

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.

Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias that prioritizes local interactions. This makes them expressive, but also computationally infeasible for long sequences, such as high-resolution images. We demonstrate how combining the effectiveness of the inductive bias of CNNs with the expressivity of transformers enables them to model and thereby synthesize high-resolution images. We show how to (i) use CNNs to learn a contextrich vocabulary of image constituents, and in turn (ii) utilize transformers to efficiently model their composition within high-resolution images. Our approach is readily applied to conditional synthesis tasks, where both non-spatial information, such as object classes, and spatial information, such as segmentations, can control the generated image. In particular, we present the first results on semanticallyguided synthesis of megapixel images with transformers and obtain the state of the art among autoregressive models on class-conditional ImageNet. Code and pretrained models can be found at https://git.io/JnyvK.

文本识别是文档数字化的长期研究问题。现有的方法通常是基于CNN构建的，以用于图像理解，并为Char-Level文本生成而建立RNN。此外，通常需要另一种语言模型来提高整体准确性作为后处理步骤。在本文中，我们提出了一种使用预训练的图像变压器和文本变压器模型（即Trocr）提出的端到端文本识别方法，该模型利用了变压器体系结构，以实现图像理解和文字级级文本生成。TROR模型很简单，但有效，可以通过大规模合成数据进行预训练，并通过人体标记的数据集进行微调。实验表明，TROR模型的表现优于印刷，手写和场景文本识别任务上的当前最新模型。Trocr模型和代码可在\ url {https://aka.ms/trocr}上公开获得。

视觉变压器正在成为解决计算机视觉问题的强大工具。最近的技术还证明了超出图像域之外的变压器来解决许多与视频相关的任务的功效。其中，由于其广泛的应用，人类的行动识别是从研究界受到特别关注。本文提供了对动作识别的视觉变压器技术的首次全面调查。我们朝着这个方向分析并总结了现有文献和新兴文献，同时突出了适应变形金刚以进行动作识别的流行趋势。由于其专业应用，我们将这些方法统称为``动作变压器''。我们的文献综述根据其架构，方式和预期目标为动作变压器提供了适当的分类法。在动作变压器的背景下，我们探讨了编码时空数据，降低维度降低，框架贴片和时空立方体构造以及各种表示方法的技术。我们还研究了变压器层中时空注意的优化，以处理更长的序列，通常通过减少单个注意操作中的令牌数量。此外，我们还研究了不同的网络学习策略，例如自我监督和零局学习，以及它们对基于变压器的行动识别的相关损失。这项调查还总结了在具有动作变压器重要基准的评估度量评分方面取得的进步。最后，它提供了有关该研究方向的挑战，前景和未来途径的讨论。

公共数据集在推进车牌识别（LPR）的最新技术方面发挥了关键作用。尽管数据集偏见在计算机视觉社区中被认为是一个严重的问题，但在LPR文献中很大程度上忽略了它。 LPR模型通常在每个数据集上进行训练和评估。在这种情况下，他们经常在接受培训的数据集中证明了强大的证明，但在看不见的数据集中表现出有限的性能。因此，这项工作研究了LPR上下文中的数据集偏差问题。我们在八个数据集上进行了实验，在巴西收集了四个，在中国大陆进行了实验，并观察到每个数据集都有一个独特的，可识别的“签名”，因为轻量级分类模型预测了车牌（LP）图像的源数据集，其图像的源95％的精度。在我们的讨论中，我们提请人们注意以下事实：大多数LPR模型可能正在利用此类签名，以以失去概括能力为代价，以改善每个数据集中的结果。这些结果强调了评估跨数据库设置中LPR模型的重要性，因为它们提供了比数据库内部的更好的概括（因此实际性能）。

机器翻译历史上的重要突破之一是变压器模型的发展。不仅对于各种翻译任务，而且对于大多数其他NLP任务都是革命性的。在本文中，我们针对一个基于变压器的系统，该系统能够将德语用源句子转换为其英语的对应目标句子。我们对WMT'13数据集的新闻评论德语 - 英语并行句子进行实验。此外，我们研究了来自IWSLT'16数据集的培训中包含其他通用域数据以改善变压器模型性能的效果。我们发现，在培训中包括IWSLT'16数据集，有助于在WMT'13数据集的测试集中获得2个BLEU得分点。引入定性分析以分析通用域数据的使用如何有助于提高产生的翻译句子的质量。

Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems. Among their salient benefits, Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence as compared to recurrent networks e.g., Long short-term memory (LSTM). Different from convolutional networks, Transformers require minimal inductive biases for their design and are naturally suited as set-functions. Furthermore, the straightforward design of Transformers allows processing multiple modalities (e.g., images, videos, text and speech) using similar processing blocks and demonstrates excellent scalability to very large capacity networks and huge datasets. These strengths have led to exciting progress on a number of vision tasks using Transformer networks. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline. We start with an introduction to fundamental concepts behind the success of Transformers i.e., self-attention, large-scale pre-training, and bidirectional feature encoding. We then cover extensive applications of transformers in vision including popular recognition tasks (e.g., image classification, object detection, action recognition, and segmentation), generative modeling, multi-modal tasks (e.g., visual-question answering, visual reasoning, and visual grounding), video processing (e.g., activity recognition, video forecasting), low-level vision (e.g., image super-resolution, image enhancement, and colorization) and 3D analysis (e.g., point cloud classification and segmentation). We compare the respective advantages and limitations of popular techniques both in terms of architectural design and their experimental value. Finally, we provide an analysis on open research directions and possible future works. We hope this effort will ignite further interest in the community to solve current challenges towards the application of transformer models in computer vision.

In this paper, we assess the viability of transformer models in end-to-end InfoSec settings, in which no intermediate feature representations or processing steps occur outside the model. We implement transformer models for two distinct InfoSec data formats - specifically URLs and PE files - in a novel end-to-end approach, and explore a variety of architectural designs, training regimes, and experimental settings to determine the ingredients necessary for performant detection models. We show that in contrast to conventional transformers trained on more standard NLP-related tasks, our URL transformer model requires a different training approach to reach high performance levels. Specifically, we show that 1) pre-training on a massive corpus of unlabeled URL data for an auto-regressive task does not readily transfer to binary classification of malicious or benign URLs, but 2) that using an auxiliary auto-regressive loss improves performance when training from scratch. We introduce a method for mixed objective optimization, which dynamically balances contributions from both loss terms so that neither one of them dominates. We show that this method yields quantitative evaluation metrics comparable to that of several top-performing benchmark classifiers. Unlike URLs, binary executables contain longer and more distributed sequences of information-rich bytes. To accommodate such lengthy byte sequences, we introduce additional context length into the transformer by providing its self-attention layers with an adaptive span similar to Sukhbaatar et al. We demonstrate that this approach performs comparably to well-established malware detection models on benchmark PE file datasets, but also point out the need for further exploration into model improvements in scalability and compute efficiency.

场景文本识别（str）是图像和文本之间的重要桥梁，吸引了丰富的研究关注。虽然卷积神经网络（CNNS）在此任务中取得了显着的进展，但大多数现有工作都需要额外的模块（上下文建模模块）来帮助CNN捕获全局依赖项来解决归纳偏差并加强文本特征之间的关系。最近，该变压器已被提出作为通过自我关注机制的全球背景建模的有希望的网络，但在应用于识别时主要缺点是效率。我们提出了一个1-D拆分来解决复杂性的挑战，并用变压器编码器替换CNN，以减少对上下文建模模块的需求。此外，最近的方法使用冻结的初始嵌入来指导解码器对文本进行解码，导致精度损失。我们建议使用从变压器编码器中学到的学习学习的可读初始嵌入，使其自适应不同的输入图像。最重要的是，我们介绍了一个新颖的文本识别架构，名为基于变压器的文本识别器，其中包含三个阶段（转换，特征提取和预测）组成的初始嵌入指导（TRIG）。广泛的实验表明，我们的方法可以在文本识别基准上实现最先进的。

在诸如对象跟踪的应用中，时间序列数据不可避免地携带缺失的观察。在基于深度学习的模型的成功之后，对于各种序列学习任务，这些模型越来越替换对象跟踪应用中的经典方法，以推断对象的运动状态。虽然传统的跟踪方法可以处理缺失的观察，但默认情况下，大多数深度同行都不适合这一点。迄今为止，本文介绍了一种基于变压器的方法，用于在可变输入长度轨迹数据中处理缺失的观察。通过连续增加所需推理任务的复杂性，间接地形成模型。从再现无噪声轨迹开始，该模型然后学会从嘈杂的输入中推断出来的轨迹。通过提供缺失的令牌，二进制编码的缺失事件，该模型将学习进入缺少数据，并且Infers在其余输入上调整完整的轨迹。在连续缺失事件序列的情况下，该模型则用作纯预测模型。该方法的能力在反映原型对象跟踪方案的综合数据和实际数据上进行了证明。