Scene text images have different shapes and are subjected to various distortions, e.g. perspective distortions. To handle these challenges, the state-of-the-art methods rely on a rectification network, which is connected to the text recognition network. They form a linear pipeline which uses text rectification on all input images, even for images that can be recognized without it. Undoubtedly, the rectification network improves the overall text recognition performance. However, in some cases, the rectification network generates unnecessary distortions on images, resulting in incorrect predictions in images that would have otherwise been correct without it. In order to alleviate the unnecessary distortions, the portmanteauing of features is proposed. The portmanteau feature, inspired by the portmanteau word, is a feature containing information from both the original text image and the rectified image. To generate the portmanteau feature, a non-linear input pipeline with a block matrix initialization is presented. In this work, the transformer is chosen as the recognition network due to its utilization of attention and inherent parallelism, which can effectively handle the portmanteau feature. The proposed method is examined on 6 benchmarks and compared with 13 state-of-the-art methods. The experimental results show that the proposed method outperforms the state-of-the-art methods on various of the benchmarks.

Scene text recognition (STR) involves the task of reading text in cropped images of natural scenes. Conventional models in STR employ convolutional neural network (CNN) followed by recurrent neural network in an encoder-decoder framework. In recent times, the transformer architecture is being widely adopted in STR as it shows strong capability in capturing long-term dependency which appears to be prominent in scene text images. Many researchers utilized transformer as part of a hybrid CNN-transformer encoder, often followed by a transformer decoder. However, such methods only make use of the long-term dependency mid-way through the encoding process. Although the vision transformer (ViT) is able to capture such dependency at an early stage, its utilization remains largely unexploited in STR. This work proposes the use of a transformer-only model as a simple baseline which outperforms hybrid CNN-transformer models. Furthermore, two key areas for improvement were identified. Firstly, the first decoded character has the lowest prediction accuracy. Secondly, images of different original aspect ratios react differently to the patch resolutions while ViT only employ one fixed patch resolution. To explore these areas, Pure Transformer with Integrated Experts (PTIE) is proposed. PTIE is a transformer model that can process multiple patch resolutions and decode in both the original and reverse character orders. It is examined on 7 commonly used benchmarks and compared with over 20 state-of-the-art methods. The experimental results show that the proposed method outperforms them and obtains state-of-the-art results in most benchmarks.

By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance. While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data. We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with high-capacity architectures that can absorb all of the diverse, robotic data. In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks. The project's website and videos can be found at robotics-transformer.github.io

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

我们为不依赖数据分布满足功能不平等的数据分布或强烈的平滑度假设提供了多项式收敛保证。假设有$ l^2 $准确的分数估计，我们可以为任何有限支撑或足够衰减的尾巴的分布获得Wasserstein距离保证，以及具有进一步平滑度假设的电视保证。

尽管进行了数十年的研究，但现有的导航系统在野外部署时仍然面临现实世界中的挑战，例如在混乱的家庭环境或人类占领的公共场所中。为了解决这个问题，我们提出了一类新的隐式控制政策，将模仿学习的好处与模型预测控制（MPC）的系统约束的强大处理结合在一起。我们的方法称为Performer-MPC，使用了通过表演者提供的视觉上下文嵌入的学习成本函数（一种低级隐式意见变压器）。我们共同训练成本函数并构建依靠它的控制器，有效地端到端解决相应的双层优化问题。我们表明，由此产生的策略通过利用一些在不同挑战的现实世界情景中利用一些专家演示来提高标准MPC绩效。与标准的MPC政策相比，表演者MPC在混乱的环境中实现了40％的目标，而在人类浏览时，社交指标的目标> 65％。

进化策略（ES）算法由于其巨大的并行能力，简单的实现，有效的参数空间探索和快速训练时间，在训练复杂的机器人控制策略中显示出令人鼓舞的结果。但是，ES的关键限制是其对大容量模型（包括现代神经网络体系结构）的可扩展性。在这项工作中，我们开发了预测信息增强随机搜索（PI-ARS），以通过利用表示表示学习来减少ES的参数搜索空间来减轻这种限制。即，PI-ARS将基于梯度的表示技术，预测信息（PI）与无梯度ES算法，增强随机搜索（ARS）结合在一起，以训练可以处理复杂机器人感觉输入并处理高度非线性机器人的策略动力学。我们在一系列具有挑战性的视觉范围任务上评估了PI-ARS，四倍的机器人需要在不平坦的踏脚石，Quincuncial Pile和移动平台上行走，并完成室内导航任务。在所有任务中，与ARS基线相比，PI-ARS表现出明显更好的学习效率和表现。我们通过证明学识渊博的政策可以成功地转移到真正的四倍机器人的情况下，进一步验证我们的算法，例如，在现实世界中的垫脚石环境上取得了100％的成功率，从而显着提高了先前的结果，从而实现了40％的成功。

由于COVID-19，许多学校通过视频会议软件在线考试已经采用了许多学校。虽然方便，但教师要同时显示的学生变焦窗口监督在线考试是具有挑战性的。在本文中，我们提出了IEXAM，这是一种智能的在线考试监测和分析系统，不仅可以使用面部检测来帮助监护人实时学生识别，而且还可以检测到常见的异常行为（包括面部消失，旋转的面部，旋转的面部，旋转，，旋转，并在考试期间用另一个人替换）通过基于面部识别后的外观后视频分析。为了建立这样的新型系统，我们克服了三个挑战。首先，我们发现了一种轻巧的方法来捕获考试视频流并实时分析它们。其次，我们利用每个学生的变焦窗口上显示的左角名称，并提出了改进的OCR（光学角色识别）技术来自动收集具有动态位置的学生面孔的地面真相。第三，我们进行了几次实验比较和优化，以有效缩短教师PC所需的训练时间和测试时间。我们的评估表明，IEXAM可以实现高精度，实时面部检测为90.4％，后验后面部识别率为98.4％，同时保持可接受的运行时性能。我们已经在https://github.com/vprlab/iexam上提供了IEXAM的源代码。

基于分数的生成建模（SGM）是一种从数据中学习概率分布并生成更多样本的非常成功的方法。我们证明了SGM背后的核心机械师的第一个多项式收敛保证：从概率密度$ p $中绘制样品估计（估计为$ \ nabla \ ln p $），该样本在$ l^2（p）中是准确的$。与以前的作品相比，我们不会产生误差，该错误会在时间上成倍增长或受到维度诅咒的影响。我们的保证对任何平滑分布都有效，并在多个一级取决于其对数sobolev常数。使用我们的保证，我们对基于分数的生成建模进行了理论分析，该模型将白色噪声输入转换为从不同噪声量表下得分估计的学习数据分布的样品。我们的分析将理论上的基础奠定了这样的观察，即在实践中需要进行退火，以生成好样品，因为我们的证明基本上取决于使用退火以在每个步骤中获得温暖的开始。此外，我们表明，与单独使用任何一部分相比，预测器 - 校正算法给出了更好的收敛性。