转移学习可以在源任务上重新使用知识来帮助学习目标任务。一种简单的转移学习形式在当前的最先进的计算机视觉模型中是常见的，即预先训练ILSVRC数据集上的图像分类模型，然后在任何目标任务上进行微调。然而，先前对转移学习的系统研究已经有限，并且预计工作的情况并不完全明白。在本文中，我们对跨越不同的图像域进行了广泛的转移学习实验探索（消费者照片，自主驾驶，空中图像，水下，室内场景，合成，特写镜头）和任务类型（语义分割，物体检测，深度估计，关键点检测）。重要的是，这些都是与现代计算机视觉应用相关的复杂的结构化的输出任务类型。总共执行超过2000年的转移学习实验，包括许多来源和目标来自不同的图像域，任务类型或两者。我们系统地分析了这些实验，了解图像域，任务类型和数据集大小对传输学习性能的影响。我们的研究导致了几个见解和具体建议：（1）对于大多数任务，存在一个显着优于ILSVRC'12预培训的来源; （2）图像领域是实现阳性转移的最重要因素; （3）源数据集应该\ \ emph {include}目标数据集的图像域以获得最佳结果; （4）与此同时，当源任务的图像域比目标的图像域时，我们只观察小的负面影响; （5）跨任务类型的转移可能是有益的，但其成功严重依赖于源和目标任务类型。

我们解决了转移学习中的集合选择问题：给出了大量的源模型，我们要选择一个模型的集合，在对目标训练集的微调后，在目标测试集上产生最佳性能。由于微调所有可能的合奏是计算禁止的，因此我们目的是使用计算上有效的可转换度量来预测目标数据集的性能。我们提出了用于此任务的几个新的可转换性指标，并在对语义细分的具有挑战性和现实的转移学习设置中进行评估：我们通过考虑涵盖各种图像域的各种数据集来创建一个大型和多样化的源模型池，两种不同架构和两个预训练计划。鉴于此池，我们自动选择子集，以在给定的目标数据集上形成良好的集合。我们将通过我们的方法选择的合奏与两个基线进行比较，该基线选择单个源模型，其中（1）与我们的方法相同;或（2）从包含大源模型的池，每个池具有与集合相似的容量。平均超过17个目标数据集，我们分别以6.0％和2.5％的相对平均值越优于这些基线。

Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques that rely less on manual annotations (e.g., image, bounding-box, and per-pixel labels), but learn useful information from unlabeled data. In this thesis, we explore five different techniques for handling AEL.

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

Image segmentation is a key topic in image processing and computer vision with applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among many others. Various algorithms for image segmentation have been developed in the literature. Recently, due to the success of deep learning models in a wide range of vision applications, there has been a substantial amount of works aimed at developing image segmentation approaches using deep learning models. In this survey, we provide a comprehensive review of the literature at the time of this writing, covering a broad spectrum of pioneering works for semantic and instance-level segmentation, including fully convolutional pixel-labeling networks, encoder-decoder architectures, multi-scale and pyramid based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the similarity, strengths and challenges of these deep learning models, examine the most widely used datasets, report performances, and discuss promising future research directions in this area.

The success of deep learning in vision can be attributed to: (a) models with high capacity; (b) increased computational power; and (c) availability of large-scale labeled data. Since 2012, there have been significant advances in representation capabilities of the models and computational capabilities of GPUs. But the size of the biggest dataset has surprisingly remained constant. What will happen if we increase the dataset size by 10× or 100×? This paper takes a step towards clearing the clouds of mystery surrounding the relationship between 'enormous data' and visual deep learning. By exploiting the JFT-300M dataset which has more than 375M noisy labels for 300M images, we investigate how the performance of current vision tasks would change if this data was used for representation learning. Our paper delivers some surprising (and some expected) findings. First, we find that the performance on vision tasks increases logarithmically based on volume of training data size. Second, we show that representation learning (or pretraining) still holds a lot of promise. One can improve performance on many vision tasks by just training a better base model. Finally, as expected, we present new state-of-theart results for different vision tasks including image classification, object detection, semantic segmentation and human pose estimation. Our sincere hope is that this inspires vision community to not undervalue the data and develop collective efforts in building larger datasets.

视频分析的图像分割在不同的研究领域起着重要作用，例如智能城市，医疗保健，计算机视觉和地球科学以及遥感应用。在这方面，最近致力于发展新的细分策略;最新的杰出成就之一是Panoptic细分。后者是由语义和实例分割的融合引起的。明确地，目前正在研究Panoptic细分，以帮助获得更多对视频监控，人群计数，自主驾驶，医学图像分析的图像场景的更细致的知识，以及一般对场景更深入的了解。为此，我们介绍了本文的首次全面审查现有的Panoptic分段方法，以获得作者的知识。因此，基于所采用的算法，应用场景和主要目标的性质，执行现有的Panoptic技术的明确定义分类。此外，讨论了使用伪标签注释新数据集的Panoptic分割。继续前进，进行消融研究，以了解不同观点的Panoptic方法。此外，讨论了适合于Panoptic分割的评估度量，并提供了现有解决方案性能的比较，以告知最先进的并识别其局限性和优势。最后，目前对主题技术面临的挑战和吸引不久的将来吸引相当兴趣的未来趋势，可以成为即将到来的研究研究的起点。提供代码的文件可用于：https：//github.com/elharroussomar/awesome-panoptic-egation

深神网络的对象探测器正在不断发展，并用于多种应用程序，每个应用程序都有自己的要求集。尽管关键安全应用需要高准确性和可靠性，但低延迟任务需要资源和节能网络。不断提出了实时探测器，在高影响现实世界中是必需的，但是它们过分强调了准确性和速度的提高，而其他功能（例如多功能性，鲁棒性，资源和能源效率）则被省略。现有网络的参考基准不存在，设计新网络的标准评估指南也不存在，从而导致比较模棱两可和不一致的比较。因此，我们对广泛的数据集进行了多个实时探测器（基于锚点，关键器和变压器）的全面研究，并报告了一系列广泛指标的结果。我们还研究了变量，例如图像大小，锚固尺寸，置信阈值和架构层对整体性能的影响。我们分析了检测网络的鲁棒性，以防止分配变化，自然腐败和对抗性攻击。此外，我们提供了校准分析来评估预测的可靠性。最后，为了强调现实世界的影响，我们对自动驾驶和医疗保健应用进行了两个独特的案例研究。为了进一步衡量关键实时应用程序中网络的能力，我们报告了在Edge设备上部署检测网络后的性能。我们广泛的实证研究可以作为工业界对现有网络做出明智选择的指南。我们还希望激发研究社区的设计和评估网络的新方向，该网络着重于更大而整体的概述，以实现深远的影响。

我们提出了一个统一的查看，即通过通用表示，一个深层神经网络共同学习多个视觉任务和视觉域。同时学习多个问题涉及最大程度地减少具有不同幅度和特征的多个损失函数的加权总和，从而导致一个损失的不平衡状态，与学习每个问题的单独模型相比，一个损失的不平衡状态主导了优化和差的结果。为此，我们提出了通过小容量适配器将多个任务/特定于域网络的知识提炼到单个深神经网络中的知识。我们严格地表明，通用表示在学习NYU-V2和CityScapes中多个密集的预测问题方面实现了最新的表现，来自视觉Decathlon数据集中的不同域中的多个图像分类问题以及MetadataSet中的跨域中的几个域中学习。最后，我们还通过消融和定性研究进行多次分析。

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.

标记数据通常昂贵且耗时，特别是对于诸如对象检测和实例分割之类的任务，这需要对图像的密集标签进行密集的标签。虽然几张拍摄对象检测是关于培训小说中的模型（看不见的）对象类具有很少的数据，但它仍然需要在许多标记的基础（见）类的课程上进行训练。另一方面，自我监督的方法旨在从未标记数据学习的学习表示，该数据转移到诸如物体检测的下游任务。结合几次射击和自我监督的物体检测是一个有前途的研究方向。在本调查中，我们审查并表征了几次射击和自我监督对象检测的最新方法。然后，我们给我们的主要外卖，并讨论未来的研究方向。https://gabrielhuang.github.io/fsod-survey/的项目页面

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

尽管进行了多年的研究，但跨域的概括仍然是深层网络的语义分割的关键弱点。先前的研究取决于静态模型的假设，即训练过程完成后，模型参数在测试时间保持固定。在这项工作中，我们通过一种自适应方法来挑战这一前提，用于语义分割，将推理过程调整为每个输入样本。自我适应在两个级别上运行。首先，它采用了自我监督的损失，该损失将网络中卷积层的参数定制为输入图像。其次，在批准层中，自适应近似于整个测试数据的平均值和方差，这是不可用的。它通过在训练和从单个测试样本得出的参考分布之间进行插值来实现这一目标。为了凭经验分析我们的自适应推理策略，我们制定并遵循严格的评估协议，以解决先前工作的严重局限性。我们的广泛分析得出了一个令人惊讶的结论：使用标准训练程序，自我适应大大优于强大的基准，并在多域基准测试方面设定了新的最先进的准确性。我们的研究表明，自适应推断可以补充培训时间的既定模型正规化实践，以改善深度网络的概括到异域数据。

State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards "small". Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are obvious: such datasets are difficult to collect and annotate. In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images. Our experiments demonstrate that training for large-scale hashtag prediction leads to excellent results. We show improvements on several image classification and object detection tasks, and report the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5). We also perform extensive experiments that provide novel empirical data on the relationship between large-scale pretraining and transfer learning performance. Name template Description train-IG-I-1.5k Instagram training set of I images and ∼1.5k hashtags from ImageNet-1k. train-IG-I-8.5k Instagram training set of I images and ∼8.5k hashtags from WordNet. train-IG-I-17k Instagram training set of I images and ∼17k hashtags from WordNet. train-IN-1M-1k The standard ImageNet-1k ILSVRC training set with 1.28M images. val-IN-50k-1k The standard ImageNet-1k ILSVRC validation set with 50k images. train-IN-I-L Extended ImageNet training set of I images and L ∈ {5k, 9k} labels. val-IN-I-L Extended ImageNet validation set of I images and L ∈ {5k, 9k} labels. train-CUB-6k-200 The Caltech-UCSD Birds-200-2011 training set. val-CUB-6k-200 The Caltech-UCSD Birds-200-2011 validation set. train-Places-1.8M-365 The Places365-Standard training set (high-resolution version). val-Places-37k-365 The Places365-Standard validation set (high-resolution version). train-COCO-135k-80 The standard COCO detection training set (2017 version). val-COCO-5k-80 The standard COCO detection validation set (2017 version). test-COCO-20k-80 The standard COCO detection test-dev set (2017 version).Table 1: Summary of image classification datasets. Each dataset is named with a template, role-source-I-L, that indicates its role (training, validation, testing), source, number of images I, and number of labels L.

X-ray imaging technology has been used for decades in clinical tasks to reveal the internal condition of different organs, and in recent years, it has become more common in other areas such as industry, security, and geography. The recent development of computer vision and machine learning techniques has also made it easier to automatically process X-ray images and several machine learning-based object (anomaly) detection, classification, and segmentation methods have been recently employed in X-ray image analysis. Due to the high potential of deep learning in related image processing applications, it has been used in most of the studies. This survey reviews the recent research on using computer vision and machine learning for X-ray analysis in industrial production and security applications and covers the applications, techniques, evaluation metrics, datasets, and performance comparison of those techniques on publicly available datasets. We also highlight some drawbacks in the published research and give recommendations for future research in computer vision-based X-ray analysis.

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub. 3 * Equal contribution; the order of first authors was randomly selected.

Self-supervised visual representation learning has seen huge progress recently, but no large scale evaluation has compared the many models now available. We evaluate the transfer performance of 13 top self-supervised models on 40 downstream tasks, including many-shot and few-shot recognition, object detection, and dense prediction. We compare their performance to a supervised baseline and show that on most tasks the best self-supervised models outperform supervision, confirming the recently observed trend in the literature. We find ImageNet Top-1 accuracy to be highly correlated with transfer to many-shot recognition, but increasingly less so for few-shot, object detection and dense prediction. No single self-supervised method dominates overall, suggesting that universal pre-training is still unsolved. Our analysis of features suggests that top self-supervised learners fail to preserve colour information as well as supervised alternatives, but tend to induce better classifier calibration, and less attentive overfitting than supervised learners.

在混合完成的多任务，多域和多模式数据上进行预训练仍然是视力感知预训练的开放挑战。在本文中，我们提出了GPPF，这是一个普遍的感知预训练框架，预先培训任务级的动态网络，该网络是由在标签的多任务和多域数据集上的各层知识“乐高”组成的。通过检查人类在复杂环境中学习的先天能力，我们识别并将三个关键要素转移到深网上：（1）同时暴露于每个批次中的各种交叉任务和跨域信息。 （2）由知识共享驱动的单独的乐高单元中的分区知识存储。 （3）用于训练和下游任务的乐高单元子集的稀疏激活。值得注意的是，由于其在输入形状，损失功能，输出格式，数据分布等方面的差异，不同视觉任务的联合培训是不平凡的。因此，我们创新地开发了插件的多任务培训算法，该培训算法是支持单个迭代多个任务（SIMT）同时培训。 Simt用大型多任务多任务数据集为预训练的基础奠定了基础，并且被证明对于我们的GPPF实验中的稳定培训至关重要。令人兴奋的是，详尽的实验表明，我们的GPPF-R50型号在GPPF-15M中的8个预训练预培训任务的强大基线上取得了显着改善，并在22个下游任务中收获了一系列SOTA，并具有相似的计算预算。我们还验证了GPPF对SOTA视觉变压器的概括能力，并具有一致的改进。这些可靠的实验结果充分证明了我们新颖的GPPF框架提供的有效的知识学习，存储，共享和转移。

数据采集​​和注释中的困难基本上限制了3D医学成像应用的训练数据集的样本尺寸。结果，在没有足够的预训练参数的情况下，构建来自划痕的高性能3D卷积神经网络仍然是一项艰巨的任务。以前关于3D预培训的努力经常依赖于自我监督的方法，它在未标记的数据上使用预测或对比学习来构建不变的3D表示。然而，由于大规模监督信息的不可用，从这些学习框架获得语义不变和歧视性表示仍然存在问题。在本文中，我们重新审视了一种创新但简单的完全监督的3D网络预训练框架，以利用来自大型2D自然图像数据集的语义监督。通过重新设计的3D网络架构，重新设计的自然图像用于解决数据稀缺问题并开发强大的3D表示。四个基准数据集上的综合实验表明，所提出的预先接受的模型可以有效地加速收敛，同时还提高了各种3D医学成像任务，例如分类，分割和检测的准确性。此外，与从头划伤的训练相比，它可以节省高达60％的注释工作。在NIH Deeplesion数据集上，它同样地实现了最先进的检测性能，优于早期的自我监督和完全监督的预训练方法，以及从头训练进行培训的方法。为了促进3D医疗模型的进一步发展，我们的代码和预先接受的模型权重在https://github.com/urmagicsmine/cspr上公开使用。

Transfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -from 1 example per class to 1 M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.