分析显微镜图像中细胞的形态可以为化合物或基因的功能提供洞察。解决此任务需要不仅可以从图像中提取生物信息的方法，而且还忽略了技术变异，即，用于收集显微镜图像的设备之间的实验过程或差异的变化。我们提出了与专家混合（团队）的嵌入学习方法提出了治疗计划，该方法学习了一组专家，专门专门捕获我们的培训集中的技术变异，然后在测试时间汇总专家的预测。因此，通过最大限度地减少每个专家的噪声，团队可以通过更少的技术变化偏差来学习强大的嵌入。要培训我们的模型，我们利用了处理样本，使我们的方法能够在每个小靶中捕获整个数据集的分布，同时仍然适用于GPU存储器。我们在三个数据集中评估了我们的方法，如药物发现，促进了识别细胞治疗的真实作用机制的表现，通过最先进的5.5-11％。

High content imaging assays can capture rich phenotypic response data for large sets of compound treatments, aiding in the characterization and discovery of novel drugs. However, extracting representative features from high content images that can capture subtle nuances in phenotypes remains challenging. The lack of high-quality labels makes it difficult to achieve satisfactory results with supervised deep learning. Self-Supervised learning methods, which learn from automatically generated labels has shown great success on natural images, offer an attractive alternative also to microscopy images. However, we find that self-supervised learning techniques underperform on high content imaging assays. One challenge is the undesirable domain shifts present in the data known as batch effects, which may be caused by biological noise or uncontrolled experimental conditions. To this end, we introduce Cross-Domain Consistency Learning (CDCL), a novel approach that is able to learn in the presence of batch effects. CDCL enforces the learning of biological similarities while disregarding undesirable batch-specific signals, which leads to more useful and versatile representations. These features are organised according to their morphological changes and are more useful for downstream tasks - such as distinguishing treatments and mode of action.

我们将WS-DINO作为一种新型框架，以从细胞的高内感荧光图像学习表型表示中使用弱标记信息。我们的模型基于具有视觉变压器骨干（Dino）的知识蒸馏方法，我们将其用作研究的基准模型。使用WS-DINO，我们对高含量显微镜屏幕（处理和化合物）中可用的弱标签信息进行了微调，并在BBBC021数据集的非同样化合物的动作预测中实现了最先进的性能（98％），并使用该化合物作为弱标签，而非类型的化合物和批处理性能（96％）。我们的方法绕过单细胞种植作为预处理步骤，并使用自发图表表明该模型学习结构上有意义的表型曲线。

Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques that rely less on manual annotations (e.g., image, bounding-box, and per-pixel labels), but learn useful information from unlabeled data. In this thesis, we explore five different techniques for handling AEL.

Contrastive learning has become a key component of self-supervised learning approaches for computer vision. By learning to embed two augmented versions of the same image close to each other and to push the embeddings of different images apart, one can train highly transferable visual representations. As revealed by recent studies, heavy data augmentation and large sets of negatives are both crucial in learning such representations. At the same time, data mixing strategies, either at the image or the feature level, improve both supervised and semi-supervised learning by synthesizing novel examples, forcing networks to learn more robust features. In this paper, we argue that an important aspect of contrastive learning, i.e. the effect of hard negatives, has so far been neglected. To get more meaningful negative samples, current top contrastive self-supervised learning approaches either substantially increase the batch sizes, or keep very large memory banks; increasing memory requirements, however, leads to diminishing returns in terms of performance. We therefore start by delving deeper into a top-performing framework and show evidence that harder negatives are needed to facilitate better and faster learning. Based on these observations, and motivated by the success of data mixing, we propose hard negative mixing strategies at the feature level, that can be computed on-the-fly with a minimal computational overhead. We exhaustively ablate our approach on linear classification, object detection, and instance segmentation and show that employing our hard negative mixing procedure improves the quality of visual representations learned by a state-of-the-art self-supervised learning method.Project page: https://europe.naverlabs.com/mochi 34th Conference on Neural Information Processing Systems (NeurIPS 2020),

State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.

Molecular and genomic properties are critical in selecting cancer treatments to target individual tumors, particularly for immunotherapy. However, the methods to assess such properties are expensive, time-consuming, and often not routinely performed. Applying machine learning to H&E images can provide a more cost-effective screening method. Dozens of studies over the last few years have demonstrated that a variety of molecular biomarkers can be predicted from H&E alone using the advancements of deep learning: molecular alterations, genomic subtypes, protein biomarkers, and even the presence of viruses. This article reviews the diverse applications across cancer types and the methodology to train and validate these models on whole slide images. From bottom-up to pathologist-driven to hybrid approaches, the leading trends include a variety of weakly supervised deep learning-based approaches, as well as mechanisms for training strongly supervised models in select situations. While results of these algorithms look promising, some challenges still persist, including small training sets, rigorous validation, and model explainability. Biomarker prediction models may yield a screening method to determine when to run molecular tests or an alternative when molecular tests are not possible. They also create new opportunities in quantifying intratumoral heterogeneity and predicting patient outcomes.

Unsupervised image representations have significantly reduced the gap with supervised pretraining, notably with the recent achievements of contrastive learning methods. These contrastive methods typically work online and rely on a large number of explicit pairwise feature comparisons, which is computationally challenging. In this paper, we propose an online algorithm, SwAV, that takes advantage of contrastive methods without requiring to compute pairwise comparisons. Specifically, our method simultaneously clusters the data while enforcing consistency between cluster assignments produced for different augmentations (or "views") of the same image, instead of comparing features directly as in contrastive learning. Simply put, we use a "swapped" prediction mechanism where we predict the code of a view from the representation of another view. Our method can be trained with large and small batches and can scale to unlimited amounts of data. Compared to previous contrastive methods, our method is more memory efficient since it does not require a large memory bank or a special momentum network. In addition, we also propose a new data augmentation strategy, multi-crop, that uses a mix of views with different resolutions in place of two full-resolution views, without increasing the memory or compute requirements. We validate our findings by achieving 75.3% top-1 accuracy on ImageNet with ResNet-50, as well as surpassing supervised pretraining on all the considered transfer tasks.

Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models. Modern batch contrastive approaches subsume or significantly outperform traditional contrastive losses such as triplet, max-margin and the N-pairs loss. In this work, we extend the self-supervised batch contrastive approach to the fully-supervised setting, allowing us to effectively leverage label information. Clusters of points belonging to the same class are pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes. We analyze two possible versions of the supervised contrastive (SupCon) loss, identifying the best-performing formulation of the loss. On ResNet-200, we achieve top-1 accuracy of 81.4% on the Ima-geNet dataset, which is 0.8% above the best number reported for this architecture. We show consistent outperformance over cross-entropy on other datasets and two ResNet variants. The loss shows benefits for robustness to natural corruptions, and is more stable to hyperparameter settings such as optimizers and data augmentations. Our loss function is simple to implement and reference TensorFlow code is released at https://t.ly/supcon 1 .

Recent methods for deep metric learning have been focusing on designing different contrastive loss functions between positive and negative pairs of samples so that the learned feature embedding is able to pull positive samples of the same class closer and push negative samples from different classes away from each other. In this work, we recognize that there is a significant semantic gap between features at the intermediate feature layer and class labels at the final output layer. To bridge this gap, we develop a contrastive Bayesian analysis to characterize and model the posterior probabilities of image labels conditioned by their features similarity in a contrastive learning setting. This contrastive Bayesian analysis leads to a new loss function for deep metric learning. To improve the generalization capability of the proposed method onto new classes, we further extend the contrastive Bayesian loss with a metric variance constraint. Our experimental results and ablation studies demonstrate that the proposed contrastive Bayesian metric learning method significantly improves the performance of deep metric learning in both supervised and pseudo-supervised scenarios, outperforming existing methods by a large margin.

图像回归任务，如骨矿物密度（BMD）估计和左心室喷射分数（LVEF）预测，在计算机辅助疾病评估中起重要作用。大多数深度回归方法用单一的回归损耗函数训练神经网络，如MSE或L1损耗。在本文中，我们提出了一种用于深度图像回归的第一个对比学习框架，即adacon，其包括通过新颖的自适应边缘对比损耗和回归预测分支的特征学习分支组成。我们的方法包含标签距离关系作为学习特征表示的一部分，这允许在下游回归任务中进行更好的性能。此外，它可以用作即插即用模块，以提高现有回归方法的性能。我们展示了adacon对来自X射线图像的骨矿物密度估计和来自超声心动图象的X射线图像和左心室喷射分数预测的骨矿物密度估计的有效性。 Adacon分别导致MAE在最先进的BMD估计和LVEF预测方法中相对提高3.3％和5.9％。

动机：近年来，基于形象的生物测定稳步成为高吞吐量，引发了快速自动化方法，以提取来自数百种图像的生物学有意义的信息。从想象成的成功取得灵感，我们驯服细胞造就花，一个公开源和弱标记的显微镜图像的大规模数据集（890K图像，894级）。预先训练的细胞造黄养箱产生了对上游显微镜分类任务的想象成特征具有竞争力的功能。我们展示了CytoImAgenet的证据表明，CytoImAgenet在想象中训练有素的功能中捕获信息不可用。数据集是在https://www.kaggle.com/stanleyhua/cyaagenet中提供的。

Image classification with small datasets has been an active research area in the recent past. However, as research in this scope is still in its infancy, two key ingredients are missing for ensuring reliable and truthful progress: a systematic and extensive overview of the state of the art, and a common benchmark to allow for objective comparisons between published methods. This article addresses both issues. First, we systematically organize and connect past studies to consolidate a community that is currently fragmented and scattered. Second, we propose a common benchmark that allows for an objective comparison of approaches. It consists of five datasets spanning various domains (e.g., natural images, medical imagery, satellite data) and data types (RGB, grayscale, multispectral). We use this benchmark to re-evaluate the standard cross-entropy baseline and ten existing methods published between 2017 and 2021 at renowned venues. Surprisingly, we find that thorough hyper-parameter tuning on held-out validation data results in a highly competitive baseline and highlights a stunted growth of performance over the years. Indeed, only a single specialized method dating back to 2019 clearly wins our benchmark and outperforms the baseline classifier.

药物重新利用可以加速鉴定有效化合物用于针对SARS-COV-2的临床使用，并具有先前存在的临床安全数据和已建立的供应链的优势。 RNA病毒（例如SARS-COV-2）操纵细胞途径并诱导亚细胞结构的重组以支持其生命周期。可以使用生物成像技术来量化这些形态学的变化。在这项工作中，我们开发了DEEMD：使用深层神经网络模型在多个实例学习框架内的计算管道，以基于对公开可用RXRX19A数据集的形态分析来确定针对SARS-COV-2有效的推定治疗方法。该数据集由SARS-COV-2未感染的细胞和受感染细胞的荧光显微镜图像组成，有或没有药物治疗。 Deemd首先提取歧视性形态学特征，以产生来自未感染和感染细胞的细胞形态特征。然后在统计模型中使用这些形态学特征，以根据与未感染细胞的相似性估算受感染细胞的应用治疗疗效。 DEEMD能够通过弱监督定位受感染的细胞，而无需任何昂贵的像素级注释。 DEEMD确定已知的SARS-COV-2抑制剂，例如Remdesivir和Aloxistatin，支持我们方法的有效性。可以在其他新兴病毒和数据集上探索DEEMD，以便将来快速识别候选抗病毒药治疗}。我们的实施可在线网络https://www.github.com/sadegh-saberian/deemd

基因组工程正在进行前所未有的发展，现在已广泛可用。为确保负责任的生物技术创新并减少滥用工程DNA序列，为识别工程型质粒的起源实验室来说是至关重要的。基因工程归因（GEA），制定序列实验室协会的能力将支持这一过程中的法医专家。在这里，我们提出了一种基于度量学习的方法，该方法将最可能的原产实验室排名，同时为质粒序列和实验室产生嵌入。这些嵌入物可用于执行各种下游任务，例如聚类DNA序列和实验室，以及在机器学习模型中使用它们作为特征。我们的方法采用了循环转移增强方法，能够在前10个预测中正确地将原产于原产的90亿美元的时间排列 - 优于所有最新的最先进的方法。我们还证明我们可以使用只需10次\％$ 10 \％$ of序列进行几次拍摄学习并获得76±10美元的准确性。这意味着，我们仅使用第十个数据表达先前的CNN方法。我们还证明我们能够在特定实验室中提取质粒序列中的关键签名，允许对模型的产出进行可解释的检查。

This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive selfsupervised learning algorithms without requiring specialized architectures or a memory bank. In order to understand what enables the contrastive prediction tasks to learn useful representations, we systematically study the major components of our framework. We show that (1) composition of data augmentations plays a critical role in defining effective predictive tasks, (2) introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and (3) contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. By combining these findings, we are able to considerably outperform previous methods for self-supervised and semi-supervised learning on ImageNet. A linear classifier trained on self-supervised representations learned by Sim-CLR achieves 76.5% top-1 accuracy, which is a 7% relative improvement over previous state-ofthe-art, matching the performance of a supervised ResNet-50. When fine-tuned on only 1% of the labels, we achieve 85.8% top-5 accuracy, outperforming AlexNet with 100× fewer labels. 1

细粒度的图像分析（FGIA）是计算机视觉和模式识别中的长期和基本问题，并为一组多种现实世界应用提供了基础。 FGIA的任务是从属类别分析视觉物体，例如汽车或汽车型号的种类。细粒度分析中固有的小阶级和阶级阶级内变异使其成为一个具有挑战性的问题。利用深度学习的进步，近年来，我们在深入学习动力的FGIA中见证了显着进展。在本文中，我们对这些进展的系统进行了系统的调查，我们试图通过巩固两个基本的细粒度研究领域 - 细粒度的图像识别和细粒度的图像检索来重新定义和扩大FGIA领域。此外，我们还审查了FGIA的其他关键问题，例如公开可用的基准数据集和相关域的特定于应用程序。我们通过突出几个研究方向和开放问题，从社区中突出了几个研究方向和开放问题。

机器学习模型通常会遇到与训练分布不同的样本。无法识别分布（OOD）样本，因此将该样本分配给课堂标签会显着损害模​​型的可靠性。由于其对在开放世界中的安全部署模型的重要性，该问题引起了重大关注。由于对所有可能的未知分布进行建模的棘手性，检测OOD样品是具有挑战性的。迄今为止，一些研究领域解决了检测陌生样本的问题，包括异常检测，新颖性检测，一级学习，开放式识别识别和分布外检测。尽管有相似和共同的概念，但分别分布，开放式检测和异常检测已被独立研究。因此，这些研究途径尚未交叉授粉，创造了研究障碍。尽管某些调查打算概述这些方法，但它们似乎仅关注特定领域，而无需检查不同领域之间的关系。这项调查旨在在确定其共同点的同时，对各个领域的众多著名作品进行跨域和全面的审查。研究人员可以从不同领域的研究进展概述中受益，并协同发展未来的方法。此外，据我们所知，虽然进行异常检测或单级学习进行了调查，但没有关于分布外检测的全面或最新的调查，我们的调查可广泛涵盖。最后，有了统一的跨域视角，我们讨论并阐明了未来的研究线，打算将这些领域更加紧密地融为一体。

对比度学习最近在无监督的视觉表示学习中显示出巨大的潜力。在此轨道中的现有研究主要集中于图像内不变性学习。学习通常使用丰富的图像内变换来构建正对，然后使用对比度损失最大化一致性。相反，相互影响不变性的优点仍然少得多。利用图像间不变性的一个主要障碍是，尚不清楚如何可靠地构建图像间的正对，并进一步从它们中获得有效的监督，因为没有配对注释可用。在这项工作中，我们提出了一项全面的实证研究，以更好地了解从三个主要组成部分的形象间不变性学习的作用：伪标签维护，采样策略和决策边界设计。为了促进这项研究，我们引入了一个统一的通用框架，该框架支持无监督的内部和间形内不变性学习的整合。通过精心设计的比较和分析，揭示了多个有价值的观察结果：1）在线标签收敛速度比离线标签更快； 2）半硬性样品比硬否定样品更可靠和公正； 3）一个不太严格的决策边界更有利于形象间的不变性学习。借助所有获得的食谱，我们的最终模型（即InterCLR）对多个标准基准测试的最先进的内图内不变性学习方法表现出一致的改进。我们希望这项工作将为设计有效的无监督间歇性不变性学习提供有用的经验。代码：https：//github.com/open-mmlab/mmselfsup。

在过去几年中，无监督的学习取得了很大的进展，特别是通过对比的自我监督学习。用于基准测试自我监督学习的主导数据集已经想象，最近的方法正在接近通过完全监督培训实现的性能。然而，ImageNet DataSet在很大程度上是以对象为中心的，并且目前尚不清楚这些方法的广泛不同的数据集和任务，这些方法是非以对象为中心的，例如数字病理学。虽然自我监督的学习已经开始在这个领域探讨了令人鼓舞的结果，但有理由看起来更接近这个环境与自然图像和想象成的不同。在本文中，我们对组织病理学进行了对比学学习的深入分析，引脚指向对比物镜的表现如何不同，由于组织病理学数据的特征。我们提出了一些考虑因素，例如对比目标和超参数调整的观点。在大量的实验中，我们分析了组织分类的下游性能如何受到这些考虑因素的影响。结果指出了对比学习如何减少数字病理中的注释工作，但需要考虑特定的数据集特征。为了充分利用对比学习目标，需要不同的视野和超参数校准。我们的结果为实现组织病理学应用的自我监督学习的全部潜力铺平了道路。