这项工作是在培训生成动作/视频识别模型上，其输出是描述视频的自由形式的特定动作标题（而不是动作类标签）。生成的方法具有实用的优势，例如生产更细粒度和人类可读的产出，并且自然而然地是开放的。为此，我们提议适应视频/动作识别的预先训练的生成视觉和语言（V＆L）基础模型。据我们所知，最近有几次尝试适应了用对比度学习（例如剪辑）训练的V＆L模型（例如剪辑），但据我们所知，我们提出了第一种设定实现这一目标的方法来实现生成模型的方法。我们首先表明，生成模型的直接微调生产具有严重过度拟合的动作类别。为了减轻这一点，我们介绍了REST，这是一个由两个关键组成部分组成的培训框架：一种无监督的方法，用于通过伪捕获生成和自我训练，将生成模型适应动作/视频，即不使用任何动作特定的标签； （b）基于剪辑的检索方法，用于为每个视频发现一套伪装的伪扣，以训练该模型。重要的是，我们表明这两个组件对于获得高精度都是必要的。我们评估零拍动识别的问题的休息，我们表明，与基于对比的学习方法相比，我们的方法非常有竞争力。代码将可用。

本文解决了有效的视频识别问题。在这一领域，视频变压器最近在效率（Top-1精度与Flops）频谱中占据了主导地位。同时，在图像域中进行了一些尝试，这些尝试挑战了变压器体系结构中自我发挥操作的必要性，主张使用更简单的方法来进行令牌混合。但是，对于视频识别的情况，尚无结果，在这种情况下，自我发项操作员对效率的影响（与图像的情况相比）明显更高。为了解决这一差距，在本文中，我们做出以下贡献：（a）我们基于移位操作员，构成的仿射偏移块构建了一个高效\＆精确的无注意块，专门为尽可能近的近似而设计变压器层的MHSA块中的操作。基于我们的仿射转移块，我们构建了我们的仿射转移变压器，并表明它已经超过了所有现有的基于移位/MLP的架构进行Imagenet分类。 （b）我们将公式扩展到视频域中，以构建视频播客变压器（vast），这是第一个纯粹无注意的基于偏移的视频变压器。 （c）我们表明，对于最流行的动作识别基准，对于具有低计算和内存足迹的模型的情况，大量的最新变压器在最流行的动作识别基准上表现出色。代码将可用。

通过自学学习的视觉表示是一项极具挑战性的任务，因为网络需要在没有监督提供的主动指导的情况下筛选出相关模式。这是通过大量数据增强，大规模数据集和过量量的计算来实现的。视频自我监督学习（SSL）面临着额外的挑战：视频数据集通常不如图像数据集那么大，计算是一个数量级，并且优化器所必须通过的伪造模式数量乘以几倍。因此，直接从视频数据中学习自我监督的表示可能会导致次优性能。为了解决这个问题，我们建议在视频表示学习框架中利用一个以自我或语言监督为基础的强大模型，并在不依赖视频标记的数据的情况下学习强大的空间和时间信息。为此，我们修改了典型的基于视频的SSL设计和目标，以鼓励视频编码器\ textit {subsume}基于图像模型的语义内容，该模型在通用域上训练。所提出的算法被证明可以更有效地学习（即在较小的时期和较小的批次中），并在单模式SSL方法中对标准下游任务进行了新的最新性能。

基于自我注意力的模型，例如视觉变压器（VIT），已经成为计算机视觉中卷积神经网络（CNN）的一种非常有竞争力的建筑。尽管越来越高的变体具有更高的识别精度，但由于自我注意力的二次复杂性，现有的VIT通常在计算和模型大小中要求。尽管已重新引入了最近的CNN的几种成功设计选择（例如，卷积和分层多阶段结构）已重新引入最近的VIT，但它们仍然不足以满足移动设备的有限资源要求。这激发了最近根据最先进的Mobilenet-V2开发光线的尝试，但仍然留下了性能差距。在这项工作中，在这个研究不足的方向上进一步推动了Edgevits，这是一个新的轻巧vits家族，这首先使基于注意力的视觉模型能够与最佳轻巧的CNN竞争，这准确性和设备效率。这是通过基于自我注意力和卷积的最佳整合而引入高度成本效益的本地 - 全球局（LGL）信息交换瓶颈来实现的。对于设备青年的评估，我们不再依赖诸如拖船或参数的不准确代理，而是采用一种实用的方法来直接专注于设备延迟，以及首次首次提供能源效率。具体而言，我们表明，当考虑准确性的延迟和准确性 - 能量折衷时，我们的模型是帕累托最佳的，在几乎所有情况下都严格占据了其他VIT并与最有效的CNN竞争的严格优势。代码可从https://github.com/saic-fi/edgevit获得。

基于Heatmap回归的深度学习模型彻底改变了面部地标定位的任务，现有模型在大型姿势，非均匀照明和阴影，闭塞和自闭合，低分辨率和模糊。然而，尽管采用了广泛的采用，Heatmap回归方法遭受与热图编码和解码过程相关的离散化引起的误差。在这项工作中，我们表明这些误差对面部对准精度具有令人惊讶的大量负面影响。为了减轻这个问题，我们通过利用底层连续分布提出了一种热爱编码和解码过程的新方法。为了充分利用新提出的编码解码机制，我们还介绍了基于暹罗的训练，该训练能够在各种几何图像变换上实施热线图一致性。我们的方法在多个数据集中提供了明显的增益，在面部地标本地化中设置新的最先进的结果。旁边的代码将在https://www.adrianbulat.com/face-alignment上提供

学习普遍面孔表示的最佳方法是什么？在面部分析领域进行深度学习的最新工作集中在监督方面的学习特定任务（例如面部识别，面部地标本地化等），但忽略了如何找到可以轻松适应面部表征的总体问题到几个面部分析任务和数据集。为此，我们做出以下4个贡献：（a）我们首次介绍面部表示学习的全面评估基准，该基准由5个重要​​的面部分析任务组成。 （b）我们系统地研究了应用于面孔的大规模表示学习的两种方式：受监督和无监督的预训练。重要的是，我们将评估重点放在几乎没有面部学习的情况下。 （c）我们研究了培训数据集的重要特性，包括其大小和质量（标记，未标记甚至未经保育）。 （d）为了得出结论，我们进行了大量实验。我们的主要两个发现是：（1）完全在野外的未经监督的预培训，未经保育的数据提供了一致的，在某些情况下，对所有面部任务进行了显着准确的改进。 （2）许多现有的面部视频数据集似乎具有大量冗余。我们将发布代码和预先培训的模型，以促进未来的研究。

We address the challenge of building domain-specific knowledge models for industrial use cases, where labelled data and taxonomic information is initially scarce. Our focus is on inductive link prediction models as a basis for practical tools that support knowledge engineers with exploring text collections and discovering and linking new (so-called open-world) entities to the knowledge graph. We argue that - though neural approaches to text mining have yielded impressive results in the past years - current benchmarks do not reflect the typical challenges encountered in the industrial wild properly. Therefore, our first contribution is an open benchmark coined IRT2 (inductive reasoning with text) that (1) covers knowledge graphs of varying sizes (including very small ones), (2) comes with incidental, low-quality text mentions, and (3) includes not only triple completion but also ranking, which is relevant for supporting experts with discovery tasks. We investigate two neural models for inductive link prediction, one based on end-to-end learning and one that learns from the knowledge graph and text data in separate steps. These models compete with a strong bag-of-words baseline. The results show a significant advance in performance for the neural approaches as soon as the available graph data decreases for linking. For ranking, the results are promising, and the neural approaches outperform the sparse retriever by a wide margin.

Artificial Intelligence (AI) and Machine Learning (ML) are weaving their way into the fabric of society, where they are playing a crucial role in numerous facets of our lives. As we witness the increased deployment of AI and ML in various types of devices, we benefit from their use into energy-efficient algorithms for low powered devices. In this paper, we investigate a scale and medium that is far smaller than conventional devices as we move towards molecular systems that can be utilized to perform machine learning functions, i.e., Molecular Machine Learning (MML). Fundamental to the operation of MML is the transport, processing, and interpretation of information propagated by molecules through chemical reactions. We begin by reviewing the current approaches that have been developed for MML, before we move towards potential new directions that rely on gene regulatory networks inside biological organisms as well as their population interactions to create neural networks. We then investigate mechanisms for training machine learning structures in biological cells based on calcium signaling and demonstrate their application to build an Analog to Digital Converter (ADC). Lastly, we look at potential future directions as well as challenges that this area could solve.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Post-hoc explanation methods are used with the intent of providing insights about neural networks and are sometimes said to help engender trust in their outputs. However, popular explanations methods have been found to be fragile to minor perturbations of input features or model parameters. Relying on constraint relaxation techniques from non-convex optimization, we develop a method that upper-bounds the largest change an adversary can make to a gradient-based explanation via bounded manipulation of either the input features or model parameters. By propagating a compact input or parameter set as symbolic intervals through the forwards and backwards computations of the neural network we can formally certify the robustness of gradient-based explanations. Our bounds are differentiable, hence we can incorporate provable explanation robustness into neural network training. Empirically, our method surpasses the robustness provided by previous heuristic approaches. We find that our training method is the only method able to learn neural networks with certificates of explanation robustness across all six datasets tested.