Vision Transformers (ViTs) have gained significant popularity in recent years and have proliferated into many applications. However, it is not well explored how varied their behavior is under different learning paradigms. We compare ViTs trained through different methods of supervision, and show that they learn a diverse range of behaviors in terms of their attention, representations, and downstream performance. We also discover ViT behaviors that are consistent across supervision, including the emergence of Offset Local Attention Heads. These are self-attention heads that attend to a token adjacent to the current token with a fixed directional offset, a phenomenon that to the best of our knowledge has not been highlighted in any prior work. Our analysis shows that ViTs are highly flexible and learn to process local and global information in different orders depending on their training method. We find that contrastive self-supervised methods learn features that are competitive with explicitly supervised features, and they can even be superior for part-level tasks. We also find that the representations of reconstruction-based models show non-trivial similarity to contrastive self-supervised models. Finally, we show how the "best" layer for a given task varies by both supervision method and task, further demonstrating the differing order of information processing in ViTs.

研究表明，当训练数据缺少注释时，对象检测器的性能下降，即稀疏注释数据。当代方法专注于缺少地面实话注释的代理，无论是伪标签的形式还是通过在训练期间重新称重梯度。在这项工作中，我们重新审视了稀疏注释物体检测的制定。我们观察到稀疏注释的物体检测可以被认为是区域级的半监督对象检测问题。在此洞察力上，我们提出了一种基于区域的半监督算法，它自动识别包含未标记的前景对象的区域。我们的算法然后以不同的方式处理标记和未标记的前景区域，在半监督方法中进行常见做法。为了评估所提出的方法的有效性，我们对普斯卡尔库尔和可可数据集的稀疏注释方法常用的五种分裂进行详尽的实验，并实现最先进的性能。除此之外，我们还表明，我们的方法在标准半监督设置上实现了竞争性能，证明了我们的方法的实力和广泛适用性。

Deploying machine learning models in production may allow adversaries to infer sensitive information about training data. There is a vast literature analyzing different types of inference risks, ranging from membership inference to reconstruction attacks. Inspired by the success of games (i.e., probabilistic experiments) to study security properties in cryptography, some authors describe privacy inference risks in machine learning using a similar game-based style. However, adversary capabilities and goals are often stated in subtly different ways from one presentation to the other, which makes it hard to relate and compose results. In this paper, we present a game-based framework to systematize the body of knowledge on privacy inference risks in machine learning.

A distribution inference attack aims to infer statistical properties of data used to train machine learning models. These attacks are sometimes surprisingly potent, but the factors that impact distribution inference risk are not well understood and demonstrated attacks often rely on strong and unrealistic assumptions such as full knowledge of training environments even in supposedly black-box threat scenarios. To improve understanding of distribution inference risks, we develop a new black-box attack that even outperforms the best known white-box attack in most settings. Using this new attack, we evaluate distribution inference risk while relaxing a variety of assumptions about the adversary's knowledge under black-box access, like known model architectures and label-only access. Finally, we evaluate the effectiveness of previously proposed defenses and introduce new defenses. We find that although noise-based defenses appear to be ineffective, a simple re-sampling defense can be highly effective. Code is available at https://github.com/iamgroot42/dissecting_distribution_inference

Object-goal navigation (Object-nav) entails searching, recognizing and navigating to a target object. Object-nav has been extensively studied by the Embodied-AI community, but most solutions are often restricted to considering static objects (e.g., television, fridge, etc.). We propose a modular framework for object-nav that is able to efficiently search indoor environments for not just static objects but also movable objects (e.g. fruits, glasses, phones, etc.) that frequently change their positions due to human intervention. Our contextual-bandit agent efficiently explores the environment by showing optimism in the face of uncertainty and learns a model of the likelihood of spotting different objects from each navigable location. The likelihoods are used as rewards in a weighted minimum latency solver to deduce a trajectory for the robot. We evaluate our algorithms in two simulated environments and a real-world setting, to demonstrate high sample efficiency and reliability.

深度学习（DL）模型越来越多地为应用程序提供多种应用。不幸的是，这种普遍性也使它们成为提取攻击的有吸引力的目标，这些目标可以窃取目标DL模型的体系结构，参数和超参数。现有的提取攻击研究观察到不同DL模型和数据集的攻击成功水平不同，但其易感性背后的根本原因通常仍不清楚。确定此类根本原因弱点将有助于促进安全的DL系统，尽管这需要在各种情况下研究提取攻击，以确定跨攻击成功和DL特征的共同点。理解，实施和评估甚至单一攻击所需的绝大部分技术努力和时间都使探索现有的大量独特提取攻击方案是不可行的，当前框架通常设计用于仅针对特定攻击类型，数据集和数据集，以及硬件平台。在本文中，我们介绍捏：一个有效且自动化的提取攻击框架，能够在异质硬件平台上部署和评估多个DL模型和攻击。我们通过经验评估大量先前未开发的提取攻击情景以及次级攻击阶段来证明捏合的有效性。我们的主要发现表明，1）多个特征影响开采攻击成功跨越DL模型体系结构，数据集复杂性，硬件，攻击类型和2）部分成功的提取攻击显着增强了进一步的对抗攻击分期的成功。

深度学习研究引起了广泛的兴趣，导致出现了各种各样的技术创新和应用。由于深度学习研究的很大比例关注基于视觉的应用，因此存在使用其中一些技术来实现低功率便携式医疗保健诊断支持解决方案的潜力。在本文中，我们提出了一个基于硬件的嵌入式软件实施显微镜诊断支持系统，用于POC案例研究：（a）厚血液涂片中的疟疾，（b）痰液样品中的结核病，以及（c）（c）粪便中的肠道寄生虫感染样品。我们使用基于挤压网络的模型来减少网络大小和计算时间。我们还利用训练有素的量化技术来进一步减少学习模型的记忆足迹。这使基于显微镜的病原体检测将实验室专家级别的精度分类为独立的嵌入式硬件平台。与基于CPU的常规实施相比，提议的实施功率更高6倍，并且推理时间为$ \ sim $ 3 ms/示例。

语言，视觉和多模式预审查的大量融合正在出现。在这项工作中，我们介绍了通用多模式基础模型BEIT-3，该模型BEIT-3，该模型在视觉和视觉任务上都实现了最新的转移性能。具体来说，我们从三个方面提出了大融合：骨干架构，预训练任务和模型扩展。我们介绍了多道路变压器进行通用建模，其中模块化体系结构可以实现深融合和模态特定的编码。基于共享的骨干，我们以统一的方式对图像（Imglish），文本（英语）和图像文本对（“平行句子”）进行蒙面的“语言”建模。实验结果表明，BEIT-3在对象检测（COCO），语义分割（ADE20K），图像分类（Imagenet），视觉推理（NLVR2），视觉询问答案（VQAV2），图像字幕上获得最先进的性能（可可）和跨模式检索（Flickr30k，可可）。

当今的计算机不仅限于笔记本电脑和台式机。手机和笔记本电脑等移动小工具也可以利用它。但是，在过去50年中没有更改的一个输入设备是QWERTY键盘。通过传感器技术和人工智能，虚拟键盘用户可以在任何表面上输入任何表面。在这项研究中，我们使用图像处理的想法来创建一个应用程序，以使用新颖的框架来查看计算机键盘，该框架可以精确地检测手势，同时也具有可持续性且在财务上可行。相机用于捕获键盘图像和手指动作，后来充当虚拟键盘。此外，本研究还描述了一种接受手指坐标为输入的可见虚拟小鼠。该系统具有降低外围成本的直接好处，减少由于外部设备而产生的电子废物，并为无法使用传统键盘和鼠标的人们提供可访问性。

最紧迫的社会问题之一是与虚假新闻的斗争。虚假的主张很难暴露，造成了很多损害。为了解决这个问题，事实验证变得至关重要，因此是不同研究社区中感兴趣的话题。仅使用数据的文本形式，我们建议解决问题的解决方案，并通过其他方法实现竞争结果。我们基于两种方法（基于训练的语言模型）基于两种方法和基于提示的方法提供解决方案。基于PLM的方法使用传统的监督学习，其中训练模型以“ X”为输入和输出预测为P（Y | X）。鉴于，基于及时的学习反映了设计输入以适合模型的想法，以便可以将原始目标重新构成（掩盖）语言建模的问题。我们可能会进一步刺激PLM提供的丰富知识，以通过采用额外提示来微调PLM，以更好地完成下游任务。我们的实验表明，所提出的方法的性能不仅仅是微调PLM。我们在Trancify数据集中获得了0.6946的F1分数，在比赛负责人板上获得了第七名。