尽管变压器已经开始在视力中占主导地位，但将它们应用于大图像仍然很困难。这样做的一个很大的原因是，自我发场的标记数二次缩放，而令牌数量又随图像大小而倍增。在较大的图像（例如1080p）上，网络中总计算的60％以上仅用于创建和应用注意矩阵。我们通过引入Hydra注意来解决这个问题，这是视觉变压器（VITS）的极有效的关注操作。自相矛盾的是，这种效率来自对其极端的多头关注：通过使用尽可能多的注意力头部，Hydra注意力在代币和没有隐藏常数的特征上是线性的，使其比标准自我注意力要快得多。在现成的VIT-B/16中，代币计数的一倍。此外，Hydra注意力保留了ImageNet上的高精度，在某些情况下实际上可以改善它。

半监督对象检测（SSOD）的最新发展显示了利用未标记数据改善对象检测器的希望。但是，到目前为止，这些方法已经假设未标记的数据不包含分布（OOD）类，这对于较大规模的未标记数据集是不现实的。在本文中，我们考虑了一个更实用但具有挑战性的问题，开放式半监督对象检测（OSSOD）。我们首先发现现有的SSOD方法在开放式条件下获得了较低的性能增长，这是由语义扩展引起的，在该语义扩展中，分散注意力的OOD对象​​被错误预测为半监督训练的分布伪标签。为了解决此问题，我们考虑与SSOD方法集成的在线和离线OOD检测模块。通过广泛的研究，我们发现，基于自我监视的视觉变压器的脱机OOD检测器对在线OOD探测器的表现良好，因为它稳健地对伪标记的干扰。在实验中，我们提出的框架有效地解决了语义扩展问题，并在许多OSSOD基准（包括大规模的可可开放图）上显示出一致的改进。我们还在不同的OSSOD条件下验证框架的有效性，包括不同数量的分布类别，不同程度的监督和不同标记集的组合。

深度神经网络的兴起为优化推荐系统提供了重要的驱动力。但是，推荐系统的成功在于精致的建筑制造，因此呼吁神经建筑搜索（NAS）进一步改善其建模。我们提出了NASREC，它是一种训练单个超级网的范式，并通过重量共享有效地产生丰富的模型/子构造。为了克服数据多模式和体系结构异质性挑战，NASREC建立了一个大型的超级网（即搜索空间），以搜索完整的体系结构，而SuperNet结合了多功能操作员的选择和密集的连接性选择，并使人类的密集连接性最小化。 Nasrec的规模和异质性在搜索中构成了挑战，例如训练效率低下，操作员不平衡和降级等级相关性。我们通过提出单操作员任何连接采样，操作员平衡互动模块和训练后微调来应对这些挑战。我们对三个点击率（CTR）预测基准测试的结果表明，NASREC可以胜过手动设计的模型和现有的NAS方法，从而实现最先进的性能。

我们解决对象检测中的域适应问题，其中在源（带有监控）和目标域（没有监督的域的域名）之间存在显着的域移位。作为广泛采用的域适应方法，自培训教师学生框架（学生模型从教师模型生成的伪标签学习）在目标域中产生了显着的精度增益。然而，由于其偏向源域，它仍然存在从教师产生的大量低质量伪标签（例如，误报）。为了解决这个问题，我们提出了一种叫做自适应无偏见教师（AUT）的自我训练框架，利用对抗的对抗学习和弱强的数据增强来解决域名。具体而言，我们在学生模型中使用特征级的对抗性培训，确保从源和目标域中提取的功能共享类似的统计数据。这使学生模型能够捕获域不变的功能。此外，我们在目标领域的教师模型和两个域上的学生模型之间应用了弱强的增强和相互学习。这使得教师模型能够从学生模型中逐渐受益，而不会遭受域移位。我们展示了AUT通过大边距显示所有现有方法甚至Oracle（完全监督）模型的优势。例如，我们在有雾的城市景观（Clipart1k）上实现了50.9％（49.3％）地图，分别比以前的最先进和甲骨文高9.2％（5.2％）和8.2％（11.0％）

神经结构搜索（NAS）已被广泛采用设计准确，高效的图像分类模型。但是，将NAS应用于新的计算机愿景任务仍然需要大量的努力。这是因为1）以前的NAS研究已经过度优先考虑图像分类，同时在很大程度上忽略了其他任务; 2）许多NAS工作侧重于优化特定于任务特定的组件，这些组件不能有利地转移到其他任务; 3）现有的NAS方法通常被设计为“Proxyless”，需要大量努力与每个新任务的培训管道集成。为了解决这些挑战，我们提出了FBNetv5，这是一个NAS框架，可以在各种视觉任务中寻找神经架构，以降低计算成本和人力努力。具体而言，我们设计1）一个简单但包容性和可转换的搜索空间; 2）用目标任务培训管道解开的多址搜索过程; 3）一种算法，用于同时搜索具有计算成本不可知的多个任务的架构到任务数。我们评估所提出的FBNetv5目标三个基本视觉任务 - 图像分类，对象检测和语义分割。 FBNETV5在单一搜索中搜索的模型在所有三个任务中都表现优于先前的议定书 - 现有技术：图像分类（例如，与FBNetv3相比，在与FBNetv3相比的同一拖鞋下的1 + 1.3％Imageet Top-1精度。 （例如，+ 1.8％较高的Ade20k Val。Miou比SegFormer为3.6倍的拖鞋），对象检测（例如，+ 1.1％Coco Val。与yolox相比，拖鞋的1.2倍的地图。

Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS) methods are computationally expensive. ConvNet architecture optimality depends on factors such as input resolution and target devices. However, existing approaches are too resource demanding for case-by-case redesigns. Also, previous work focuses primarily on reducing FLOPs, but FLOP count does not always reflect actual latency. To address these, we propose a differentiable neural architecture search (DNAS) framework that uses gradient-based methods to optimize Con-vNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. FBNets (Facebook-Berkeley-Nets), a family of models discovered by DNAS surpass state-of-the-art models both designed manually and generated automatically. FBNet-B achieves 74.1% top-1 accuracy on ImageNet with 295M FLOPs and 23.1 ms latency on a Samsung S8 phone, 2.4x smaller and 1.5x faster than MobileNetV2-1.3[17] with similar accuracy. Despite higher accuracy and lower latency than MnasNet[20], we estimate FBNet-B's search cost is 420x smaller than MnasNet's, at only 216 GPUhours. Searched for different resolutions and channel sizes, FBNets achieve 1.5% to 6.4% higher accuracy than Mo-bileNetV2. The smallest FBNet achieves 50.2% accuracy and 2.9 ms latency (345 frames per second) on a Samsung S8. Over a Samsung-optimized FBNet, the iPhone-Xoptimized model achieves a 1.4x speedup on an iPhone X. FBNet models are open-sourced at https://github. com/facebookresearch/mobile-vision. * Work done while interning at Facebook.… Figure 1. Differentiable neural architecture search (DNAS) for ConvNet design. DNAS explores a layer-wise space that each layer of a ConvNet can choose a different block. The search space is represented by a stochastic super net. The search process trains the stochastic super net using SGD to optimize the architecture distribution. Optimal architectures are sampled from the trained distribution. The latency of each operator is measured on target devices and used to compute the loss for the super net.

When using LiDAR semantic segmentation models for safety-critical applications such as autonomous driving, it is essential to understand and improve their robustness with respect to a large range of LiDAR corruptions. In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. To rigorously evaluate the robustness and generalizability of current approaches, we propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic segmentation models, especially spanning different input representations (e.g., point clouds, voxels, projected images, and etc.), network architectures and training schemes. Through this study, we obtain two insights: 1) We find out that the input representation plays a crucial role in robustness. Specifically, under specific corruptions, different representations perform variously. 2) Although state-of-the-art methods on LiDAR semantic segmentation achieve promising results on clean data, they are less robust when dealing with noisy data. Finally, based on the above observations, we design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications. It is promising that our benchmark, comprehensive analysis, and observations can boost future research in robust LiDAR semantic segmentation for safety-critical applications.

In recent years, arbitrary image style transfer has attracted more and more attention. Given a pair of content and style images, a stylized one is hoped that retains the content from the former while catching style patterns from the latter. However, it is difficult to simultaneously keep well the trade-off between the content details and the style features. To stylize the image with sufficient style patterns, the content details may be damaged and sometimes the objects of images can not be distinguished clearly. For this reason, we present a new transformer-based method named STT for image style transfer and an edge loss which can enhance the content details apparently to avoid generating blurred results for excessive rendering on style features. Qualitative and quantitative experiments demonstrate that STT achieves comparable performance to state-of-the-art image style transfer methods while alleviating the content leak problem.

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.

Gaze estimation is the fundamental basis for many visual tasks. Yet, the high cost of acquiring gaze datasets with 3D annotations hinders the optimization and application of gaze estimation models. In this work, we propose a novel Head-Eye redirection parametric model based on Neural Radiance Field, which allows dense gaze data generation with view consistency and accurate gaze direction. Moreover, our head-eye redirection parametric model can decouple the face and eyes for separate neural rendering, so it can achieve the purpose of separately controlling the attributes of the face, identity, illumination, and eye gaze direction. Thus diverse 3D-aware gaze datasets could be obtained by manipulating the latent code belonging to different face attributions in an unsupervised manner. Extensive experiments on several benchmarks demonstrate the effectiveness of our method in domain generalization and domain adaptation for gaze estimation tasks.