Unlike traditional distributed machine learning, federated learning stores data locally for training and then aggregates the models on the server, which solves the data security problem that may arise in traditional distributed machine learning. However, during the training process, the transmission of model parameters can impose a significant load on the network bandwidth. It has been pointed out that the vast majority of model parameters are redundant during model parameter transmission. In this paper, we explore the data distribution law of selected partial model parameters on this basis, and propose a deep hierarchical quantization compression algorithm, which further compresses the model and reduces the network load brought by data transmission through the hierarchical quantization of model parameters. And we adopt a dynamic sampling strategy for the selection of clients to accelerate the convergence of the model. Experimental results on different public datasets demonstrate the effectiveness of our algorithm.

In recent years, vision-centric perception has flourished in various autonomous driving tasks, including 3D detection, semantic map construction, motion forecasting, and depth estimation. Nevertheless, the latency of vision-centric approaches is too high for practical deployment (e.g., most camera-based 3D detectors have a runtime greater than 300ms). To bridge the gap between ideal research and real-world applications, it is necessary to quantify the trade-off between performance and efficiency. Traditionally, autonomous-driving perception benchmarks perform the offline evaluation, neglecting the inference time delay. To mitigate the problem, we propose the Autonomous-driving StreAming Perception (ASAP) benchmark, which is the first benchmark to evaluate the online performance of vision-centric perception in autonomous driving. On the basis of the 2Hz annotated nuScenes dataset, we first propose an annotation-extending pipeline to generate high-frame-rate labels for the 12Hz raw images. Referring to the practical deployment, the Streaming Perception Under constRained-computation (SPUR) evaluation protocol is further constructed, where the 12Hz inputs are utilized for streaming evaluation under the constraints of different computational resources. In the ASAP benchmark, comprehensive experiment results reveal that the model rank alters under different constraints, suggesting that the model latency and computation budget should be considered as design choices to optimize the practical deployment. To facilitate further research, we establish baselines for camera-based streaming 3D detection, which consistently enhance the streaming performance across various hardware. ASAP project page: https://github.com/JeffWang987/ASAP.

Artificial intelligence is to teach machines to take actions like humans. To achieve intelligent teaching, the machine learning community becomes to think about a promising topic named machine teaching where the teacher is to design the optimal (usually minimal) teaching set given a target model and a specific learner. However, previous works usually require numerous teaching examples along with large iterations to guide learners to converge, which is costly. In this paper, we consider a more intelligent teaching paradigm named one-shot machine teaching which costs fewer examples to converge faster. Different from typical teaching, this advanced paradigm establishes a tractable mapping from the teaching set to the model parameter. Theoretically, we prove that this mapping is surjective, which serves to an existence guarantee of the optimal teaching set. Then, relying on the surjective mapping from the teaching set to the parameter, we develop a design strategy of the optimal teaching set under appropriate settings, of which two popular efficiency metrics, teaching dimension and iterative teaching dimension are one. Extensive experiments verify the efficiency of our strategy and further demonstrate the intelligence of this new teaching paradigm.

Contrastive Language-Image Pre-trained (CLIP) models have zero-shot ability of classifying an image belonging to "[CLASS]" by using similarity between the image and the prompt sentence "a [CONTEXT] of [CLASS]". Based on exhaustive text cues in "[CONTEXT]", CLIP model is aware of different contexts, e.g. background, style, viewpoint, and exhibits unprecedented robustness against a wide range of distribution shifts. However, recent works find further fine-tuning of CLIP models improves accuracy but sacrifices the robustness on downstream tasks. We conduct an empirical investigation to show fine-tuning will corrupt the context-aware ability of pre-trained CLIP features. To solve this problem, we propose Context-Aware Robust Fine-tuning (CAR-FT). CAR-FT regularizes the model during fine-tuning to capture the context information. Specifically, we use zero-shot prompt weights to get the context distribution contained in the image. By minimizing the Kullback-Leibler Divergence (KLD) between context distributions induced by original/fine-tuned CLIP models, CAR-FT makes the context-aware ability of CLIP inherited into downstream tasks, and achieves both higher In-Distribution (ID) and Out-Of-Distribution (OOD) accuracy. The experimental results show CAR-FT achieves superior robustness on five OOD test datasets of ImageNet, and meanwhile brings accuracy gains on nine downstream tasks. Additionally, CAR-FT surpasses previous Domain Generalization (DG) methods and gets 78.5% averaged accuracy on DomainBed benchmark, building the new state-of-the-art.

Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

域的概括旨在学习一个可以很好地概括在看不见的测试数据集（即分布数据集）上的模型，该数据与培训数据集不同。为了解决计算机视觉中的领域概括，我们将损失景观理论引入该领域。具体而言，我们从损失景观的角度从四个方面（包括骨干，正则化，训练范式和学习率）引起了深度学习模型的概括能力。我们通过进行广泛的消融研究和可视化来验证有关NICO ++，PAC和VLCS数据集的提议理论。此外，我们将该理论应用于ECCV 2022 NICO挑战1，并在不使用任何域不变方法的情况下获得第三名。

对抗性训练（AT）通常被认为是防御对抗性例子的最有效的方法之一，可能会在很大程度上损害标准绩效，因此对工业规模的生产和应用的有用性有限。令人惊讶的是，这种现象在自然语言处理（NLP）任务中完全相反，在该任务中甚至可以从中受益。我们注意到NLP任务中AT的优点可能来自离散和符号输入空间。为了借用NLP风格的优势，我们提出了离散的对抗训练（DAT）。 DAT利用VQGAN改革图像数据以离散类似文本的输入，即视觉单词。然后，它可以最大程度地减少这种离散图像的最大风险，并具有符号对抗扰动。我们从分布的角度进一步提供了解释，以证明DAT的有效性。作为增强视觉表示的插件技术，DAT可以在多个任务上取得重大改进，包括图像分类，对象检测和自我监督学习。尤其是，该模型通过胶带自动编码（MAE）预先训练并由我们的DAT进行微调，而没有额外的数据可以在Imagenet-C上获得31.40 MCE，并且在Stylized-Imagenet上进行了32.77％的TOP-1准确性，建立了新的状态 - 艺术。该代码将在https://github.com/alibaba/easyrobust上找到。

这项研究提出了一种基于深度学习的超声（US）图像引导放射疗法的跟踪方法。拟议的级联深度学习模型由注意力网络，基于掩模区域的卷积神经网络（Mask R-CNN）和长期短期记忆（LSTM）网络组成。注意网络从美国图像到可疑的具有里程碑意义的运动区域，以减少搜索区域。然后，面膜R-CNN在减少区域中产生多个利益区域（ROI）建议，并通过三个网络头确定拟议的地标：边界框回归，提案分类和地标分段。 LSTM网络对连续的图像框架之间的时间关系建模，以进行边界框回归和建议分类。为了合并最终建议，根据顺序框架之间的相似性设计选择方法。该方法在肝脏美国跟踪数据集中测试了医疗图像计算和计算机辅助干预措施（MICCAI）2015年的挑战，其中有三位经验丰富的观察者注释了地标，以获得其平均位置。在24个鉴于我们具有地面真相的序列的24个序列上，所有地标的平均跟踪误差为0.65 +/- 0.56毫米，所有地标的误差均在2 mm之内。我们进一步测试了从测试数据集中的69个地标上提出的模型，该模型具有与训练模式相似的图像模式，从而导致平均跟踪误差为0.94 +/- 0.83 mm。我们的实验结果表明，我们提出的方法使用US图像跟踪肝解剖学地标的可行性和准确性，为放射治疗期间的主动运动管理提供了潜在的解决方案。

本文回顾了AIM 2022上压缩图像和视频超级分辨率的挑战。这项挑战包括两条曲目。轨道1的目标是压缩图像的超分辨率，轨迹〜2靶向压缩视频的超分辨率。在轨道1中，我们使用流行的数据集DIV2K作为培训，验证和测试集。在轨道2中，我们提出了LDV 3.0数据集，其中包含365个视频，包括LDV 2.0数据集（335个视频）和30个其他视频。在这一挑战中，有12支球队和2支球队分别提交了赛道1和赛道2的最终结果。所提出的方法和解决方案衡量了压缩图像和视频上超分辨率的最先进。提出的LDV 3.0数据集可在https://github.com/renyang-home/ldv_dataset上找到。此挑战的首页是在https://github.com/renyang-home/aim22_compresssr。

自我监督的单眼方法可以有效地了解弱纹理表面或反射性对象的深度信息。但是，由于单眼几何建模的固有歧义，深度精度受到限制。相反，由于多视图立体声（MVS）的成功，多帧深度估计方法提高了深度准确性，后者直接使用几何约束。不幸的是，MV经常患有无纹理区域，非斜角表面和移动物体，尤其是在没有已知的相机运动和深度监督的现实世界视频序列中。因此，我们提出了MoveEpth，它利用了单眼线索和速度指导来改善多帧深度学习。与现有的MVS深度和单眼深度之间一致性的方法不同，MoveEpth通过直接解决MV的固有问题来增强多帧深度学习。我们方法的关键是利用单眼深度作为几何优先级来构建MVS成本量，并根据预测的相机速度的指导来调整成本量的深度候选。我们通过学习成本量的不确定性来进一步融合单眼深度和MVS深度，从而导致深度估计多视图几何形状的歧义。广泛的实验表明，移动eptth达到了最先进的性能：与monodepth2和packnet相比，我们的方法相对地将深度准确性提高了20 \％和19.8 \％，而Kitti基准测试的方法则提高了。 MoveEpth还推广到更具挑战性的DDAD基准测试，相对超过7.2 \％。该代码可在https://github.com/jeffwang987/movedepth上获得。