Recently, the joint learning framework (JOINT) integrates matching based transductive reasoning and online inductive learning to achieve accurate and robust semi-supervised video object segmentation (SVOS). However, using the mask embedding as the label to guide the generation of target features in the two branches may result in inadequate target representation and degrade the performance. Besides, how to reasonably fuse the target features in the two different branches rather than simply adding them together to avoid the adverse effect of one dominant branch has not been investigated. In this paper, we propose a novel framework that emphasizes Learning to Learn Better (LLB) target features for SVOS, termed LLB, where we design the discriminative label generation module (DLGM) and the adaptive fusion module to address these issues. Technically, the DLGM takes the background-filtered frame instead of the target mask as input and adopts a lightweight encoder to generate the target features, which serves as the label of the online few-shot learner and the value of the decoder in the transformer to guide the two branches to learn more discriminative target representation. The adaptive fusion module maintains a learnable gate for each branch, which reweighs the element-wise feature representation and allows an adaptive amount of target information in each branch flowing to the fused target feature, thus preventing one branch from being dominant and making the target feature more robust to distractor. Extensive experiments on public benchmarks show that our proposed LLB method achieves state-of-the-art performance.

为了获得下游图像信号过程（ISP）的高质量的原始图像，在本文中，我们提出了一个有效的本地乘法变压器，称为ELMFORMER，用于原始图像恢复。 Elmformer包含两个核心设计，尤其是针对原始属性是单渠道的原始图像。第一个设计是双向融合投影（BFP）模块，我们考虑了原始图像的颜色特征和单渠道的空间结构。第二个是我们提出了一个本地乘法自我注意力（L-MSA）方案，以有效地从当地空间传递信息到相关部分。 Elmformer可以有效地减少计算消耗，并在原始图像恢复任务上表现良好。通过这两种核心设计，Elmformer提高了最高的性能，并且与最先进的机构相比，原始DeNoising和原始Deblurring基准测试最低。广泛的实验证明了Elmformer的优势和概括能力。在SIDD基准测试中，我们的方法比基于ISP的方法具有更好的降解性能，这些方法需要大量的额外的SRGB培训图像。这些代码在https://github.com/leonmakise/elmformer上发布。

以前的多任务密集预测研究开发了复杂的管道，例如在多个阶段进行多模式蒸馏或为每个任务寻找任务关系上下文。这些方法以外的核心洞察力是最大程度地利用每个任务之间的相互作用。受到最近基于查询的变压器的启发，我们提出了一条更简单的管道，称为Multi-Querti-Transformer（MQTRANSFORMER），该管道配备了来自不同任务的多个查询，以促进多个任务之间的推理并简化交叉任务管道。我们没有在不同任务之间建模每个像素上下文的密集上下文，而是寻求特定于任务的代理，以通过每个查询编码与任务相关的上下文进行编码的多个查询执行交叉任务推理。 MQTRANSFORMER由三个关键组件组成：共享编码器，交叉任务注意和共享解码器。我们首先将每个任务与任务相关且具有比例意识的查询对每个任务进行建模，然后将功能提取器的图像功能输出和与任务相关的查询功能都馈入共享编码器，从而从图像功能中编码查询功能。其次，我们设计了一个交叉任务注意模块，以从两个角度来推理多个任务和特征量表之间的依赖项，包括相同尺度的不同任务和同一任务的不同尺度。然后，我们使用共享解码器逐渐使用来自不同任务的合理查询功能来逐步完善图像功能。对两个密集的预测数据集（NYUD-V2和Pascal-Context）的广泛实验结果表明，该方法是一种有效的方法，并实现了最新结果。

半监控视频对象分割（VOS）是指在近年来在第一帧中的注释中分割剩余帧中的目标对象，该帧近年来已经积极研究。关键挑战在于找到利用过去框架的时空上下文的有效方法来帮助学习当前帧的判别目标表示。在本文中，我们提出了一种具有专门设计的交互式变压器的新型暹罗网络，称为SITVOS，以实现从历史到当前帧的有效上下文传播。从技术上讲，我们使用变换器编码器和解码器单独处理过去的帧和当前帧，即，编码器从过去的帧中对目标对象的强大的时空上下文进行编码，而解码器将当前帧的特征嵌入为查询。从编码器输出检索目标。为了进一步增强目标表示，设计了一种特征交互模块（FIM）以促进编码器和解码器之间的信息流。此外，我们使用暹罗架构来提取过去和当前帧的骨干功能，它能够重用并且比现有方法更有效。三个挑战基准测试的实验结果验证了SITVOS在最先进的方法上的优越性。

最近提出的深度感知视频Panoptic分段（DVPS）旨在预测视频中的Panoptic分段结果和深度映射，这是一个具有挑战性的场景理解问题。在本文中，我们提供了多相变压器，揭示了DVPS任务下的所有子任务。我们的方法通过基于查询的学习探讨了深度估计与Panoptic分割的关系。特别是，我们设计三个不同的查询，包括查询，填写询问和深度查询的东西。然后我们建议通过门控融合来学习这些查询之间的相关性。从实验中，我们从深度估计和Panoptic分割方面证明了我们设计的好处。由于每个物品查询还对实例信息进行了编码，因此通过具有外观学习的裁剪实例掩码功能来执行跟踪是自然的。我们的方法在ICCV-2021 BMTT挑战视频+深度轨道上排名第一。据报道，消融研究表明我们如何提高性能。代码将在https://github.com/harboryuan/polyphonicformer提供。

估计可驱动表面和周围环境的3D结构是辅助和自主驾驶的重要任务。通过使用昂贵的3D传感器（例如LIDAR）或通过深度学习预测点深度来常见的是。而不是遵循现有的方法，我们提出道路平面视差关注网络（RPANET），这是一种基于平面视差的单眼图像序列的新型神经网络，这是驾驶场景中常见的道路平面几何形状的充分优势。 RPANET需要一对图像由道路平面的定址为对齐的图像，作为输入，输出3D重建的$ \ Gamma $地图。除了估计深度或高度之外，$ \ Gamma $ MAP的可能性在两个连续帧之间构造二维变换，同时可以容易地导出深度或高度。通过使用道路平面作为参考的连续帧，可以从平面视差和残余图像位移估计3D结构。此外，为了使网络更好地了解由平面视差引起的位移，我们引入了一种新颖的跨关注模块。我们从Waymo Open DataSet中示机数据并构建与平面视差相关的数据。在采样的数据集上进行综合实验，以展示我们在具有挑战性的情况下的方法的三维重建准确性。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.