室内场景云的无监督对比学习取得了巨大的成功。但是，室外场景中无监督的学习点云仍然充满挑战，因为以前的方法需要重建整个场景并捕获对比度目标的部分视图。这在带有移动物体，障碍物和传感器的室外场景中是不可行的。在本文中，我们提出了CO^3，即合作对比度学习和上下文形状的预测，以无监督的方式学习3D表示室外景点云。与现有方法相比，Co^3具有几种优点。 （1）它利用了从车辆侧和基础架构侧来的激光点云来构建差异，但同时维护对比度学习的通用语义信息，这比以前的方法构建的视图更合适。 （2）在对比度目标的同时，提出了形状上下文预测作为预训练目标，并为无监督的3D点云表示学习带来了更多与任务相关的信息，这在将学习的表示形式转移到下游检测任务时是有益的。 （3）与以前的方法相比，CO^3学到的表示形式可以通过不同类型的LIDAR传感器收集到不同的室外场景数据集。 （4）CO^3将一次和Kitti数据集的当前最新方法提高到2.58地图。代码和模型将发布。我们认为Co^3将有助于了解室外场景中的LiDar Point云。

机器的图像编码（ICM）旨在压缩图像进行AI任务分析，而不是满足人类的看法。学习一种既是一般（用于AI任务）的特征，也是紧凑的（用于压缩）的功能，这对于其成功而言至关重要。在本文中，我们试图通过学习通用功能，同时考虑压缩来开发ICM框架。我们将诸如无所不能功能和相应框架的功能命名为Omni-ICM。考虑到自我监督学习（SSL）提高了特征的概括，我们将其与压缩任务集成到OMNI-ICM框架中，以学习无所不能的功能。但是，在SSL中协调语义建模并在压缩中删除冗余是不平凡的，因此我们通过合作实例区分和熵最小化以自适应掉落的信息来设计新颖的信息过滤（如果）模块，以较弱相关的信息执行AI任务（例如，某些纹理冗余）。与以前的特定解决方案不同，Omni-ICM可以直接基于学习的无能功能的AI任务分析，而无需联合培训或额外的转换。尽管简单而直观，但Omni-ICM在多个基本愿景任务上大大优于现有的传统和基于学习的编解码器。

在本文中，我们介绍了第一个神经视频编解码器，可以在用于低延迟模式的UVG数据集上的SRGB PSNR方面与最新编码标准H.266 / VVC竞争。现有的神经混合视频编码方法依赖于用于预测的光流或高斯尺度流，这不能支持对不同运动内容的细粒度适应性。为了更具内容 - 自适应预测，我们提出了一种新颖的跨尺度预测模块，实现更有效的运动补偿。具体地，一方面，我们生产参考特征金字塔作为预测源，然后传输利用特征尺度的横级流来控制预测的精度。另一方面，我们将加权预测的机制介绍到具有单个参考帧的预测场景的机制，其中发送交叉尺度权重映射以合成精细预测结果。除了串尺度预测模块之外，我们还提出了一种多级量化策略，这提高了在推理期间没有额外计算惩罚的速率失真性能。我们展示了我们有效的神经视频编解码器（ENVC）对几个常见的基准数据集的令人鼓舞的表现，并详细分析了每个重要组成部分的有效性。

学习的视频压缩方法在赶上其速率 - 失真（R-D）性能时，追赶传统视频编解码器的许多承诺。然而，现有的学习视频压缩方案受预测模式和固定网络框架的绑定限制。它们无法支持各种帧间预测模式，从而不适用于各种场景。在本文中，为了打破这种限制，我们提出了一种多功能学习的视频压缩（VLVC）框架，它使用一个模型来支持所有可能的预测模式。具体而言，为了实现多功能压缩，我们首先构建一个运动补偿模块，该模块应用用于在空间空间中的加权三线性翘曲的多个3D运动矢量字段（即，Voxel流量）。 Voxel流量传达了时间参考位置的信息，有助于与框架设计中的帧间预测模式分离。其次，在多参考帧预测的情况下，我们应用流预测模块以预测具有统一多项式函数的准确运动轨迹。我们表明流量预测模块可以大大降低体素流的传输成本。实验结果表明，我们提出的VLVC不仅支持各种设置中的多功能压缩，而且还通过MS-SSIM的最新VVC标准实现了可比的R-D性能。

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.

In this chapter, we review and discuss the transformation of AI technology in HCI/UX work and assess how AI technology will change how we do the work. We first discuss how AI can be used to enhance the result of user research and design evaluation. We then discuss how AI technology can be used to enhance HCI/UX design. Finally, we discuss how AI-enabled capabilities can improve UX when users interact with computing systems, applications, and services.

As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via https://mea-lab-421.github.io.

Face Anti-spoofing (FAS) is essential to secure face recognition systems from various physical attacks. However, recent research generally focuses on short-distance applications (i.e., phone unlocking) while lacking consideration of long-distance scenes (i.e., surveillance security checks). In order to promote relevant research and fill this gap in the community, we collect a large-scale Surveillance High-Fidelity Mask (SuHiFiMask) dataset captured under 40 surveillance scenes, which has 101 subjects from different age groups with 232 3D attacks (high-fidelity masks), 200 2D attacks (posters, portraits, and screens), and 2 adversarial attacks. In this scene, low image resolution and noise interference are new challenges faced in surveillance FAS. Together with the SuHiFiMask dataset, we propose a Contrastive Quality-Invariance Learning (CQIL) network to alleviate the performance degradation caused by image quality from three aspects: (1) An Image Quality Variable module (IQV) is introduced to recover image information associated with discrimination by combining the super-resolution network. (2) Using generated sample pairs to simulate quality variance distributions to help contrastive learning strategies obtain robust feature representation under quality variation. (3) A Separate Quality Network (SQN) is designed to learn discriminative features independent of image quality. Finally, a large number of experiments verify the quality of the SuHiFiMask dataset and the superiority of the proposed CQIL.

When using LiDAR semantic segmentation models for safety-critical applications such as autonomous driving, it is essential to understand and improve their robustness with respect to a large range of LiDAR corruptions. In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. To rigorously evaluate the robustness and generalizability of current approaches, we propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic segmentation models, especially spanning different input representations (e.g., point clouds, voxels, projected images, and etc.), network architectures and training schemes. Through this study, we obtain two insights: 1) We find out that the input representation plays a crucial role in robustness. Specifically, under specific corruptions, different representations perform variously. 2) Although state-of-the-art methods on LiDAR semantic segmentation achieve promising results on clean data, they are less robust when dealing with noisy data. Finally, based on the above observations, we design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications. It is promising that our benchmark, comprehensive analysis, and observations can boost future research in robust LiDAR semantic segmentation for safety-critical applications.

Panoptic Part Segmentation (PPS) unifies panoptic segmentation and part segmentation into one task. Previous works utilize separated approaches to handle thing, stuff, and part predictions without shared computation and task association. We aim to unify these tasks at the architectural level, designing the first end-to-end unified framework named Panoptic-PartFormer. Moreover, we find the previous metric PartPQ biases to PQ. To handle both issues, we make the following contributions: Firstly, we design a meta-architecture that decouples part feature and things/stuff feature, respectively. We model things, stuff, and parts as object queries and directly learn to optimize all three forms of prediction as a unified mask prediction and classification problem. We term our model as Panoptic-PartFormer. Secondly, we propose a new metric Part-Whole Quality (PWQ) to better measure such task from both pixel-region and part-whole perspectives. It can also decouple the error for part segmentation and panoptic segmentation. Thirdly, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross attention scheme to further boost part segmentation qualities. We design a new part-whole interaction method using masked cross attention. Finally, the extensive ablation studies and analysis demonstrate the effectiveness of both Panoptic-PartFormer and Panoptic-PartFormer++. Compared with previous Panoptic-PartFormer, our Panoptic-PartFormer++ achieves 2% PartPQ and 3% PWQ improvements on the Cityscapes PPS dataset and 5% PartPQ on the Pascal Context PPS dataset. On both datasets, Panoptic-PartFormer++ achieves new state-of-the-art results with a significant cost drop of 70% on GFlops and 50% on parameters. Our models can serve as a strong baseline and aid future research in PPS. Code will be available.