We present X-Decoder, a generalized decoding model that can predict pixel-level segmentation and language tokens seamlessly. X-Decodert takes as input two types of queries: (i) generic non-semantic queries and (ii) semantic queries induced from text inputs, to decode different pixel-level and token-level outputs in the same semantic space. With such a novel design, X-Decoder is the first work that provides a unified way to support all types of image segmentation and a variety of vision-language (VL) tasks. Further, our design enables seamless interactions across tasks at different granularities and brings mutual benefits by learning a common and rich pixel-level visual-semantic understanding space, without any pseudo-labeling. After pretraining on a mixed set of a limited amount of segmentation data and millions of image-text pairs, X-Decoder exhibits strong transferability to a wide range of downstream tasks in both zero-shot and finetuning settings. Notably, it achieves (1) state-of-the-art results on open-vocabulary segmentation and referring segmentation on eight datasets; (2) better or competitive finetuned performance to other generalist and specialist models on segmentation and VL tasks; and (3) flexibility for efficient finetuning and novel task composition (e.g., referring captioning and image editing). Code, demo, video, and visualization are available at https://x-decoder-vl.github.io.

Developing autonomous vehicles (AVs) helps improve the road safety and traffic efficiency of intelligent transportation systems (ITS). Accurately predicting the trajectories of traffic participants is essential to the decision-making and motion planning of AVs in interactive scenarios. Recently, learning-based trajectory predictors have shown state-of-the-art performance in highway or urban areas. However, most existing learning-based models trained with fixed datasets may perform poorly in continuously changing scenarios. Specifically, they may not perform well in learned scenarios after learning the new one. This phenomenon is called "catastrophic forgetting". Few studies investigate trajectory predictions in continuous scenarios, where catastrophic forgetting may happen. To handle this problem, first, a novel continual learning (CL) approach for vehicle trajectory prediction is proposed in this paper. Then, inspired by brain science, a dynamic memory mechanism is developed by utilizing the measurement of traffic divergence between scenarios, which balances the performance and training efficiency of the proposed CL approach. Finally, datasets collected from different locations are used to design continual training and testing methods in experiments. Experimental results show that the proposed approach achieves consistently high prediction accuracy in continuous scenarios without re-training, which mitigates catastrophic forgetting compared to non-CL approaches. The implementation of the proposed approach is publicly available at https://github.com/BIT-Jack/D-GSM

Remote photoplethysmography (rPPG) enables non-contact heart rate (HR) estimation from facial videos which gives significant convenience compared with traditional contact-based measurements. In the real-world long-term health monitoring scenario, the distance of the participants and their head movements usually vary by time, resulting in the inaccurate rPPG measurement due to the varying face resolution and complex motion artifacts. Different from the previous rPPG models designed for a constant distance between camera and participants, in this paper, we propose two plug-and-play blocks (i.e., physiological signal feature extraction block (PFE) and temporal face alignment block (TFA)) to alleviate the degradation of changing distance and head motion. On one side, guided with representative-area information, PFE adaptively encodes the arbitrary resolution facial frames to the fixed-resolution facial structure features. On the other side, leveraging the estimated optical flow, TFA is able to counteract the rPPG signal confusion caused by the head movement thus benefit the motion-robust rPPG signal recovery. Besides, we also train the model with a cross-resolution constraint using a two-stream dual-resolution framework, which further helps PFE learn resolution-robust facial rPPG features. Extensive experiments on three benchmark datasets (UBFC-rPPG, COHFACE and PURE) demonstrate the superior performance of the proposed method. One highlight is that with PFE and TFA, the off-the-shelf spatio-temporal rPPG models can predict more robust rPPG signals under both varying face resolution and severe head movement scenarios. The codes are available at https://github.com/LJW-GIT/Arbitrary_Resolution_rPPG.

复杂的流量分析，例如加密的流量分析和未知的恶意软件检测，强调需要进行高级方法来分析网络流量。使用固定模式，签名匹配和检测网络流量中已知模式的规则的传统方法已被AI（人工智能）驱动算法取代。但是，缺乏高性能AI网络特定的框架使得不可能在网络工作负载中部署基于AI的实时处理。在本文中，我们描述了流量分析开发工具包（TADK）的设计，这是一个针对基于AI的网络工作负载处理的行业标准框架。 TADK可以在数据中心到边缘的网络设备中基于实时的AI网络工作负载处理，而无需专门硬件（例如GPU，神经处理单元等）。我们已经在商品WAF和5G UPF中部署了TADK，评估结果表明，Tadk可以在流量功能提取时达到每个核心最多35.3Gbps的吞吐量，每核6.5Gbps在流量分类中，并且可以减少SQLI/XSS检测到下降至4.5us每个请求的精度比固定模式解决方案更高。

在城市环境中，复杂和不确定的交叉场景对于自动驾驶而言是挑战性的。为了确保安全，建立可以处理与其他车辆互动的自适应决策系统至关重要。在常见方案中，手动设计的基于模型的方法是可靠的。但是在不确定的环境中，它们不是可靠的，因此提出了基于学习的方法，尤其是强化学习（RL）方法。但是，当场景更改时，当前的RL方法需要重新培训。换句话说，当前的RL方法无法重复使用积累的知识。他们忘记了新场景时学到的知识。为了解决这个问题，我们提出了一个可以自主积累和重用知识的层次结构框架。所提出的方法将运动原语（MP）的概念与分层增强学习（HRL）结合在一起。它将复杂的问题分解为多个基本子任务以减少难度。提出的方法和其他基线方法在基于CARLA模拟器的具有挑战性的交点方案中进行了测试。相交场景包含三个不同的子任务，可以反映出真实交通流的复杂性和不确定性。在离线学习和测试之后，事实证明，所提出的方法在所有方法中具有最佳性能。

旨在自动进行工程增强政策的自动数据扩展最近引起了不断增长的研究兴趣。许多以前的自动启发方法通过评估测试时间增强性能来评估策略，利用了密度匹配策略。在本文中，我们从理论上和经验上证明了火车和小规模医学图像数据集的验证集之间的不一致，称为内域采样偏差。接下来，我们证明了域中采样偏置可能导致密度匹配的效率低下。为了解决这个问题，提出了一种改进的增强搜索策略，称为增强密度匹配，是通过从先前的培训分布中随机采样策略提出的。此外，提出了有效的自动机器学习（AUTOML）算法，通过统一数据增强和神经体系结构的搜索来提出。实验结果表明，所提出的方法优于MedMnist的最先进方法，MedMnist是一种开拓性的基准测试，旨在在医学图像分析中进行自动。

痤疮检测对于解释性诊断和对皮肤疾病的精确治疗至关重要。任意边界和痤疮病变的尺寸较小，导致在两阶段检测中大量质量较差的建议。在本文中，我们提出了一个针对地区建议网络的新型头部结构，以两种方式提高建议的质量。首先，提出了一个空间意识的双头（SADH）结构，以从两个不同的空间角度从分类和本地化进行分类和本地化的表示。拟议的SADH确保了更陡峭的分类信心梯度，并抑制了与匹配的地面真理相交（IOU）低相交（IOU）的建议。然后，我们提出了一个归一化的Wasserstein距离预测分支，以改善提议分类评分与IOU之间的相关性。此外，为了促进痤疮检测的进一步研究，我们构建了一个名为Acnescu的新数据集，具有高分辨率成像，精确的注释和细粒度的病变类别。对AcnesCU和公共数据集Acne04进行了广泛的实验，结果表明该方法可以提高建议的质量，始终超过最先进的方法。代码和收集的数据集可在https://github.com/pingguokiller/acnedetection中找到。

在自然性和可读性方面，韵律边界在文本到语音综合（TTS）中起着重要作用。但是，获得韵律边界标签的获取依赖于手动注释，这是昂贵且耗时的。在本文中，我们建议通过带有预训练的音频编码器的神经文本语音模型自动从文本审计数据中提取韵律边界标签。该模型分别对文本和语音数据进行了预先训练，并以三重态格式对TTS数据进行了微调：{语音，文本，韵律}。自动评估和人类评估的实验结果表明：1）提出的文本言论韵律注释框架极大地超过了文本基本线；2）自动韵律边界注释的质量与人类注释相当；3）经过模型通知边界训练的TTS系统比使用手动系统的系统要好得多。

从自然语言监督中学习视觉表示，最近在许多开创性的作品中表现出了巨大的希望。通常，这些具有语言的视觉模型表现出对各种数据集和任务的强大可传递性。但是，由于缺乏易于使用的评估工具包和公共基准，评估这些模型的可转让性仍然很具有挑战性。为了解决这个问题，我们构建了高级版（评估语言的视觉任务级传输），这是用于评估（预训练）语言增强视觉模型的第一个基准和工具包。升华由三个组成部分组成。 （i）数据集。作为下游评估套件，它由20个图像分类数据集和35个对象检测数据集组成，每个数据集都用外部知识来增强。 （ii）工具包。开发了自动高参数调谐工具包，以促进下游任务的模型评估。 （iii）指标。多种评估指标用于测量样品效率（零射击和少量）和参数效率（线性探测和完整模型微调）。我们在https://computer-vision-in-the-wild.github.io/elevater/上公开发布leverater

In computer vision, it has achieved great transfer learning performance via adapting large-scale pretrained vision models (e.g., vision transformers) to downstream tasks. Common approaches for model adaptation either update all model parameters or leverage linear probes. In this paper, we aim to study parameter-efficient model adaptation strategies for vision transformers on the image classification task. We formulate efficient model adaptation as a subspace training problem and perform a comprehensive benchmarking over different efficient adaptation methods. We conduct an empirical study on each efficient model adaptation method focusing on its performance alongside parameter cost. Furthermore, we propose a parameter-efficient model adaptation framework, which first selects submodules by measuring local intrinsic dimensions and then projects them into subspace for further decomposition via a novel Kronecker Adaptation (KAdaptation) method. We analyze and compare our method with a diverse set of baseline model adaptation methods (including state-of-the-art methods for pretrained language models). Our method performs the best in terms of the tradeoff between accuracy and parameter efficiency across 20 image classification datasets under the few-shot setting and 7 image classification datasets under the full-shot setting.