To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2.0, focusing on the use of pre-trained language models (PLMs). To be comprehensive, our library covers $13$ common text generation tasks and their corresponding $83$ datasets and further incorporates $45$ PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLMs. We also implement $4$ efficient training strategies and provide $4$ generation objectives for pre-training new PLMs from scratch. To be unified, we design the interfaces to support the entire research pipeline (from data loading to training and evaluation), ensuring that each step can be fulfilled in a unified way. Despite the rich functionality, it is easy to use our library, either through the friendly Python API or command line. To validate the effectiveness of our library, we conduct extensive experiments and exemplify four types of research scenarios. The project is released at the link: https://github.com/RUCAIBox/TextBox.

Prompt tuning has been employed as an efficient way to adapt large vision-language pre-trained models (e.g. CLIP) to various downstream tasks in data-limited or label-limited settings. Nonetheless, visual data (e.g., images) is by default prerequisite for learning prompts in existing methods. In this work, we advocate that the effectiveness of image-text contrastive learning in aligning the two modalities (for training CLIP) further makes it feasible to treat texts as images for prompt tuning and introduce TaI prompting. In contrast to the visual data, text descriptions are easy to collect, and their class labels can be directly derived. Particularly, we apply TaI prompting to multi-label image recognition, where sentences in the wild serve as alternatives to images for prompt tuning. Moreover, with TaI, double-grained prompt tuning (TaI-DPT) is further presented to extract both coarse-grained and fine-grained embeddings for enhancing the multi-label recognition performance. Experimental results show that our proposed TaI-DPT outperforms zero-shot CLIP by a large margin on multiple benchmarks, e.g., MS-COCO, VOC2007, and NUS-WIDE, while it can be combined with existing methods of prompting from images to improve recognition performance further. Code is released at https://github.com/guozix/TaI-DPT.

Artificial Intelligence (AI) is having a tremendous impact across most areas of science. Applications of AI in healthcare have the potential to improve our ability to detect, diagnose, prognose, and intervene on human disease. For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e.g. geometry, physiology, physics) of medical data being processed. This work introduces MONAI, a freely available, community-supported, and consortium-led PyTorch-based framework for deep learning in healthcare. MONAI extends PyTorch to support medical data, with a particular focus on imaging, and provide purpose-specific AI model architectures, transformations and utilities that streamline the development and deployment of medical AI models. MONAI follows best practices for software-development, providing an easy-to-use, robust, well-documented, and well-tested software framework. MONAI preserves the simple, additive, and compositional approach of its underlying PyTorch libraries. MONAI is being used by and receiving contributions from research, clinical and industrial teams from around the world, who are pursuing applications spanning nearly every aspect of healthcare.

在各种图像处理和计算机视觉任务中经常遇到颜色图像Denoising。一种传统的策略是将RGB图像转换为较小相关的颜色空间，并分别将新空间的每个通道定义。但是，这种策略无法完全利用渠道之间的相关信息，并且不足以获得令人满意的结果。为了解决这个问题，本文提出了一个新的多通道优化模型，用于在核定标准下减去Frobenius规范最小化框架下的颜色图像Deno。具体而言，基于块匹配，将颜色图像分解为重叠的RGB补丁。对于每个补丁，我们堆叠其相似的邻居以形成相应的补丁矩阵。提出的模型是在补丁矩阵上执行的，以恢复其无噪声版本。在恢复过程中，a）引入权重矩阵以充分利用通道之间的噪声差； b）单数值是自适应缩小的，而无需分配权重。有了他们，提议的模型可以在保持简单的同时取得有希望的结果。为了解决提出的模型，基于乘数框架的交替方向方法构建了准确有效的算法。每个更新步骤的解决方案可以在封闭式中分析表达。严格的理论分析证明了所提出的算法产生的解决方案序列会收敛到其各自的固定点。合成和真实噪声数据集的实验结果证明了所提出的模型优于最先进的模型。

在标签噪声下训练深神网络的能力很有吸引力，因为不完美的注释数据相对便宜。最先进的方法基于半监督学习（SSL），该学习选择小损失示例为清洁，然后应用SSL技术来提高性能。但是，选择步骤主要提供一个中等大小的清洁子集，该子集可俯瞰丰富的干净样品。在这项工作中，我们提出了一个新颖的嘈杂标签学习框架Promix，试图最大程度地提高清洁样品的实用性以提高性能。我们方法的关键是，我们提出了一种匹配的高信心选择技术，该技术选择了那些具有很高置信的示例，并与给定标签进行了匹配的预测。结合小损失选择，我们的方法能够达到99.27的精度，并在检测CIFAR-10N数据集上的干净样品时召回98.22。基于如此大的清洁数据，Promix将最佳基线方法提高了CIFAR-10N的 +2.67％，而CIFAR-100N数据集则提高了 +1.61％。代码和数据可从https://github.com/justherozen/promix获得

多通道多扬声器的自动语音识别（ASR）重叠的语音仍然是语音社区最具挑战性的任务之一。在本文中，我们首次利用3D空间中的目标扬声器的位置信息来研究挑战。为了探讨所提出的3D空间特征的强度，研究了两个范例。 1）带有多通道语音分离模块的流水线系统，后跟最先进的单通道ASR模块; 2）3D空间特征直接用作无明确分离模块的ASR系统的输入的“一体化”模型。它们都是完全可分辨的，并且可以回到倒端的端到端。我们在模拟重叠的语音和实际录音上测试它们。实验结果表明，1）所提出的一体化模型对流水线系统实现了类似的误码率，同时将推理时间减少一半; 2）所提出的3D空间特征显着优于（31 \％CERR）所有先前的应用程序在两个范例中使用的所有先前作品。

公制学习旨在学习一个距离度量，以便在将不同的实例推开时将语义上相似的实例放在一起。许多现有方法考虑在特征空间中最大化或至少限制距离距离的距离，以分离相似和不同的实例对以保证其概括能力。在本文中，我们主张在输入空间中施加对抗边缘，以改善公制学习算法的概括和稳健性。我们首先表明，对抗边缘定义为训练实例与其最接近的对手示例之间的距离，它既考虑了特征空间中的距离差距以及指标和三重限制之间的相关性。接下来，为了增强实例扰动的鲁棒性，我们建议通过最大程度地减少称为扰动损失的新型损失函数来扩大对抗缘。提出的损失可以看作是数据依赖性的正规器，并轻松地插入任何现有的度量学习方法中。最后，我们表明扩大边缘通过使用算法鲁棒性的理论技术对概括能力有益。 16个数据集的实验结果证明了所提出的方法比现有的最新方法具有歧视精度和鲁棒性，以抵抗可能的噪声。

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.

Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above mentioned challenges and address problems of supervised, zero-shot, generalized zero-shot and open set recognition using a unified framework. Specifically, we propose a weighted maximum margin framework for semantic manifold-based recognition that incorporates distance constraints from (both supervised and unsupervised) vocabulary atoms. Distance constraints ensure that labeled samples are projected closer to their correct prototypes, in the embedding space, than to others. We illustrate that resulting model shows improvements in supervised, zero-shot, generalized zero-shot, and large open set recognition, with up to 310K class vocabulary on Animal with Attributes and ImageNet datasets.

Advances in computer vision and machine learning techniques have led to significant development in 2D and 3D human pose estimation from RGB cameras, LiDAR, and radars. However, human pose estimation from images is adversely affected by occlusion and lighting, which are common in many scenarios of interest. Radar and LiDAR technologies, on the other hand, need specialized hardware that is expensive and power-intensive. Furthermore, placing these sensors in non-public areas raises significant privacy concerns. To address these limitations, recent research has explored the use of WiFi antennas (1D sensors) for body segmentation and key-point body detection. This paper further expands on the use of the WiFi signal in combination with deep learning architectures, commonly used in computer vision, to estimate dense human pose correspondence. We developed a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. The results of the study reveal that our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches, by utilizing WiFi signals as the only input. This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing.