知识基础问题答案（KBQA）的最新著作检索了子图，以更容易推理。所需的子图至关重要，因为很小的子图可能会排除答案，但是大型答案可能会引入更多的声音。但是，现有的检索要么是启发式的，要么与推理交织在一起，从而导致部分子图的推理，这在缺少中间监督时会增加推理偏见。本文提出了一个与后续推理过程分离的可训练子图检索器（SR），该过程使插件框架可以增强任何面向子图的KBQA模型。广泛的实验表明，与现有检索方法相比，SR的检索和质量检查的性能明显更好。通过弱监督的预训练以及端到端的微调，SRL与基于嵌入基于嵌入的KBQA方法NSM结合使用NSM结合使用NSM时实现了新的最先进性能。

基于规则的分类器，其提取诱导规则的子集，以便在保留可辨别信息的同时有效地学习/挖掘，在人工可解释的人工智能中起着至关重要的作用。但是，在这个大数据的时代，整个数据集上的规则感应是计算密集的。到目前为止，据我们所知，报道了没有针对加速规则诱导的已知方法。这是首先要考虑减少规则感应规模的加速技术的研究。我们提出了一种基于模糊粗略理论的规则感应的加速器;加速器可以避免冗余计算并加速规则分类器的构建。首先，提出基于一致程度的规则感应方法，称为基于一致的基于值（CVR），并用作加速的基础。其次，我们引入了一个被称为关键集的压实的搜索空间，其只包含更新诱导规则所需的关键实例，以减少值。关键集的单调性可确保我们的加速器的可行性。第三，基于密钥集设计了规则感应加速器，从理论上保证将与未被插布的版本的结果相同的结果。具体地，键集的等级保存属性可确保通过加速器和未燃道的方法实现的规则感应之间的一致性。最后，广泛的实验表明，所提出的加速器可以比未被基于规则的分类器方法更快地执行，特别是在具有许多实例的数据集上。

基于嵌入的方法是知识库问题应答（KBQA）的流行，但目前的型号很少有数值推理技能，从而努力回答序数受约束的问题。本文提出了一种新的基于嵌入的KBQA框架，特别是考虑了数值推理。我们在NSM的顶部呈现NumericalTransformer，以最先进的基于嵌入的KBQA模型，以创建NT-NSM。为了实现更好的培训，我们提出了两个在两个生成的训练数据集上具有明确的数字导向损失功能的预训练任务和基于模板的数据增强方法，用于丰富序数约束QA数据集。关于KBQA基准测试的广泛实验表明，在我们的培训算法的帮助下，NT-NSM具有数值推理技能，并且基本上优于基本的基线在回答序数受约束的问题中。

冷启动问题是推荐任务的根本挑战。最近的自我监督学习（SSL）图形神经网络（GNNS）模型，PT-GNN，预先列出GNN模型以重建冷启动嵌入，并为冷启动推荐表示了很大的潜力。然而，由于过平滑的问题，PT-GNN只能捕获多达3阶关系，这不能提供许多有用的辅助信息来描绘目标冷启动用户或项目。此外，嵌入重建任务仅考虑用户和项目的子图内的相关性，同时忽略不同子图之间的相关间。为解决上述挑战，我们提出了一种基于多策略的冷启动推荐（MPT）的预训练方法，它从模型架构和借口任务的角度扩展了PT-GNN，以提高冷启动推荐性能。具体地，在模型架构方面，除了由GNN编码器捕获的用户和项目的短程依赖性之外，我们还引入变压器编码器以捕获远程依赖性。在借口任务方面，除了通过嵌入重建任务考虑用户和项目的相关性，我们还添加了嵌入对比学习任务以捕获用户和项目的相关性。我们在元学习设置下培训GNN和变压器编码，在这些借口任务下，以模拟真实的冷启动方案，使模型轻松迅速，适应新的冷启动用户和项目。三个公共推荐数据集的实验显示了对Vanilla GNN模型的提议MPT模型的优势，预先培训了用户/项目嵌入推断和推荐任务的GNN模型。

我们研究推荐项目到偶尔群体的问题（A.K.A.冷启动群体），偶尔群体形成ad-hoc，并且有很少或没有历史互动的物品。由于偶尔群体的极端稀疏问题，偶尔与物品的互动，很难为这些偶尔群体学习高质量的嵌入。尽管图表神经网络（GNN）的最近进步包括高阶协作信号来缓解问题，但在GNN的图表卷积期间未明确考虑高阶冷启动邻居。本文提出了一个自我监督的图表学习范式，该图解范例共同列举了骨干GNN模型来重建元学习环境下的群体/用户/项目嵌入，使得它可以直接提高嵌入质量，并且可以轻松调整新偶尔群体。为了进一步减少来自冷启动邻居的影响，我们纳入了一种基于自我关注的元聚合器，以增强每个图卷积步骤的聚合能力。此外，我们添加了对比学习（CL）适配器，明确地考虑组和非组成员之间的相关性。三个公共推荐数据集的实验结果表明我们拟议模型的优势，以防止最先进的集团推荐方法。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.