Single-cell technologies are revolutionizing the entire field of biology. The large volumes of data generated by single-cell technologies are high-dimensional, sparse, heterogeneous, and have complicated dependency structures, making analyses using conventional machine learning approaches challenging and impractical. In tackling these challenges, deep learning often demonstrates superior performance compared to traditional machine learning methods. In this work, we give a comprehensive survey on deep learning in single-cell analysis. We first introduce background on single-cell technologies and their development, as well as fundamental concepts of deep learning including the most popular deep architectures. We present an overview of the single-cell analytic pipeline pursued in research applications while noting divergences due to data sources or specific applications. We then review seven popular tasks spanning through different stages of the single-cell analysis pipeline, including multimodal integration, imputation, clustering, spatial domain identification, cell-type deconvolution, cell segmentation, and cell-type annotation. Under each task, we describe the most recent developments in classical and deep learning methods and discuss their advantages and disadvantages. Deep learning tools and benchmark datasets are also summarized for each task. Finally, we discuss the future directions and the most recent challenges. This survey will serve as a reference for biologists and computer scientists, encouraging collaborations.

为了自动纠正手写作业，传统方法是使用OCR模型来识别字符并将其与答案进行比较。 OCR模型在识别手写的汉字时很容易混淆，并且在模型推断过程中缺少答案的文本信息。但是，教师总是考虑到这些答案来审查和纠正作业。在本文中，我们专注于中国披肩测试校正并提出一种多模式方法（命名为AIM）。答案的编码表示与学生笔迹的视觉信息进行了交互。我们没有预测“正确”或“错误”，而是在答案文本上执行序列标记，以推断哪个答案字符与手写内容以细粒度的方式不同。我们将OCR数据集的样本作为此任务的正样本，并开发一种负面样本增强方法来扩展培训数据。实验结果表明，目标的范围优于基于OCR的方法。广泛的研究证明了我们多模式方法的有效性。

最近，基于深度学习的图像降级方法在测试数据上具有与训练集相同的测试数据的有希望的性能，在该数据中，已经学习了基于合成或收集的现实世界训练数据的各种denoising模型。但是，在处理真实世界的嘈杂图像时，Denoising的性能仍然受到限制。在本文中，我们提出了一种简单而有效的贝叶斯深集合（BDE）方法，用于真实世界图像denoising，其中可以融合使用各种训练数据设置进行预训练的几位代表性的深层Denoiser，以提高稳健性。 BDE的基础是，现实世界的图像噪声高度取决于信号依赖性，并且在现实世界中的嘈杂图像中的异质噪声可以由不同的Deoisiser分别处理。特别是，我们将受过良好训练的CBDNET，NBNET，HINET，UFORFORMER和GMSNET进入Denoiser池，并采用U-NET来预测Pixel的加权图以融合这些DeOisiser。引入了贝叶斯深度学习策略，而不是仅仅学习像素的加权地图，而是为了预测加权不确定性和加权图，可以通过该策略来建模预测差异，以改善现实世界中的嘈杂图像的鲁棒性。广泛的实验表明，可以通过融合现有的DINOISER而不是训练一个以昂贵的成本来训练一个大的Denoiser来更好地消除现实世界的噪音。在DND数据集上，我们的BDE实现了 +0.28〜dB PSNR的增益，而不是最先进的denoising方法。此外，我们注意到，在应用于现实世界嘈杂的图像时，基于不同高斯噪声水平的BDE Denoiser优于最先进的CBDNET。此外，我们的BDE可以扩展到其他图像恢复任务，并在基准数据集上获得 +0.30dB， +0.18dB和 +0.12dB PSNR的收益，以分别用于图像去除图像，图像降低和单个图像超级分辨率。

图形神经网络（GNNS）将深度神经网络（DNN）的成功扩展到非欧几里德图数据，实现了各种任务的接地性能，例如节点分类和图形属性预测。尽管如此，现有系统效率低，培训数十亿节点和GPU的节点和边缘训练大图。主要瓶颈是准备GPU数据的过程 - 子图采样和特征检索。本文提出了一个分布式GNN培训系统的BGL，旨在解决一些关键思想的瓶颈。首先，我们提出了一种动态缓存引擎，以最小化特征检索流量。通过协同设计缓存政策和抽样顺序，我们发现低开销和高缓存命中率的精美斑点。其次，我们改善了曲线图分区算法，以减少子图采样期间的交叉分区通信。最后，仔细资源隔离减少了不同数据预处理阶段之间的争用。关于各种GNN模型和大图数据集的广泛实验表明，BGL平均明显优于现有的GNN训练系统20.68倍。

Knowledge tracing (KT) aims to leverage students' learning histories to estimate their mastery levels on a set of pre-defined skills, based on which the corresponding future performance can be accurately predicted. In practice, a student's learning history comprises answers to sets of massed questions, each known as a session, rather than merely being a sequence of independent answers. Theoretically, within and across these sessions, students' learning dynamics can be very different. Therefore, how to effectively model the dynamics of students' knowledge states within and across the sessions is crucial for handling the KT problem. Most existing KT models treat student's learning records as a single continuing sequence, without capturing the sessional shift of students' knowledge state. To address the above issue, we propose a novel hierarchical transformer model, named HiTSKT, comprises an interaction(-level) encoder to capture the knowledge a student acquires within a session, and a session(-level) encoder to summarise acquired knowledge across the past sessions. To predict an interaction in the current session, a knowledge retriever integrates the summarised past-session knowledge with the previous interactions' information into proper knowledge representations. These representations are then used to compute the student's current knowledge state. Additionally, to model the student's long-term forgetting behaviour across the sessions, a power-law-decay attention mechanism is designed and deployed in the session encoder, allowing it to emphasize more on the recent sessions. Extensive experiments on three public datasets demonstrate that HiTSKT achieves new state-of-the-art performance on all the datasets compared with six state-of-the-art KT models.

Learning with noisy labels is a vital topic for practical deep learning as models should be robust to noisy open-world datasets in the wild. The state-of-the-art noisy label learning approach JoCoR fails when faced with a large ratio of noisy labels. Moreover, selecting small-loss samples can also cause error accumulation as once the noisy samples are mistakenly selected as small-loss samples, they are more likely to be selected again. In this paper, we try to deal with error accumulation in noisy label learning from both model and data perspectives. We introduce mean point ensemble to utilize a more robust loss function and more information from unselected samples to reduce error accumulation from the model perspective. Furthermore, as the flip images have the same semantic meaning as the original images, we select small-loss samples according to the loss values of flip images instead of the original ones to reduce error accumulation from the data perspective. Extensive experiments on CIFAR-10, CIFAR-100, and large-scale Clothing1M show that our method outperforms state-of-the-art noisy label learning methods with different levels of label noise. Our method can also be seamlessly combined with other noisy label learning methods to further improve their performance and generalize well to other tasks. The code is available in https://github.com/zyh-uaiaaaa/MDA-noisy-label-learning.

Deep learning, especially convolutional neural networks, has triggered accelerated advancements in computer vision, bringing changes into our daily practice. Furthermore, the standardized deep learning modules (also known as backbone networks), i.e., ResNet and EfficientNet, have enabled efficient and rapid development of new computer vision solutions. Yet, deep learning methods still suffer from several drawbacks. One of the most concerning problems is the high memory and computational cost, such that dedicated computing units, typically GPUs, have to be used for training and development. Therefore, in this paper, we propose a quantifiable evaluation method, the convolutional kernel redundancy measure, which is based on perceived image differences, for guiding the network structure simplification. When applying our method to the chest X-ray image classification problem with ResNet, our method can maintain the performance of the network and reduce the number of parameters from over $23$ million to approximately $128$ thousand (reducing $99.46\%$ of the parameters).

Graph neural networks (GNNs) have demonstrated excellent performance in a wide range of applications. However, the enormous size of large-scale graphs hinders their applications under real-time inference scenarios. Although existing scalable GNNs leverage linear propagation to preprocess the features and accelerate the training and inference procedure, these methods still suffer from scalability issues when making inferences on unseen nodes, as the feature preprocessing requires the graph is known and fixed. To speed up the inference in the inductive setting, we propose a novel adaptive propagation order approach that generates the personalized propagation order for each node based on its topological information. This could successfully avoid the redundant computation of feature propagation. Moreover, the trade-off between accuracy and inference latency can be flexibly controlled by simple hyper-parameters to match different latency constraints of application scenarios. To compensate for the potential inference accuracy loss, we further propose Inception Distillation to exploit the multi scale reception information and improve the inference performance. Extensive experiments are conducted on four public datasets with different scales and characteristics, and the experimental results show that our proposed inference acceleration framework outperforms the SOTA graph inference acceleration baselines in terms of both accuracy and efficiency. In particular, the advantage of our proposed method is more significant on larger-scale datasets, and our framework achieves $75\times$ inference speedup on the largest Ogbn-products dataset.

在线知识蒸馏（OKD）通过相互利用教师和学生之间的差异来改善所涉及的模型。它们之间的差距上有几个关键的瓶颈 - 例如，为什么以及何时以及何时损害表现，尤其是对学生的表现？如何量化教师和学生之间的差距？ - 接受了有限的正式研究。在本文中，我们提出了可切换的在线知识蒸馏（Switokd），以回答这些问题。 Switokd的核心思想不是专注于测试阶段的准确性差距，而是通过两种模式之间的切换策略来适应训练阶段的差距，即蒸馏差距 - 专家模式（暂停老师，同时暂停教师保持学生学习）和学习模式（重新启动老师）。为了拥有适当的蒸馏差距，我们进一步设计了一个自适应开关阈值，该阈值提供了有关何时切换到学习模式或专家模式的正式标准，从而改善了学生的表现。同时，老师从我们的自适应切换阈值中受益，并基本上与其他在线艺术保持同步。我们进一步将Switokd扩展到具有两个基础拓扑的多个网络。最后，广泛的实验和分析验证了Switokd在最新面前的分类的优点。我们的代码可在https://github.com/hfutqian/switokd上找到。

数十年来，计算机系统持有大量个人数据。一方面，这种数据丰度允许在人工智能（AI），尤其是机器学习（ML）模型中突破。另一方面，它可能威胁用户的隐私并削弱人类与人工智能之间的信任。最近的法规要求，可以从一般情况下从计算机系统中删除有关用户的私人信息，特别是根据要求从ML模型中删除（例如，“被遗忘的权利”）。虽然从后端数据库中删除数据应该很简单，但在AI上下文中，它不够，因为ML模型经常“记住”旧数据。现有的对抗攻击证明，我们可以从训练有素的模型中学习私人会员或培训数据的属性。这种现象要求采用新的范式，即机器学习，以使ML模型忘记了特定的数据。事实证明，由于缺乏共同的框架和资源，最近在机器上学习的工作无法完全解决问题。在本调查文件中，我们试图在其定义，场景，机制和应用中对机器进行彻底的研究。具体而言，作为最先进的研究的类别集合，我们希望为那些寻求机器未学习的入门及其各种表述，设计要求，删除请求，算法和用途的人提供广泛的参考。 ML申请。此外，我们希望概述范式中的关键发现和趋势，并突出显示尚未看到机器无法使用的新研究领域，但仍可以受益匪浅。我们希望这项调查为ML研究人员以及寻求创新隐私技术的研究人员提供宝贵的参考。我们的资源是在https://github.com/tamlhp/awesome-machine-unlearning上。