具有病理注释的计算机断层扫描（CT）样品很难获得。结果，计算机辅助诊断（CAD）算法在小型数据集（例如带有1,018个样本的LIDC-IDRI）上进行了培训，从而限制了其准确性和可靠性。在过去的五年中，通过二维（2D）和三维（3D）自我监督学习（SSL）算法为CT病变的无监督表示量身定制了几项作品。 2D算法很难捕获3D信息，并且现有的3D算法在计算上很重。轻巧的3D SSL仍然是要探索的边界。在本文中，我们提出了螺旋形对比度学习（SCL），该学习以计算有效的方式产生3D表示。 SCL首先使用信息保护螺旋变换将3D病变转换为2D平面，然后使用2D对比度学习学习转换不变的特征。为了进行增强，我们考虑自然图像增强和医疗图像增强。我们通过在嵌入层上训练分类头来评估SCL。实验结果表明，对于无监督的代表性学习，SCL在LIDC-IDRI（89.72％），LNDB（82.09％）和天奇（90.16％）上实现了最先进的准确性。使用10％的带计算的注释数据，SCL的性能与监督学习算法的性能相当（Lidc-Idri的85.75％比85.03％，78.20％vs. 73.44％的LNDB和87.85％vs. 83.34％vs. 83.34％and。天奇，分别）。同时，与其他3D SSL算法相比，SCL将计算工作减少了66.98％，这证明了该方法在无监督的预训练中的效率。

尽管深入学习算法已被深入开发用于计算机辅助结核病诊断（CTD），但它们主要依赖于精心注释的数据集，从而导致了大量时间和资源消耗。弱监督的学习（WSL）利用粗粒标签来完成精细的任务，具有解决此问题的潜力。在本文中，我们首先提出了一个新的大规模结核病（TB）胸部X射线数据集，即结核病胸部X射线属性数据集（TBX-ATT），然后建立一个属性辅助的弱点监督的框架来分类并通过利用属性信息来克服WSL方案中的监督不足来定位结核病。具体而言，首先，TBX-ATT数据集包含2000个X射线图像，其中具有七种用于TB关系推理的属性，这些属性由经验丰富的放射科医生注释。它还包括带有11200 X射线图像的公共TBX11K数据集，以促进弱监督检测。其次，我们利用一个多尺度特征交互模型，用于TB区域分类和属性关系推理检测。在TBX-ATT数据集上评估了所提出的模型，并将作为未来研究的稳固基准。代码和数据将在https://github.com/gangmingzhao/tb-attribute-weak-localization上获得。

在胸部X射线图像中定位疾病很少仔细注释可以节省大量的人类努力。最近的作品通过创新的弱监督算法（例如多稳定学习（MIL）和类激活图（CAM））处理了这项任务，但是，这些方法通常会产生不准确或不完整的区域。原因之一是忽视了每个图像内部解剖区域的关系中隐藏的病理意义以及跨图像的关系。在本文中，我们认为，作为上下文和补偿信息的跨区域和跨图像关系对于获得更一致和更一致的区域至关重要。为了建模关系，我们提出了图形正则嵌入网络（GREN），该网络（GREN）利用图像和图像间信息来定位胸部X射线图像上的疾病。 Gren使用预先训练的U-NET来分割肺裂片，然后使用图像内图形图对肺裂片之间的内图像进行建模以比较不同的区域。同时，内部图像之间的关系是通过图像间图建模的，以比较多个图像。此过程模仿了放射科医生的训练和决策过程：比较多个区域和图像进行诊断。为了使神经网络的深层嵌入层保留结构信息（在本地化任务中很重要），我们使用哈希编码和锤击距离来计算图形，这些图形用作正规化器来促进训练。通过这种情况，我们的方法实现了NIH胸部X射线数据集的最新结果，以实现弱监督疾病的定位。我们的代码可在线访问（https://github.com/qibaolian/gren）。

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

We introduce Argoverse 2 (AV2) - a collection of three datasets for perception and forecasting research in the self-driving domain. The annotated Sensor Dataset contains 1,000 sequences of multimodal data, encompassing high-resolution imagery from seven ring cameras, and two stereo cameras in addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain 3D cuboid annotations for 26 object categories, all of which are sufficiently-sampled to support training and evaluation of 3D perception models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point clouds and map-aligned pose. This dataset is the largest ever collection of lidar sensor data and supports self-supervised learning and the emerging task of point cloud forecasting. Finally, the Motion Forecasting Dataset contains 250,000 scenarios mined for interesting and challenging interactions between the autonomous vehicle and other actors in each local scene. Models are tasked with the prediction of future motion for "scored actors" in each scenario and are provided with track histories that capture object location, heading, velocity, and category. In all three datasets, each scenario contains its own HD Map with 3D lane and crosswalk geometry - sourced from data captured in six distinct cities. We believe these datasets will support new and existing machine learning research problems in ways that existing datasets do not. All datasets are released under the CC BY-NC-SA 4.0 license.

The surrogate loss of variational autoencoders (VAEs) poses various challenges to their training, inducing the imbalance between task fitting and representation inference. To avert this, the existing strategies for VAEs focus on adjusting the tradeoff by introducing hyperparameters, deriving a tighter bound under some mild assumptions, or decomposing the loss components per certain neural settings. VAEs still suffer from uncertain tradeoff learning.We propose a novel evolutionary variational autoencoder (eVAE) building on the variational information bottleneck (VIB) theory and integrative evolutionary neural learning. eVAE integrates a variational genetic algorithm into VAE with variational evolutionary operators including variational mutation, crossover, and evolution. Its inner-outer-joint training mechanism synergistically and dynamically generates and updates the uncertain tradeoff learning in the evidence lower bound (ELBO) without additional constraints. Apart from learning a lossy compression and representation of data under the VIB assumption, eVAE presents an evolutionary paradigm to tune critical factors of VAEs and deep neural networks and addresses the premature convergence and random search problem by integrating evolutionary optimization into deep learning. Experiments show that eVAE addresses the KL-vanishing problem for text generation with low reconstruction loss, generates all disentangled factors with sharp images, and improves the image generation quality,respectively. eVAE achieves better reconstruction loss, disentanglement, and generation-inference balance than its competitors.

Surgical robot automation has attracted increasing research interest over the past decade, expecting its huge potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied AI has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant researchers. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how human demonstrations would affect policy learning. In this paper, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. With these, we further propose to collect human demonstrations and imitate the action patterns to achieve more effective policy learning. We showcase the improvement of our simulation environment with the designed new features and tasks, and validate state-of-the-art reinforcement learning algorithms using the interactive environment. Promising results are obtained, with which we hope to pave the way for future research on surgical embodied intelligence. Our platform is released and will be continuously updated in the website: https://med-air.github.io/SurRoL/

Brain midline shift (MLS) is one of the most critical factors to be considered for clinical diagnosis and treatment decision-making for intracranial hemorrhage. Existing computational methods on MLS quantification not only require intensive labeling in millimeter-level measurement but also suffer from poor performance due to their dependence on specific landmarks or simplified anatomical assumptions. In this paper, we propose a novel semi-supervised framework to accurately measure the scale of MLS from head CT scans. We formulate the MLS measurement task as a deformation estimation problem and solve it using a few MLS slices with sparse labels. Meanwhile, with the help of diffusion models, we are able to use a great number of unlabeled MLS data and 2793 non-MLS cases for representation learning and regularization. The extracted representation reflects how the image is different from a non-MLS image and regularization serves an important role in the sparse-to-dense refinement of the deformation field. Our experiment on a real clinical brain hemorrhage dataset has achieved state-of-the-art performance and can generate interpretable deformation fields.

Audio-Visual scene understanding is a challenging problem due to the unstructured spatial-temporal relations that exist in the audio signals and spatial layouts of different objects and various texture patterns in the visual images. Recently, many studies have focused on abstracting features from convolutional neural networks while the learning of explicit semantically relevant frames of sound signals and visual images has been overlooked. To this end, we present an end-to-end framework, namely attentional graph convolutional network (AGCN), for structure-aware audio-visual scene representation. First, the spectrogram of sound and input image is processed by a backbone network for feature extraction. Then, to build multi-scale hierarchical information of input features, we utilize an attention fusion mechanism to aggregate features from multiple layers of the backbone network. Notably, to well represent the salient regions and contextual information of audio-visual inputs, the salient acoustic graph (SAG) and contextual acoustic graph (CAG), salient visual graph (SVG), and contextual visual graph (CVG) are constructed for the audio-visual scene representation. Finally, the constructed graphs pass through a graph convolutional network for structure-aware audio-visual scene recognition. Extensive experimental results on the audio, visual and audio-visual scene recognition datasets show that promising results have been achieved by the AGCN methods. Visualizing graphs on the spectrograms and images have been presented to show the effectiveness of proposed CAG/SAG and CVG/SVG that could focus on the salient and semantic relevant regions.