越来越多的人期望在对象属性具有高感知不确定性的越来越多的非结构化环境中操纵对象。这直接影响成功的对象操纵。在这项工作中，我们提出了一个基于增强的学习动作计划框架，用于对象操纵，该框架既利用了在现有的多感觉反馈，也可以使用学习的注意力引导的深层负担能力模型作为感知状态。可承受的模型是从多种感官方式中学到的，包括视觉和触摸（触觉和力/扭矩），旨在预测和指示具有相似外观的物体的多个负担能力（即抓地力和推动力）的可操作区域属性（例如，质量分布）。然后，对基于DQN的深钢筋学习算法进行培训，以选择成功对象操纵的最佳动作。为了验证提出的框架的性能，使用开放数据集和收集的数据集对我们的方法进行评估和基准测试。结果表明，所提出的方法和整体框架的表现优于现有方法，并实现更好的准确性和更高的效率。

Class-incremental learning (CIL) learns a classification model with training data of different classes arising progressively. Existing CIL either suffers from serious accuracy loss due to catastrophic forgetting, or invades data privacy by revisiting used exemplars. Inspired by linear learning formulations, we propose an analytic class-incremental learning (ACIL) with absolute memorization of past knowledge while avoiding breaching of data privacy (i.e., without storing historical data). The absolute memorization is demonstrated in the sense that class-incremental learning using ACIL given present data would give identical results to that from its joint-learning counterpart which consumes both present and historical samples. This equality is theoretically validated. Data privacy is ensured since no historical data are involved during the learning process. Empirical validations demonstrate ACIL's competitive accuracy performance with near-identical results for various incremental task settings (e.g., 5-50 phases). This also allows ACIL to outperform the state-of-the-art methods for large-phase scenarios (e.g., 25 and 50 phases).

Surgical robot automation has attracted increasing research interest over the past decade, expecting its huge potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied AI has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant researchers. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how human demonstrations would affect policy learning. In this paper, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. With these, we further propose to collect human demonstrations and imitate the action patterns to achieve more effective policy learning. We showcase the improvement of our simulation environment with the designed new features and tasks, and validate state-of-the-art reinforcement learning algorithms using the interactive environment. Promising results are obtained, with which we hope to pave the way for future research on surgical embodied intelligence. Our platform is released and will be continuously updated in the website: https://med-air.github.io/SurRoL/

Accurate airway extraction from computed tomography (CT) images is a critical step for planning navigation bronchoscopy and quantitative assessment of airway-related chronic obstructive pulmonary disease (COPD). The existing methods are challenging to sufficiently segment the airway, especially the high-generation airway, with the constraint of the limited label and cannot meet the clinical use in COPD. We propose a novel two-stage 3D contextual transformer-based U-Net for airway segmentation using CT images. The method consists of two stages, performing initial and refined airway segmentation. The two-stage model shares the same subnetwork with different airway masks as input. Contextual transformer block is performed both in the encoder and decoder path of the subnetwork to finish high-quality airway segmentation effectively. In the first stage, the total airway mask and CT images are provided to the subnetwork, and the intrapulmonary airway mask and corresponding CT scans to the subnetwork in the second stage. Then the predictions of the two-stage method are merged as the final prediction. Extensive experiments were performed on in-house and multiple public datasets. Quantitative and qualitative analysis demonstrate that our proposed method extracted much more branches and lengths of the tree while accomplishing state-of-the-art airway segmentation performance. The code is available at https://github.com/zhaozsq/airway_segmentation.

Autoregressive language modeling (ALM) have been successfully used in self-supervised pre-training in Natural language processing (NLP). However, this paradigm has not achieved comparable results with other self-supervised approach in computer vision (e.g., contrastive learning, mask image modeling). In this paper, we try to find the reason why autoregressive modeling does not work well on vision tasks. To tackle this problem, we fully analyze the limitation of visual autoregressive methods and proposed a novel stochastic autoregressive image modeling (named SAIM) by the two simple designs. First, we employ stochastic permutation strategy to generate effective and robust image context which is critical for vision tasks. Second, we create a parallel encoder-decoder training process in which the encoder serves a similar role to the standard vision transformer focus on learning the whole contextual information, and meanwhile the decoder predicts the content of the current position, so that the encoder and decoder can reinforce each other. By introducing stochastic prediction and the parallel encoder-decoder, SAIM significantly improve the performance of autoregressive image modeling. Our method achieves the best accuracy (83.9%) on the vanilla ViT-Base model among methods using only ImageNet-1K data. Transfer performance in downstream tasks also show that our model achieves competitive performance.

Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks(winning tickets) which are capable of reaching accuracy comparable to the original models. However, these tickets are proved to be notrobust to adversarial examples, and even worse than their PLM counterparts. To address this problem, we propose a novel method based on learning binary weight masks to identify robust tickets hidden in the original PLMs. Since the loss is not differentiable for the binary mask, we assign the hard concrete distribution to the masks and encourage their sparsity using a smoothing approximation of L0 regularization.Furthermore, we design an adversarial loss objective to guide the search for robust tickets and ensure that the tickets perform well bothin accuracy and robustness. Experimental results show the significant improvement of the proposed method over previous work on adversarial robustness evaluation.

变压器架构已成为广泛的自然语言处理〜（NLP）模型的基本要素。随着大型NLP模型的趋势，增加的内存和计算成本阻碍了其在资源有限设备上的有效部署。因此，变压器量化吸引了广泛的研究兴趣。最近的工作认识到结构化的离群值是量化性能的关键瓶颈。但是，他们提出的方法增加了开销的计算，仍然将异常值留在那里。为了从根本上解决这个问题，本文深入研究了异常值的固有诱因和重要性。我们发现$ \ boldsymbol \ gamma $ in LaiserNorm（ln）充当异常值的有罪放大器，而异常值的重要性差异很大，其中一些代币提供的一些异常值覆盖了大面积，但可以牢固地夹住一个大面积，但可以将其夹住，而没有负面影响。 。在这些发现的激励下，我们提出了一个异常抑制框架，其中包括两个组成部分：伽玛迁移和象征性的剪辑。伽马迁移将异常放大器迁移到等效转换中的后续模块，从而导致更量化的模型而没有任何额外的负担。令牌的剪辑利用了令牌范围的较大差异，并设计了代币的粗到精细管道，以有效的方式获得了具有最小的最终量化损失的剪辑范围。该框架有效地抑制了异常值，可以在插件模式下使用。广泛的实验证明，我们的框架超过了现有作品，并且首次将6位训练后的BERT量化量化推向完整精确度（FP）级别。我们的代码可在https://github.com/wimh966/outlier_suppression上找到。

在本文中，我们提出了一种新颖的，通用的数据驱动方法，用于伺服控制连续机器人的3-D形状，并嵌入了纤维bragg光栅（FBG）传感器。 3D形状感知和控制技术的发展对于连续机器人在手术干预中自主执行任务至关重要。但是，由于连续机器人的非线性特性，主要难度在于它们的建模，尤其是对于具有可变刚度的软机器人。为了解决这个问题，我们通过利用FBG形状反馈和神经网络（NNS）提出了一个新的健壮自适应控制器，该反馈和神经网络（NNS）可以在线估算连续机器人的未知模型，并说明了意外的干扰以及NN近似错误，该错误表现出适应性行为对适应性行为呈现没有先验数据探索的未建模系统。基于新的复合适应算法，Lyapunov理论证明了具有NNS学习参数的闭环系统的渐近收敛。为了验证所提出的方法，我们通过使用两个连续机器人进行了一项全面的实验研究，这些连续机器人都与多核FBG集成，包括机器人辅助结肠镜和多部分可扩展的软操纵剂。结果表明，在各种非结构化环境以及幻影实验中，我们的控制器的可行性，适应性和优越性。