Masked Modeling (MM) has demonstrated widespread success in various vision challenges, by reconstructing masked visual patches. Yet, applying MM for large-scale 3D scenes remains an open problem due to the data sparsity and scene complexity. The conventional random masking paradigm used in 2D images often causes a high risk of ambiguity when recovering the masked region of 3D scenes. To this end, we propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points, effectively enhancing the pretext masking task for 3D scene understanding. Integrated with a progressive reconstruction manner, our method can concentrate on modeling regional geometry and enjoy less ambiguity for masked reconstruction. Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas. By elegantly combining informative-preserved reconstruction on masked areas and consistency self-distillation from unmasked areas, a unified framework called MM-3DScene is yielded. We conduct comprehensive experiments on a host of downstream tasks. The consistent improvement (e.g., +6.1 mAP@0.5 on object detection and +2.2% mIoU on semantic segmentation) demonstrates the superiority of our approach.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Various types of Multi-Agent Reinforcement Learning (MARL) methods have been developed, assuming that agents' policies are based on true states. Recent works have improved the robustness of MARL under uncertainties from the reward, transition probability, or other partners' policies. However, in real-world multi-agent systems, state estimations may be perturbed by sensor measurement noise or even adversaries. Agents' policies trained with only true state information will deviate from optimal solutions when facing adversarial state perturbations during execution. MARL under adversarial state perturbations has limited study. Hence, in this work, we propose a State-Adversarial Markov Game (SAMG) and make the first attempt to study the fundamental properties of MARL under state uncertainties. We prove that the optimal agent policy and the robust Nash equilibrium do not always exist for an SAMG. Instead, we define the solution concept, robust agent policy, of the proposed SAMG under adversarial state perturbations, where agents want to maximize the worst-case expected state value. We then design a gradient descent ascent-based robust MARL algorithm to learn the robust policies for the MARL agents. Our experiments show that adversarial state perturbations decrease agents' rewards for several baselines from the existing literature, while our algorithm outperforms baselines with state perturbations and significantly improves the robustness of the MARL policies under state uncertainties.

The problem of covariate-shift generalization has attracted intensive research attention. Previous stable learning algorithms employ sample reweighting schemes to decorrelate the covariates when there is no explicit domain information about training data. However, with finite samples, it is difficult to achieve the desirable weights that ensure perfect independence to get rid of the unstable variables. Besides, decorrelating within stable variables may bring about high variance of learned models because of the over-reduced effective sample size. A tremendous sample size is required for these algorithms to work. In this paper, with theoretical justification, we propose SVI (Sparse Variable Independence) for the covariate-shift generalization problem. We introduce sparsity constraint to compensate for the imperfectness of sample reweighting under the finite-sample setting in previous methods. Furthermore, we organically combine independence-based sample reweighting and sparsity-based variable selection in an iterative way to avoid decorrelating within stable variables, increasing the effective sample size to alleviate variance inflation. Experiments on both synthetic and real-world datasets demonstrate the improvement of covariate-shift generalization performance brought by SVI.

Designing safety-critical control for robotic manipulators is challenging, especially in a cluttered environment. First, the actual trajectory of a manipulator might deviate from the planned one due to the complex collision environments and non-trivial dynamics, leading to collision; Second, the feasible space for the manipulator is hard to obtain since the explicit distance functions between collision meshes are unknown. By analyzing the relationship between the safe set and the controlled invariant set, this paper proposes a data-driven control barrier function (CBF) construction method, which extracts CBF from distance samples. Specifically, the CBF guarantees the controlled invariant property for considering the system dynamics. The data-driven method samples the distance function and determines the safe set. Then, the CBF is synthesized based on the safe set by a scenario-based sum of square (SOS) program. Unlike most existing linearization based approaches, our method reserves the volume of the feasible space for planning without approximation, which helps find a solution in a cluttered environment. The control law is obtained by solving a CBF-based quadratic program in real time, which works as a safe filter for the desired planning-based controller. Moreover, our method guarantees safety with the proven probabilistic result. Our method is validated on a 7-DOF manipulator in both real and virtual cluttered environments. The experiments show that the manipulator is able to execute tasks where the clearance between obstacles is in millimeters.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

本文考虑了有损神经图像压缩（NIC）的问题。当前的最新方法（SOTA）方法采用近似量化噪声的后部均匀的后方，单样本估计量近似于证据下限（ELBO）的梯度。在本文中，我们建议用多个样本重要性加权自动编码器（IWAE）目标训练NIC，该目标比Elbo更紧，并随着样本量的增加而收敛至对数的可能性。首先，我们确定NIC的均匀后验具有特殊的特性，这会影响IWAE目标的Pathiswise和得分函数估计器的方差和偏差。此外，从梯度差异的角度来看，我们提供了有关NIC中通常采用的技巧的见解。基于这些分析，我们进一步提出了多样本NIC（MS-NIC），这是NIC的IWAE靶标。实验结果表明，它改善了SOTA NIC方法。我们的MS-NIC是插件，可以轻松扩展到其他神经压缩任务。

大规模发光点云的快速有效语义分割是自主驾驶中的一个基本问题。为了实现这一目标，现有的基于点的方法主要选择采用随机抽样策略来处理大规模点云。但是，我们的数量和定性研究发现，随机抽样可能不适合自主驾驶场景，因为LiDAR点遵循整个空间的不均匀甚至长尾巴分布，这阻止了模型从从中捕获足够的信息，从而从中捕获了足够的信息不同的距离范围并降低了模型的学习能力。为了减轻这个问题，我们提出了一种新的极性缸平衡的随机抽样方法，该方法使下采样的点云能够保持更平衡的分布并改善不同空间分布下的分割性能。此外，引入了采样一致性损失，以进一步提高分割性能并降低模型在不同采样方法下的方差。广泛的实验证实，我们的方法在Semantickitti和Semanticposs基准测试中都产生了出色的性能，分别提高了2.8％和4.0％。

我们提出了一种新型的交通轨迹编辑方法，该方法使用时空钥匙帧在模拟过程中控制车辆以生成所需的交通轨迹。通过考虑自我动机，遵循和避免碰撞的路径，提出的基于力的交通模拟框架更新了FRENET坐标和笛卡尔坐标中车辆的动作。使用用户的路点，可以通过参考路径计划生成车道级导航。使用给定的钥匙帧，提出了粗到1的优化，以有效地生成可满足时空约束的合理轨迹。首先，沿参考路径构建的有向状态图用于通过将密钥帧映射为目标来搜索粗粒轨迹。然后，使用从粗轨迹作为初始化提取的信息，基于基于伴随的优化来生成基于我们基于力的仿真的平滑运动的较好轨迹。我们通过广泛的实验来验证我们的方法。

在本文中，我们考虑了神经视频压缩（NVC）中位分配的问题。由于帧参考结构，使用相同的R-D（速率）权衡参数$ \ lambda $的当前NVC方法是次优的，这带来了位分配的需求。与以前基于启发式和经验R-D模型的方法不同，我们建议通过基于梯度的优化解决此问题。具体而言，我们首先提出了一种基于半损坏的变异推理（SAVI）的连续位实现方法。然后，我们通过更改SAVI目标，使用迭代优化提出了一个像素级隐式分配方法。此外，我们基于NVC的可区分特征得出了精确的R-D模型。我们通过使用精确的R-D模型证明其等效性与位分配的等效性来展示我们的方法的最佳性。实验结果表明，我们的方法显着改善了NVC方法，并且胜过现有的位分配方法。我们的方法是所有可区分NVC方法的插件，并且可以直接在现有的预训练模型上采用。