科学和工程领域广泛使用计算机模拟。这些模拟通常以多个级别的复杂性运行，以平衡准确性和效率。多保真替代建模通过融合不同的仿真输出来降低计算成本。低保真模拟器产生的廉价数据可以与昂贵的高保真模拟器生成的有限高质量数据结合使用。基于高斯流程的现有方法依赖于内核函数的强烈假设，并且几乎不能扩展到高维设置。我们提出了多保真层次神经过程（MF-HNP），这是一种用于多效率替代模型的统一神经潜在变量模型。 MF-HNP继承了神经过程的灵活性和可扩展性。潜在变量将不同的保真度水平之间的相关性从观测到潜在空间。鉴于潜在状态，跨忠诚度之间的预测是有条件独立的。它有助于缓解现有方法中的错误传播问题。 MF-HNP足够灵活，可以在不同的保真度水平下处理非巢高维数据，并具有不同的输入和输出尺寸。我们评估了MF-HNP关于流行病学和气候建模任务的评估，从而在准确性和不确定性估计方面实现了竞争性能。与仅具有低维度（<10）任务的Deep Gaussian过程相反，我们的方法显示出巨大的希望，可以加速高维复杂模拟（用于流行病学建模的7000多个超过7000个，对于气候建模45000）。

Multi-view representation learning has developed rapidly over the past decades and has been applied in many fields. However, most previous works assumed that each view is complete and aligned. This leads to an inevitable deterioration in their performance when encountering practical problems such as missing or unaligned views. To address the challenge of representation learning on partially aligned multi-view data, we propose a new cross-view graph contrastive learning framework, which integrates multi-view information to align data and learn latent representations. Compared with current approaches, the proposed method has the following merits: (1) our model is an end-to-end framework that simultaneously performs view-specific representation learning via view-specific autoencoders and cluster-level data aligning by combining multi-view information with the cross-view graph contrastive learning; (2) it is easy to apply our model to explore information from three or more modalities/sources as the cross-view graph contrastive learning is devised. Extensive experiments conducted on several real datasets demonstrate the effectiveness of the proposed method on the clustering and classification tasks.

考虑到来源提供准确的信息（可信度）的可能性，加权多数投票（WMV）是集体决策制定的众所周知的最佳决策规则。但是，实际上，可信度不是对决策者的已知数量 - 他们必须依靠称为信托的估计。计算信任的（机器学习）算法在具有系统上不高估或低估可信度的属性时称为无偏见。为了正式分析决策过程的不确定性，我们介绍和分析了这种无偏信任值的两个重要特性：正确性的稳定性和最优性的稳定性。正确性的稳定性意味着决策者认为自己所获得的决策准确性等于实际准确性。我们证明了正确性的稳定性。最优性的稳定性意味着基于信任做出的决策与基于可信赖性的情况一样好。最佳稳定性不达到。我们分析了两者之间的差异及其界限。我们还概述了对信任和可信度变化的敏感决策正确性的敏感性。

在本文中，我们考虑了在不完整视图上的多视图聚类问题。与完整的多视图聚类相比，视图缺失的问题会增加学习不同视图的常见表示的难度。为了解决挑战，我们提出了一种新颖的不完整的多视图聚类框架，该框架包含跨视网围传输和多视图融合学习。具体地，基于在多视图数据中存在的一致性，我们设计了一种基于跨视网围的转移转移的完成模块，该完成模块将已知与缺失视图的已知相似的相互关系的关系传输，并根据传输的图形网络恢复丢失的数据关系图。然后，设计特定于特定的编码器以提取恢复的多视图数据，引入基于注意的融合层以获得公共表示。此外，为了减少由视图之间不一致并获得更好的聚类结构引起的误差的影响，引入了联合聚类层以同时优化恢复和聚类。在几个真实数据集上进行的广泛实验证明了该方法的有效性。

基于图形的多视图聚类，旨在跨多种视图获取数据分区，近年来接受了相当大的关注。虽然已经为基于图形的多视图群集进行了巨大努力，但它对各种视图融合特征仍然是一个挑战，以学习聚类的常见表示。在本文中，我们提出了一种新的一致多曲线图嵌入聚类框架（CMGEC）。具体地，设计了一种多图自动编码器（M-GAE），用于使用多图注意融合编码器灵活地编码多视图数据的互补信息。为了引导所学过的公共表示维护每个视图中相邻特征的相似性，引入了多视图相互信息最大化模块（MMIM）。此外，设计了一个图形融合网络（GFN），以探讨来自不同视图的图表之间的关系，并提供M-GAE所需的常见共识图。通过联合训练这些模型，可以获得共同的潜在表示，其从多个视图中编码更多互补信息，并更全面地描绘数据。三种类型的多视图数据集的实验表明CMGEC优于最先进的聚类方法。

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pretrained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-Shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.