我们提出了神经引导的形状解析器（NGSP），一种方法，该方法学习如何将细粒度语义标签分配给3D形状的区域。 NGSP通过MAP推断解决了这个问题，在输入形状上建模了标签分配的后验概率，其具有学习的似然函数。为了使这次搜索易于进行，NGSP采用神经指南网络，了解近似后部。 NGSP通过使用引导网络的第一次采样提案找到高概率标签分配，然后在完全可能性下评估每个提案。我们评估NGSP从Partnet的制造3D形状的细粒度语义分割任务，其中形状被分解成对应于零件实例过分分割的区域。我们发现NGSP通过比较方法提供显着的性能改进，（i）使用区域对分组每点预测，（ii）使用区域作为自我监督信号或（iii）将标签分配给替代配方下的区域。此外，我们表明，即使具有有限的标记数据或作为形状区域经历人为腐败，NGSP即使具有有限的人为腐败，也会保持强劲的性能。最后，我们证明了NGSP可以直接应用于在线存储库中的CAD形状，并验证其效力与感知研究。

我们提出切碎，这是一种3D形状区域分解的方法。 Shred将3D点云作为输入，并使用学习的本地操作来产生近似细粒零件实例的分割。我们将切碎的分解操作赋予了三个分解操作：分裂区域，固定区域之间的边界，并将区域合并在一起。模块经过独立和本地培训，使切碎可以为在培训过程中未见的类别生成高质量的细分。我们通过Partnet的细粒细分进行训练和评估切碎；使用其合并 - 阈值超参数，我们表明，在任何所需的分解粒度下，切碎的分割可以更好地尊重与基线方法相比，更好地尊重地面真相的注释。最后，我们证明切碎对于下游应用非常有用，在零弹药细粒的零件实例分割上的所有基准都超过了所有基准，并且当与学习标记形状区域的方法结合使用时，几乎没有发射细粒的语义分割。

We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information. Our dataset consists of 573,585 part instances over 26,671 3D models covering 24 object categories. This dataset enables and serves as a catalyst for many tasks such as shape analysis, dynamic 3D scene modeling and simulation, affordance analysis, and others. Using our dataset, we establish three benchmarking tasks for evaluating 3D part recognition: fine-grained semantic segmentation, hierarchical semantic segmentation, and instance segmentation. We benchmark four state-ofthe-art 3D deep learning algorithms for fine-grained semantic segmentation and three baseline methods for hierarchical semantic segmentation. We also propose a novel method for part instance segmentation and demonstrate its superior performance over existing methods.

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.

我们建议在2D域中利用自我监督的技术来实现细粒度的3D形状分割任务。这是受到观察的启发：基于视图的表面表示比基于点云或体素占用率的3D对应物更有效地建模高分辨率表面细节和纹理。具体而言，给定3D形状，我们将其从多个视图中渲染，并在对比度学习框架内建立密集的对应学习任务。结果，与仅在2D或3D中使用自学的替代方案相比，学到的2D表示是视图不变和几何一致的，在对有限的标记形状进行培训时，可以更好地概括概括。对纹理（渲染peple）和未纹理（partnet）3D数据集的实验表明，我们的方法在细粒部分分割中优于最先进的替代方案。当仅一组稀疏的视图可供训练或形状纹理时，对基准的改进就会更大，这表明MVDecor受益于2D处理和3D几何推理。

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

我们通过执行基于接触的推理，提供了一种形状部分插槽机，一种用于组装来自现有部件的新型3D形状。我们的方法表示每个形状作为“槽”的图形，其中每个槽是两个形状部件之间的接触区域。基于此表示，我们设计了一种基于图形 - 神经网络的模型，用于生成新的插槽图和检索兼容部分，以及基于梯度 - 下降的优化方案，用于将检索到的部分组装成尊重所生成的完整形状插槽图。这种方法不需要任何语义部分标签;有趣的是，它还不需要完整的部分几何形状 - 推理零件连接的区域足以产生新颖的，高质量的3D形状。我们展示了我们的方法在质量，多样性和结构复杂性方面产生了优于现有的逐个拟合方法的形状。

我们呈现FURTIT，这是一种简单的3D形状分割网络的高效学习方法。FURTIT基于自我监督的任务，可以将3D形状的表面分解成几何基元。可以很容易地应用于用于3D形状分割的现有网络架构，并提高了几张拍摄设置中的性能，因为我们在广泛使用的ShapEnet和Partnet基准中展示。FISHIT在这种环境中优于现有的现有技术，表明对基元的分解是在学习对语义部分预测的陈述之前的有用。我们提出了许多实验，改变了几何基元和下游任务的选择，以证明该方法的有效性。

最近对隐含形状表示的兴趣日益增长。与明确的陈述相反，他们没有解决局限性，他们很容易处理各种各样的表面拓扑。为了了解这些隐式表示，电流方法依赖于一定程度的形状监督（例如，内部/外部信息或距离形状知识），或者至少需要密集点云（以近似距离 - 到 - 到 - 形状）。相比之下，我们介绍{\方法}，一种用于学习形状表示的自我监督方法，从可能极其稀疏的点云。就像在水牛的针问题一样，我们在点云上“掉落”（样本）针头，认为，静统计地靠近表面，针端点位于表面的相对侧。不需要形状知识，点云可以高稀疏，例如，作为车辆获取的Lidar点云。以前的自我监督形状表示方法未能在这种数据上产生良好的结果。我们获得定量结果与现有的形状重建数据集上现有的监督方法标准，并在Kitti等硬自动驾驶数据集中显示有前途的定性结果。

Generalizable 3D part segmentation is important but challenging in vision and robotics. Training deep models via conventional supervised methods requires large-scale 3D datasets with fine-grained part annotations, which are costly to collect. This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP, which achieves superior performance on open-vocabulary 2D detection. We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm. We also utilize multi-view 3D priors and few-shot prompt tuning to boost performance significantly. Extensive evaluation on PartNet and PartNet-Mobility datasets shows that our method enables excellent zero-shot 3D part segmentation. Our few-shot version not only outperforms existing few-shot approaches by a large margin but also achieves highly competitive results compared to the fully supervised counterpart. Furthermore, we demonstrate that our method can be directly applied to iPhone-scanned point clouds without significant domain gaps.

本地化隐式功能的最新进展使神经隐式表示能够可扩展到大型场景。然而，这些方法采用的3D空间的定期细分未能考虑到表面占用的稀疏性和几何细节的变化粒度。结果，其内存占地面积与输入体积均别较大，即使在适度密集的分解中也导致禁止的计算成本。在这项工作中，我们为3D表面，编码OCTFIELD提供了一种学习的分层隐式表示，允许具有低内存和计算预算的复杂曲面的高精度编码。我们方法的关键是仅在感兴趣的表面周围分发本地隐式功能的3D场景的自适应分解。我们通过引入分层Octree结构来实现这一目标，以根据表面占用和部件几何形状的丰富度自适应地细分3D空间。随着八十六是离散和不可分辨性的，我们进一步提出了一种新颖的等级网络，其模拟八偏细胞的细分作为概率的过程，并以可差的方式递归地编码和解码八叠结构和表面几何形状。我们展示了Octfield的一系列形状建模和重建任务的价值，显示出在替代方法方面的优越性。

3D点云的卷积经过广泛研究，但在几何深度学习中却远非完美。卷积的传统智慧在3D点之间表现出特征对应关系，这是对差的独特特征学习的内在限制。在本文中，我们提出了自适应图卷积（AGCONV），以供点云分析的广泛应用。 AGCONV根据其动态学习的功能生成自适应核。与使用固定/各向同性核的解决方案相比，AGCONV提高了点云卷积的灵活性，有效，精确地捕获了不同语义部位的点之间的不同关系。与流行的注意力体重方案不同，AGCONV实现了卷积操作内部的适应性，而不是简单地将不同的权重分配给相邻点。广泛的评估清楚地表明，我们的方法优于各种基准数据集中的点云分类和分割的最新方法。同时，AGCONV可以灵活地采用更多的点云分析方法来提高其性能。为了验证其灵活性和有效性，我们探索了基于AGCONV的完成，DeNoing，Upsmpling，注册和圆圈提取的范式，它们与竞争对手相当甚至优越。我们的代码可在https://github.com/hrzhou2/adaptconv-master上找到。

我们呈现NESF，一种用于单独从构成的RGB图像中生成3D语义场的方法。代替经典的3D表示，我们的方法在最近的基础上建立了隐式神经场景表示的工作，其中3D结构被点亮功能捕获。我们利用这种方法来恢复3D密度领域，我们然后在其中培训由构成的2D语义地图监督的3D语义分段模型。尽管仅在2D信号上培训，我们的方法能够从新颖的相机姿势生成3D一致的语义地图，并且可以在任意3D点查询。值得注意的是，NESF与产生密度场的任何方法兼容，并且随着密度场的质量改善，其精度可提高。我们的实证分析在复杂的实际呈现的合成场景中向竞争性2D和3D语义分割基线表现出可比的质量。我们的方法是第一个提供真正密集的3D场景分段，需要仅需要2D监督培训，并且不需要任何关于新颖场景的推论的语义输入。我们鼓励读者访问项目网站。

We introduce a novel deep learning-based framework to interpret 3D urban scenes represented as textured meshes. Based on the observation that object boundaries typically align with the boundaries of planar regions, our framework achieves semantic segmentation in two steps: planarity-sensible over-segmentation followed by semantic classification. The over-segmentation step generates an initial set of mesh segments that capture the planar and non-planar regions of urban scenes. In the subsequent classification step, we construct a graph that encodes the geometric and photometric features of the segments in its nodes and the multi-scale contextual features in its edges. The final semantic segmentation is obtained by classifying the segments using a graph convolutional network. Experiments and comparisons on two semantic urban mesh benchmarks demonstrate that our approach outperforms the state-of-the-art methods in terms of boundary quality, mean IoU (intersection over union), and generalization ability. We also introduce several new metrics for evaluating mesh over-segmentation methods dedicated to semantic segmentation, and our proposed over-segmentation approach outperforms state-of-the-art methods on all metrics. Our source code is available at \url{https://github.com/WeixiaoGao/PSSNet}.

Point cloud completion is a generation and estimation issue derived from the partial point clouds, which plays a vital role in the applications in 3D computer vision. The progress of deep learning (DL) has impressively improved the capability and robustness of point cloud completion. However, the quality of completed point clouds is still needed to be further enhanced to meet the practical utilization. Therefore, this work aims to conduct a comprehensive survey on various methods, including point-based, convolution-based, graph-based, and generative model-based approaches, etc. And this survey summarizes the comparisons among these methods to provoke further research insights. Besides, this review sums up the commonly used datasets and illustrates the applications of point cloud completion. Eventually, we also discussed possible research trends in this promptly expanding field.

我们呈现Mix3D，一种用于分割大规模3D场景的数据增强技术。由于场景上下文有助于推理对象语义，因此当前的工作侧重于具有大容量和接收字段的模型，可以完全捕获输入3D场景的全局上下文。然而，强烈的背景前瞻可能会有不利的影响，就像错过了一个穿过街道的行人。在这项工作中，我们专注于平衡全球场景和局部几何形状的重要性，以概括在培训集中的上下文前方之外的目标。特别是，我们提出了一种“混合”技术，通过组合两个增强的场景来创造新的训练样本。通过这样做，对象实例被隐式地放入新颖的外观环境中，因此模型更难地依赖场景上下文，而是从本地结构推断出语义。我们进行详细的分析以了解全球背景，局部结构，局部结构和混合场景效果的重要性。在实验中，我们展示了Mix3D培训的模型从室内（Scannet，S3DIS）和室外数据集（Semantickitti）上的显着性能提升。 Mix3D可以逐渐与任何现有方法一起使用，例如，用Mix3D培训，MinkowsWinet在SCANNet测试基准78.1 Miou的显着边际占据了所有现有最先进的方法。代码可用：https://nekrasov.dev/mix3d/

Current 3D object detection methods are heavily influenced by 2D detectors. In order to leverage architectures in 2D detectors, they often convert 3D point clouds to regular grids (i.e., to voxel grids or to bird's eye view images), or rely on detection in 2D images to propose 3D boxes. Few works have attempted to directly detect objects in point clouds. In this work, we return to first principles to construct a 3D detection pipeline for point cloud data and as generic as possible. However, due to the sparse nature of the data -samples from 2D manifolds in 3D space -we face a major challenge when directly predicting bounding box parameters from scene points: a 3D object centroid can be far from any surface point thus hard to regress accurately in one step. To address the challenge, we propose VoteNet, an end-to-end 3D object detection network based on a synergy of deep point set networks and Hough voting. Our model achieves state-of-the-art 3D detection on two large datasets of real 3D scans, ScanNet and SUN RGB-D with a simple design, compact model size and high efficiency. Remarkably, VoteNet outperforms previous methods by using purely geometric information without relying on color images.

我们介绍了PartGolot，神经框架和相关架构，用于学习3D形几何的语义部分分割，仅基于部分参照语言。我们利用形状的语言描述可以提供形状的部分的前瞻性 - 因为自然语言已经发展，以反映对物体的组成结构的人类感知，对其认可和使用至关重要。对于培训，我们使用Shapeglot工作中收集的成对几何/语言数据来为其参考游戏，其中扬声器创建话语以区分从两个牵引器的目标形状，并且听众必须基于这种话语找到目标。我们的网络旨在解决此目标辨别问题，仔细介绍基于变压器的注意模块，以便输出注意力可以精确地突出显示语言中描述的语义部件或零件。此外，网络在3D几何形状本身上没有任何直接监督。令人惊讶的是，我们进一步证明学习部分信息是概括的，可以在训练期间形状看不见。我们的方法打开了单独从语言学习3D形状的可能性，而无需大规模部分几何注释，从而促进注释采集。

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset. Code, data and trained models are available at https://wentaoyuan.github.io/pcn.

现实的3D室内场景数据集在计算机视觉，场景理解，自主导航和3D重建中启用了最近的最近进展。但是，现有数据集的规模，多样性和可定制性有限，并且扫描和注释更多的耗时和昂贵。幸运的是，组合者在我们方面：现有3D场景数据集有足够的个别房间，如果有一种方法可以将它们重新组合成新的布局。在本文中，我们提出了从现有3D房间生成新型3D平面图的任务。我们确定了这个问题的三个子任务：生成2D布局，检索兼容3D房间，以及3D房间的变形，以适应布局。然后，我们讨论解决问题的不同策略，设计两个代表性管道：一个使用可用的2D楼层计划，以指导3D房间的选择和变形;另一个学习检索一组兼容的3D房间，并将它们与新颖的布局相结合。我们设计一组指标，可评估所生成的结果与三个子任务中的每一个，并显示不同的方法在这些子任务上交易性能。最后，我们调查从生成的3D场景中受益的下游任务，并讨论选择最适合这些任务的需求的方法。