Standard convolution is inherently limited for semantic segmentation of point cloud due to its isotropy about features. It neglects the structure of an object, results in poor object delineation and small spurious regions in the segmentation result. This paper proposes a novel graph attention convolution (GAC), whose kernels can be dynamically carved into specific shapes to adapt to the structure of an object. Specifically, by assigning proper attentional weights to different neighboring points, GAC is designed to selectively focus on the most relevant part of them according to their dynamically learned features. The shape of the convolution kernel is then determined by the learned distribution of the attentional weights. Though simple, GAC can capture the structured features of point clouds for finegrained segmentation and avoid feature contamination between objects. Theoretically, we provided a thorough analysis on the expressive capabilities of GAC to show how it can learn about the features of point clouds. Empirically, we evaluated the proposed GAC on challenging indoor and outdoor datasets and achieved the state-of-the-art results in both scenarios.

3D点云的卷积经过广泛研究，但在几何深度学习中却远非完美。卷积的传统智慧在3D点之间表现出特征对应关系，这是对差的独特特征学习的内在限制。在本文中，我们提出了自适应图卷积（AGCONV），以供点云分析的广泛应用。 AGCONV根据其动态学习的功能生成自适应核。与使用固定/各向同性核的解决方案相比，AGCONV提高了点云卷积的灵活性，有效，精确地捕获了不同语义部位的点之间的不同关系。与流行的注意力体重方案不同，AGCONV实现了卷积操作内部的适应性，而不是简单地将不同的权重分配给相邻点。广泛的评估清楚地表明，我们的方法优于各种基准数据集中的点云分类和分割的最新方法。同时，AGCONV可以灵活地采用更多的点云分析方法来提高其性能。为了验证其灵活性和有效性，我们探索了基于AGCONV的完成，DeNoing，Upsmpling，注册和圆圈提取的范式，它们与竞争对手相当甚至优越。我们的代码可在https://github.com/hrzhou2/adaptconv-master上找到。

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

We propose a novel deep learning-based framework to tackle the challenge of semantic segmentation of largescale point clouds of millions of points. We argue that the organization of 3D point clouds can be efficiently captured by a structure called superpoint graph (SPG), derived from a partition of the scanned scene into geometrically homogeneous elements. SPGs offer a compact yet rich representation of contextual relationships between object parts, which is then exploited by a graph convolutional network. Our framework sets a new state of the art for segmenting outdoor LiDAR scans (+11.9 and +8.8 mIoU points for both Semantic3D test sets), as well as indoor scans (+12.4 mIoU points for the S3DIS dataset).

通过当地地区的点特征聚合来捕获的细粒度几何是对象识别和场景理解在点云中的关键。然而，现有的卓越点云骨架通常包含最大/平均池用于局部特征聚集，这在很大程度上忽略了点的位置分布，导致细粒结构组装不足。为了缓解这一瓶颈，我们提出了一个有效的替代品，可以使用新颖的图形表示明确地模拟了本地点之间的空间关系，并以位置自适应方式聚合特征，从而实现位置敏感的表示聚合特征。具体而言，Papooling分别由两个关键步骤，图形结构和特征聚合组成，分别负责构造与将中心点连接的边缘与本地区域中的每个相邻点连接的曲线图组成，以将它们的相对位置信息映射到通道 - 明智的细心权重，以及基于通过图形卷积网络（GCN）的生成权重自适应地聚合局部点特征。 Papooling简单而且有效，并且足够灵活，可以随时为PointNet ++和DGCNN等不同的流行律源，作为即插即说运算符。关于各种任务的广泛实验，从3D形状分类，部分分段对场景分割良好的表明，伪装可以显着提高预测准确性，而具有最小的额外计算开销。代码将被释放。

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.

We present Kernel Point Convolution 1 (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KP-Conv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.

Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and density functions through kernel density estimation. The most important contribution of this work is a novel reformulation proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure.

机载激光扫描（ALS）点云的分类是遥感和摄影测量场的关键任务。尽管最近基于深度学习的方法取得了令人满意的表现，但他们忽略了接受场的统一性，这使得ALS点云分类对于区分具有复杂结构和极端规模变化的区域仍然具有挑战性。在本文中，为了配置多受感受性的场特征，我们提出了一个新型的接受场融合和分层网络（RFFS-NET）。以新颖的扩张图卷积（DGCONV）及其扩展环形扩张卷积（ADCONV）作为基本的构建块，使用扩张和环形图融合（Dagfusion）模块实现了接受场融合过程，该模块获得了多受感染的场特征代表通过捕获带有各种接收区域的扩张和环形图。随着计算碱基的计算基础，使用嵌套在RFFS-NET中的多级解码器进行的接收场的分层，并由多层接受场聚集损失（MRFALOSS）驱动，以驱动网络驱动网络以学习在具有不同分辨率的监督标签的方向。通过接受场融合和分层，RFFS-NET更适应大型ALS点云中具有复杂结构和极端尺度变化区域的分类。在ISPRS Vaihingen 3D数据集上进行了评估，我们的RFFS-NET显着优于MF1的基线方法5.3％，而MIOU的基线方法的总体准确性为82.1％，MF1的总准确度为71.6％，MIOU的MF1和MIOU为58.2％。此外，LASDU数据集和2019 IEEE-GRSS数据融合竞赛数据集的实验显示，RFFS-NET可以实现新的最新分类性能。

点云的语义分割通过密集预测每个点的类别来产生对场景的全面理解。由于接收场的一致性，点云的语义分割对于多受感受性场特征的表达仍然具有挑战性，这会导致对具有相似空间结构的实例的错误分类。在本文中，我们提出了一个植根于扩张图特征聚集（DGFA）的图形卷积网络DGFA-NET，该图由通过金字塔解码器计算出的多基质聚集损失（Maloss）引导。为了配置多受感受性字段特征，将建议的扩张图卷积（DGCONV）作为其基本构建块，旨在通过捕获带有各种接收区域的扩张图来汇总多尺度特征表示。通过同时考虑用不同分辨率的点集作为计算碱基的点集惩罚接收场信息，我们引入了由Maloss驱动的金字塔解码器，以了解接受田间的多样性。结合这两个方面，DGFA-NET显着提高了具有相似空间结构的实例的分割性能。 S3DIS，ShapenetPart和Toronto-3D的实验表明，DGFA-NET优于基线方法，实现了新的最新细分性能。

A number of problems can be formulated as prediction on graph-structured data. In this work, we generalize the convolution operator from regular grids to arbitrary graphs while avoiding the spectral domain, which allows us to handle graphs of varying size and connectivity. To move beyond a simple diffusion, filter weights are conditioned on the specific edge labels in the neighborhood of a vertex. Together with the proper choice of graph coarsening, we explore constructing deep neural networks for graph classification. In particular, we demonstrate the generality of our formulation in point cloud classification, where we set the new state of the art, and on a graph classification dataset, where we outperform other deep learning approaches. The source code is available at https://github.com/mys007/ecc.

标准空间卷积假设具有常规邻域结构的输入数据。现有方法通常通过修复常规“视图”来概括对不规则点云域的卷积。固定的邻域大小，卷积内核大小对于每个点保持不变。然而，由于点云不是像图像的结构，所以固定邻权给出了不幸的感应偏压。我们提出了一个名为digress图卷积（diffconv）的新图表卷积，不依赖常规视图。DiffConv在空间 - 变化和密度扩张的邻域上操作，其进一步由学习屏蔽的注意机制进行了进一步调整。我们在ModelNet40点云分类基准测试中验证了我们的模型，获得最先进的性能和更稳健的噪声，以及更快的推广速度。

We present an approach to semantic scene analysis using deep convolutional networks. Our approach is based on tangent convolutions -a new construction for convolutional networks on 3D data. In contrast to volumetric approaches, our method operates directly on surface geometry. Crucially, the construction is applicable to unstructured point clouds and other noisy real-world data. We show that tangent convolutions can be evaluated efficiently on large-scale point clouds with millions of points. Using tangent convolutions, we design a deep fully-convolutional network for semantic segmentation of 3D point clouds, and apply it to challenging real-world datasets of indoor and outdoor 3D environments. Experimental results show that the presented approach outperforms other recent deep network constructions in detailed analysis of large 3D scenes.

在本文中，我们提出了一个全面的点云语义分割网络，该网络汇总了本地和全球多尺度信息。首先，我们提出一个角度相关点卷积（ACPCONV）模块，以有效地了解点的局部形状。其次，基于ACPCONV，我们引入了局部多规模拆分（MSS）块，该块从一个单个块中连接到一个单个块中的特征，并逐渐扩大了接受场，这对利用本地上下文是有益的。第三，受HRNET的启发，在2D图像视觉任务上具有出色的性能，我们构建了一个针对Point Cloud的HRNET，以学习全局多尺度上下文。最后，我们介绍了一种融合多分辨率预测并进一步改善点云语义分割性能的点上的注意融合方法。我们在几个基准数据集上的实验结果和消融表明，与现有方法相比，我们提出的方法有效，能够实现最先进的性能。

This paper presents PointWeb, a new approach to extract contextual features from local neighborhood in a point cloud. Unlike previous work, we densely connect each point with every other in a local neighborhood, aiming to specify feature of each point based on the local region characteristics for better representing the region. A novel module, namely Adaptive Feature Adjustment (AFA) module, is presented to find the interaction between points. For each local region, an impact map carrying element-wise impact between point pairs is applied to the feature difference map. Each feature is then pulled or pushed by other features in the same region according to the adaptively learned impact indicators. The adjusted features are well encoded with region information, and thus benefit the point cloud recognition tasks, such as point cloud segmentation and classification. Experimental results show that our model outperforms the state-of-the-arts on both semantic segmentation and shape classification datasets.

Convolutional Neural Networks (CNNs) achieve impressive performance in a wide variety of fields. Their success benefited from a massive boost when very deep CNN models were able to be reliably trained. Despite their merits, CNNs fail to properly address problems with non-Euclidean data. To overcome this challenge, Graph Convolutional Networks (GCNs) build graphs to represent non-Euclidean data, borrow concepts from CNNs, and apply them in training. GCNs show promising results, but they are usually limited to very shallow models due to the vanishing gradient problem (see Figure 1). As a result, most state-of-the-art GCN models are no deeper than 3 or 4 layers. In this work, we present new ways to successfully train very deep GCNs. We do this by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures. Extensive experiments show the positive effect of these deep GCN frameworks. Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3.7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation. We believe that the community can greatly benefit from this work, as it opens up many opportunities for advancing GCN-based research.

Raw point clouds data inevitably contains outliers or noise through acquisition from 3D sensors or reconstruction algorithms. In this paper, we present a novel endto-end network for robust point clouds processing, named PointASNL, which can deal with point clouds with noise effectively. The key component in our approach is the adaptive sampling (AS) module. It first re-weights the neighbors around the initial sampled points from farthest point sampling (FPS), and then adaptively adjusts the sampled points beyond the entire point cloud. Our AS module can not only benefit the feature learning of point clouds, but also ease the biased effect of outliers. To further capture the neighbor and long-range dependencies of the sampled point, we proposed a local-nonlocal (L-NL) module inspired by the nonlocal operation. Such L-NL module enables the learning process insensitive to noise. Extensive experiments verify the robustness and superiority of our approach in point clouds processing tasks regardless of synthesis data, indoor data, and outdoor data with or without noise. Specifically, PointASNL achieves state-of-theart robust performance for classification and segmentation tasks on all datasets, and significantly outperforms previous methods on real-world outdoor SemanticKITTI dataset with considerate noise. Our code is released through https: //github.com/yanx27/PointASNL.

随着激光雷达传感器和3D视觉摄像头的扩散，3D点云分析近年来引起了重大关注。经过先驱工作点的成功后，基于深度学习的方法越来越多地应用于各种任务，包括3D点云分段和3D对象分类。在本文中，我们提出了一种新颖的3D点云学习网络，通过选择性地执行具有动态池的邻域特征聚合和注意机制来提出作为动态点特征聚合网络（DPFA-NET）。 DPFA-Net有两个可用于三维云的语义分割和分类的变体。作为DPFA-NET的核心模块，我们提出了一个特征聚合层，其中每个点的动态邻域的特征通过自我注意机制聚合。与其他分割模型相比，来自固定邻域的聚合特征，我们的方法可以在不同层中聚合来自不同邻居的特征，在不同层中为查询点提供更具选择性和更广泛的视图，并更多地关注本地邻域中的相关特征。此外，为了进一步提高所提出的语义分割模型的性能，我们提出了两种新方法，即两级BF-Net和BF-Rengralization来利用背景前台信息。实验结果表明，所提出的DPFA-Net在S3DIS数据集上实现了最先进的整体精度分数，在S3DIS数据集上进行了语义分割，并在不同的语义分割，部分分割和3D对象分类中提供始终如一的令人满意的性能。与其他方法相比，它也在计算上更有效。

我们提出了一种基于注意力的新型机制，可以学习用于点云处理任务的增强点特征，例如分类和分割。与先前的作品不同，该作品经过培训以优化预选的一组注意点的权重，我们的方法学会了找到最佳的注意点，以最大程度地提高特定任务的性能，例如点云分类。重要的是，我们主张使用单个注意点来促进语义理解在点特征学习中。具体而言，我们制定了一种新的简单卷积，该卷积结合了输入点及其相应学习的注意点或膝盖的卷积特征。我们的注意机制可以轻松地纳入最新的点云分类和分割网络中。对诸如ModelNet40，ShapenetPart和S3DIS之类的常见基准测试的广泛实验都表明，我们的支持LAP的网络始终优于各自的原始网络，以及其他竞争性替代方案，这些替代方案在我们的膝盖下采用了多个注意力框架。