3D重建问题中的一个关键问题是如何训练机器人或机器人以模型3D对象。在实时系统（例如自动驾驶汽车）中导航等许多任务直接取决于此问题。这些系统通常具有有限的计算能力。尽管近年来3D重建系统在3D重建系统中取得了长足的进展，但由于现有方法的高复杂性和计算需求，将它们应用于自动驾驶汽车中的导航系统等实时系统仍然具有挑战性。这项研究解决了以更快（实时）方式重建单视图像中显示的对象的当前问题。为此，开发了一个简单而强大的深度神经框架。提出的框架由两个组件组成：特征提取器模块和3D发电机模块。我们将点云表示为我们的重建模块的输出。将Shapenet数据集用于将方法与计算时间和准确性方面的现有结果进行比较。模拟证明了所提出的方法的出色性能。索引术语现实时间3D重建，单视图重建，监督学习，深神经网络

Point cloud completion is a generation and estimation issue derived from the partial point clouds, which plays a vital role in the applications in 3D computer vision. The progress of deep learning (DL) has impressively improved the capability and robustness of point cloud completion. However, the quality of completed point clouds is still needed to be further enhanced to meet the practical utilization. Therefore, this work aims to conduct a comprehensive survey on various methods, including point-based, convolution-based, graph-based, and generative model-based approaches, etc. And this survey summarizes the comparisons among these methods to provoke further research insights. Besides, this review sums up the commonly used datasets and illustrates the applications of point cloud completion. Eventually, we also discussed possible research trends in this promptly expanding field.

单视图3D对象重建是一项基本且具有挑战性的计算机视觉任务，旨在从单视RGB图像中恢复3D形状。大多数现有的基于深度学习的重建方法都是​​在同一类别上培训和评估的，并且在处理训练过程中未见的新颖类别的物体时，它们无法正常工作。本文着眼于这个问题，解决了零照片的单视3D网格重建，以研究对看不见类别的模型概括，并鼓励模型从字面上重建对象。具体而言，我们建议一个端到端的两阶段网络Zeromesh，以打破重建中的类别边界。首先，我们将复杂的图像到网格映射分解为两个较简单的映射，即图像对点映射和点对点映射，而后者主要是几何问题，而不是对象类别的依赖。其次，我们在2D和3D特征空间中设计了局部特征采样策略，以捕获跨对象共享的局部几何形状，以增强模型概括。第三，除了传统的点对点监督外，我们还引入了多视图轮廓损失以监督表面生成过程，该过程提供了其他正则化，并进一步缓解了过度拟合的问题。实验结果表明，我们的方法在不同方案和各种指标下，特别是对于新颖对象而言，在Shapenet和Pix3D上的现有作品显着优于Shapenet和Pix3D的现有作品。

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset. Code, data and trained models are available at https://wentaoyuan.github.io/pcn.

Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to represent and generate 3D shapes, as well as a vast number of use cases. However, single-view reconstruction remains a challenging topic that can unlock various interesting use cases such as interactive design. In this work, we propose a novel framework that leverages the intermediate latent spaces of Vision Transformer (ViT) and a joint image-text representational model, CLIP, for fast and efficient Single View Reconstruction (SVR). More specifically, we propose a novel mapping network architecture that learns a mapping between deep features extracted from ViT and CLIP, and the latent space of a base 3D generative model. Unlike previous work, our method enables view-agnostic reconstruction of 3D shapes, even in the presence of large occlusions. We use the ShapeNetV2 dataset and perform extensive experiments with comparisons to SOTA methods to demonstrate our method's effectiveness.

您将如何通过一些错过来修复物理物体？您可能会想象它的原始形状从先前捕获的图像中，首先恢复其整体（全局）但粗大的形状，然后完善其本地细节。我们有动力模仿物理维修程序以解决点云完成。为此，我们提出了一个跨模式的形状转移双转化网络（称为CSDN），这是一种带有全循环参与图像的粗到精细范式，以完成优质的点云完成。 CSDN主要由“ Shape Fusion”和“ Dual-Refinect”模块组成，以应对跨模式挑战。第一个模块将固有的形状特性从单个图像传输，以指导点云缺失区域的几何形状生成，在其中，我们建议iPadain嵌入图像的全局特征和部分点云的完成。第二个模块通过调整生成点的位置来完善粗糙输出，其中本地改进单元通过图卷积利用了小说和输入点之间的几何关系，而全局约束单元则利用输入图像来微调生成的偏移。与大多数现有方法不同，CSDN不仅探讨了图像中的互补信息，而且还可以在整个粗到精细的完成过程中有效利用跨模式数据。实验结果表明，CSDN对十个跨模式基准的竞争对手表现出色。

Intelligent mesh generation (IMG) refers to a technique to generate mesh by machine learning, which is a relatively new and promising research field. Within its short life span, IMG has greatly expanded the generalizability and practicality of mesh generation techniques and brought many breakthroughs and potential possibilities for mesh generation. However, there is a lack of surveys focusing on IMG methods covering recent works. In this paper, we are committed to a systematic and comprehensive survey describing the contemporary IMG landscape. Focusing on 110 preliminary IMG methods, we conducted an in-depth analysis and evaluation from multiple perspectives, including the core technique and application scope of the algorithm, agent learning goals, data types, targeting challenges, advantages and limitations. With the aim of literature collection and classification based on content extraction, we propose three different taxonomies from three views of key technique, output mesh unit element, and applicable input data types. Finally, we highlight some promising future research directions and challenges in IMG. To maximize the convenience of readers, a project page of IMG is provided at \url{https://github.com/xzb030/IMG_Survey}.

基于单个草图图像重建3D形状是由于稀疏，不规则的草图和常规，密集的3D形状之间的较大域间隙而具有挑战性的。现有的作品尝试采用从草图提取的全局功能来直接预测3D坐标，但通常会遭受失去对输入草图不忠心的细节。通过分析3D到2D投影过程，我们注意到表征2D点云分布的密度图（即，投影平面每个位置的点的概率）可以用作代理，以促进该代理重建过程。为此，我们首先通过图像翻译网络将草图翻译成一个更有信息的2D表示，可用于生成密度映射。接下来，通过两个阶段的概率采样过程重建一个3D点云：首先通过对密度映射进行采样，首先恢复2D点（即X和Y坐标）；然后通过在每个2D点确定的射线处采样深度值来预测深度​​（即Z坐标）。进行了广泛的实验，定量和定性结果都表明，我们提出的方法显着优于其他基线方法。

Figure 1. Given input as either a 2D image or a 3D point cloud (a), we automatically generate a corresponding 3D mesh (b) and its atlas parameterization (c). We can use the recovered mesh and atlas to apply texture to the output shape (d) as well as 3D print the results (e).

最近的研究表明，MMWave雷达感测在低可见性环境中对象检测的有效性，这使其成为自主导航系统中的理想技术。在本文中，我们将雷达介绍给点云（R2P），这是一个深度学习模型，该模型基于具有不正确点的粗糙和稀疏点云，生成具有精细几何细节的3D对象的平滑，密集且高度准确的点云表示。来自mmwave雷达。这些输入点云是从由原始MMWave雷达传感器数据生成的2D深度图像转换的，其特征是不一致，方向和形状误差。 R2P利用两个顺序的深度学习编码器块的体系结构在从多个角度观察到对象的基于雷达的输入点云的基本特征，并确保生成的输出点云及其准确的内部一致性和原始对象的详细形状重建。我们实施R2P来替换我们最近提出的3DRIMR（通过MMWave Radar）系统的第2阶段。我们的实验证明了R2P在流行的现有方法（例如PointNet，PCN和原始3DRIMR设计）上的显着性能提高。

Point clouds are characterized by irregularity and unstructuredness, which pose challenges in efficient data exploitation and discriminative feature extraction. In this paper, we present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology as a completely regular 2D point geometry image (PGI) structure, in which coordinates of spatial points are captured in colors of image pixels. \mr{Intuitively, Flattening-Net implicitly approximates a locally smooth 3D-to-2D surface flattening process while effectively preserving neighborhood consistency.} \mr{As a generic representation modality, PGI inherently encodes the intrinsic property of the underlying manifold structure and facilitates surface-style point feature aggregation.} To demonstrate its potential, we construct a unified learning framework directly operating on PGIs to achieve \mr{diverse types of high-level and low-level} downstream applications driven by specific task networks, including classification, segmentation, reconstruction, and upsampling. Extensive experiments demonstrate that our methods perform favorably against the current state-of-the-art competitors. We will make the code and data publicly available at https://github.com/keeganhk/Flattening-Net.

Point clouds captured by scanning devices are often incomplete due to occlusion. Point cloud completion aims to predict the complete shape based on its partial input. Existing methods can be classified into supervised and unsupervised methods. However, both of them require a large number of 3D complete point clouds, which are difficult to capture. In this paper, we propose Cross-PCC, an unsupervised point cloud completion method without requiring any 3D complete point clouds. We only utilize 2D images of the complete objects, which are easier to capture than 3D complete and clean point clouds. Specifically, to take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features and design a fusion module to fuse the 2D and 3D features extracted from the partial point cloud. To guide the shape of predicted point clouds, we project the predicted points of the object to the 2D plane and use the foreground pixels of its silhouette maps to constrain the position of the projected points. To reduce the outliers of the predicted point clouds, we propose a view calibrator to move the points projected to the background into the foreground by the single-view silhouette image. To the best of our knowledge, our approach is the first point cloud completion method that does not require any 3D supervision. The experimental results of our method are superior to those of the state-of-the-art unsupervised methods by a large margin. Moreover, compared to some supervised methods, our method achieves similar performance. We will make the source code publicly available at https://github.com/ltwu6/cross-pcc.

Generation of 3D data by deep neural network has been attracting increasing attention in the research community. The majority of extant works resort to regular representations such as volumetric grids or collection of images; however, these representations obscure the natural invariance of 3D shapes under geometric transformations, and also suffer from a number of other issues. In this paper we address the problem of 3D reconstruction from a single image, generating a straight-forward form of output -point cloud coordinates. Along with this problem arises a unique and interesting issue, that the groundtruth shape for an input image may be ambiguous. Driven by this unorthodox output form and the inherent ambiguity in groundtruth, we design architecture, loss function and learning paradigm that are novel and effective. Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image. In experiments not only can our system outperform state-ofthe-art methods on single image based 3d reconstruction benchmarks; but it also shows strong performance for 3d shape completion and promising ability in making multiple plausible predictions.

Generative models, as an important family of statistical modeling, target learning the observed data distribution via generating new instances. Along with the rise of neural networks, deep generative models, such as variational autoencoders (VAEs) and generative adversarial network (GANs), have made tremendous progress in 2D image synthesis. Recently, researchers switch their attentions from the 2D space to the 3D space considering that 3D data better aligns with our physical world and hence enjoys great potential in practice. However, unlike a 2D image, which owns an efficient representation (i.e., pixel grid) by nature, representing 3D data could face far more challenges. Concretely, we would expect an ideal 3D representation to be capable enough to model shapes and appearances in details, and to be highly efficient so as to model high-resolution data with fast speed and low memory cost. However, existing 3D representations, such as point clouds, meshes, and recent neural fields, usually fail to meet the above requirements simultaneously. In this survey, we make a thorough review of the development of 3D generation, including 3D shape generation and 3D-aware image synthesis, from the perspectives of both algorithms and more importantly representations. We hope that our discussion could help the community track the evolution of this field and further spark some innovative ideas to advance this challenging task.

Implicit shape representations, such as Level Sets, provide a very elegant formulation for performing computations involving curves and surfaces. However, including implicit representations into canonical Neural Network formulations is far from straightforward. This has consequently restricted existing approaches to shape inference, to significantly less effective representations, perhaps most commonly voxels occupancy maps or sparse point clouds.To overcome this limitation we propose a novel formulation that permits the use of implicit representations of curves and surfaces, of arbitrary topology, as individual layers in Neural Network architectures with end-to-end trainability. Specifically, we propose to represent the output as an oriented level set of a continuous and discretised embedding function. We investigate the benefits of our approach on the task of 3D shape prediction from a single image and demonstrate its ability to produce a more accurate reconstruction compared to voxel-based representations. We further show that our model is flexible and can be applied to a variety of shape inference problems.

Figure 1: DeepSDF represents signed distance functions (SDFs) of shapes via latent code-conditioned feed-forward decoder networks. Above images are raycast renderings of DeepSDF interpolating between two shapes in the learned shape latent space. Best viewed digitally.

从单视图重建3D形状是一个长期的研究问题。在本文中，我们展示了深度隐式地面网络，其可以通过预测底层符号距离场来从2D图像产生高质量的细节的3D网格。除了利用全局图像特征之外，禁止2D图像上的每个3D点的投影位置，并从图像特征映射中提取本地特征。结合全球和局部特征显着提高了符合距离场预测的准确性，特别是对于富含细节的区域。据我们所知，伪装是一种不断捕获从单视图图像中存在于3D形状中存在的孔和薄结构等细节的方法。 Disn在从合成和真实图像重建的各种形状类别上实现最先进的单视性重建性能。代码可在https://github.com/xharlie/disn提供补充可以在https://xharlie.github.io/images/neUrips_2019_Supp.pdf中找到补充

许多涉及某种形式的3D视觉感知的机器人任务极大地受益于对工作环境的完整知识。但是，机器人通常必须应对非结构化的环境，并且由于工作空间有限，混乱或对象自我划分，它们的车载视觉传感器只能提供不完整的信息。近年来，深度学习架构的形状完成架构已开始将牵引力作为从部分视觉数据中推断出完整的3D对象表示的有效手段。然而，大多数现有的最新方法都以体素电网形式提供了固定的输出分辨率，这与神经网络输出阶段的大小严格相关。尽管这足以完成某些任务，例如导航，抓握和操纵的障碍需要更精细的分辨率，并且简单地扩大神经网络输出在计算上是昂贵的。在本文中，我们通过基于隐式3D表示的对象形状完成方法来解决此限制，该方法为每个重建点提供了置信值。作为第二个贡献，我们提出了一种基于梯度的方法，用于在推理时在任意分辨率下有效地采样这种隐式函数。我们通过将重建的形状与地面真理进行比较，并通过在机器人握把管道中部署形状完成算法来实验验证我们的方法。在这两种情况下，我们将结果与最先进的形状完成方法进行了比较。

We propose an end-to-end deep learning architecture that produces a 3D shape in triangular mesh from a single color image. Limited by the nature of deep neural network, previous methods usually represent a 3D shape in volume or point cloud, and it is non-trivial to convert them to the more ready-to-use mesh model. Unlike the existing methods, our network represents 3D mesh in a graph-based convolutional neural network and produces correct geometry by progressively deforming an ellipsoid, leveraging perceptual features extracted from the input image. We adopt a coarse-to-fine strategy to make the whole deformation procedure stable, and define various of mesh related losses to capture properties of different levels to guarantee visually appealing and physically accurate 3D geometry. Extensive experiments show that our method not only qualitatively produces mesh model with better details, but also achieves higher 3D shape estimation accuracy compared to the state-of-the-art.

本文探讨了使用深神经网络从多个拼字图中重建的当前对象重建的最新。它提出了两种算法来从单个图像中提取多个视图。本文提出了一个基于像素对齐的隐式函数（PIFU）的系统，并制定了一个高级采样策略来生成签名的距离样品。它还将这种方法与从多个视图中的深度图回归进行了比较。此外，本文使用了一个新颖的数据集来进行赛车游戏Assetto Corsa的车辆重建，该数据集的质量比常用的Shapenet数据集更高。受过训练的神经网络很好地概括了现实世界的输入，并创建了合理且详细的重建。