基于单个草图图像重建3D形状是由于稀疏，不规则的草图和常规，密集的3D形状之间的较大域间隙而具有挑战性的。现有的作品尝试采用从草图提取的全局功能来直接预测3D坐标，但通常会遭受失去对输入草图不忠心的细节。通过分析3D到2D投影过程，我们注意到表征2D点云分布的密度图（即，投影平面每个位置的点的概率）可以用作代理，以促进该代理重建过程。为此，我们首先通过图像翻译网络将草图翻译成一个更有信息的2D表示，可用于生成密度映射。接下来，通过两个阶段的概率采样过程重建一个3D点云：首先通过对密度映射进行采样，首先恢复2D点（即X和Y坐标）；然后通过在每个2D点确定的射线处采样深度值来预测深度​​（即Z坐标）。进行了广泛的实验，定量和定性结果都表明，我们提出的方法显着优于其他基线方法。

从单视图重建3D形状是一个长期的研究问题。在本文中，我们展示了深度隐式地面网络，其可以通过预测底层符号距离场来从2D图像产生高质量的细节的3D网格。除了利用全局图像特征之外，禁止2D图像上的每个3D点的投影位置，并从图像特征映射中提取本地特征。结合全球和局部特征显着提高了符合距离场预测的准确性，特别是对于富含细节的区域。据我们所知，伪装是一种不断捕获从单视图图像中存在于3D形状中存在的孔和薄结构等细节的方法。 Disn在从合成和真实图像重建的各种形状类别上实现最先进的单视性重建性能。代码可在https://github.com/xharlie/disn提供补充可以在https://xharlie.github.io/images/neUrips_2019_Supp.pdf中找到补充

单视图3D对象重建是一项基本且具有挑战性的计算机视觉任务，旨在从单视RGB图像中恢复3D形状。大多数现有的基于深度学习的重建方法都是​​在同一类别上培训和评估的，并且在处理训练过程中未见的新颖类别的物体时，它们无法正常工作。本文着眼于这个问题，解决了零照片的单视3D网格重建，以研究对看不见类别的模型概括，并鼓励模型从字面上重建对象。具体而言，我们建议一个端到端的两阶段网络Zeromesh，以打破重建中的类别边界。首先，我们将复杂的图像到网格映射分解为两个较简单的映射，即图像对点映射和点对点映射，而后者主要是几何问题，而不是对象类别的依赖。其次，我们在2D和3D特征空间中设计了局部特征采样策略，以捕获跨对象共享的局部几何形状，以增强模型概括。第三，除了传统的点对点监督外，我们还引入了多视图轮廓损失以监督表面生成过程，该过程提供了其他正则化，并进一步缓解了过度拟合的问题。实验结果表明，我们的方法在不同方案和各种指标下，特别是对于新颖对象而言，在Shapenet和Pix3D上的现有作品显着优于Shapenet和Pix3D的现有作品。

您将如何通过一些错过来修复物理物体？您可能会想象它的原始形状从先前捕获的图像中，首先恢复其整体（全局）但粗大的形状，然后完善其本地细节。我们有动力模仿物理维修程序以解决点云完成。为此，我们提出了一个跨模式的形状转移双转化网络（称为CSDN），这是一种带有全循环参与图像的粗到精细范式，以完成优质的点云完成。 CSDN主要由“ Shape Fusion”和“ Dual-Refinect”模块组成，以应对跨模式挑战。第一个模块将固有的形状特性从单个图像传输，以指导点云缺失区域的几何形状生成，在其中，我们建议iPadain嵌入图像的全局特征和部分点云的完成。第二个模块通过调整生成点的位置来完善粗糙输出，其中本地改进单元通过图卷积利用了小说和输入点之间的几何关系，而全局约束单元则利用输入图像来微调生成的偏移。与大多数现有方法不同，CSDN不仅探讨了图像中的互补信息，而且还可以在整个粗到精细的完成过程中有效利用跨模式数据。实验结果表明，CSDN对十个跨模式基准的竞争对手表现出色。

我们提出了一个新的框架，以重建整体3D室内场景，包括单视图像的房间背景和室内对象。由于室内场景的严重阻塞，现有方法只能产生具有有限几何质量的室内物体的3D形状。为了解决这个问题，我们提出了一个与实例一致的隐式函数（InstPifu），以进行详细的对象重建。与实例对齐的注意模块结合使用，我们的方法有权将混合的局部特征与遮挡实例相结合。此外，与以前的方法不同，该方法仅代表房间背景为3D边界框，深度图或一组平面，我们通过隐式表示恢复了背景的精细几何形状。在E SUN RGB-D，PIX3D，3D-FUTURE和3D-FRONT数据集上进行的广泛实验表明，我们的方法在背景和前景对象重建中均优于现有方法。我们的代码和模型将公开可用。

Point clouds captured by scanning devices are often incomplete due to occlusion. Point cloud completion aims to predict the complete shape based on its partial input. Existing methods can be classified into supervised and unsupervised methods. However, both of them require a large number of 3D complete point clouds, which are difficult to capture. In this paper, we propose Cross-PCC, an unsupervised point cloud completion method without requiring any 3D complete point clouds. We only utilize 2D images of the complete objects, which are easier to capture than 3D complete and clean point clouds. Specifically, to take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features and design a fusion module to fuse the 2D and 3D features extracted from the partial point cloud. To guide the shape of predicted point clouds, we project the predicted points of the object to the 2D plane and use the foreground pixels of its silhouette maps to constrain the position of the projected points. To reduce the outliers of the predicted point clouds, we propose a view calibrator to move the points projected to the background into the foreground by the single-view silhouette image. To the best of our knowledge, our approach is the first point cloud completion method that does not require any 3D supervision. The experimental results of our method are superior to those of the state-of-the-art unsupervised methods by a large margin. Moreover, compared to some supervised methods, our method achieves similar performance. We will make the source code publicly available at https://github.com/ltwu6/cross-pcc.

Generation of 3D data by deep neural network has been attracting increasing attention in the research community. The majority of extant works resort to regular representations such as volumetric grids or collection of images; however, these representations obscure the natural invariance of 3D shapes under geometric transformations, and also suffer from a number of other issues. In this paper we address the problem of 3D reconstruction from a single image, generating a straight-forward form of output -point cloud coordinates. Along with this problem arises a unique and interesting issue, that the groundtruth shape for an input image may be ambiguous. Driven by this unorthodox output form and the inherent ambiguity in groundtruth, we design architecture, loss function and learning paradigm that are novel and effective. Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image. In experiments not only can our system outperform state-ofthe-art methods on single image based 3d reconstruction benchmarks; but it also shows strong performance for 3d shape completion and promising ability in making multiple plausible predictions.

Point cloud completion is a generation and estimation issue derived from the partial point clouds, which plays a vital role in the applications in 3D computer vision. The progress of deep learning (DL) has impressively improved the capability and robustness of point cloud completion. However, the quality of completed point clouds is still needed to be further enhanced to meet the practical utilization. Therefore, this work aims to conduct a comprehensive survey on various methods, including point-based, convolution-based, graph-based, and generative model-based approaches, etc. And this survey summarizes the comparisons among these methods to provoke further research insights. Besides, this review sums up the commonly used datasets and illustrates the applications of point cloud completion. Eventually, we also discussed possible research trends in this promptly expanding field.

Generative models, as an important family of statistical modeling, target learning the observed data distribution via generating new instances. Along with the rise of neural networks, deep generative models, such as variational autoencoders (VAEs) and generative adversarial network (GANs), have made tremendous progress in 2D image synthesis. Recently, researchers switch their attentions from the 2D space to the 3D space considering that 3D data better aligns with our physical world and hence enjoys great potential in practice. However, unlike a 2D image, which owns an efficient representation (i.e., pixel grid) by nature, representing 3D data could face far more challenges. Concretely, we would expect an ideal 3D representation to be capable enough to model shapes and appearances in details, and to be highly efficient so as to model high-resolution data with fast speed and low memory cost. However, existing 3D representations, such as point clouds, meshes, and recent neural fields, usually fail to meet the above requirements simultaneously. In this survey, we make a thorough review of the development of 3D generation, including 3D shape generation and 3D-aware image synthesis, from the perspectives of both algorithms and more importantly representations. We hope that our discussion could help the community track the evolution of this field and further spark some innovative ideas to advance this challenging task.

在本文中，我们从功能学习的角度解决了点云完成的具有挑战性的问题。我们的主要观察结果是，要恢复基础结构以及表面细节，给定部分输入，基本组件是一个很好的特征表示，可以同时捕获全球结构和局部几何细节。因此，我们首先提出了FSNET，这是一个功能结构模块，可以通过从本地区域学习多个潜在图案来适应汇总点的点功能。然后，我们将FSNET集成到粗线管道中，以完成点云完成。具体而言，采用2D卷积神经网络将特征图从FSNET解码为粗且完整的点云。接下来，使用一个点云UP抽样网络来从部分输入和粗糙的中间输出中生成密集的点云。为了有效利用局部结构并增强点分布均匀性，我们提出了IFNET，该点具有自校正机制的点提升模块，该模块可以逐步完善生成的密集点云的细节。我们已经在Shapenet，MVP和Kitti数据集上进行了定性和定量实验，这些实验表明我们的方法优于最先进的点云完成方法。

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset. Code, data and trained models are available at https://wentaoyuan.github.io/pcn.

最近的研究表明，MMWave雷达感测在低可见性环境中对象检测的有效性，这使其成为自主导航系统中的理想技术。在本文中，我们将雷达介绍给点云（R2P），这是一个深度学习模型，该模型基于具有不正确点的粗糙和稀疏点云，生成具有精细几何细节的3D对象的平滑，密集且高度准确的点云表示。来自mmwave雷达。这些输入点云是从由原始MMWave雷达传感器数据生成的2D深度图像转换的，其特征是不一致，方向和形状误差。 R2P利用两个顺序的深度学习编码器块的体系结构在从多个角度观察到对象的基于雷达的输入点云的基本特征，并确保生成的输出点云及其准确的内部一致性和原始对象的详细形状重建。我们实施R2P来替换我们最近提出的3DRIMR（通过MMWave Radar）系统的第2阶段。我们的实验证明了R2P在流行的现有方法（例如PointNet，PCN和原始3DRIMR设计）上的显着性能提高。

Intelligent mesh generation (IMG) refers to a technique to generate mesh by machine learning, which is a relatively new and promising research field. Within its short life span, IMG has greatly expanded the generalizability and practicality of mesh generation techniques and brought many breakthroughs and potential possibilities for mesh generation. However, there is a lack of surveys focusing on IMG methods covering recent works. In this paper, we are committed to a systematic and comprehensive survey describing the contemporary IMG landscape. Focusing on 110 preliminary IMG methods, we conducted an in-depth analysis and evaluation from multiple perspectives, including the core technique and application scope of the algorithm, agent learning goals, data types, targeting challenges, advantages and limitations. With the aim of literature collection and classification based on content extraction, we propose three different taxonomies from three views of key technique, output mesh unit element, and applicable input data types. Finally, we highlight some promising future research directions and challenges in IMG. To maximize the convenience of readers, a project page of IMG is provided at \url{https://github.com/xzb030/IMG_Survey}.

We propose an end-to-end deep learning architecture that produces a 3D shape in triangular mesh from a single color image. Limited by the nature of deep neural network, previous methods usually represent a 3D shape in volume or point cloud, and it is non-trivial to convert them to the more ready-to-use mesh model. Unlike the existing methods, our network represents 3D mesh in a graph-based convolutional neural network and produces correct geometry by progressively deforming an ellipsoid, leveraging perceptual features extracted from the input image. We adopt a coarse-to-fine strategy to make the whole deformation procedure stable, and define various of mesh related losses to capture properties of different levels to guarantee visually appealing and physically accurate 3D geometry. Extensive experiments show that our method not only qualitatively produces mesh model with better details, but also achieves higher 3D shape estimation accuracy compared to the state-of-the-art.

点云的语义场景重建是3D场景理解的必不可少的任务。此任务不仅需要识别场景中的每个实例，而且还需要根据部分观察到的点云恢复其几何形状。现有方法通常尝试基于基于检测的主链的不完整点云建议直接预测完整对象的占用值。但是，由于妨碍了各种检测到的假阳性对象建议以及对完整对象学习占用值的不完整点观察的歧义，因此该框架始终无法重建高保真网格。为了绕开障碍，我们提出了一个分离的实例网格重建（DIMR）框架，以了解有效的点场景。采用基于分割的主链来减少假阳性对象建议，这进一步使我们对识别与重建之间关系的探索有益。根据准确的建议，我们利用网状意识的潜在代码空间来解开形状完成和网格生成的过程，从而缓解了由不完整的点观测引起的歧义。此外，通过在测试时间访问CAD型号池，我们的模型也可以通过在没有额外训练的情况下执行网格检索来改善重建质量。我们用多个指标彻底评估了重建的网格质量，并证明了我们在具有挑战性的扫描仪数据集上的优越性。代码可在\ url {https://github.com/ashawkey/dimr}上获得。

精确地重建由单个图像的各种姿势和服装引起的精确复杂的人类几何形状非常具有挑战性。最近，基于像素对齐的隐式函数（PIFU）的作品已迈出了一步，并在基于图像的3D人数数字化上实现了最先进的保真度。但是，PIFU的培训在很大程度上取决于昂贵且有限的3D地面真相数据（即合成数据），从而阻碍了其对更多样化的现实世界图像的概括。在这项工作中，我们提出了一个名为selfpifu的端到端自我监督的网络，以利用丰富和多样化的野外图像，在对无约束的内部图像进行测试时，在很大程度上改善了重建。 SelfPifu的核心是深度引导的体积/表面感知的签名距离领域（SDF）学习，它可以自欺欺人地学习PIFU，而无需访问GT网格。整个框架由普通估计器，深度估计器和基于SDF的PIFU组成，并在训练过程中更好地利用了额外的深度GT。广泛的实验证明了我们自我监督框架的有效性以及使用深度作为输入的优越性。在合成数据上，与PIFUHD相比，我们的交叉点（IOU）达到93.5％，高18％。对于野外图像，我们对重建结果进行用户研究，与其他最先进的方法相比，我们的结果的选择率超过68％。

In recent years, substantial progress has been achieved in learning-based reconstruction of 3D objects. At the same time, generative models were proposed that can generate highly realistic images. However, despite this success in these closely related tasks, texture reconstruction of 3D objects has received little attention from the research community and state-of-the-art methods are either limited to comparably low resolution or constrained experimental setups. A major reason for these limitations is that common representations of texture are inefficient or hard to interface for modern deep learning techniques. In this paper, we propose Texture Fields, a novel texture representation which is based on regressing a continuous 3D function parameterized with a neural network. Our approach circumvents limiting factors like shape discretization and parameterization, as the proposed texture representation is independent of the shape representation of the 3D object. We show that Texture Fields are able to represent high frequency texture and naturally blend with modern deep learning techniques. Experimentally, we find that Texture Fields compare favorably to state-of-the-art methods for conditional texture reconstruction of 3D objects and enable learning of probabilistic generative models for texturing unseen 3D models. We believe that Texture Fields will become an important building block for the next generation of generative 3D models.

从单个2D图像推断3D位置和多个对象的形状是计算机视觉的长期目标。大多数现有的作品都预测这些3D属性之一或专注于解决单个对象。一个基本挑战在于如何学习适合3D检测和重建的图像的有效表示。在这项工作中，我们建议从输入图像中学习3D体素特征的常规网格，其通过3D特征升降操作员与3D场景空间对齐。基于3D体素特征，我们的新型中心-3D检测头在3D空间中配制了3D检测作为关键点检测。此外，我们设计了一种高效的粗致细重建模块，包括粗级体轴和新的本地PCA-SDF形状表示，其能够精细的细节重建和比现有方法更快地推理的阶数。通过3D检测和重建的互补监督，可以使3D体素特征成为几何和上下文保留，从而通过单个对象中的3D检测和重建来证明我们的方法的有效性和多个对象场景。

Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to represent and generate 3D shapes, as well as a vast number of use cases. However, single-view reconstruction remains a challenging topic that can unlock various interesting use cases such as interactive design. In this work, we propose a novel framework that leverages the intermediate latent spaces of Vision Transformer (ViT) and a joint image-text representational model, CLIP, for fast and efficient Single View Reconstruction (SVR). More specifically, we propose a novel mapping network architecture that learns a mapping between deep features extracted from ViT and CLIP, and the latent space of a base 3D generative model. Unlike previous work, our method enables view-agnostic reconstruction of 3D shapes, even in the presence of large occlusions. We use the ShapeNetV2 dataset and perform extensive experiments with comparisons to SOTA methods to demonstrate our method's effectiveness.