深度学习在多个领域取得了显着的性能突破,尤其是在语音识别,自然语言处理和计算机视觉方面。特别地,卷积神经网络(CNN)架构目前在诸如对象检测和识别的各种图像分析任务上产生最先进的性能。到目前为止,大多数深度学习研究都集中在处理一维,二维或三维核素结构数据,如声学信号,图像或视频。最近,人们越来越关注几何深度学习,试图将深度学习方法推广到非欧几里德结构数据,如图形和流形,具有网络分析,计算社会科学或计算机图形领域的各种应用。在本文中,我们提出了一个统一的框架,允许将CNN架构推广到非欧几里德域(图和流形),并学习本地,静态和组合任务特定的功能。我们表明,文献中先前提出的各种非欧几里德CNN方法可以被认为是我们框架的特定实例。我们从图像,图形和三维形状分析领域测试了标准任务的建议方法,并表明它始终优于以前的方法。
translated by 谷歌翻译
许多科学领域研究具有非欧几里德空间的基础结构的数据。一些例子包括计算社会科学中的社交网络,通信中的传感器网络,脑内成像的功能网络,遗传学中的调节网络以及计算机图形中的网状表面。在许多应用中,这样的几何数据是大而复杂的(在社交网络的情况下,在数十亿的规模上),并且是机器学习技术的自然目标。特别是,我们希望使用深度神经网络,这种网络最近被证明是解决计算机视觉,自然语言处理和音频分析等广泛问题的强大工具。然而,这些工具在具有基于欧几里得或网格状结构的数据上是最成功的,并且在这些结构的不变性被构建到用于对其进行建模的网络中的情况下。几何深度学习是新兴技术试图将(结构化的)深层神经模型推广到非欧几里德域(如图和流形)的总称。本文的目的是概述不同的几何深度学习问题的例子,并提供这个新生领域的可用解决方案,关键难点,应用和未来的研究方向。
translated by 谷歌翻译
我们提出了一种在曲面上进行信号卷积的新方法,并展示了它在各种几何深度学习应用中的实用性。我们构造的关键是在表面上定义的定向函数的概念,它扩展了经典的实值信号,并且可以与实值模板函数自然地卷积。因此,我们不是试图修正规范方向或仅在表面的每个点上保持2D模板的所有对齐的最大响应,如在先前的工作中所做的那样,我们展示了如何跨越神经的不同层保持跨旋转的信息。网络。我们称之为多向测地卷积,简称或定向卷积的我们的构造特别允许跨越层传播和关联方向信息,从而在形状上传播不同的区域。我们首先在连续设置中定义定向卷积,证明关键属性,然后展示如何在实践中实现它,表示为三角形网格的forshapes。我们在各种学习场景中评估定向卷积,包括表面信号分类,形状分割和形状匹配,我们在几个基线上显示出显着的改进。
translated by 谷歌翻译
深度学习是图像分类大幅改进的基础。为了提高预测的稳健性,贝叶斯近似已被用于学习深度神经网络中的参数。我们采用另一种方法,通过使用高斯过程作为贝叶斯深度学习模型的构建模块,由于卷积和深层结构的推断,这种模型最近变得可行。我们研究了深度卷积高斯过程,并确定了一个保持逆流性能的问题。为了解决这个问题,我们引入了一个转换敏感卷积内核,它消除了对相同补丁输入的要求相同输出的限制。我们凭经验证明,这种卷积核可以改善浅层和深层模型的性能。在ONMNIST,FASHION-MNIST和CIFAR-10上,我们在准确性方面改进了以前的GP模型,增加了更简单的DNN模型的校准预测概率。
translated by 谷歌翻译
We present a practical way of introducing convolutional structure intoGaussian processes, making them more suited to high-dimensional inputs likeimages. The main contribution of our work is the construction of aninter-domain inducing point approximation that is well-tailored to theconvolutional kernel. This allows us to gain the generalisation benefit of aconvolutional kernel, together with fast but accurate posterior inference. Weinvestigate several variations of the convolutional kernel, and apply it toMNIST and CIFAR-10, which have both been known to be challenging for Gaussianprocesses. We also show how the marginal likelihood can be used to find anoptimal weighting between convolutional and RBF kernels to further improveperformance. We hope that this illustration of the usefulness of a marginallikelihood will help automate discovering architectures in larger models.
translated by 谷歌翻译
We present Spline-based Convolutional Neural Networks (SplineCNNs), a variant of deep neural networks for irregular structured and geometric input, e.g., graphs or meshes. Our main contribution is a novel convolution operator based on B-splines, that makes the computation time independent from the kernel size due to the local support property of the B-spline basis functions. As a result, we obtain a generalization of the traditional CNN convolution operator by using continuous kernel functions parametrized by a fixed number of trainable weights. In contrast to related approaches that filter in the spectral domain, the proposed method aggregates features purely in the spatial domain. In addition, SplineCNN allows entire end-to-end training of deep architectures, using only the geometric structure as input, instead of handcrafted feature descriptors. For validation, we apply our method on tasks from the fields of image graph classification, shape correspondence and graph node classification, and show that it outperforms or pars state-of-the-art approaches while being significantly faster and having favorable properties like domain-independence. Our source code is available on GitHub 1 .
translated by 谷歌翻译
建立形状之间的对应关系是几何学处理中的基本问题,在各种各样的应用中产生。问题在于非等长变形的设置以及拓扑噪声和缺失部分的存在尤其困难,这主要是由于在公理上对这种变形进行建模的有限能力。最近的一些工作表明,可以从实例中学习复杂形状变换的不变性。在本文中,我们介绍了一种基于各向异性扩散核的内在卷积神经网络结构,我们称之为各向异性卷积神经网络(ACNN)。在我们的构造中,我们通过构造一组定向的各向异性扩散核,将卷积推广到非欧几里德域,以这种方式创建数据的局部特征极性表示(“补丁”),然后与过滤器相关联。这些滤波器,线性和非线性算子的若干级联被堆叠以形成深度神经网络,其参数通过最小化任务特定成本来学习。我们使用ACNN在非常具有挑战性的环境中有效地学习可变形状之间的内在对应关系,在一些最困难的最近对应基准上实现最先进的结果。
translated by 谷歌翻译
特征描述符在各种几何分析和处理应用中起着至关重要的作用,包括形状对应,检索和分割。在本文中,我们介绍了测地卷积神经网络(GCNN),卷积网络(CNN)范式的泛化到非欧几里德流形。我们的构造基于极坐标的局部测地系统来提取“斑块”,然后通过级联滤波器和线性和非线性算子传递。滤波器和线性组合权重的系数是优化变量,其被学习以最小化任务特定的成本函数。我们使用GCNN来学习各种形状特征,从而可以在形状描述,检索和对应等问题中实现最先进的性能。
translated by 谷歌翻译
Spectral methods for mesh processing and analysis rely on the eigenvalues, eigenvectors, or eigenspace projections derived from appropriately defined mesh operators to carry out desired tasks. Early work in this area can be traced back to the seminal paper by Taubin in 1995, where spectral analysis of mesh geometry based on a combinatorial Laplacian aids our understanding of the low-pass filtering approach to mesh smoothing. Over the past fifteen years, the list of applications in the area of geometry processing which utilize the eigenstructures of a variety of mesh operators in different manners have been growing steadily. Many works presented so far draw parallels from developments in fields such as graph theory, computer vision, machine learning, graph drawing, numerical linear algebra, and high-performance computing. This paper aims to provide a comprehensive survey on the spectral approach, focusing on its power and versatility in solving geometry processing problems and attempting to bridge the gap between relevant research in computer graphics and other fields. Necessary theoretical background is provided. Existing works covered are classified according to different criteria: the operators or eigenstructures employed, application domains, or the dimensionality of the spectral embeddings used. Despite much empirical success, there still remain many open questions pertaining to the spectral approach. These are discussed as we conclude the survey and provide our perspective on possible future research.
translated by 谷歌翻译
Convolutional neural networks (CNNs) have massively impacted visual recognition in 2D images, and are now ubiquitous in state-of-the-art approaches. CNNs do not easily extend, however, to data that are not represented by regular grids, such as 3D shape meshes or other graph-structured data, to which traditional local convolution operators do not directly apply. To address this problem, we propose a novel graph-convolution operator to establish correspondences between filter weights and graph neighborhoods with arbitrary connectivity. The key novelty of our approach is that these correspondences are dynamically computed from features learned by the network, rather than relying on predefined static coordinates over the graph as in previous work. We obtain excellent experimental results that significantly improve over previous state-of-the-art shape correspondence results. This shows that our approach can learn effective shape representations from raw input coordinates, without relying on shape descriptors.
translated by 谷歌翻译
Spectral methods have recently gained popularity in many domains of computer graphics and geometry processing, especially shape processing, computation of shape descriptors, distances, and correspondence. Spectral geometric structures are intrinsic and thus invariant to isometric deformations, are efficiently computed, and can be constructed on shapes in different representations. A notable drawback of these constructions, however, is that they are isotropic, i.e., insensitive to direction. In this paper, we show how to construct direction-sensitive spectral feature descriptors using anisotropic diffusion on meshes and point clouds. The core of our construction are directed local kernels acting similarly to steerable filters, which are learned in a task-specific manner. Remarkably, while being intrinsic, our descriptors allow to disambiguate reflection symmetries. We show the application of anisotropic descriptors for problems of shape correspondence on meshes and point clouds, achieving results significantly better than state-of-the-art methods.
translated by 谷歌翻译
3D几何数据的生成模型出现在3D计算机视觉和图形的许多重要应用中。在本文中,我们关注具有共同拓扑结构的3D变形形状,例如人脸和人体。 Morphable Models是最初为这种形状创建紧凑表示的尝试之一;尽管这些模型具有有效性和简单性,但由于它们的线性公式,它们具有有限的表示能力。近来,已经提出了非线性可学习方法,尽管它们大多数采用中间表示,例如体素或2Dview的3D网格。在本文中,我们引入了卷积网格自动编码器和基于螺旋卷积算子的GAN架构,直接作用于主题并利用其基础几何结构。我们对卷积算子进行了分析,并与线性可变模型和最近提出的COMA模型相比,展示了3D形状数据集的最新结果。
translated by 谷歌翻译
我们将平面和规则域(例如2D图像)上的卷积神经网络(CNN)扩展到嵌入3D欧几里德空间中的曲面,这些曲面被分离为不规则网格,并广泛用于表示计算机视觉和图形中的几何数据。我们在表面域的切线空间上定义表面卷积,其中卷积具有两个期望的属性:1)表面域信号的偏差在被投影到切线空间时局部最小,以及2)平移等方差属性通过对齐而局部保持切线空间与规范的平行传输,保留度量。为了计算,我们依赖于表面上的并行N方向帧场,其最小化场变化,因此尽可能与并行传输兼容并近似。在装有平行框架的切线空间上,表面卷积的计算成为标准的例程。框架具有旋转对称性,我们通过构造由平行框架引起的表面覆盖空间并将特征图分组为N组来消除歧义;在覆盖空间的N个分支上使用相应的特征映射计算卷积,同时共享核心权重。为了在共享内核权重的同时处理离散网格的不规则点,我们使卷积半离散,即卷积内核是多项式函数,并且它们与离散表面点的卷积变为采样和加权求和。通过简化构建的网格层次计算池化和解池操作。所呈现的表面CNN允许对网格进行有效的深度学习。我们展示了对于分类,分割和非刚性配准的任务,仅使用原始输入信号的表面CNN比使用复杂输入特征的先前模型实现了优越的性能。
translated by 谷歌翻译
In this work, we present intrinsic shape context (ISC) descriptors for 3D shapes. We generalize to surfaces the polar sampling of the image domain used in shape contexts: for this purpose, we chart the surface by shooting geodesic outwards from the point being analyzed; 'angle' is treated as tantamount to geodesic shooting direction, and radius as geodesic distance. To deal with orientation ambiguity, we exploit properties of the Fourier transform. Our charting method is intrinsic, i.e., invariant to isometric shape transformations. The resulting descriptor is a meta-descriptor that can be applied to any photometric or geometric property field defined on the shape, in particular, we can leverage recent developments in intrinsic shape analysis and construct ISC based on state-of-the-art dense shape descriptors such as heat kernel signatures. Our experiments demonstrate a notable improvement in shape matching on standard benchmarks.
translated by 谷歌翻译
Meshes have become a widespread and popular representation of models in computer graphics. Morphing techniques aim at transforming a given source shape into a target shape. Morphing techniques have various applications ranging from special effects in television and movies to medical imaging and scientific visualization. Not surprisingly, morphing techniques for meshes have received a lot of interest lately. Thiswork sums up recent developments in the area of mesh morphing. It presents a consistent framework to classify and compare various techniques approaching the same underlying problems from different angles.
translated by 谷歌翻译
研究界已经观察到卷积神经网络(CNN)在视觉识别任务中取得了巨大成功。然而,这种强大的CNN不能很好地扩展到任意形状的主折叠域。因此,如果有的话,在任意流形上定义的许多视觉识别问题仍然不能从CNN的成功中获益很多。阻碍CNN普遍化的技术困难的根源在于缺乏规范的网格式表示,一致的方向,以及跨域的兼容本地拓扑。不幸的是,除了一些开创性的作品外,在这方面只有很少的研究。为此,在本文中,我们提出了一种新的数学公式,将CNN扩展到二维(2D)流形域。更具体地,我们使用正交基函数(称为Zernike多项式)在局部的空间上近似在amanifold上定义的张量场。我们证明了两个函数的卷积可以表示为Zernike多项式系数之间的简单点积。我们还证明了卷积核的旋转等于应用于Zernike多项式系数的2乘2旋转矩阵,这在多流域中是关键的。因此,这项工作的关键贡献在于对CNN构件的完善和严格的数学推广。此外,与其他最先进的方法相比,我们的方法在分类和回归任务上都表现出明显更好的表现。
translated by 谷歌翻译
我们提出了深度卷积高斯过程,一种具有卷积结构的深度高斯过程体系结构。该模型是原型贝叶斯框架,用于检测图像分类的局部特征的分层组合。与MNIST和CIFAR-10数据集上的当前高斯过程方法相比,我们展示了大大改进的图像分类性能。特别是,我们将CIFAR-10精度提高了10个百分点以上。
translated by 谷歌翻译
在这项工作中,我们感兴趣的是将卷积神经网络(CNN)从低维规则网格(其中包含图像,视频和语音)推广到高维不规则域,例如社交网络,脑连接词或单词'嵌入,表示通过图表。我们在谱图理论的背景下提出了CNN的形式,它提供了必要的数学背景和有效的数值方案,以在图上设计快速局部卷积滤波器。重要的是,所提出的技术提供了与经典CNN相同的线性计算复杂度和恒定学习复杂度,同时对任何图形结构都是通用的。对MNIST和20NEWS的实验证明了这种新颖的学习系统能够学习图形上的局部,静止和组合特征。
translated by 谷歌翻译
This paper introduces a novel spectral framework for solving Markov decision processes (MDPs) by jointly learning representations and optimal policies. The major components of the framework described in this paper include: (i) A general scheme for constructing representations or basis functions by diagonalizing symmetric diffusion operators (ii) A specific instantiation of this approach where global basis functions called proto-value functions (PVFs) are formed using the eigenvectors of the graph Laplacian on an undirected graph formed from state transitions induced by the MDP (iii) A three-phased procedure called representation policy iteration comprising of a sample collection phase, a representation learning phase that constructs basis functions from samples, and a final parameter estimation phase that determines an (approximately) optimal policy within the (linear) subspace spanned by the (current) basis functions. (iv) A specific instantiation of the RPI framework using least-squares policy iteration (LSPI) as the parameter estimation method (v) Several strategies for scaling the proposed approach to large discrete and continuous state spaces, including the Nyström extension for out-of-sample interpolation of eigenfunctions, and the use of Kronecker sum factorization to construct compact eigenfunctions in product spaces such as factored MDPs (vi) Finally, a series of illustrative discrete and continuous control tasks, which both illustrate the concepts and provide a benchmark for evaluating the proposed approach. Many challenges remain to be addressed in scaling the proposed framework to large MDPs, and several elaboration of the proposed framework are briefly summarized at the end.
translated by 谷歌翻译