We introduce a simple permutation equivariant layer for deep learning with set structure. This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit summation, where in both cases the output is invariant to permutations of the input. In a semi-supervised setting, where the goal is make predictions for each instance within a set, we demonstrate the usefulness of this type of layer in set-outlier detection as well as semi-supervised learning with clustering side-information.
translated by 谷歌翻译
机器学习中最基本的问题之一是比较例子:给定一对对象,我们想要返回一个表示(dis)相似度的值。相似性通常是特定于任务的,并且预定义的距离可能表现不佳,从而导致在度量学习中工作。然而,能够学习相似性敏感距离函数也预先假定对于手头的对象的丰富的,有辨别力的表示。在本论文中,我们提出了两端的贡献。在论文的第一部分中,假设数据具有良好的表示,我们提出了一种用于度量学习的公式,与先前的工作相比,它更直接地尝试优化k-NN精度。我们还提出了这个公式的扩展,用于kNN回归的度量学习,不对称相似学习和汉明距离的判别学习。在第二部分中,我们考虑我们处于有限计算预算的情况,即在可能度量的空间上进行优化是不可行的,但是仍然需要访问标签感知距离度量。我们提出了一种简单,计算成本低廉的方法,用于估计仅依靠梯度估计,讨论理论和实验结果的良好动机。在最后一部分,我们讨论代表性问题,考虑组等变卷积神经网络(GCNN)。等效tosymmetry转换在GCNNs中明确编码;经典的CNN是最简单的例子。特别地,我们提出了一种用于球形数据的SO(3) - 等变神经网络架构,它完全在傅立叶空间中运行,同时也为完全傅立叶神经网络的设计提供了形式,这与任何连续紧凑组的动作是等效的。
translated by 谷歌翻译
在本文中,我们报告了我们对文本数据密集分布表示的研究结果。我们提出了两种新颖的神经模型来学习这种表征。第一个模型学习文档级别的表示,而第二个模型学习单词级表示。对于文档级表示,我们提出二进制段落向量:用于学习文本文档的二进制表示的神经网络模型,其可用于快速文档检索。我们对这些模型进行了全面评估,并证明它们在信息检索任务中的表现优于该领域的开创性方法。我们还报告了强有力的结果转换学习设置,其中我们的模型在通用textcorpus上训练,然后用于从特定于域的数据集推断文档的代码。与先前提出的方法相反,二进制段落矢量模型直接从原始文本数据学习嵌入。对于词级表示,我们提出消歧Skip-gram:用于学习多义词嵌入的神经网络模型。通过该模型学习的表示可以用于下游任务,例如词性标记或语义关系的识别。在单词意义上感应任务Disambiguated Skip-gram在三个基准测试数据集上优于最先进的模型。我们的模型具有优雅的概率解释。此外,与以前的这种模型不同,它在所有参数方面都是不同的,并且可以用反向传播进行训练。除了定量结果,我们还提出消除歧义的Skip-gram的定性评估,包括选定的词义嵌入的二维可视化。
translated by 谷歌翻译
点云是一种重要的几何数据结构。由于其规则的格式,大多数研究人员将这些数据转换为常规的3D体素网格或图像集合。然而,这会使数据不必要地变得多样化并导致问题。在本文中,我们设计了一种新型的神经网络,它直接消耗点云,并且很好地尊重输入中点的排列不变性。我们的网络名为PointNet,为从对象分类,部分分割到场景语义分析等应用程序提供了统一的体系结构。虽然简单,但PointNet非常高​​效和有效。从经验上看,它表现出比现有技术更好的表现。从理论上讲,我们提供分析,以了解网络学到了什么,以及为什么网络在输入扰动和腐败方面是健壮的。
translated by 谷歌翻译
A recent ''third wave'' of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this paper, Kezban Dilek Onal and Ye Zhang contributed equally. Maarten de Rijke and Matthew Lease contributed equally. we survey the current landscape of Neural IR research, paying special attention to the use of learned distributed representations of textual units. We highlight the successes of neural IR thus far, catalog obstacles to its wider adoption, and suggest potentially promising directions for future research.
translated by 谷歌翻译
深度学习提出了希望和期望,作为许多应用程序的一般解决方案;事实证明它已被证明是有效的,但它也显示出对大量数据的强烈依赖性。幸运的是,已经证明,即使数据稀缺,也可以通过重复使用priorknowledge来训练成功的模型。因此,在最广泛的定义中,开发转移学习技术是部署有效和准确的智能系统的关键因素。本文将重点研究一系列适用于视觉目标识别任务的转移学习方法,特别是图像分类。转移学习是一个通用术语,并且特定设置已经给出了特定的名称:当学习者只能访问来自目标域的标记数据和来自不同域(源)的标记数据时,问题被称为“无监督域适应”。 (DA)。这项工作的第一部分将集中在这个设置的三种方法:其中一种方法涉及特征,一种是图像,而第三种方法同时使用两种。第二部分将重点关注机器人感知的现实生活问题,特别是RGB-D识别。机器人平台通常不仅限于色彩感知;他们经常带着Depthcamera。不幸的是,深度模态很少用于视觉识别,因为缺乏预先训练的模型,从中可以传输并且很少有数据从头开始。将提出两种处理这种情况的方法:一种使用合成数据,另一种利用跨模态转移学习。
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
在过去几年中,神经网络重新成为强大的机器学习模型,在图像识别和语音处理等领域产生了最先进的结果。最近,神经网络模型开始应用于文本自然语言信号,同样具有非常有希望的结果。本教程从自然语言处理研究的角度对神经网络模型进行了调查,试图通过神经技术使自然语言研究人员加快速度。本教程介绍了自然语言任务,前馈网络,卷积网络,循环网络和递归网络的输入编码,以及自动梯度计算的计算图形抽象。
translated by 谷歌翻译
无监督学习的核心目标是从未标记的数据或经验中获取表示,这些表示可用于从适量的标记数据中更有效地学习下游任务。许多先前的无监督学习工作旨在通过基于重构,解开,预测和其他指标开发代理目标来实现这一目标。相反,我们开发了一种无监督的学习方法,该方法明确优化了从少量数据中学习各种任务的能力。为此,我们以自动方式从未标记数据构建任务,并对构建的任务运行元学习。令人惊讶的是,我们发现,当与学习相结合时,相对简单的任务设计机制,例如群集无监督表示,可以在各种下游任务中获得良好的性能。我们对四个图像数据集的实验表明,我们的无监督元学习方法获得了一种学习算法,没有任何适用于各种下游分类任务的标记数据,改进了四种先前无监督学习方法所学习的表示。
translated by 谷歌翻译
最近关于集合上函数表示的工作考虑了在潜在空间中使用求和来强制置换不变性。特别是,有人推测,随着所考虑的集合的基数增加,这个潜在空间的维度可能会保持不变。但是,我们证明导致这种猜想的分析需要高度不连续的映射,并认为这仅仅是有限的实际用途。 。在这种观察的启发下,我们证明了通过连续映射(例如由神经网络或高斯过程提供)实现该模型实际上对该空间的维度施加了约束。用于设定输入的实际通用函数表示通常至少具有输入元素的最大数量的大小的潜在维度来实现。
translated by 谷歌翻译
三维几何数据为研究代表性学习和生成建模提供了一个很好的领域。在本文中,我们将看到表示为点云的几何数据。我们引入了深度自动编码器(AE)网络,具有最先进的重建质量和可扩展性。学习的表示优于现有的3D识别任务方法,并通过简单的代数操作实现形状编辑,例如语义部分编辑,形状类比和形状插值,以及形状完成。我们对不同的生成模型进行了彻底的研究,包括在原始点云上运行的GAN,在我们的AE的固定潜在空间中显着改进的GAN应变,以及高斯混合模型(GMM)。为了定量评估生成模型,我们引入了基于点云集之间匹配的样本保真度和多样性的度量。有趣的是,我们对泛化,保真度和多样性的评估表明,在我们的AE的潜在空间中训练的GMM产生最好的结果。
translated by 谷歌翻译
In 2007, Labeled Faces in the Wild was released in an effort to spur research in face recognition, specifically for the problem of face verification with un-constrained images. Since that time, more than 50 papers have been published that improve upon this benchmark in some respect. A remarkably wide variety of innovative methods have been developed to overcome the challenges presented in this database. As performance on some aspects of the benchmark approaches 100% accuracy , it seems appropriate to review this progress, derive what general principles we can from these works, and identify key future challenges in face recognition. In this survey, we review the contributions to LFW for which the authors have provided results to the curators (results found on the LFW results web page). We also review the cross cutting topic of alignment and how it is used in various methods. We end with a brief discussion of recent databases designed to challenge the next generation of face recognition algorithms.
translated by 谷歌翻译
计算机图形学,3D计算机视觉和机器人社区已经产生了多种方法来表示用于渲染和重建的3D几何。这些提供了保真度,效率和压缩功能之间的权衡。在这项工作中,我们介绍了DeepSDF,一种学习的连续符号距离函数(SDF)表示的一类形状,可以实现高质量的形状表示,插值和完成部分和有噪声的3D输入数据。与经典对应物一样,DeepSDF通过连续的体积场表示形状的表面:场中点的大小表示到表面边界的距离,而标志表示该区域是否在形状的内部( - )或外部(+)因此,我们的表示隐式地将形状的边界编码为学习函数的零级集合,同时明确地将空间的分类表示为内部形状的一部分。虽然经典SDF在分析或离散体素形式中通常表示单个形状的表面,但DeepSDF可以表示整个形状类别。此外,我们展示了学习3D形状表示和完成的最先进性能,同时与之前的工作相比将模型尺寸减小了一个数量级。
translated by 谷歌翻译
许多科学领域研究具有非欧几里德空间的基础结构的数据。一些例子包括计算社会科学中的社交网络,通信中的传感器网络,脑内成像的功能网络,遗传学中的调节网络以及计算机图形中的网状表面。在许多应用中,这样的几何数据是大而复杂的(在社交网络的情况下,在数十亿的规模上),并且是机器学习技术的自然目标。特别是,我们希望使用深度神经网络,这种网络最近被证明是解决计算机视觉,自然语言处理和音频分析等广泛问题的强大工具。然而,这些工具在具有基于欧几里得或网格状结构的数据上是最成功的,并且在这些结构的不变性被构建到用于对其进行建模的网络中的情况下。几何深度学习是新兴技术试图将(结构化的)深层神经模型推广到非欧几里德域(如图和流形)的总称。本文的目的是概述不同的几何深度学习问题的例子,并提供这个新生领域的可用解决方案,关键难点,应用和未来的研究方向。
translated by 谷歌翻译