We introduce a simple permutation equivariant layer for deep learning with set structure. This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit summation, where in both cases the output is invariant to permutations of the input. In a semi-supervised setting, where the goal is make predictions for each instance within a set, we demonstrate the usefulness of this type of layer in set-outlier detection as well as semi-supervised learning with clustering side-information.
translated by 谷歌翻译
机器学习中最基本的问题之一是比较例子:给定一对对象,我们想要返回一个表示(dis)相似度的值。相似性通常是特定于任务的,并且预定义的距离可能表现不佳,从而导致在度量学习中工作。然而,能够学习相似性敏感距离函数也预先假定对于手头的对象的丰富的,有辨别力的表示。在本论文中,我们提出了两端的贡献。在论文的第一部分中,假设数据具有良好的表示,我们提出了一种用于度量学习的公式,与先前的工作相比,它更直接地尝试优化k-NN精度。我们还提出了这个公式的扩展,用于kNN回归的度量学习,不对称相似学习和汉明距离的判别学习。在第二部分中,我们考虑我们处于有限计算预算的情况,即在可能度量的空间上进行优化是不可行的,但是仍然需要访问标签感知距离度量。我们提出了一种简单,计算成本低廉的方法,用于估计仅依靠梯度估计,讨论理论和实验结果的良好动机。在最后一部分,我们讨论代表性问题,考虑组等变卷积神经网络(GCNN)。等效tosymmetry转换在GCNNs中明确编码;经典的CNN是最简单的例子。特别地,我们提出了一种用于球形数据的SO(3) - 等变神经网络架构,它完全在傅立叶空间中运行,同时也为完全傅立叶神经网络的设计提供了形式,这与任何连续紧凑组的动作是等效的。
translated by 谷歌翻译
在本文中,我们报告了我们对文本数据密集分布表示的研究结果。我们提出了两种新颖的神经模型来学习这种表征。第一个模型学习文档级别的表示,而第二个模型学习单词级表示。对于文档级表示,我们提出二进制段落向量:用于学习文本文档的二进制表示的神经网络模型,其可用于快速文档检索。我们对这些模型进行了全面评估,并证明它们在信息检索任务中的表现优于该领域的开创性方法。我们还报告了强有力的结果转换学习设置,其中我们的模型在通用textcorpus上训练,然后用于从特定于域的数据集推断文档的代码。与先前提出的方法相反,二进制段落矢量模型直接从原始文本数据学习嵌入。对于词级表示,我们提出消歧Skip-gram:用于学习多义词嵌入的神经网络模型。通过该模型学习的表示可以用于下游任务,例如词性标记或语义关系的识别。在单词意义上感应任务Disambiguated Skip-gram在三个基准测试数据集上优于最先进的模型。我们的模型具有优雅的概率解释。此外,与以前的这种模型不同,它在所有参数方面都是不同的,并且可以用反向传播进行训练。除了定量结果,我们还提出消除歧义的Skip-gram的定性评估,包括选定的词义嵌入的二维可视化。
translated by 谷歌翻译
点云是一种重要的几何数据结构。由于其规则的格式,大多数研究人员将这些数据转换为常规的3D体素网格或图像集合。然而,这会使数据不必要地变得多样化并导致问题。在本文中,我们设计了一种新型的神经网络,它直接消耗点云,并且很好地尊重输入中点的排列不变性。我们的网络名为PointNet,为从对象分类,部分分割到场景语义分析等应用程序提供了统一的体系结构。虽然简单,但PointNet非常高​​效和有效。从经验上看,它表现出比现有技术更好的表现。从理论上讲,我们提供分析,以了解网络学到了什么,以及为什么网络在输入扰动和腐败方面是健壮的。
translated by 谷歌翻译
A recent ''third wave'' of neural network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing. Because these modern NNs often comprise multiple interconnected layers, work in this area is often referred to as deep learning. Recent years have witnessed an explosive growth of research into NN-based approaches to information retrieval (IR). A significant body of work has now been created. In this paper, Kezban Dilek Onal and Ye Zhang contributed equally. Maarten de Rijke and Matthew Lease contributed equally. we survey the current landscape of Neural IR research, paying special attention to the use of learned distributed representations of textual units. We highlight the successes of neural IR thus far, catalog obstacles to its wider adoption, and suggest potentially promising directions for future research.
translated by 谷歌翻译
深度学习提出了希望和期望,作为许多应用程序的一般解决方案;事实证明它已被证明是有效的,但它也显示出对大量数据的强烈依赖性。幸运的是,已经证明,即使数据稀缺,也可以通过重复使用priorknowledge来训练成功的模型。因此,在最广泛的定义中,开发转移学习技术是部署有效和准确的智能系统的关键因素。本文将重点研究一系列适用于视觉目标识别任务的转移学习方法,特别是图像分类。转移学习是一个通用术语,并且特定设置已经给出了特定的名称:当学习者只能访问来自目标域的标记数据和来自不同域(源)的标记数据时,问题被称为“无监督域适应”。 (DA)。这项工作的第一部分将集中在这个设置的三种方法:其中一种方法涉及特征,一种是图像,而第三种方法同时使用两种。第二部分将重点关注机器人感知的现实生活问题,特别是RGB-D识别。机器人平台通常不仅限于色彩感知;他们经常带着Depthcamera。不幸的是,深度模态很少用于视觉识别,因为缺乏预先训练的模型,从中可以传输并且很少有数据从头开始。将提出两种处理这种情况的方法:一种使用合成数据,另一种利用跨模态转移学习。
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
声学数据提供从生物学和通信到海洋和地球科学等领域的科学和工程见解。我们调查了机器学习(ML)的进步和变革潜力,包括声学领域的深度学习。 ML是用于自动检测和利用模式印度的广泛的统计技术家族。相对于传统的声学和信号处理,ML是数据驱动的。给定足够的训练数据,ML可以发现特征之间的复杂关系。通过大量的训练数据,ML candiscover模型描述复杂的声学现象,如人类语音和混响。声学中的ML正在迅速发展,具有令人瞩目的成果和未来的重大前景。我们首先介绍ML,然后在五个声学研究领域强调MLdevelopments:语音处理中的源定位,海洋声学中的源定位,生物声学,地震探测和日常场景中的环境声音。
translated by 谷歌翻译
对输入对象进行排序是许多机器学习管道中的重要步骤。然而,排序运算符在输入方面是不可微分的,这禁止基于端到端梯度的优化。在这项工作中,我们提出NeuralSort,一种通用的连续松弛排序运算符的输出,从置换矩阵到一组单模 - 随机矩阵,其中每行总和为1并具有不同的argmax。这种放松允许任何涉及分类操作的计算图的直接优化。此外,我们使用这种弛豫,通过在排列上导出分布的派拉克 - 卢斯分布族的重新参数化梯度估计,在组合大的空间上实现基于梯度的随机优化。我们在需要学习高维对象的语义排序的三个任务中证明了我们的框架的有用性,包括k-最近邻算法的完全可微分的参数化扩展。
translated by 谷歌翻译
在过去几年中,神经网络重新成为强大的机器学习模型,在图像识别和语音处理等领域产生了最先进的结果。最近,神经网络模型开始应用于文本自然语言信号,同样具有非常有希望的结果。本教程从自然语言处理研究的角度对神经网络模型进行了调查,试图通过神经技术使自然语言研究人员加快速度。本教程介绍了自然语言任务,前馈网络,卷积网络,循环网络和递归网络的输入编码,以及自动梯度计算的计算图形抽象。
translated by 谷歌翻译
最近关于集合上函数表示的工作考虑了在潜在空间中使用求和来强制置换不变性。特别是,有人推测,随着所考虑的集合的基数增加,这个潜在空间的维度可能会保持不变。但是,我们证明导致这种猜想的分析需要高度不连续的映射,并认为这仅仅是有限的实际用途。 。在这种观察的启发下,我们证明了通过连续映射(例如由神经网络或高斯过程提供)实现该模型实际上对该空间的维度施加了约束。用于设定输入的实际通用函数表示通常至少具有输入元素的最大数量的大小的潜在维度来实现。
translated by 谷歌翻译
In 2007, Labeled Faces in the Wild was released in an effort to spur research in face recognition, specifically for the problem of face verification with un-constrained images. Since that time, more than 50 papers have been published that improve upon this benchmark in some respect. A remarkably wide variety of innovative methods have been developed to overcome the challenges presented in this database. As performance on some aspects of the benchmark approaches 100% accuracy , it seems appropriate to review this progress, derive what general principles we can from these works, and identify key future challenges in face recognition. In this survey, we review the contributions to LFW for which the authors have provided results to the curators (results found on the LFW results web page). We also review the cross cutting topic of alignment and how it is used in various methods. We end with a brief discussion of recent databases designed to challenge the next generation of face recognition algorithms.
translated by 谷歌翻译
大图中节点的低维嵌入已被证明在从内容推荐到识别蛋白质功能的各种预测任务中极其有用。然而,大多数现有方法要求在嵌入训练期间存在图中的所有节点;这些以前的方法本质上是转换性的,并不自然地普遍认为看不见的节点。在这里,我们提出GraphSAGE,一种通用的归纳框架,它利用节点特征信息(例如,文本属性)来有效地为先前看不见的数据生成节点嵌入。我们学习了一种函数,通过对节点的localneighborhood中的特征进行采样和聚合来生成嵌入,而不是为每个节点进行单独的嵌入。我们的算法在三个归纳节点分类基准上优于强基线:我们根据引用和Reddit后期数据对信息图中看不见的节点类别进行分类,并且我们展示了我们的算法使用蛋白质 - 蛋白质相互作用的多图形数据集推广到完全看不见的图形。 。
translated by 谷歌翻译
新颖性检测是识别与训练集明显不同的数据异常的无监督问题。新颖性检测是机器学习中的经典挑战之一,也是欺诈检测,入侵检测,医疗诊断,数据清理和故障预防等几个研究领域的核心组成部分。虽然设计了许多算法来解决这个问题,但大多数方法仅适用于模拟连续数值数据。处理由混合类型特征(例如数值和分类数据)或描述离散事件序列的时间数据集组成的数据集是一项具有挑战性的任务。除了支持的数据类型之外,有效新颖性检测方法的关键标准是能够准确地将新颖性与标称样本分离,可解释性,可扩展性以及对位于训练数据中的异常的鲁棒性。在本文中,我们研究了解决这些问题的新方法。特别地,我们提出(i)混合型数据的新颖性检测方法的实验比较(ii)序列数据的新颖检测方法的实验比较,(iii)基于Dirichlet过程混合和指数的混合型数据的概率非参数奇异检测方法。 - 家庭分布和(iv)基于自动编码器的新奇检测模型,其编码器/解码器被建模为深度高斯过程。
translated by 谷歌翻译
We introduce a very general method for high dimensional classification, based on careful combination of the results of applying an arbitrary base classifier to random projections of the feature vectors into a lower dimensional space. In one special case that we study in detail, the random projections are divided into disjoint groups, and within each group we select the projection yielding the smallest estimate of the test error. Our random-projection ensemble classifier then aggregates the results of applying the base classifier on the selected projections, with a data-driven voting threshold to determine the final assignment. Our theoretical results elucidate the effect on performance of increasing the number of projections. Moreover, under a boundary condition that is implied by the sufficient dimension reduction assumption, we show that the test excess risk of the random-projection ensemble classifier can be controlled by terms that do not depend on the original data dimension and a term that becomes negligible as the number of projections increases. The classifier is also compared empirically with several other popular high dimensional classifiers via an extensive simulation study, which reveals its excellent finite sample performance.
translated by 谷歌翻译
计算机图形学,3D计算机视觉和机器人社区已经产生了多种方法来表示用于渲染和重建的3D几何。这些提供了保真度,效率和压缩功能之间的权衡。在这项工作中,我们介绍了DeepSDF,一种学习的连续符号距离函数(SDF)表示的一类形状,可以实现高质量的形状表示,插值和完成部分和有噪声的3D输入数据。与经典对应物一样,DeepSDF通过连续的体积场表示形状的表面:场中点的大小表示到表面边界的距离,而标志表示该区域是否在形状的内部( - )或外部(+)因此,我们的表示隐式地将形状的边界编码为学习函数的零级集合,同时明确地将空间的分类表示为内部形状的一部分。虽然经典SDF在分析或离散体素形式中通常表示单个形状的表面,但DeepSDF可以表示整个形状类别。此外,我们展示了学习3D形状表示和完成的最先进性能,同时与之前的工作相比将模型尺寸减小了一个数量级。
translated by 谷歌翻译
近年来,复杂文档和文本的数量呈指数增长,需要更深入地了解机器学习方法,才能在许多应用程序中准确地对文本进行分类。许多机器学习方法在自然语言处理方面取得了超越的成果。这些学习算法的成功依赖于它们能够理解数据中的复杂模型和非线性关系。然而,为文本分类找到合适的结构,体系结构和技术对研究人员来说是一个挑战。在本文中,讨论了文本分类算法的简要概述。本概述涵盖了不同的文本特征提取,降维方法,现有算法和技术以及评估方法。最后,讨论了每种技术的局限性及其在现实问题中的应用。
translated by 谷歌翻译