到目前为止,深度学习和深层体系结构正在成为许多实际应用中最好的机器学习方法,例如降低数据的维度,图像分类,语音识别或对象分割。事实上,许多领先的技术公司,如谷歌,微软或IBM,正在研究和使用他们系统中的深层架构来取代其他传统模型。因此,提高这些模型的性能可以在机器学习领域产生强烈的影响。然而,深度学习是一个快速发展的研究领域,在过去几年中发现了许多核心方法和范例。本文将首先作为深度学习的简短总结,试图包括本研究领域中所有最重要的思想。基于这一知识,我们提出并进行了一些实验,以研究基于自动编程(ADATE)改进深度学习的可能性。尽管我们的实验确实产生了良好的结果,但由于时间有限以及当前ADATE版本的局限性,我们还有更多的可能性无法尝试。我希望这篇论文可以促进关于这个主题的未来工作,特别是在ADATE的下一个版本中。本文还简要分析了ADATEsystem的功能,这对于想要了解其功能的其他研究人员非常有用。
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
机器学习中最基本的问题之一是比较例子:给定一对对象,我们想要返回一个表示(dis)相似度的值。相似性通常是特定于任务的,并且预定义的距离可能表现不佳,从而导致在度量学习中工作。然而,能够学习相似性敏感距离函数也预先假定对于手头的对象的丰富的,有辨别力的表示。在本论文中,我们提出了两端的贡献。在论文的第一部分中,假设数据具有良好的表示,我们提出了一种用于度量学习的公式,与先前的工作相比,它更直接地尝试优化k-NN精度。我们还提出了这个公式的扩展,用于kNN回归的度量学习,不对称相似学习和汉明距离的判别学习。在第二部分中,我们考虑我们处于有限计算预算的情况,即在可能度量的空间上进行优化是不可行的,但是仍然需要访问标签感知距离度量。我们提出了一种简单,计算成本低廉的方法,用于估计仅依靠梯度估计,讨论理论和实验结果的良好动机。在最后一部分,我们讨论代表性问题,考虑组等变卷积神经网络(GCNN)。等效tosymmetry转换在GCNNs中明确编码;经典的CNN是最简单的例子。特别地,我们提出了一种用于球形数据的SO(3) - 等变神经网络架构,它完全在傅立叶空间中运行,同时也为完全傅立叶神经网络的设计提供了形式,这与任何连续紧凑组的动作是等效的。
translated by 谷歌翻译
Deep Learning methods are currently the state-of-the
translated by 谷歌翻译
深度神经网络(DNN)正在成为现代计算应用中的重要工具。加速他们的培训是一项重大挑战,技术范围从分布式算法到低级电路设计。在这项调查中,我们从理论的角度描述了这个问题,然后是并行化的方法。我们介绍了DNN体系结构的趋势以及由此产生的对并行化策略的影响。然后,我们回顾并模拟DNN中不同类型的并发性:从单个运算符,到网络推理和训练中的并行性,再到分布式深度学习。我们讨论异步随机优化,分布式系统架构,通信方案和神经架构搜索。基于这些方法,我们推断了在深度学习中并行性的潜在方向。
translated by 谷歌翻译
深度神经网络(DNN)目前广泛用于许多人工智能(AI)应用,包括计算机视觉,语音识别和机器人技术。虽然DNN在许多AI任务上提供最先进的准确性,但却以高计算复杂性为代价。因此,技术要求能够有效地处理DNN以提高能量效率和吞吐量,而不会牺牲应用精度或增加对AI系统中DNN的广泛部署至关重要的硬件成本。本文旨在提供有关实现DNN高效处理目标的最新进展的综合指南和调查。具体而言,它将提供DNN的概述,讨论支持DNN的各种硬件平台和体系结构,并突出降低计算成本的关键趋势通过联合硬件设计和DNN算法变化,仅通过硬件设计变更或DNN。它还将总结各种开发资源,使研究人员和从业人员能够快速开始这一领域,并突出重要的基准测量指标和设计考虑因素,用于评估快速增长的DNN硬件设计数量,可选择包括算法设计,在学术界和行业。读者将从本文中删除以下概念:了解DNN的关键设计注意事项;能够使用基准和比较指标评估不同的DNN硬件实现;了解各种硬件架构和平台之间的权衡;能够评估各种DNN设计技术在高效处理中的实用性;并了解最近的实施趋势和机会。
translated by 谷歌翻译
声学数据提供从生物学和通信到海洋和地球科学等领域的科学和工程见解。我们调查了机器学习(ML)的进步和变革潜力,包括声学领域的深度学习。 ML是用于自动检测和利用模式印度的广泛的统计技术家族。相对于传统的声学和信号处理,ML是数据驱动的。给定足够的训练数据,ML可以发现特征之间的复杂关系。通过大量的训练数据,ML candiscover模型描述复杂的声学现象,如人类语音和混响。声学中的ML正在迅速发展,具有令人瞩目的成果和未来的重大前景。我们首先介绍ML,然后在五个声学研究领域强调MLdevelopments:语音处理中的源定位,海洋声学中的源定位,生物声学,地震探测和日常场景中的环境声音。
translated by 谷歌翻译
Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.
translated by 谷歌翻译
不同传感器捕获的数据可用性的急剧和近期增加加上其相当多的异质性质对于有效和高效地处理远程数据提出了严峻挑战。然而,遥感和辅助数据集的这种增加开辟了以联合方式利用多模态数据集的可能性,以进一步改善处理方法的性能,而不是考虑到手头的应用。因此,多源数据融合受到全世界研究人员的广泛关注,适用于各种各样的应用。此外,由于几个空间传感器的重新访问能力,时间信息与遥感数据的空间和/或光谱/反向散射信息的集成是可能的,并有助于从2D / 3D数据的表示转移到4D数据结构,其中时间变量为信息提取算法添加了新信息和挑战。有大量研究工作涉及多源和多时相数据融合,但不同模式的融合方法已根据各研究界的不同路径扩展。本文汇集了多源和多时相数据融合方法在不同研究社区方面的进展,并为不同层次的研究人员(即学生,研究人员和资深研究人员)提供了一个全面的,针对学科的起点,愿意对这一具有挑战性的话题进行新的研究。提供足够的细节和参考。
translated by 谷歌翻译
我们提出并讨论了物理层深度学习的几种新应用。通过将通信系统解释为自动编码器,我们开发了一种将通信系统设计视为端到端重建任务的基本新方法,该任务旨在在单个过程中联合优化发送器和接收器组件。我们展示了如何将这种想法扩展到多个发射器和接收器的网络,并将无线电变压器网络的概念作为在机器学习模型中结合专家领域知识的手段。最后,我们展示了卷积神经网络在原始IQ样本上的应用,用于调制分类,相对于依赖于专家特征的传统方案,它实现了竞争准确性。本文最后讨论了未来调查的开放性挑战和领域。
translated by 谷歌翻译
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques that diminish noise in the stochastic directions and methods that make use of second-order derivative approximations.
translated by 谷歌翻译
随着数字信息的快速发展,人和机器产生的数据量呈指数级增长。随着这种趋势,机器学习算法不断形成和发展,以发现来自不同数据源的新信息和知识。使用超级盒作为基本代表和构建块的学习算法是机器学习方法的缩写。这些算法具有巨大的潜力,可以实现高可扩展性和以这种方式构建的预测器的在线适应,以适应动态变化的环境和流数据。本文旨在对基于超级盒机器学习模型的文献进行全面的调查。通常,根据所得模型的架构和特征,现有的基于超盒的学习算法可以分为三大类:模糊最小 - 最大神经网络,基于超盒的混合模型,以及基于超盒表示的其他算法。在这些组中,本文简要描述了模型的结构,相关的学习算法,以及它们的优缺点。本文还介绍了这些基于高箱的模型在实际问题中的主要应用。最后,我们讨论了一些未解决的问题,并确定了该领域潜在的未来研究方向。
translated by 谷歌翻译
This is the pre-acceptance version, to read the final version please go to IEEE Geoscience and Remote Sensing Magazine on IEEE XPlore. Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a "black-box" solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization. Deep learning, remote sensing, machine learning, big data, Earth observation I. MOTIVATION Deep learning is the fastest-growing trend in big data analysis and has been deemed one of the 10 breakthrough technologies of 2013 [1]. It is characterized by neural networks (NNs) involving usually more than two layers (for this reason, they are called deep). As their shallow counterpart, deep neural networks exploit feature representations learned exclusively from data, instead of hand-crafting features that are mostly designed based on domain-specific knowledge. Deep learning research has been extensively pushed by Internet companies, such as Google, Baidu, Microsoft, and Facebook for several image analysis tasks, including image indexing, segmentation, and object detection. Recent advances in the field have proven deep learning a very successful set of tools, sometimes even able to surpass human ability to solve highly computational tasks (see, for instance, the highly mediatized Go match between Google's AlphaGo AI and the World Go Champion Lee Sedol. Motivated by those exciting advances, deep learning is becoming the model of choice in many fields of application. For instance, convolutional neural networks (CNNs) have proven to be good at extracting mid-and high-level abstract features from raw images, by interleaving convolutional and pooling layers, (i.e., spatially shrinking the feature maps layer by layer). Recent studies indicate that the feature representations learned by CNNs are greatly effective in large-scale image recognition [2-4], object detection [5, 6], and semantic segmentation [7, 8]. Furthermore, as an important branch of the deep learning family, recurrent neural networks (RNNs) have been shown to be very successful on a variety of tasks involved in sequential data analysis, such as action recognition [9, 10] and image captioning [11]. Following this wave of success and thanks to the increased availability of data and computational reso
translated by 谷歌翻译
In the era of the Internet of Things (IoT), an enormous amount of sensing devices collect and/or generate various sensory data over time for a wide range of fields and applications. Based on the nature of the application, these devices will result in big or fast/real-time data streams. Applying analytics over such data streams to discover new information, predict future insights, and make control decisions is a crucial process that makes IoT a worthy paradigm for businesses and a quality-of-life improving technology. In this paper, we provide a thorough overview on using a class of advanced machine learning techniques, namely Deep Learning (DL), to facilitate the analytics and learning in the IoT domain. We start by articulating IoT data characteristics and identifying two major treatments for IoT data from a machine learning perspective, namely IoT big data analytics and IoT streaming data analytics. We also discuss why DL is a promising approach to achieve the desired analytics in these types of data and applications. The potential of using emerging DL techniques for IoT data analytics are then discussed, and its promises and challenges are introduced. We present a comprehensive background on different DL architectures and algorithms. We also analyze and summarize major reported research attempts that leveraged DL in the IoT domain. The smart IoT devices that have incorporated DL in their intelligence background are also discussed. DL implementation approaches on the fog and cloud centers in support of IoT applications are also surveyed. Finally, we shed light on some challenges and potential directions for future research. At the end of each section, we highlight the lessons learned based on our experiments and review of the recent literature.
translated by 谷歌翻译
We present an analysis of three possible strategies for exploiting the power of existing convo-lutional neural networks (ConvNets) in different scenarios from the ones they were trained: full training, fine tuning, and using ConvNets as feature extractors. In many applications, especially including remote sensing, it is not feasible to fully design and train a new ConvNet, as this usually requires a considerable amount of labeled data and demands high computational costs. Therefore, it is important to understand how to obtain the best profit from existing ConvNets. We perform experiments with six popular ConvNets using three remote sensing datasets. We also compare ConvNets in each strategy with existing descriptors and with state-of-the-art baselines. Results point that fine tuning tends to be the best performing strategy. In fact, using the features from the fine-tuned ConvNet with linear SVM obtains the best results. We also achieved state-of-the-art results for the three datasets used.
translated by 谷歌翻译
Objective: Most current Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) are based on machine learning algorithms. There is a large diversity of classifier types that are used in this field, as described in our 2007 review paper. Now, approximately 10 years after this review publication, many new algorithms have been developed and tested to classify EEG signals in BCIs. The time is therefore ripe for an updated review of EEG classification algorithms for BCIs. Approach: We surveyed the BCI and machine learning literature from 2007 to 2017 to identify the new classification approaches that have been investigated to design BCIs. We synthesize these studies in order to present such algorithms, to report how they were used for BCIs, what were the outcomes, and to identify their pros and cons. Main results: We found that the recently designed classification algorithms for EEG-based BCIs can be divided into four main categories: adaptive classifiers, matrix and tensor classifiers, transfer learning and deep learning, plus a few other miscellaneous classifiers. Among these, adaptive classifiers were demonstrated to be generally superior to static ones, even with unsupervised adaptation. Transfer learning can also prove useful although the benefits of transfer learning remain unpredictable. Riemannian geometry-based methods have reached state-of-the-art performances on multiple BCI problems and deserve to be explored more thoroughly, along with tensor-based methods. Shrinkage linear discriminant analysis and random forests also appear particularly useful for small training samples settings. On the other hand, deep learning methods have not yet shown convincing improvement over state-of-the-art BCI methods. Significance: This paper provides a comprehensive overview of the modern classification algorithms used in EEG-based BCIs, presents the principles of these Review of Classification Algorithms for EEG-based BCI 2 methods and guidelines on when and how to use them. It also identifies a number of challenges to further advance EEG classification in BCI.
translated by 谷歌翻译
深度学习提出了希望和期望,作为许多应用程序的一般解决方案;事实证明它已被证明是有效的,但它也显示出对大量数据的强烈依赖性。幸运的是,已经证明,即使数据稀缺,也可以通过重复使用priorknowledge来训练成功的模型。因此,在最广泛的定义中,开发转移学习技术是部署有效和准确的智能系统的关键因素。本文将重点研究一系列适用于视觉目标识别任务的转移学习方法,特别是图像分类。转移学习是一个通用术语,并且特定设置已经给出了特定的名称:当学习者只能访问来自目标域的标记数据和来自不同域(源)的标记数据时,问题被称为“无监督域适应”。 (DA)。这项工作的第一部分将集中在这个设置的三种方法:其中一种方法涉及特征,一种是图像,而第三种方法同时使用两种。第二部分将重点关注机器人感知的现实生活问题,特别是RGB-D识别。机器人平台通常不仅限于色彩感知;他们经常带着Depthcamera。不幸的是,深度模态很少用于视觉识别,因为缺乏预先训练的模型,从中可以传输并且很少有数据从头开始。将提出两种处理这种情况的方法:一种使用合成数据,另一种利用跨模态转移学习。
translated by 谷歌翻译
Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks have a long history, recent advances have greatly improved their performance in computer vision, natural language processing, etc. From the statistical and scientific perspective, it is natural to ask: What is deep learning? What are the new characteristics of deep learning, compared with classical methods? What are the theoretical foundations of deep learning? To answer these questions, we introduce common neural network models (e.g., convolutional neural nets, recurrent neural nets, generative adversarial nets) and training techniques (e.g., stochastic gradient descent, dropout, batch normalization) from a statistical point of view. Along the way, we highlight new characteristics of deep learning (including depth and over-parametrization) and explain their practical and theoretical benefits. We also sample recent results on theories of deep learning, many of which are only suggestive. While a complete understanding of deep learning remains elusive, we hope that our perspectives and discussions serve as a stimulus for new statistical research.
translated by 谷歌翻译