音频或视觉数据分析任务通常必须处理高维和非负信号。然而,当数据具有多维数减少预处理时,大多数数据分析方法遭受过度拟合和数值问题。此外,关于如何以及为什么滤波器为音频或可视应用的方式工作是所需的属性,特别是当涉及能量或频谱信号时。在这些情况下,由于这些信号的性质,滤波器重量的非承诺是所需的性质,以更好地理解其工作。由于这两个必需品,我们提出了不同的方法来减少数据的维度,而保证溶液的非承诺和可解释性。特别是,我们提出了一种广义方法,以在处理非负数据的应用程序中以监督方式设计过滤器银行,并且我们探讨了解决所提出的目标函数的不同方式,包括非负面的部分最小二乘法的非负图。我们分析了通过拟议的两种不同和广泛研究的应用方法获得的特征的辨别力:纹理和音乐类型分类。此外,我们比较我们的方法实现的滤波器银行,具体设计用于特征提取的其他最先进的方法。
translated by 谷歌翻译
多变量分析(MVA)包括用于特征提取的众所周知的方法,该方法提取,其利用表示数据的输入变量之间的相关性。大多数此类方法享有的一个重要属性是提取特征之间的不相关性。最近,MVA方法的正则化版本在文献中出现,主要是为了获得解决方案的解释性。在这些情况下,不再以封闭的方式获得解决方案,并且经常使用更复杂的优化方法,依赖于两个步骤的迭代。本文回到了替代方法来解决这个迭代问题。这种方法的主要新颖性在于保持原始方法的几个属性,最值得注意的是提取特征的不相关性。在此框架下,我们提出了一种新的方法,该方法利用L-21规范在特征提取过程中执行变量选择。不同问题的实验结果证实了与现有化配方的拟议配方的优点。
translated by 谷歌翻译
我们介绍了一种确定全局特征解耦的方法,并显示其适用于提高数据分析性能的适用性,并开放了新的场所以进行功能传输。我们提出了一种新的形式主义,该形式主义是基于沿特征梯度遵循轨迹来定义对子曼群的转换的。通过这些转换,我们定义了一个归一化,我们证明,它允许解耦可区分的特征。通过将其应用于采样矩,我们获得了用于正骨的准分析溶液,正尾肌肉是峰度的归一化版本,不仅与平均值和方差相关,而且还与偏度相关。我们将此方法应用于原始数据域和过滤器库的输出中,以基于全局描述符的回归和分类问题,与使用经典(未删除)描述符相比,性能得到一致且显着的改进。
translated by 谷歌翻译
The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.
translated by 谷歌翻译
自适应滤波器处于许多信号处理应用的核心,从声噪声繁殖到回声消除,阵列波束形成,信道均衡,以更新的传感器网络应用在监控,目标本地化和跟踪中。沿着该方向的趋势方法是重复到网络内分布式处理,其中各个节点实现适应规则并将它们的估计扩散到网络。当关于过滤方案的先验知识有限或不精确时,选择最适当的过滤器结构并调整其参数变得有挑战性的任务,并且错误的选择可能导致性能不足。为了解决这个困难,一种有用的方法是依赖自适应结构的组合。自适应滤波器的组合在某种程度上利用相同的鸿沟和征服机器学习界(例如,袋装或升级)成功利用的原则。特别地,在不同的视角下,在计算学习领域中研究了组合若干学习算法的输出(专家的混合):而不是研究混合物的预期性能,衍生出适用于各个序列的确定性范围因此,反映了最糟糕的情况。这些界限需要与通常在自适应滤波中使用的那些不同的假设,这是该概述文章的重点。我们审查了这些组合计划背后的关键思想和原则,重点是设计规则。我们还通过各种示例说明了它们的性能。
translated by 谷歌翻译
监督主体组件分析(SPCA)的方法旨在将标签信息纳入主成分分析(PCA),以便提取的功能对于预测感兴趣的任务更有用。SPCA的先前工作主要集中在优化预测误差上,并忽略了提取功能解释的最大化方差的价值。我们为SPCA提出了一种新的方法,该方法共同解决了这两个目标,并从经验上证明我们的方法主导了现有方法,即在预测误差和变异方面都超越了它们的表现。我们的方法可容纳任意监督的学习损失,并通过统计重新制定提供了广义线性模型的新型低级扩展。
translated by 谷歌翻译
情感计算在人与机器之间的关系中非常重要。在本文中,提出了一种基于语音信号的语音情感识别(SER)的系统,其利用不同处理阶段的新技术。该系统由三个阶段组成:功能提取,功能选择,最终要素分类。在第一阶段,使用新的和多样性特征(如韵律,光谱和频谱)特征,从语音信号和光泽 - 波形信号中提取复杂的长期统计特征。 SER系统的挑战之一是区分相关情绪。这些特征是言语情绪的好鉴别者,并提高Ser识别类似和不同情绪的能力。此特征向量具有大量维度自然具有冗余。在第二阶段,使用经典特征选择技术以及用于减少特征向量维度的新量子启发技术,减少了特征向量尺寸的数量。在第三阶段,优化的特征向量由加权深稀疏的极端学习机(ELM)分类器分类。分类器以三个步骤执行分类:稀疏随机特征学习,使用奇异值分解(SVD)技术的正交随机投影,以及使用广义Tikhonov正规技术的最后一步中的鉴别分类。此外,许多现有的情绪数据集遭受数据不平衡分布的问题,这反过来增加了分类误差并降低了系统性能。在本文中,还提出了一种新的加权方法来处理类别不平衡,比现有的加权方法更有效。所提出的方法是在三个标准情绪数据库上进行评估。
translated by 谷歌翻译
Explainable AI transforms opaque decision strategies of ML models into explanations that are interpretable by the user, for example, identifying the contribution of each input feature to the prediction at hand. Such explanations, however, entangle the potentially multiple factors that enter into the overall complex decision strategy. We propose to disentangle explanations by finding relevant subspaces in activation space that can be mapped to more abstract human-understandable concepts and enable a joint attribution on concepts and input features. To automatically extract the desired representation, we propose new subspace analysis formulations that extend the principle of PCA and subspace analysis to explanations. These novel analyses, which we call principal relevant component analysis (PRCA) and disentangled relevant subspace analysis (DRSA), optimize relevance of projected activations rather than the more traditional variance or kurtosis. This enables a much stronger focus on subspaces that are truly relevant for the prediction and the explanation, in particular, ignoring activations or concepts to which the prediction model is invariant. Our approach is general enough to work alongside common attribution techniques such as Shapley Value, Integrated Gradients, or LRP. Our proposed methods show to be practically useful and compare favorably to the state of the art as demonstrated on benchmarks and three use cases.
translated by 谷歌翻译
大多数机器学习算法由一个或多个超参数配置,必须仔细选择并且通常会影响性能。为避免耗时和不可递销的手动试验和错误过程来查找性能良好的超参数配置,可以采用各种自动超参数优化(HPO)方法,例如,基于监督机器学习的重新采样误差估计。本文介绍了HPO后,本文审查了重要的HPO方法,如网格或随机搜索,进化算法,贝叶斯优化,超带和赛车。它给出了关于进行HPO的重要选择的实用建议,包括HPO算法本身,性能评估,如何将HPO与ML管道,运行时改进和并行化结合起来。这项工作伴随着附录,其中包含关于R和Python的特定软件包的信息,以及用于特定学习算法的信息和推荐的超参数搜索空间。我们还提供笔记本电脑,这些笔记本展示了这项工作的概念作为补充文件。
translated by 谷歌翻译
被动射频(RF)感测和对老年护理房屋的人类日常活动监测是一个新兴的话题。微多普勒雷达是一种吸引人的解决方案,考虑到它们的非侵入性,深渗透和高距离范围。尽管在真实情景中未标记或较差的活动的情况下,但是使用多普勒雷达数据的无监督活动识别尚未得到注意。本研究提出了使用多普勒流的人类活动监测的两个无监督特征提取方法。这些包括基于局部离散余弦变换(DCT)的特征提取方法和基于局部熵的特征提取方法。此外,对于多普勒雷达数据,首次采用了卷积变分性自动化器(CVAE)特征提取的新应用。将三种特征提取架构与先前使用的卷积AutoEncoder(CAE)和基于主成分分析(PCA)和2DPCA的线性特征提取进行比较。使用K-Means和K-METOIDS进行无监督的聚类。结果表明,与CAE,PCA和2DPCA相比,基于DCT的方法,基于熵的方法和CVAE特征的优越性,具有超过5 \%-20 \%的平均精度。关于计算时间,两个提出的方法明显比现有的CVAE快得多。此外,对于高维数据可视化,考虑了三种歧管学习技术。比较方法,以对原始数据的投影以及编码的CVAE特征进行比较。当应用于编码的CVAE特征时,所有三种方法都显示出改善的可视化能力。
translated by 谷歌翻译
In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signal-atoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, regularization in inverse problems, feature extraction, and more. Recent activity in this field has concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. Designing dictionaries to better fit the above model can be done by either selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Both of these techniques have been considered, but this topic is largely still open. In this paper we propose a novel algorithm for adapting dictionaries in order to achieve sparse signal representations. Given a set of training signals, we seek the dictionary that leads to the best representation for each member in this set, under strict sparsity constraints. We present a new method-the K-SVD algorithm-generalizing the K-means clustering process. K-SVD is an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data. The update of the dictionary columns is combined with an update of the sparse representations, thereby accelerating convergence. The K-SVD algorithm is flexible and can work with any pursuit method (e.g., basis pursuit, FOCUSS, or matching pursuit). We analyze this algorithm and demonstrate its results both on synthetic tests and in applications on real image data.
translated by 谷歌翻译
这是一门专门针对STEM学生开发的介绍性机器学习课程。我们的目标是为有兴趣的读者提供基础知识,以在自己的项目中使用机器学习,并将自己熟悉术语作为进一步阅读相关文献的基础。在这些讲义中,我们讨论受监督,无监督和强化学习。注释从没有神经网络的机器学习方法的说明开始,例如原理分析,T-SNE,聚类以及线性回归和线性分类器。我们继续介绍基本和先进的神经网络结构,例如密集的进料和常规神经网络,经常性的神经网络,受限的玻尔兹曼机器,(变性)自动编码器,生成的对抗性网络。讨论了潜在空间表示的解释性问题,并使用梦和对抗性攻击的例子。最后一部分致力于加强学习,我们在其中介绍了价值功能和政策学习的基本概念。
translated by 谷歌翻译
大量的数据和创新算法使数据驱动的建模成为现代行业的流行技术。在各种数据驱动方法中,潜在变量模型(LVM)及其对应物占主要份额,并在许多工业建模领域中起着至关重要的作用。 LVM通常可以分为基于统计学习的经典LVM和基于神经网络的深层LVM(DLVM)。我们首先讨论经典LVM的定义,理论和应用,该定义和应用既是综合教程,又是对经典LVM的简短申请调查。然后,我们对当前主流DLVM进行了彻底的介绍,重点是其理论和模型体系结构,此后不久就提供了有关DLVM的工业应用的详细调查。上述两种类型的LVM具有明显的优势和缺点。具体而言,经典的LVM具有简洁的原理和良好的解释性,但是它们的模型能力无法解决复杂的任务。基于神经网络的DLVM具有足够的模型能力,可以在复杂的场景中实现令人满意的性能,但它以模型的解释性和效率为例。旨在结合美德并减轻这两种类型的LVM的缺点,并探索非神经网络的举止以建立深层模型,我们提出了一个新颖的概念,称为“轻量级Deep LVM(LDLVM)”。在提出了这个新想法之后,该文章首先阐述了LDLVM的动机和内涵,然后提供了两个新颖的LDLVM,并详尽地描述了其原理,建筑和优点。最后,讨论了前景和机会,包括重要的开放问题和可能的研究方向。
translated by 谷歌翻译
Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.Real-life document recognition systems are composed of multiple modules including eld extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure.Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the exibility of Graph Transformer Networks.A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.
translated by 谷歌翻译
The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning.
translated by 谷歌翻译
卷积神经网络已在图像分类方面取得了成功的结果,从而实现了超过人类水平的实时结果。但是,纹理图像仍然对这些模型构成一些挑战,例如,在出现这些图像,高层间相似性,没有代表对象的全局观点的几个问题中,培训的数据可用性有限,并且其他。在这种情况下,本文的重点是提高纹理分类中卷积神经网络的准确性。这是通过从验证的神经网络的多个卷积层中提取特征并使用Fisher载体聚集此类特征来完成的。使用较早卷积层的特征的原因是获得了较少域的信息。我们验证方法对基准数据集的纹理分类以及巴西植物物种识别的实际任务的有效性。在这两种情况下,在多层上计算出的Fisher矢量都优于制作方法,证实早期卷积层提供了有关分类纹理图像的重要信息。
translated by 谷歌翻译
Deep neural networks provide unprecedented performance gains in many real world problems in signal and image processing. Despite these gains, future development and practical deployment of deep networks is hindered by their blackbox nature, i.e., lack of interpretability, and by the need for very large training sets. An emerging technique called algorithm unrolling or unfolding offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are used widely in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention and is rapidly growing both in theoretic investigations and practical applications. The growing popularity of unrolled deep networks is due in part to their potential in developing efficient, high-performance and yet interpretable network architectures from reasonable size training sets. In this article, we review algorithm unrolling for signal and image processing. We extensively cover popular techniques for algorithm unrolling in various domains of signal and image processing including imaging, vision and recognition, and speech processing. By reviewing previous works, we reveal the connections between iterative algorithms and neural networks and present recent theoretical results. Finally, we provide a discussion on current limitations of unrolling and suggest possible future research directions.
translated by 谷歌翻译
This paper reviews the recent progress of remote sensing image scene classification, proposes a large-scale benchmark dataset, and evaluates a number of state-of-the-art methods using the proposed dataset.
translated by 谷歌翻译
The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.
translated by 谷歌翻译
最近有一项激烈的活动在嵌入非常高维和非线性数据结构的嵌入中,其中大部分在数据科学和机器学习文献中。我们分四部分调查这项活动。在第一部分中,我们涵盖了非线性方法,例如主曲线,多维缩放,局部线性方法,ISOMAP,基于图形的方法和扩散映射,基于内核的方法和随机投影。第二部分与拓扑嵌入方法有关,特别是将拓扑特性映射到持久图和映射器算法中。具有巨大增长的另一种类型的数据集是非常高维网络数据。第三部分中考虑的任务是如何将此类数据嵌入中等维度的向量空间中,以使数据适合传统技术,例如群集和分类技术。可以说,这是算法机器学习方法与统计建模(所谓的随机块建模)之间的对比度。在论文中,我们讨论了两种方法的利弊。调查的最后一部分涉及嵌入$ \ mathbb {r}^ 2 $,即可视化中。提出了三种方法:基于第一部分,第二和第三部分中的方法,$ t $ -sne,UMAP和大节。在两个模拟数据集上进行了说明和比较。一个由嘈杂的ranunculoid曲线组成的三胞胎,另一个由随机块模型和两种类型的节点产生的复杂性的网络组成。
translated by 谷歌翻译