We propose a novel Multi-Task Learning with Low Rank Attribute Embedding (MTL-LORAE) framework for person re-identification. Re-identifications from multiple cameras are regarded as related tasks to exploit shared information to improve re-identification accuracy. Both low level features and semantic/data-driven attributes are utilized. Since attributes are generally correlated, we introduce a low rank attribute embedding into the MTL formulation to embed original binary attributes to a continuous attribute space, where incorrect and incomplete attributes are rectified and recovered to better describe people. The learning objective function consists of a quadratic loss regarding class labels and an attribute embedding error, which is solved by an alternating optimization procedure. Experiments on four person re-identification datasets have demonstrated that MTL-LORAE outperforms existing approaches by a large margin and produces promising results.
translated by 谷歌翻译
多任务学习(MTL)是机器学习的学习范例,其目的是利用多个相关任务中包含的有用信息来帮助提高所有任务的泛化性能。在本文中,我们对MTL进行了调查。首先,我们将不同的MTL算法分为几类,包括特征学习方法,低秩方法,任务聚类方法,任务关系学习方法和分解方法,然后讨论每种方法的特征。为了进一步提高学习任务的表现,MTL可以与其他学习范式相结合,包括半监督学习,主动学习,无监督学习,强化学习,多视图学习和图形模型。当任务数量很大或数据维数很高时,批量MTL模型很难处理这种情况,并且审查在线,并行和分布式MTL模型以及降维和特征散列以揭示其计算和存储优势。许多真实世界的应用程序使用MTL来提高其性能,并审查代表性的工作。最后,我们提出了理论分析,并讨论了MTL的未来几个方向。
translated by 谷歌翻译
机器学习中最基本的问题之一是比较例子:给定一对对象,我们想要返回一个表示(dis)相似度的值。相似性通常是特定于任务的,并且预定义的距离可能表现不佳,从而导致在度量学习中工作。然而,能够学习相似性敏感距离函数也预先假定对于手头的对象的丰富的,有辨别力的表示。在本论文中,我们提出了两端的贡献。在论文的第一部分中,假设数据具有良好的表示,我们提出了一种用于度量学习的公式,与先前的工作相比,它更直接地尝试优化k-NN精度。我们还提出了这个公式的扩展,用于kNN回归的度量学习,不对称相似学习和汉明距离的判别学习。在第二部分中,我们考虑我们处于有限计算预算的情况,即在可能度量的空间上进行优化是不可行的,但是仍然需要访问标签感知距离度量。我们提出了一种简单,计算成本低廉的方法,用于估计仅依靠梯度估计,讨论理论和实验结果的良好动机。在最后一部分,我们讨论代表性问题,考虑组等变卷积神经网络(GCNN)。等效tosymmetry转换在GCNNs中明确编码;经典的CNN是最简单的例子。特别地,我们提出了一种用于球形数据的SO(3) - 等变神经网络架构,它完全在傅立叶空间中运行,同时也为完全傅立叶神经网络的设计提供了形式,这与任何连续紧凑组的动作是等效的。
translated by 谷歌翻译
We present a novel attribute learning framework named Hypergraph-based Attribute Predictor (HAP). In HAP, a hy-pergraph is leveraged to depict the attribute relations in the data. Then the attribute prediction problem is casted as a regularized hypergraph cut problem, in which a collection of attribute projections is jointly learnt from the feature space to a hypergraph embedding space aligned with the attributes. The learned projections directly act as attribute classifiers (linear and kernelized). This formulation leads to a very efficient approach. By considering our model as a multi-graph cut task, our framework can flexibly incorporate other available information, in particular class label. We apply our approach to attribute prediction, Zero-shot and N-shot learning tasks. The results on AWA, USAA and CUB databases demonstrate the value of our methods in comparison with the state-of-the-art approaches.
translated by 谷歌翻译
The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural computation. Here we describe locally linear embedding (LLE), an unsupervised learning algorithm that computes low dimensional, neighborhood preserving embeddings of high dimensional data. The data, assumed to be sampled from an underlying manifold, are mapped into a single global coordinate system of lower dimensionality. The mapping is derived from the symmetries of locally linear reconstructions, and the actual computation of the embedding reduces to a sparse eigen-value problem. Notably, the optimizations in LLE-though capable of generating highly nonlinear embeddings-are simple to implement, and they do not involve local minima. In this paper, we describe the implementation of the algorithm in detail and discuss several extensions that enhance its performance. We present results of the algorithm applied to data sampled from known manifolds, as well as to collections of images of faces, lips, and handwritten digits. These examples are used to provide extensive illustrations of the algorithm's performance-both successes and failures-and to relate the algorithm to previous and ongoing work in nonlinear dimensionality reduction.
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
We are honored to welcome you to the 2nd International Workshop on Advanced Analyt-ics and Learning on Temporal Data (AALTD), which is held in Riva del Garda, Italy, on September 19th, 2016, co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2016). The aim of this workshop is to bring together researchers and experts in machine learning, data mining, pattern analysis and statistics to share their challenging issues and advance researches on temporal data analysis. Analysis and learning from temporal data cover a wide scope of tasks including learning metrics, learning representations, unsupervised feature extraction, clustering and classification. This volume contains the conference program, an abstract of the invited keynotes and the set of regular papers accepted to be presented at the conference. Each of the submitted papers was reviewed by at least two independent reviewers, leading to the selection of eleven papers accepted for presentation and inclusion into the program and these proceedings. The contributions are given by the alphabetical order, by surname. The keynote given by Marco Cuturi on "Regularized DTW Divergences for Time Se-ries" focuses on the definition of alignment kernels for time series that can later be used at the core of standard machine learning algorithms. The one given by Tony Bagnall on "The Great Time Series Classification Bake Off" presents an important attempt to experimentally compare performance of a wide range of time series classifiers, together with ensemble classifiers that aim at combining existing classifiers to improve classification quality. Accepted papers spanned from innovative ideas on analytic of temporal data, including promising new approaches and covering both practical and theoretical issues. We wish to thank the ECML PKDD council members for giving us the opportunity to hold the AALTD workshop within the framework of the ECML/PKDD Conference and the members of the local organizing committee for their support. The organizers of the AALTD conference gratefully thank the financial support of the Université de Rennes 2, MODES and Universidade da Coruña. Last but not least, we wish to thank the contributing authors for the high quality works and all members of the Reviewing Committee for their invaluable assistance in the iii selection process. All of them have significantly contributed to the success of AALTD 2106. We sincerely hope that the workshop participants have a great and fruitful time at the conference.
translated by 谷歌翻译
Relative attributes learning aims to learn ranking functions describing the relative strength of attributes. Most of current learning approaches learn ranking functions for each attribute independently without considering possible intrinsic relatedness among the attributes. For a problem involving multiple attributes, it is reasonable to assume that utilizing such relatedness among the attributes would benefit learning, especially when the number of labeled training pairs are very limited. In this paper, we proposed a relative multi-attribute learning framework that integrates relative attributes into a multi-task learning scheme. The formulation allows us to exploit the advantages of the state-of-the-art regularization-based multi-task learning for improved attribute learning. In particular, using joint feature learning as the case studies, we evaluated our framework with both synthetic data and two real datasets. Experimental results suggest that the proposed framework has clear performance gain in ranking accuracy and zero-shot learning accuracy over existing methods of independent relative attributes learning and multi-task learning.
translated by 谷歌翻译
We study large-scale image classification methods that can incorporate new classes and training images continuously over time at negligible cost. To this end we consider two distance-based classifiers, the k-nearest neighbor (k-NN) and nearest class mean (NCM) classifiers, and introduce a new metric learning approach for the latter. We also introduce an extension of the NCM classifier to allow for richer class representations. Experiments on the ImageNet 2010 challenge dataset, which contains over 10 6 training images of 1,000 classes, show that, surprisingly, the NCM classifier compares favorably to the more flexible k-NN classifier. Moreover, the NCM performance is comparable to that of linear SVMs which obtain current state-of-the-art performance. Experimentally we study the generalization performance to classes that were not used to learn the metrics. Using a metric learned on 1,000 classes, we show results for the ImageNet-10K dataset which contains 10,000 classes, and obtain performance that is competitive with the current state-of-the-art, while being orders of magnitude faster. Furthermore, we show how a zero-shot class prior based on the ImageNet hierarchy can improve performance when few training images are available.
translated by 谷歌翻译
We study the problem of recovery of matrices that are simultaneously low rank and row and/or column sparse. Such matrices appear in recent applications in cognitive neuroscience, imaging, computer vision, macroeconomics, and genetics. We propose a GDT (Gradient Descent with hard Thresholding) algorithm to efficiently recover matrices with such structure, by minimizing a bi-convex function over a nonconvex set of constraints. We show linear convergence of the iterates obtained by GDT to a region within statistical error of an optimal solution. As an application of our method, we consider multi-task learning problems and show that the statistical error rate obtained by GDT is near optimal compared to minimax rate. Experiments demonstrate competitive performance and much faster running speed compared to existing methods, on both simulations and real data sets.
translated by 谷歌翻译
Metric and kernel learning arise in several machine learning applications. However, most existing metric learning algorithms are limited to learning metrics over low-dimensional data, while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points. In this paper, we study the connections between metric learning and kernel learning that arise when studying metric learning as a linear transformation learning problem. In particular, we propose a general optimization framework for learning metrics via linear transformations, and analyze in detail a special case of our framework-that of minimizing the LogDet divergence subject to linear constraints. We then propose a general regularized framework for learning a kernel matrix, and show it to be equivalent to our metric learning framework. Our theoretical connections between metric and kernel learning have two main consequences: 1) the learned kernel matrix parameterizes a linear transformation kernel function and can be applied inductively to new data points, 2) our result yields a constructive method for kernelizing most existing Mahalanobis metric learning formulations. We demonstrate our learning approach by applying it to large-scale real world problems in computer vision, text mining and semi-supervised kernel dimensionality reduction.
translated by 谷歌翻译
我们提出深度非对称多任务特征学习(Deep-AMTFL),它可以学习在多个任务中共享的深层表示,同时有效地防止在特征共享过程中可能发生的负面转移。具体而言,我们引入了一个非对称自动编码器术语,允许可靠的预测器使简单任务具有对特征学习的贡献很大,同时抑制不可靠预测因子对更困难任务的影响。这允许学习较少噪声的表示,并且使得不可靠的预测器能够通过共享的潜在特征来利用来自可靠预测器的知识。通过共享特征进行的这种不对称知识转移也比任务间不对称转移更具可扩展性和有效性。我们在多个基准数据集上验证我们的Deep-AMTFL模型,即形式学习和图像分类,通过有效防止深度特征学习中的负迁移,它显着地优于现有的对称和非对称多任务学习模型。
translated by 谷歌翻译
转移学习的方法试图结合来自几个相关任务(或域)的知识来提高测试任务的性能。受到因果方法学的启发,我们放松了通常的协变量偏移假设,并假设对于预测变量的子集,itholds为真:给定此预测变量子集的目标变量的条件分布对于所有预测变量是不变的。我们展示了如何从这种假设领域的想法中推动出这种假设。我们关注域泛化的问题,其中没有观察到来自测试任务的示例。我们证明在使用该子集进行预测的对抗性设置在域泛化中是最优的;我们进一步提供了示例,其中任务是充分多样化的,因此估计器优于汇集数据,即使平均也是如此。来自测试任务的示例可用,我们还提供了一种从训练任务中转移知识并利用所有可用特征进行预测的方法。但是,我们不保证此方法。我们介绍了允许自动推断上述子集并提供相应代码的实用方法。我们在合成数据集和agene删除数据集上提供结果。
translated by 谷歌翻译
We review machine learning methods employing positive definite kernels. Thesemethods formulate learning and estimation problems in a reproducing kernelHilbert space (RKHS) of functions defined on the data domain, expanded in termsof a kernel. Working in linear spaces of function has the benefit offacilitating the construction and analysis of learning algorithms while at thesame time allowing large classes of functions. The latter include nonlinearfunctions as well as functions defined on nonvectorial data. We cover a widerange of methods, ranging from binary classifiers to sophisticated methods forestimation with structured data.
translated by 谷歌翻译
In 2007, Labeled Faces in the Wild was released in an effort to spur research in face recognition, specifically for the problem of face verification with un-constrained images. Since that time, more than 50 papers have been published that improve upon this benchmark in some respect. A remarkably wide variety of innovative methods have been developed to overcome the challenges presented in this database. As performance on some aspects of the benchmark approaches 100% accuracy , it seems appropriate to review this progress, derive what general principles we can from these works, and identify key future challenges in face recognition. In this survey, we review the contributions to LFW for which the authors have provided results to the curators (results found on the LFW results web page). We also review the cross cutting topic of alignment and how it is used in various methods. We end with a brief discussion of recent databases designed to challenge the next generation of face recognition algorithms.
translated by 谷歌翻译
许多现实世界的大规模回归问题可以被表述为具有大量任务的多任务学习(MTL)问题,如在零售运输领域。然而,现有的MTL方法仍然无法提供泛化性能和这些问题的可扩展性。将MTL方法应用于大量任务的问题是一个很大的挑战。在这里,我们提出了一种新的算法,名为凸集聚多任务回归学习(CCMTL),它与凸集群在预测模型的k-最近邻图上进行整合。此外,CCMTL利用新提出的优化方法有效地解决了潜在的凸问题。 CCMTL准确,高效地训练,并且在任务数量上线性地实现经验。在合成和现实世界的数据集中,提出的CCMTL在预测精度和计算效率方面优于七种最先进的(SoA)多任务学习方法。在具有23,812个任务的真实零售数据集中,CCMTL仅需要大约30秒即可在单个线程上进行训练,而SoA方法需要长达数小时甚至数天。
translated by 谷歌翻译
This paper proposes a joint multi-task learning algorithm to better predict attributes in images using deep convolutional neural networks (CNN). We consider learning binary semantic attributes through a multi-task CNN model, where each CNN will predict one binary attribute. The multi-task learning allows CNN models to simultaneously share visual knowledge among different attribute categories. Each CNN will generate attribute-specific feature representations, and then we apply multi-task learning on the features to predict their attributes. In our multi-task framework, we propose a method to decompose the overall model's parameters into a latent task matrix and combination matrix. Furthermore, under-sampled classifiers can leverage shared statistics from other classifiers to improve their performance. Natural grouping of attributes is applied such that attributes in the same group are encouraged to share more knowledge. Meanwhile, attributes in different groups will generally compete with each other, and consequently share less knowledge. We show the effectiveness of our method on two popular attribute datasets.
translated by 谷歌翻译
Multi-task learning (MTL) learns multiple related tasks simultaneously to improve generalization performance. Alternating structure optimization (ASO) is a popular MTL method that learns a shared low-dimensional predictive structure on hypothesis spaces from multiple related tasks. It has been applied successfully in many real world applications. As an alternative MTL approach, clustered multi-task learning (CMTL) assumes that multiple tasks follow a clustered structure, i.e., tasks are partitioned into a set of groups where tasks in the same group are similar to each other, and that such a clustered structure is unknown a priori. The objectives in ASO and CMTL differ in how multiple tasks are related. Interestingly, we show in this paper the equivalence relationship between ASO and CMTL, providing significant new insights into ASO and CMTL as well as their inherent relationship. The CMTL formulation is non-convex, and we adopt a convex relaxation to the CMTL formulation. We further establish the equivalence relationship between the proposed convex relaxation of CMTL and an existing convex relaxation of ASO, and show that the proposed convex CMTL formulation is significantly more efficient especially for high-dimensional data. In addition, we present three algorithms for solving the convex CMTL formulation. We report experimental results on benchmark datasets to demonstrate the efficiency of the proposed algorithms .
translated by 谷歌翻译
多视图数据已经在各个科学和工程领域中定期收集。一般问题是研究多变量响应和多视图预测器集之间的预测关联,所有这些都可以具有高维度。很可能只有少数观点是相关的预测,并且每个相关观点中的预测因子集体而非稀疏地有助于预测。我们在熟悉的多元回归框架下投射这个新问题,并提出一个综合减少秩回归(iRRR),其中每个视图都有自己的低秩系数矩阵。因此,以无监督的方式从每个视图中提取潜在特征。对于模型估计,我们开发了一种凸复合核心惩罚方法,它通过乘法器的交替方向方法允许一种有效的算法。讨论了非高斯和不完整数据的扩展。理论上,我们在受限的特征值条件下导出iRRR的非渐近oracle边界。我们的结果恢复了iRRR的几个特殊病例的甲骨文界限,包括Lasso,Lasso组和核同源化回归。因此,iRRR无缝地桥接组稀疏和低秩方法,并且在多视图学习的现实设置下可以实现明显更快的收敛速度。模拟研究和老化纵向研究中的应用进一步展示了所提出方法的有效性。
translated by 谷歌翻译