The notion of concept drift refers to the phenomenon that the distribution generating the observed data changes over time. If drift is present, machine learning models may become inaccurate and need adjustment. Many technologies for learning with drift rely on the interleaved test-train error (ITTE) as a quantity which approximates the model generalization error and triggers drift detection and model updates. In this work, we investigate in how far this procedure is mathematically justified. More precisely, we relate a change of the ITTE to the presence of real drift, i.e., a changed posterior, and to a change of the training result under the assumption of optimality. We support our theoretical findings by empirical evidence for several learning algorithms, models, and datasets.
translated by 谷歌翻译
所有著名的机器学习算法构成了受监督和半监督的学习工作,只有在一个共同的假设下:培训和测试数据遵循相同的分布。当分布变化时,大多数统计模型必须从新收集的数据中重建,对于某些应用程序,这些数据可能是昂贵或无法获得的。因此,有必要开发方法,以减少在相关领域中可用的数据并在相似领域中进一步使用这些数据,从而减少需求和努力获得新的标签样品。这引起了一个新的机器学习框架,称为转移学习:一种受人类在跨任务中推断知识以更有效学习的知识能力的学习环境。尽管有大量不同的转移学习方案,但本调查的主要目的是在特定的,可以说是最受欢迎的转移学习中最受欢迎的次级领域,概述最先进的理论结果,称为域适应。在此子场中,假定数据分布在整个培训和测试数据中发生变化,而学习任务保持不变。我们提供了与域适应性问题有关的现有结果的首次最新描述,该结果涵盖了基于不同统计学习框架的学习界限。
translated by 谷歌翻译
Concept drift describes unforeseeable changes in the underlying distribution of streaming data over time. Concept drift research involves the development of methodologies and techniques for drift detection, understanding and adaptation. Data analysis has revealed that machine learning in a concept drift environment will result in poor learning results if the drift is not addressed. To help researchers identify which research topics are significant and how to apply related techniques in data analysis tasks, it is necessary that a high quality, instructive review of current research developments and trends in the concept drift field is conducted. In addition, due to the rapid development of concept drift in recent years, the methodologies of learning under concept drift have become noticeably systematic, unveiling a framework which has not been mentioned in literature. This paper reviews over 130 high quality publications in concept drift related research areas, analyzes up-to-date developments in methodologies and techniques, and establishes a framework of learning under concept drift including three main components: concept drift detection, concept drift understanding, and concept drift adaptation. This paper lists and discusses 10 popular synthetic datasets and 14 publicly available benchmark datasets used for evaluating the performance of learning algorithms aiming at handling concept drift. Also, concept drift related research directions are covered and discussed. By providing state-of-the-art knowledge, this survey will directly support researchers in their understanding of research developments in the field of learning under concept drift.
translated by 谷歌翻译
使用历史观察数据的政策学习是发现广泛应用程序的重要问题。示例包括选择优惠,价格,要发送给客户的广告,以及选择要开出患者的药物。但是,现有的文献取决于这样一个关键假设,即将在未来部署学习策略的未来环境与生成数据的过去环境相同 - 这个假设通常是错误或太粗糙的近似值。在本文中,我们提高了这一假设,并旨在通过不完整的观察数据来学习一项稳健的策略。我们首先提出了一个政策评估程序,该程序使我们能够评估政策在最坏情况下的转变下的表现。然后,我们为此建议的政策评估计划建立了中心限制定理类型保证。利用这种评估方案,我们进一步提出了一种新颖的学习算法,该算法能够学习一项对对抗性扰动和未知协变量转移的策略,并根据统一收敛理论的性能保证进行了绩效保证。最后,我们从经验上测试了合成数据集中提出的算法的有效性,并证明它提供了使用标准策略学习算法缺失的鲁棒性。我们通过在现实世界投票数据集的背景下提供了我们方法的全面应用来结束本文。
translated by 谷歌翻译
Concept drift primarily refers to an online supervised learning scenario when the relation between the input data and the target variable changes over time. Assuming a general knowledge of supervised learning in this paper we characterize adaptive learning process, categorize existing strategies for handling concept drift, overview the most representative, distinct and popular techniques and algorithms, discuss evaluation methodology of adaptive algorithms, and present a set of illustrative applications. The survey covers the different facets of concept drift in an integrated way to reflect on the existing scattered state-of-the-art. Thus, it aims at providing a comprehensive introduction to the concept drift adaptation for researchers, industry analysts and practitioners.
translated by 谷歌翻译
挖掘数据流姿势存在许多挑战,包括数据的连续和非静止性质,待处理的大量信息和限制计算资源。虽然在文献中提出了一些针对这个问题的监督解决方案,但大多数人都假定访问地面真理(以类标签的形式)是无限的,并且在更新学习系统时可以立即使用此类信息。这远非现实,因为必须考虑获取标签的基本成本。因此,需要解决流方案中实际真相要求的解决方案。在本文中,通过组合来自主动学习和自我标签的信息,提出了一种用于预算的挖水数据流的新框架。我们介绍了几种策略,可以利用智能实例选择和半监督程序,同时考虑到概念漂移的潜在存在。这种混合方法允许有效的探索和利用在现实标记预算中的流数据结构。由于我们的框架工作为包装器,因此它可以应用于不同的学习算法。实验研究,在具有各种类型的概念漂移的多样化现实数据流中进行的实验研究,证明了在处理对类标签的高度限制时拟议的策略的有用性。当一个人不能增加标签或更换低效分类器的预算时,呈现的混合方法尤其可行。我们为我们的战略提供了一套关于适用性领域的建议。
translated by 谷歌翻译
收购数据是机器学习的许多应用中的一项艰巨任务,只有一个人希望并且预期人口风险在单调上汇率增加(更好的性能)。事实证明,甚至对于最小化经验风险的最大限度的算法,甚至不令人惊讶的情况。在训练中的风险和不稳定的非单调行为表现出并出现在双重血统描述中的流行深度学习范式中。这些问题突出了目前对学习算法和泛化的理解缺乏了解。因此,追求这种行为的表征是至关重要的,这是至关重要的。在本文中,我们在弱假设下获得了一致和风险的单调算法,从而解决了一个打开问题Viering等。 2019关于如何避免风险曲线的非单调行为。我们进一步表明,风险单调性不一定以更糟糕的风险率的价格出现。为实现这一目标,我们推出了持有某些非I.I.D的独立利益的新经验伯恩斯坦的浓度不等式。鞅差异序列等进程。
translated by 谷歌翻译
我们在非参数二进制分类的一个对抗性训练问题之间建立了等价性,以及规范器是非识别范围功能的正则化风险最小化问题。由此产生的正常风险最小化问题允许在图像分析和基于图形学习中常常研究的$ L ^ 1 + $(非本地)$ \ Operatorvers {TV} $的精确凸松弛。这种重构揭示了丰富的几何结构,这反过来允许我们建立原始问题的最佳解决方案的一系列性能,包括存在最小和最大解决方案(以合适的意义解释),以及常规解决方案的存在(也以合适的意义解释)。此外,我们突出了对抗性训练和周长最小化问题的联系如何为涉及周边/总变化的正规风险最小化问题提供一种新颖的直接可解释的统计动机。我们的大部分理论结果与用于定义对抗性攻击的距离无关。
translated by 谷歌翻译
对抗性鲁棒性是各种现代机器学习应用中的关键财产。虽然它是最近几个理论研究的主题,但与对抗性稳健性有关的许多重要问题仍然是开放的。在这项工作中,我们研究了有关对抗对抗鲁棒性的贝叶斯最优性的根本问题。我们提供了一般的充分条件,可以保证贝叶斯最佳分类器的存在,以满足对抗性鲁棒性。我们的结果可以提供一种有用的工具,用于随后研究对抗性鲁棒性及其一致性的替代损失。这份稿件是“关于普通贝叶斯分类器的存在”在神经潮端中发表的延伸版本。原始纸张的结果不适用于一些非严格凸的规范。在这里,我们将结果扩展到所有可能的规范。
translated by 谷歌翻译
我们证明了由例如He等人提出的广泛使用的方法。(2015年)并使用梯度下降对最小二乘损失进行训练并不普遍。具体而言,我们描述了一大批一维数据生成分布,较高的概率下降只会发现优化景观的局部最小值不好,因为它无法将其偏离偏差远离其初始化,以零移动。。事实证明,在这些情况下,即使目标函数是非线性的,发现的网络也基本执行线性回归。我们进一步提供了数值证据,表明在实际情况下,对于某些多维分布而发生这种情况,并且随机梯度下降表现出相似的行为。我们还提供了有关初始化和优化器的选择如何影响这种行为的经验结果。
translated by 谷歌翻译
We define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave-one-out error. The methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real-valued function. We study the stability properties of large classes of learning algorithms such as regularization based algorithms. In particular we focus on Hilbert space regularization and Kullback-Leibler regularization. We demonstrate how to apply the results to SVM for regression and classification.1. For a qualitative discussion about sensitivity analysis with links to other resources see e.g. http://sensitivity-analysis.jrc.cec.eu.int/
translated by 谷歌翻译
到目前为止,可解释的人工智能(XAI)主要集中在静态学习方案上。我们对逐步采样数据的动态场景感兴趣,学习是以增量而不是批处理模式进行的。我们寻求有效的增量算法来计算特征重要性(FI)度量,具体来说,基于缺乏特征的特征边缘化的增量FI度量,类似于置换功能的特征重要性(PFI)。我们提出了一种称为IPFI的高效,模型不足的算法,以逐步估算此度量,并在包括概念漂移(概念漂移)在内的动态建模条件下进行估算。我们证明了关于期望和差异方面的近似质量的理论保证。为了验证我们的理论发现和与传统批处理PFI相比,我们的方法的疗效,我们对具有和没有概念漂移的基准数据进行了多项实验研究。
translated by 谷歌翻译
由于其出色的经验表现,随机森林是过去十年中使用的机器学习方法之一。然而,由于其黑框的性质,在许多大数据应用中很难解释随机森林的结果。量化各个特征在随机森林中的实用性可以大大增强其解释性。现有的研究表明,一些普遍使用的特征对随机森林的重要性措施遭受了偏见问题。此外,对于大多数现有方法,缺乏全面的规模和功率分析。在本文中,我们通过假设检验解决了问题,并提出了一个自由化特征 - 弥散性相关测试(事实)的框架,以评估具有偏见性属性的随机森林模型中给定特征的重要性,我们零假设涉及该特征是否与所有其他特征有条件地独立于响应。关于高维随机森林一致性的一些最新发展,对随机森林推断的这种努力得到了赋予的能力。在存在功能依赖性的情况下,我们的事实测试的香草版可能会遇到偏见问题。我们利用偏置校正的不平衡和调节技术。我们通过增强功率的功能转换将合奏的想法进一步纳入事实统计范围。在相当普遍的具有依赖特征的高维非参数模型设置下,我们正式确定事实可以提供理论上合理的随机森林具有P值,并通过非催化分析享受吸引人的力量。新建议的方法的理论结果和有限样本优势通过几个模拟示例和与Covid-19的经济预测应用进行了说明。
translated by 谷歌翻译
当训练概率分类器和校准时,可以容易地忽略校准损耗的所谓分组损耗组件。分组损失是指观察信息与实际校准运动中的信息之间的差距。我们调查分组损失与充足概念之间的关系,将Conoonotonics识别为充足的有用标准。我们重新审视Langford&Zadrozny(2005)的探测方法,发现它产生了减少分组损失的概率分类器的估计。最后,我们将Brier曲线讨论为支持培训的工具和“足够”概率分类器的校准。
translated by 谷歌翻译
The literature on machine learning in the context of data streams is vast and growing. However, many of the defining assumptions regarding data-stream learning tasks are too strong to hold in practice, or are even contradictory such that they cannot be met in the contexts of supervised learning. Algorithms are chosen and designed based on criteria which are often not clearly stated, for problem settings not clearly defined, tested in unrealistic settings, and/or in isolation from related approaches in the wider literature. This puts into question the potential for real-world impact of many approaches conceived in such contexts, and risks propagating a misguided research focus. We propose to tackle these issues by reformulating the fundamental definitions and settings of supervised data-stream learning with regard to contemporary considerations of concept drift and temporal dependence; and we take a fresh look at what constitutes a supervised data-stream learning task, and a reconsideration of algorithms that may be applied to tackle such tasks. Through and in reflection of this formulation and overview, helped by an informal survey of industrial players dealing with real-world data streams, we provide recommendations. Our main emphasis is that learning from data streams does not impose a single-pass or online-learning approach, or any particular learning regime; and any constraints on memory and time are not specific to streaming. Meanwhile, there exist established techniques for dealing with temporal dependence and concept drift, in other areas of the literature. For the data streams community, we thus encourage a shift in research focus, from dealing with often-artificial constraints and assumptions on the learning mode, to issues such as robustness, privacy, and interpretability which are increasingly relevant to learning in data streams in academic and industrial settings.
translated by 谷歌翻译
现在通常用于高风险设置,如医疗诊断,如医疗诊断,那么需要不确定量化,以避免后续模型失败。无分发的不确定性量化(无分布UQ)是用户友好的范式,用于为这种预测创建统计上严格的置信区间/集合。批判性地,间隔/集合有效而不进行分布假设或模型假设,即使具有最多许多DataPoints也具有显式保证。此外,它们适应输入的难度;当输入示例很困难时,不确定性间隔/集很大,信号传达模型可能是错误的。在没有多大的工作和没有再培训的情况下,可以在任何潜在的算法(例如神经网络)上使用无分​​发方法,以产生置信度集,以便包含用户指定概率,例如90%。实际上,这些方法易于理解和一般,应用于计算机视觉,自然语言处理,深度加强学习等领域出现的许多现代预测问题。这种实践介绍是针对对无需统计学家的免费UQ的实际实施感兴趣的读者。我们通过实际的理论和无分发UQ的应用领导读者,从保形预测开始,并使无关的任何风险的分布控制,如虚假发现率,假阳性分布检测,等等。我们将包括Python中的许多解释性插图,示例和代码样本,具有Pytorch语法。目标是提供读者对无分配UQ的工作理解,使它们能够将置信间隔放在算法上,其中包含一个自包含的文档。
translated by 谷歌翻译
我们基于电子价值开发假设检测理论,这是一种与p值不同的证据,允许毫不费力地结合来自常见场景中的几项研究的结果,其中决定执行新研究可能取决于以前的结果。基于E-V值的测试是安全的,即它们在此类可选的延续下保留I型错误保证。我们将增长速率最优性(GRO)定义为可选的连续上下文中的电力模拟,并且我们展示了如何构建GRO E-VARIABLE,以便为复合空缺和替代,强调模型的常规测试问题,并强调具有滋扰参数的模型。 GRO E值采取具有特殊前瞻的贝叶斯因子的形式。我们使用几种经典示例说明了该理论,包括一个样本安全T检验(其中右哈尔前方的右手前锋为GE)和2x2差价表(其中GRE之前与标准前沿不同)。分享渔业,奈曼和杰弗里斯·贝叶斯解释,电子价值观和相应的测试可以提供所有三所学校的追随者可接受的方法。
translated by 谷歌翻译
Existing generalization bounds fail to explain crucial factors that drive generalization of modern neural networks. Since such bounds often hold uniformly over all parameters, they suffer from over-parametrization, and fail to account for the strong inductive bias of initialization and stochastic gradient descent. As an alternative, we propose a novel optimal transport interpretation of the generalization problem. This allows us to derive instance-dependent generalization bounds that depend on the local Lipschitz regularity of the earned prediction function in the data space. Therefore, our bounds are agnostic to the parametrization of the model and work well when the number of training samples is much smaller than the number of parameters. With small modifications, our approach yields accelerated rates for data on low-dimensional manifolds, and guarantees under distribution shifts. We empirically analyze our generalization bounds for neural networks, showing that the bound values are meaningful and capture the effect of popular regularization methods during training.
translated by 谷歌翻译
State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels. This tendency to memorize the labels of the training data is not explained by existing theoretical analyses. Memorization of the training data also presents significant privacy risks when the training data contains sensitive personal information and thus it is important to understand whether such memorization is necessary for accurate learning.We provide the first conceptual explanation and a theoretical model for this phenomenon. Specifically, we demonstrate that for natural data distributions memorization of labels is necessary for achieving closeto-optimal generalization error. Crucially, even labels of outliers and noisy labels need to be memorized. The model is motivated and supported by the results of several recent empirical works. In our model, data is sampled from a mixture of subpopulations and our results show that memorization is necessary whenever the distribution of subpopulation frequencies is long-tailed. Image and text data is known to be long-tailed and therefore our results establish a formal link between these empirical phenomena. Our results allow to quantify the cost of limiting memorization in learning and explain the disparate effects that privacy and model compression have on different subgroups.
translated by 谷歌翻译
尽管U统计量在现代概率和统计学中存在着无处不在的,但其在依赖框架中的非反应分析可能被忽略了。在最近的一项工作中,已经证明了对统一的马尔可夫链的U级统计数据的新浓度不平等。在本文中,我们通过在三个不同的研究领域中进一步推动了当前知识状态,将这一理论突破付诸实践。首先,我们为使用MCMC方法估算痕量类积分运算符光谱的新指数不平等。新颖的是,这种结果适用于具有正征和负征值的内核,据我们所知,这是新的。此外,我们研究了使用成对损失函数和马尔可夫链样品的在线算法的概括性能。我们通过展示如何从任何在线学习者产生的假设序列中提取低风险假设来提供在线到批量转换结果。我们最终对马尔可夫链的不变度度量的密度进行了拟合优度测试的非反应分析。我们确定了一些类别的替代方案,基于$ L_2 $距离的测试具有规定的功率。
translated by 谷歌翻译