异常检测是识别数据中意外事件或AB差异的过程,并且已在许多不同领域(例如系统监控,欺诈检测,医疗保健,入侵检测等)应用。提供实时,轻量级和主动的异常情况对于人类干预和领域知识的时间序列的检测,由于它减少了人类的努力,并在发生灾难性事件发生之前可以进行适当的对策,因此既不具有人为干预和领域知识。据我们所知,Repad(实时主动的异常检测算法)是所有上述特征的通用方法。为了实现实时和轻质检测,重新使用长期记忆(LSTM)来检测每个即将到来的数据点是否基于短期历史数据点是异常的。但是,目前尚不清楚不同数量的历史数据点如何影响续期的性能。因此,在本文中,我们通过引入一组涵盖新颖的检测准确性措施,时间效率,准备和资源消耗等的绩效指标来研究不同历史数据对重新播放的影响。进行时间序列数据集以评估不同情况下的重新播放,并提出和讨论实验结果。
translated by 谷歌翻译
Anomaly detection is an active research topic in many different fields such as intrusion detection, network monitoring, system health monitoring, IoT healthcare, etc. However, many existing anomaly detection approaches require either human intervention or domain knowledge, and may suffer from high computation complexity, consequently hindering their applicability in real-world scenarios. Therefore, a lightweight and ready-to-go approach that is able to detect anomalies in real-time is highly sought-after. Such an approach could be easily and immediately applied to perform time series anomaly detection on any commodity machine. The approach could provide timely anomaly alerts and by that enable appropriate countermeasures to be undertaken as early as possible. With these goals in mind, this paper introduces ReRe, which is a Real-time Ready-to-go proactive Anomaly Detection algorithm for streaming time series. ReRe employs two lightweight Long Short-Term Memory (LSTM) models to predict and jointly determine whether or not an upcoming data point is anomalous based on short-term historical data points and two long-term self-adaptive thresholds. Experiments based on real-world time-series datasets demonstrate the good performance of ReRe in real-time anomaly detection without requiring human intervention or domain knowledge.
translated by 谷歌翻译
During the past decade, many anomaly detection approaches have been introduced in different fields such as network monitoring, fraud detection, and intrusion detection. However, they require understanding of data pattern and often need a long off-line period to build a model or network for the target data. Providing real-time and proactive anomaly detection for streaming time series without human intervention and domain knowledge is highly valuable since it greatly reduces human effort and enables appropriate countermeasures to be undertaken before a disastrous damage, failure, or other harmful event occurs. However, this issue has not been well studied yet. To address it, this paper proposes RePAD, which is a Real-time Proactive Anomaly Detection algorithm for streaming time series based on Long Short-Term Memory (LSTM). RePAD utilizes short-term historic data points to predict and determine whether or not the upcoming data point is a sign that an anomaly is likely to happen in the near future. By dynamically adjusting the detection threshold over time, RePAD is able to tolerate minor pattern change in time series and detect anomalies either proactively or on time. Experiments based on two time series datasets collected from the Numenta Anomaly Benchmark demonstrate that RePAD is able to proactively detect anomalies and provide early warnings in real time without human intervention and domain knowledge.
translated by 谷歌翻译
时间序列的异常提供了各个行业的关键方案的见解,从银行和航空航天到信息技术,安全和医学。但是,由于异常的定义,经常缺乏标签以及此类数据中存在的极为复杂的时间相关性,因此识别时间序列数据中的异常尤其具有挑战性。LSTM自动编码器是基于长期短期内存网络的异常检测的编码器传统方案,该方案学会重建时间序列行为,然后使用重建错误来识别异常。我们将Denoising Architecture作为对该LSTM编码模型模型的补充,并研究其对现实世界以及人为生成的数据集的影响。我们证明了所提出的体系结构既提高了准确性和训练速度,从而使LSTM自动编码器更有效地用于无监督的异常检测任务。
translated by 谷歌翻译
Unsupervised anomaly detection in time-series has been extensively investigated in the literature. Notwithstanding the relevance of this topic in numerous application fields, a complete and extensive evaluation of recent state-of-the-art techniques is still missing. Few efforts have been made to compare existing unsupervised time-series anomaly detection methods rigorously. However, only standard performance metrics, namely precision, recall, and F1-score are usually considered. Essential aspects for assessing their practical relevance are therefore neglected. This paper proposes an original and in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series. Instead of relying solely on standard performance metrics, additional yet informative metrics and protocols are taken into account. In particular, (1) more elaborate performance metrics specifically tailored for time-series are used; (2) the model size and the model stability are studied; (3) an analysis of the tested approaches with respect to the anomaly type is provided; and (4) a clear and unique protocol is followed for all experiments. Overall, this extensive analysis aims to assess the maturity of state-of-the-art time-series anomaly detection, give insights regarding their applicability under real-world setups and provide to the community a more complete evaluation protocol.
translated by 谷歌翻译
迁移率和加热部门的连续电气化将对分布网格运行引入新的挑战。不协调的灵活单元激活,例如,电动车辆同时充电作为对价格信号的反应,可以系统地触发变压器或线路保护。实时识别这种快速升高的灵活性激活将允许抵消以避免潜在的社会和财务成本。在这项工作中,提出了一种用于识别快速升高灵活性激活事件的新型数据处理流水线。管道结合了无监督事件检测和开放式分类的技术。实际负载数据的系统评估演示了所提出的管道的主要构建块可以通过满足分布式事件检测架构中应用的重要要求的方法来实现。为了检测灵活性激活事件,识别了上部性能限制。此外,证明了与广泛应用的闭合分类器相比,用于分类的开放式分类器的应用可以提高性能。
translated by 谷歌翻译
The detection of anomalies in time series data is crucial in a wide range of applications, such as system monitoring, health care or cyber security. While the vast number of available methods makes selecting the right method for a certain application hard enough, different methods have different strengths, e.g. regarding the type of anomalies they are able to find. In this work, we compare six unsupervised anomaly detection methods with different complexities to answer the questions: Are the more complex methods usually performing better? And are there specific anomaly types that those method are tailored to? The comparison is done on the UCR anomaly archive, a recent benchmark dataset for anomaly detection. We compare the six methods by analyzing the experimental results on a dataset- and anomaly type level after tuning the necessary hyperparameter for each method. Additionally we examine the ability of individual methods to incorporate prior knowledge about the anomalies and analyse the differences of point-wise and sequence wise features. We show with broad experiments, that the classical machine learning methods show a superior performance compared to the deep learning methods across a wide range of anomaly types.
translated by 谷歌翻译
今天的网络世界难以多变量。在极端品种中收集的指标需要多变量算法以正确检测异常。然而,基于预测的算法,如被广泛证明的方法,通常在数据集中进行次优或不一致。一个关键的常见问题是他们努力成为一个尺寸适合的,但异常在自然中是独特的。我们提出了一种裁定到这种区别的方法。提出FMUAD - 一种基于预测,多方面,无监督的异常检测框架。FMUAD明确,分别捕获异常类型的签名性状 - 空间变化,时间变化和相关变化 - 与独立模块。然后,模块共同学习最佳特征表示,这是非常灵活和直观的,与类别中的大多数其他模型不同。广泛的实验表明我们的FMUAD框架始终如一地优于其他最先进的预测的异常探测器。
translated by 谷歌翻译
Anomaly detection on time series data is increasingly common across various industrial domains that monitor metrics in order to prevent potential accidents and economic losses. However, a scarcity of labeled data and ambiguous definitions of anomalies can complicate these efforts. Recent unsupervised machine learning methods have made remarkable progress in tackling this problem using either single-timestamp predictions or time series reconstructions. While traditionally considered separately, these methods are not mutually exclusive and can offer complementary perspectives on anomaly detection. This paper first highlights the successes and limitations of prediction-based and reconstruction-based methods with visualized time series signals and anomaly scores. We then propose AER (Auto-encoder with Regression), a joint model that combines a vanilla auto-encoder and an LSTM regressor to incorporate the successes and address the limitations of each method. Our model can produce bi-directional predictions while simultaneously reconstructing the original time series by optimizing a joint objective function. Furthermore, we propose several ways of combining the prediction and reconstruction errors through a series of ablation studies. Finally, we compare the performance of the AER architecture against two prediction-based methods and three reconstruction-based methods on 12 well-known univariate time series datasets from NASA, Yahoo, Numenta, and UCR. The results show that AER has the highest averaged F1 score across all datasets (a 23.5% improvement compared to ARIMA) while retaining a runtime similar to its vanilla auto-encoder and regressor components. Our model is available in Orion, an open-source benchmarking tool for time series anomaly detection.
translated by 谷歌翻译
存在几种数据驱动方法,使我们的模型时间序列数据能够包括传统的基于回归的建模方法(即,Arima)。最近,在时间序列分析和预测的背景下介绍和探索了深度学习技术。询问的主要研究问题是在预测时间序列数据中的深度学习技术中的这些变化的性能。本文比较了两个突出的深度学习建模技术。比较了经常性的神经网络(RNN)长的短期记忆(LSTM)和卷积神经网络(CNN)基于基于TCN的时间卷积网络(TCN),并报告了它们的性能和训练时间。根据我们的实验结果,两个建模技术都表现了相当具有基于TCN的模型优于LSTM略微。此外,基于CNN的TCN模型比基于RNN的LSTM模型更快地构建了稳定的模型。
translated by 谷歌翻译
作为在Internet交换路由到达性信息的默认协议,边界网关协议(BGP)的流量异常行为与互联网异常事件密切相关。 BGP异常检测模型通过其实时监控和警报功能确保互联网上的稳定路由服务。以前的研究要么专注于特征选择问题或数据中的内存特征,同时忽略特征之间的关系和特征中的精确时间相关(无论是长期还是短期依赖性)。在本文中,我们提出了一种用于捕获来自BGP更新流量的异常行为的多视图模型,其中使用黄土(STL)方法的季节性和趋势分解来减少原始时间序列数据中的噪声和图表网络中的噪声(GAT)用于分别发现功能中的特征关系和时间相关性。我们的结果优于异常检测任务的最先进的方法,平均F1分别在平衡和不平衡数据集上得分高达96.3%和93.2%。同时,我们的模型可以扩展以对多个异常进行分类并检测未知事件。
translated by 谷歌翻译
时间序列异常检测已被认为对现实世界系统的可靠和有效运行至关重要。基于对异常特征的各种假设,已经开发了许多异常检测方法。但是,由于现实世界数据的复杂性质,时间序列中的不同异常通常具有支持不同异常假设的不同曲线。这使得很难找到一个可以始终如一的其他模型的异常检测器。在这项工作中,为了利用不同基本模型的好处,我们提出了一个基于增强学习的模型选择框架。具体而言,我们首先学习了不同异常检测模型的池,然后利用强化学习从这些基本模型中动态选择候选模型。关于现实世界数据的实验表明,就整体绩效而言,提出的策略确实可以超过所有基线模型。
translated by 谷歌翻译
分层时间记忆(HTM)是一种无监督的学习算法,其灵感来自Neocortex的功能,可用于连续处理流数据并检测异常,而无需大量数据进行培训,也不需要标记数据。 HTM还能够从样本中不断学习,提供一个始终是关于观察的模型。这些特性使HTM特别适用于支持云系统中的在线故障预测,这是具有动态变化行为的系统必须监视以预测问题。本文介绍了在故障预测的背景下评估HTM的第一个系统研究。考虑到72个HTM配置所获得的HTM配置到Clearwater云系统中引入的12种不同类型的故障表明,HTM可以帮助预测具有足够有效性(F-Measure = 0.76)的失败,代表有趣的实际替代方案(半 - )监督算法。
translated by 谷歌翻译
近年来,深度学习的时间序列增加了。对于时间序列的异常检测方案,例如金融,物联网,数据中心操作等,时间序列通常会根据各种外部因素显示非常灵活的基线。异常通过躺在远离基线的情况下揭示自己。但是,由于一些挑战,包括基线转换,缺乏标签,噪声干扰,流数据中的实时检测,可解释性等。从时间序列,即深基线网络(DBLN)。通过使用此深层网络,我们可以轻松地定位基线位置,然后提供可靠且可解释的异常检测结果。对合成和公共现实世界数据集的经验评估表明,我们纯粹的无监督算法与最新方法相比,实现了卓越的性能,并且具有良好的实际应用。
translated by 谷歌翻译
给定传感器读数随着时间的推移从电网上,我们如何在发生异常时准确地检测?实现这一目标的关键部分是使用电网传感器网络在电网上实时地在实时检测到自然故障或恶意的任何不寻常的事件。行业中现有的坏数据探测器缺乏鲁布布利地检测广泛类型的异常,特别是由于新兴网络攻击而造成的复杂性,因为它们一次在网格的单个测量快照上运行。新的ML方法更广泛适用,但通常不会考虑拓扑变化对传感器测量的影响,因此无法适应历史数据中的定期拓扑调整。因此,我们向DynWatch,基于域知识和拓扑知识算法用于使用动态网格上的传感器进行异常检测。我们的方法准确,优于实验中的现有方法20%以上(F-Measure);快速,在60K +分支机用中的每次传感器上平均运行小于1.7ms,使用笔记本电脑,并在图表的大小上线性缩放。
translated by 谷歌翻译
Aiot技术的最新进展导致利用机器学习算法来检测网络物理系统(CPS)的操作失败的越来越受欢迎。在其基本形式中,异常检测模块从物理工厂监控传感器测量和致动器状态,并检测这些测量中的异常以识别异常操作状态。然而,由于该模型必须在存在高度复杂的系统动态和未知量的传感器噪声的情况下准确地检测异常,构建有效的异常检测模型是挑战性的。在这项工作中,我们提出了一种新的时序序列异常检测方法,称为神经系统识别和贝叶斯滤波(NSIBF),其中特制的神经网络架构被构成系统识别,即捕获动态状态空间中CP的动态模型;然后,通过跟踪系统的隐藏状态的不确定性随着时间的推移,自然地施加贝叶斯滤波算法的顶部。我们提供定性的和定量实验,并在合成和三个现实世界CPS数据集上具有所提出的方法,表明NSIBF对最先进的方法比较了对CPS中异常检测的最新方法。
translated by 谷歌翻译
在能源系统的数字化中,传感器和智能电表越来越多地用于监视生产,运行和需求。基于智能电表数据的异常检测对于在早期阶段识别潜在的风险和异常事件至关重要,这可以作为及时启动适当动作和改善管理的参考。但是,来自能源系统的智能电表数据通常缺乏标签,并且包含噪声和各种模式,而没有明显的周期性。同时,在不同的能量场景中对异常的模糊定义和高度复杂的时间相关性对异常检测构成了巨大的挑战。许多传统的无监督异常检测算法(例如基于群集或基于距离的模型)对噪声不强大,也不完全利用时间序列中的时间依赖性以及在多个变量(传感器)中的其他依赖关系。本文提出了一种基于带有注意机制的变异复发自动编码器的无监督异常检测方法。凭借来自智能电表的“肮脏”数据,我们的方法预示了缺失的值和全球异常,以在训练中缩小其贡献。本文与基于VAE的基线方法和其他四种无监督的学习方法进行了定量比较,证明了其有效性和优势。本文通过一项实际案例研究进一步验证了所提出的方法,该研究方法是检测工业加热厂的供水温度异常。
translated by 谷歌翻译
Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of deep anomaly detection in time series across various application domains in recent years. It finally summarises open issues in research and challenges faced while adopting deep anomaly detection models.
translated by 谷歌翻译
在智能交通系统中,交通拥堵异常检测至关重要。运输机构的目标有两个方面:监视感兴趣领域的一般交通状况,并在异常拥堵状态下定位道路细分市场。建模拥塞模式可以实现这些目标,以实现全市道路的目标,相当于学习多元时间序列(MTS)的分布。但是,现有作品要么不可伸缩,要么无法同时捕获MTS中的空间信息。为此,我们提出了一个由数据驱动的生成方法组成的原则性和全面的框架,该方法可以执行可拖动的密度估计来检测流量异常。我们的方法在特征空间中的第一群段段,然后使用条件归一化流以在无监督的设置下在群集级别识别异常的时间快照。然后,我们通过在异常群集上使用内核密度估计器来识别段级别的异常。关于合成数据集的广泛实验表明,我们的方法在召回和F1得分方面显着优于几种最新的拥塞异常检测和诊断方法。我们还使用生成模型来采样标记的数据,该数据可以在有监督的环境中训练分类器,从而减轻缺乏在稀疏设置中进行异常检测的标记数据。
translated by 谷歌翻译
多元时间序列中的异常检测在监视各种现实世界系统(例如IT系统运营或制造业)的行为方面起着重要作用。先前的方法对关节分布进行建模,而无需考虑多元时间序列的潜在机制,使它们变得复杂且饥饿。在本文中,我们从因果的角度提出异常检测问题,并将异常视为未遵循常规因果机制来生成多元数据的情况。然后,我们提出了一种基于因果关系的异常检测方法,该方法首先从数据中学习因果结构,然后渗透实例是否是相对于局部因果机制的异常,以从其直接原因产生每个变量,其条件分布可以直接估计从数据。鉴于因果系统的模块化特性,原始问题被分为一系列单独的低维异常检测问题,因此可以直接识别出异常的地方。我们通过模拟和公共数据集以及有关现实世界中AIOPS应用程序的案例研究评估我们的方法,显示其功效,鲁棒性和实际可行性。
translated by 谷歌翻译