Levenberg-Marquardt(LM)优化算法已广泛用于解决机器学习问题。文学评论表明,当网络中的权重数不超过几百个时,LM对中等函数近似问题的LM非常强大而有效。相比之下,在处理模式识别或分类问题时,LM似乎并不表现,并且当网络变大时效率低(例如,超过500重量)。在本文中,我们利用一些现实世界飞机数据集利用LM算法的真正力量。在这些数据集上,大多数其他常用的优化器无法检测到飞机发动机的变化条件引起的异常。数据集的具有挑战性是时间序列数据的突然变化。我们发现LM优化器具有更好的近似突然变化的能力,并检测除其他优化器的异常。我们比较LM和几个其他优化器的这种异常/更改检测问题的性能。我们基于一系列措施评估了相对性能,包括网络复杂性(即权重的数量),拟合精度,拟合,培训时间,GPU和内存要求等的使用等措施。我们还讨论了Matlab中强大的LM实现问题Tensorflow用于推广LM算法的更多流行使用以及LM优化器的潜在使用进行大规模问题。
translated by 谷歌翻译
燃气轮机发动机是复杂的机器,通常产生大量数据,并且需要仔细监控,以允许具有成本效益的预防性维护。在航空航天应用中,将所有测量数据返回到地面是昂贵的,通常会导致有用,高值,要丢弃的数据。因此,在实时检测,优先级和返回有用数据的能力是至关重要的。本文提出了由卷积神经网络常态模型描述的系统输出测量,实时优先考虑预防性维护决策者。由于燃气轮机发动机时变行为的复杂性,导出精确的物理模型难以困难,并且通常导致预测精度低的模型和与实时执行不相容。数据驱动的建模是一种理想的替代方案,生产高精度,资产特定模型,而无需从第一原理推导。我们提出了一种用于在线检测和异常数据的优先级的数据驱动系统。通过集成到深神经预测模型中的不确定管理,避免了偏离新的操作条件的数据评估。测试是对实际和合成数据进行的,显示对真实和合成故障的敏感性。该系统能够在低功耗嵌入式硬件上实时运行,目前正在部署Rolls-Royce Pearl 15发动机飞行试验。
translated by 谷歌翻译
时间序列的异常提供了各个行业的关键方案的见解,从银行和航空航天到信息技术,安全和医学。但是,由于异常的定义,经常缺乏标签以及此类数据中存在的极为复杂的时间相关性,因此识别时间序列数据中的异常尤其具有挑战性。LSTM自动编码器是基于长期短期内存网络的异常检测的编码器传统方案,该方案学会重建时间序列行为,然后使用重建错误来识别异常。我们将Denoising Architecture作为对该LSTM编码模型模型的补充,并研究其对现实世界以及人为生成的数据集的影响。我们证明了所提出的体系结构既提高了准确性和训练速度,从而使LSTM自动编码器更有效地用于无监督的异常检测任务。
translated by 谷歌翻译
存在几种数据驱动方法,使我们的模型时间序列数据能够包括传统的基于回归的建模方法(即,Arima)。最近,在时间序列分析和预测的背景下介绍和探索了深度学习技术。询问的主要研究问题是在预测时间序列数据中的深度学习技术中的这些变化的性能。本文比较了两个突出的深度学习建模技术。比较了经常性的神经网络(RNN)长的短期记忆(LSTM)和卷积神经网络(CNN)基于基于TCN的时间卷积网络(TCN),并报告了它们的性能和训练时间。根据我们的实验结果,两个建模技术都表现了相当具有基于TCN的模型优于LSTM略微。此外,基于CNN的TCN模型比基于RNN的LSTM模型更快地构建了稳定的模型。
translated by 谷歌翻译
现代高性能计算(HPC)系统的复杂性日益增加,需要引入自动化和数据驱动的方法,以支持系统管理员为增加系统可用性的努力。异常检测是改善可用性不可或缺的一部分,因为它减轻了系统管理员的负担,并减少了异常和解决方案之间的时间。但是,对当前的最新检测方法进行了监督和半监督,因此它们需要具有异常的人体标签数据集 - 在生产HPC系统中收集通常是不切实际的。基于聚类的无监督异常检测方法,旨在减轻准确的异常数据的需求,到目前为止的性能差。在这项工作中,我们通过提出RUAD来克服这些局限性,RUAD是一种新型的无监督异常检测模型。 Ruad比当前的半监督和无监督的SOA方法取得了更好的结果。这是通过考虑数据中的时间依赖性以及在模型体系结构中包括长短期限内存单元的实现。提出的方法是根据tier-0系统(带有980个节点的Cineca的Marconi100的完整历史)评估的。 RUAD在半监督训练中达到曲线(AUC)下的区域(AUC)为0.763,在无监督的训练中达到了0.767的AUC,这改进了SOA方法,在半监督训练中达到0.747的AUC,无需训练的AUC和0.734的AUC在无处不在的AUC中提高了AUC。训练。它还大大优于基于聚类的当前SOA无监督的异常检测方法,其AUC为0.548。
translated by 谷歌翻译
智能制造系统以越来越多的速度部署,因为它们能够解释各种各样的感知信息并根据系统观察收集的知识采取行动。在许多情况下,智能制造系统的主要目标是快速检测(或预期)失败以降低运营成本并消除停机时间。这通常归结为检测从系统中获取的传感器日期内的异常。智能制造应用域构成了某些显着的技术挑战。特别是,通常有多种具有不同功能和成本的传感器。传感器数据特性随环境或机器的操作点而变化,例如电动机的RPM。因此,必须在工作点附近校准异常检测过程。在本文中,我们分析了从制造测试台部署的传感器中的四个数据集。我们评估了几种基于传统和ML的预测模型的性能,以预测传感器数据的时间序列。然后,考虑到一种传感器的稀疏数据,我们从高数据速率传感器中执行传输学习来执行缺陷类型分类。综上所述,我们表明可以实现预测性故障分类,从而为预测维护铺平了道路。
translated by 谷歌翻译
During the past decade, many anomaly detection approaches have been introduced in different fields such as network monitoring, fraud detection, and intrusion detection. However, they require understanding of data pattern and often need a long off-line period to build a model or network for the target data. Providing real-time and proactive anomaly detection for streaming time series without human intervention and domain knowledge is highly valuable since it greatly reduces human effort and enables appropriate countermeasures to be undertaken before a disastrous damage, failure, or other harmful event occurs. However, this issue has not been well studied yet. To address it, this paper proposes RePAD, which is a Real-time Proactive Anomaly Detection algorithm for streaming time series based on Long Short-Term Memory (LSTM). RePAD utilizes short-term historic data points to predict and determine whether or not the upcoming data point is a sign that an anomaly is likely to happen in the near future. By dynamically adjusting the detection threshold over time, RePAD is able to tolerate minor pattern change in time series and detect anomalies either proactively or on time. Experiments based on two time series datasets collected from the Numenta Anomaly Benchmark demonstrate that RePAD is able to proactively detect anomalies and provide early warnings in real time without human intervention and domain knowledge.
translated by 谷歌翻译
无监督的异常检测对于未来在大型数据集中搜索稀有现象的分析可能至关重要,例如在LHC收集的。为此,我们介绍了一个受到物理启发的变量自动编码器(VAE)体系结构,该体系结构在LHC奥运会机器学习挑战数据集中竞争性和稳健性。我们证明了如何将某些物理可观察物直接嵌入VAE潜在空间中,同时使分类器显然是不可知的,可以帮助识别和表征测得的光谱中的特征,这是由于数据集中存在异常而引起的。
translated by 谷歌翻译
我们考虑为移动机器人构建视觉异常检测系统的问题。标准异常检测模型是使用仅由非异常数据组成的大型数据集训练的。但是,在机器人技术应用中,通常可以使用(可能很少)的异常示例。我们解决了利用这些数据以通过与Real-NVP损失共同使辅助外离群损失损失共同使实际NVP异常检测模型的性能提高性能的问题。我们在新的数据集(作为补充材料)上进行定量实验,该数据集在室内巡逻方案中设计为异常检测。在不连接测试集中,我们的方法优于替代方案,并表明即使少数异常框架也可以实现重大的性能改进。
translated by 谷歌翻译
Recently, there has been a significant amount of interest in satellite telemetry anomaly detection (AD) using neural networks (NN). For AD purposes, the current approaches focus on either forecasting or reconstruction of the time series, and they cannot measure the level of reliability or the probability of correct detection. Although the Bayesian neural network (BNN)-based approaches are well known for time series uncertainty estimation, they are computationally intractable. In this paper, we present a tractable approximation for BNN based on the Monte Carlo (MC) dropout method for capturing the uncertainty in the satellite telemetry time series, without sacrificing accuracy. For time series forecasting, we employ an NN, which consists of several Long Short-Term Memory (LSTM) layers followed by various dense layers. We employ the MC dropout inside each LSTM layer and before the dense layers for uncertainty estimation. With the proposed uncertainty region and by utilizing a post-processing filter, we can effectively capture the anomaly points. Numerical results show that our proposed time series AD approach outperforms the existing methods from both prediction accuracy and AD perspectives.
translated by 谷歌翻译
A Digital Twin (DT) is a simulation of a physical system that provides information to make decisions that add economic, social or commercial value. The behaviour of a physical system changes over time, a DT must therefore be continually updated with data from the physical systems to reflect its changing behaviour. For resource-constrained systems, updating a DT is non-trivial because of challenges such as on-board learning and the off-board data transfer. This paper presents a framework for updating data-driven DTs of resource-constrained systems geared towards system health monitoring. The proposed solution consists of: (1) an on-board system running a light-weight DT allowing the prioritisation and parsimonious transfer of data generated by the physical system; and (2) off-board robust updating of the DT and detection of anomalous behaviours. Two case studies are considered using a production gas turbine engine system to demonstrate the digital representation accuracy for real-world, time-varying physical systems.
translated by 谷歌翻译
从限制黑暗部门的暗物质颗粒的生产可能导致许多新颖的实验签名。根据理论的细节,质子 - 质子碰撞中的黑暗夸克生产可能导致颗粒的半衰期:黑暗强度的准直喷雾,其中颗粒碰撞器实验只有一些。实验签名的特征在于,具有与喷射器的可见部件相结合的重建缺失的动量。这种复杂的拓扑对检测器效率低下和错误重建敏感,从而产生人为缺失的势头。通过这项工作,我们提出了一种信号不可知的策略来拒绝普通喷射,并通过异常检测技术鉴定半衰期喷射。具有喷射子结构变量的深度神经自动化器网络作为输入,证明了对分析异常喷射的非常有用。该研究重点介绍了半意射流签名;然而,该技术可以适用于任何新的物理模型,该模型预测来自非SM粒子的喷射器的签名。
translated by 谷歌翻译
装有传感器,执行器和电子控制单元(ECU)的现代车辆可以分为几个称为功能工作组(FWGS)的操作子系统。这些FWG的示例包括发动机系统,变速箱,燃油系统,制动器等。每个FWG都有相关的传感器通道,可以衡量车辆操作条件。这种丰富的数据环境有利于预测维护(PDM)技术的开发。削弱各种PDM技术的是需要强大的异常检测模型,该模型可以识别出明显偏离大多数数据的事件或观察结果,并且不符合正常车辆操作行为的明确定义的概念。在本文中,我们介绍了车辆性能,可靠性和操作(VEPRO)数据集,并使用它来创建一种基于多阶段的异常检测方法。利用时间卷积网络(TCN),我们的异常检测系统可以达到96%的检测准确性,并准确预测91%的真实异常。当利用来自多个FWG的传感器通道时,我们的异常检测系统的性能会改善。
translated by 谷歌翻译
Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of deep anomaly detection in time series across various application domains in recent years. It finally summarises open issues in research and challenges faced while adopting deep anomaly detection models.
translated by 谷歌翻译
我们描述了作为黑暗机器倡议和LES Houches 2019年物理学研讨会进行的数据挑战的结果。挑战的目标是使用无监督机器学习算法检测LHC新物理学的信号。首先,我们提出了如何实现异常分数以在LHC搜索中定义独立于模型的信号区域。我们定义并描述了一个大型基准数据集,由> 10亿美元的Muton-Proton碰撞,其中包含> 10亿美元的模拟LHC事件组成。然后,我们在数据挑战的背景下审查了各种异常检测和密度估计算法,我们在一组现实分析环境中测量了它们的性能。我们绘制了一些有用的结论,可以帮助开发无监督的新物理搜索在LHC的第三次运行期间,并为我们的基准数据集提供用于HTTPS://www.phenomldata.org的未来研究。重现分析的代码在https://github.com/bostdiek/darkmachines-unsupervisedChallenge提供。
translated by 谷歌翻译
在2015年和2019年之间,地平线的成员2020年资助的创新培训网络名为“Amva4newphysics”,研究了高能量物理问题的先进多变量分析方法和统计学习工具的定制和应用,并开发了完全新的。其中许多方法已成功地用于提高Cern大型Hadron撞机的地图集和CMS实验所执行的数据分析的敏感性;其他几个人,仍然在测试阶段,承诺进一步提高基本物理参数测量的精确度以及新现象的搜索范围。在本文中,在研究和开发的那些中,最相关的新工具以及对其性能的评估。
translated by 谷歌翻译
A new Lossy Causal Temporal Convolutional Neural Network Autoencoder for anomaly detection is proposed in this work. Our framework uses a rate-distortion loss and an entropy bottleneck to learn a compressed latent representation for the task. The main idea of using a rate-distortion loss is to introduce representation flexibility that ignores or becomes robust to unlikely events with distinctive patterns, such as anomalies. These anomalies manifest as unique distortion features that can be accurately detected in testing conditions. This new architecture allows us to train a fully unsupervised model that has high accuracy in detecting anomalies from a distortion score despite being trained with some portion of unlabelled anomalous data. This setting is in stark contrast to many of the state-of-the-art unsupervised methodologies that require the model to be only trained on "normal data". We argue that this partially violates the concept of unsupervised training for anomaly detection as the model uses an informed decision that selects what is normal from abnormal for training. Additionally, there is evidence to suggest it also effects the models ability at generalisation. We demonstrate that models that succeed in the paradigm where they are only trained on normal data fail to be robust when anomalous data is injected into the training. In contrast, our compression-based approach converges to a robust representation that tolerates some anomalous distortion. The robust representation achieved by a model using a rate-distortion loss can be used in a more realistic unsupervised anomaly detection scheme.
translated by 谷歌翻译
无监督的异常检测旨在通过在正常数据上训练来建立模型以有效地检测看不见的异常。尽管以前的基于重建的方法取得了富有成效的进展,但由于两个危急挑战,他们的泛化能力受到限制。首先,训练数据集仅包含正常模式,这限制了模型泛化能力。其次,现有模型学到的特征表示通常缺乏代表性,妨碍了保持正常模式的多样性的能力。在本文中,我们提出了一种称为自适应存储器网络的新方法,具有自我监督的学习(AMSL)来解决这些挑战,并提高无监督异常检测中的泛化能力。基于卷积的AutoEncoder结构,AMSL包含一个自我监督的学习模块,以学习一般正常模式和自适应内存融合模块来学习丰富的特征表示。四个公共多变量时间序列数据集的实验表明,与其他最先进的方法相比,AMSL显着提高了性能。具体而言,在具有9亿个样本的最大帽睡眠阶段检测数据集上,AMSL以精度和F1分数\ TextBF {4} \%+优于第二个最佳基线。除了增强的泛化能力之外,AMSL还针对输入噪声更加强大。
translated by 谷歌翻译
System logs are a common source of monitoring data for analyzing computing systems' behavior. Due to the complexity of modern computing systems and the large size of collected monitoring data, automated analysis mechanisms are required. Numerous machine learning and deep learning methods are proposed to address this challenge. However, due to the existence of sensitive data in system logs their analysis and storage raise serious privacy concerns. Anonymization methods could be used to clean the monitoring data before analysis. However, anonymized system logs, in general, do not provide adequate usefulness for the majority of behavioral analysis. Content-aware anonymization mechanisms such as PaRS preserve the correlation of system logs even after anonymization. This work evaluates the usefulness of anonymized system logs taken from the Taurus HPC cluster anonymized using PaRS, for behavioral analysis via recurrent neural network models.
translated by 谷歌翻译
Detecting anomalous data within time series is a very relevant task in pattern recognition and machine learning, with many possible applications that range from disease prevention in medicine, e.g., detecting early alterations of the health status before it can clearly be defined as "illness" up to monitoring industrial plants. Regarding this latter application, detecting anomalies in an industrial plant's status firstly prevents serious damages that would require a long interruption of the production process. Secondly, it permits optimal scheduling of maintenance interventions by limiting them to urgent situations. At the same time, they typically follow a fixed prudential schedule according to which components are substituted well before the end of their expected lifetime. This paper describes a case study regarding the monitoring of the status of Laser-guided Vehicles (LGVs) batteries, on which we worked as our contribution to project SUPER (Supercomputing Unified Platform, Emilia Romagna) aimed at establishing and demonstrating a regional High-Performance Computing platform that is going to represent the main Italian supercomputing environment for both computing power and data volume.
translated by 谷歌翻译