Ongoing risks from climate change have impacted the livelihood of global nomadic communities, and are likely to lead to increased migratory movements in coming years. As a result, mobility considerations are becoming increasingly important in energy systems planning, particularly to achieve energy access in developing countries. Advanced Plug and Play control strategies have been recently developed with such a decentralized framework in mind, more easily allowing for the interconnection of nomadic communities, both to each other and to the main grid. In light of the above, the design and planning strategy of a mobile multi-energy supply system for a nomadic community is investigated in this work. Motivated by the scale and dimensionality of the associated uncertainties, impacting all major design and decision variables over the 30-year planning horizon, Deep Reinforcement Learning (DRL) is implemented for the design and planning problem tackled. DRL based solutions are benchmarked against several rigid baseline design options to compare expected performance under uncertainty. The results on a case study for ger communities in Mongolia suggest that mobile nomadic energy systems can be both technically and economically feasible, particularly when considering flexibility, although the degree of spatial dispersion among households is an important limiting factor. Key economic, sustainability and resilience indicators such as Cost, Equivalent Emissions and Total Unmet Load are measured, suggesting potential improvements compared to available baselines of up to 25%, 67% and 76%, respectively. Finally, the decomposition of values of flexibility and plug and play operation is presented using a variation of real options theory, with important implications for both nomadic communities and policymakers focused on enabling their energy access.
translated by 谷歌翻译
智能能源网络提供了一种有效的手段,可容纳可变可再生能源(例如太阳能和风能)的高渗透率,这是能源生产深度脱碳的关键。但是,鉴于可再生能源以及能源需求的可变性,必须制定有效的控制和能源存储方案来管理可变的能源产生并实现所需的系统经济学和环境目标。在本文中,我们引入了由电池和氢能存储组成的混合储能系统,以处理与电价,可再生能源生产和消费有关的不确定性。我们旨在提高可再生能源利用率,并最大程度地减少能源成本和碳排放,同时确保网络内的能源可靠性和稳定性。为了实现这一目标,我们提出了一种多代理的深层确定性政策梯度方法,这是一种基于强化的基于强化学习的控制策略,可实时优化混合能源存储系统和能源需求的调度。提出的方法是无模型的,不需要明确的知识和智能能源网络环境的严格数学模型。基于现实世界数据的仿真结果表明:(i)混合储能系统和能源需求的集成和优化操作可将碳排放量减少78.69%,将成本节省的成本储蓄提高23.5%,可续订的能源利用率比13.2%以上。其他基线模型和(ii)所提出的算法优于最先进的自学习算法,例如Deep-Q网络。
translated by 谷歌翻译
在迅速增长的海上风电场市场中出现了增加风力涡轮机尺寸和距离的全球趋势。在英国,海上风电业于2019年生产了英国最多的电力,前一年增加了19.6%。目前,英国将进一步增加产量,旨在增加安装的涡轮机容量74.7%,如最近的冠村租赁轮次反映。通过如此巨大的增长,该部门现在正在寻求机器人和人工智能(RAI),以解决生命周期服务障碍,以支持可持续和有利可图的海上风能生产。如今,RAI应用主要用于支持运营和维护的短期目标。然而,前进,RAI在海上风基础设施的全部生命周期中有可能发挥关键作用,从测量,规划,设计,物流,运营支持,培训和退役。本文介绍了离岸可再生能源部门的RAI的第一个系统评论之一。在当前和未来的要求方面,在行业和学术界的离岸能源需求分析了rai的最先进的。我们的评论还包括对支持RAI的投资,监管和技能开发的详细评估。通过专利和学术出版数据库进行详细分析确定的关键趋势,提供了对安全合规性和可靠性的自主平台认证等障碍的见解,这是自主车队中可扩展性的数字架构,适应性居民运营和优化的适应性规划人机互动对人与自治助理的信赖伙伴关系。
translated by 谷歌翻译
Energy consumption in buildings, both residential and commercial, accounts for approximately 40% of all energy usage in the U.S., and similar numbers are being reported from countries around the world. This significant amount of energy is used to maintain a comfortable, secure, and productive environment for the occupants. So, it is crucial that the energy consumption in buildings must be optimized, all the while maintaining satisfactory levels of occupant comfort, health, and safety. Recently, Machine Learning has been proven to be an invaluable tool in deriving important insights from data and optimizing various systems. In this work, we review the ways in which machine learning has been leveraged to make buildings smart and energy-efficient. For the convenience of readers, we provide a brief introduction of several machine learning paradigms and the components and functioning of each smart building system we cover. Finally, we discuss challenges faced while implementing machine learning algorithms in smart buildings and provide future avenues for research at the intersection of smart buildings and machine learning.
translated by 谷歌翻译
蒙特卡洛树搜索(MCT)是设计游戏机器人或解决顺序决策问题的强大方法。该方法依赖于平衡探索和开发的智能树搜索。MCT以模拟的形式进行随机抽样,并存储动作的统计数据,以在每个随后的迭代中做出更有教育的选择。然而,该方法已成为组合游戏的最新技术,但是,在更复杂的游戏(例如那些具有较高的分支因素或实时系列的游戏)以及各种实用领域(例如,运输,日程安排或安全性)有效的MCT应用程序通常需要其与问题有关的修改或与其他技术集成。这种特定领域的修改和混合方法是本调查的主要重点。最后一项主要的MCT调查已于2012年发布。自发布以来出现的贡献特别感兴趣。
translated by 谷歌翻译
Global power systems are increasingly reliant on wind energy as a mitigation strategy for climate change. However, the variability of wind energy causes system reliability to erode, resulting in the wind being curtailed and, ultimately, leading to substantial economic losses for wind farm owners. Wind curtailment can be reduced using battery energy storage systems (BESS) that serve as onsite backup sources. Yet, this auxiliary role may significantly hamper the BESS's capacity to generate revenues from the electricity market, particularly in conducting energy arbitrage in the Spot market and providing frequency control ancillary services (FCAS) in the FCAS markets. Ideal BESS scheduling should effectively balance the BESS's role in absorbing onsite wind curtailment and trading in the electricity market, but it is difficult in practice because of the underlying coordination complexity and the stochastic nature of energy prices and wind generation. In this study, we investigate the bidding strategy of a wind-battery system co-located and participating simultaneously in both the Spot and Regulation FCAS markets. We propose a deep reinforcement learning (DRL)-based approach that decouples the market participation of the wind-battery system into two related Markov decision processes for each facility, enabling the BESS to absorb onsite wind curtailment while simultaneously bidding in the wholesale Spot and FCAS markets to maximize overall operational revenues. Using realistic wind farm data, we validated the coordinated bidding strategy for the wind-battery system and find that our strategy generates significantly higher revenue and responds better to wind curtailment compared to an optimization-based benchmark. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems compared to individual market participation.
translated by 谷歌翻译
多阶段随机线性问题(MSLP)的解决方案代表了许多应用程序的挑战。长期水热调度计划(LHDP)在影响全球电力市场,经济和自然资源的现实世界中实现了这一挑战。没有用于MSLP的封闭式解决方案,并且具有高质量的非预期策略的定义是至关重要的。线性决策规则(LDR)提供了一个有趣的基于模拟的框架,可通过两阶段随机模型为MSLP找到高质量的策略。但是,在实际应用中,使用LDR时要估计的参数数量可能接近或高于样本平均近似问题的场景数量,从而在样本外产生样本外的过度效果和差的表现不佳模拟。在本文中,我们提出了一个新型的正则LDR来基于Adalasso(自适应最少的绝对收缩和选择算子)求解MSLP。目的是使用高维线性回归模型中所研究的简约原理,以获得应用于MSLP的LDR的更好的样本外部性能。计算实验表明,使用经典的非规范LDR来求解LHDP时,过度合适的威胁是不可忽略的,这是研究最多的MSLP之一,其中具有相关应用在行业中。我们的分析强调了拟议框架与非规范化基准相比的以下好处:1)非零系数的数量显着减少(模型简约),2)2)大幅度降低样本外评估的成本降低, 3)改善了现货价格概况。
translated by 谷歌翻译
As various city agencies and mobility operators navigate toward innovative mobility solutions, there is a need for strategic flexibility in well-timed investment decisions in the design and timing of mobility service regions, i.e. cast as "real options" (RO). This problem becomes increasingly challenging with multiple interacting RO in such investments. We propose a scalable machine learning based RO framework for multi-period sequential service region design & timing problem for mobility-on-demand services, framed as a Markov decision process with non-stationary stochastic variables. A value function approximation policy from literature uses multi-option least squares Monte Carlo simulation to get a policy value for a set of interdependent investment decisions as deferral options (CR policy). The goal is to determine the optimal selection and timing of a set of zones to include in a service region. However, prior work required explicit enumeration of all possible sequences of investments. To address the combinatorial complexity of such enumeration, we propose a new variant "deep" RO policy using an efficient recurrent neural network (RNN) based ML method (CR-RNN policy) to sample sequences to forego the need for enumeration, making network design & timing policy tractable for large scale implementation. Experiments on multiple service region scenarios in New York City (NYC) shows the proposed policy substantially reduces the overall computational cost (time reduction for RO evaluation of > 90% of total investment sequences is achieved), with zero to near-zero gap compared to the benchmark. A case study of sequential service region design for expansion of MoD services in Brooklyn, NYC show that using the CR-RNN policy to determine optimal RO investment strategy yields a similar performance (0.5% within CR policy value) with significantly reduced computation time (about 5.4 times faster).
translated by 谷歌翻译
由于数据量增加,金融业的快速变化已经彻底改变了数据处理和数据分析的技术,并带来了新的理论和计算挑战。与古典随机控制理论和解决财务决策问题的其他分析方法相比,解决模型假设的财务决策问题,强化学习(RL)的新发展能够充分利用具有更少模型假设的大量财务数据并改善复杂的金融环境中的决策。该调查纸目的旨在审查最近的资金途径的发展和使用RL方法。我们介绍了马尔可夫决策过程,这是许多常用的RL方法的设置。然后引入各种算法,重点介绍不需要任何模型假设的基于价值和基于策略的方法。连接是用神经网络进行的,以扩展框架以包含深的RL算法。我们的调查通过讨论了这些RL算法在金融中各种决策问题中的应用,包括最佳执行,投资组合优化,期权定价和对冲,市场制作,智能订单路由和Robo-Awaring。
translated by 谷歌翻译
在本文中,我们介绍了有关典型乘车共享系统中决策优化问题的强化学习方法的全面,深入的调查。涵盖了有关乘车匹配,车辆重新定位,乘车,路由和动态定价主题的论文。在过去的几年中,大多数文献都出现了,并且要继续解决一些核心挑战:模型复杂性,代理协调和多个杠杆的联合优化。因此,我们还引入了流行的数据集和开放式仿真环境,以促进进一步的研发。随后,我们讨论了有关该重要领域的强化学习研究的许多挑战和机会。
translated by 谷歌翻译
作为行业4.0时代的一项新兴技术,数字双胞胎因其承诺进一步优化流程设计,质量控制,健康监测,决策和政策制定等,通过全面对物理世界进行建模,以进一步优化流程设计,质量控制,健康监测,决策和政策,因此获得了前所未有的关注。互连的数字模型。在一系列两部分的论文中,我们研究了不同建模技术,孪生启用技术以及数字双胞胎常用的不确定性量化和优化方法的基本作用。第二篇论文介绍了数字双胞胎的关键启示技术的文献综述,重点是不确定性量化,优化方法,开源数据集和工具,主要发现,挑战和未来方向。讨论的重点是当前的不确定性量化和优化方法,以及如何在数字双胞胎的不同维度中应用它们。此外,本文介绍了一个案例研究,其中构建和测试了电池数字双胞胎,以说明在这两部分评论中回顾的一些建模和孪生方法。 GITHUB上可以找到用于生成案例研究中所有结果和数字的代码和预处理数据。
translated by 谷歌翻译
在本文中,多种子体增强学习用于控制混合能量存储系统,通过最大化可再生能源和交易的价值来降低微电网的能量成本。该代理商必须学习在波动需求,动态批发能源价格和不可预测的可再生能源中,控制三种不同类型的能量存储系统。考虑了两种案例研究:首先看能量存储系统如何在动态定价下更好地整合可再生能源发电,第二种与这些同一代理商如何与聚合剂一起使用,以向自私外部微电网销售能量的能量减少自己的能源票据。这项工作发现,具有分散执行的多代理深度确定性政策梯度的集中学习及其最先进的变体允许多种代理方法显着地比来自单个全局代理的控制更好。还发现,在多种子体方法中使用单独的奖励功能比使用单个控制剂更好。还发现能够与其他微电网交易,而不是卖回实用电网,也发现大大增加了网格的储蓄。
translated by 谷歌翻译
无线电接入网络(RAN)技术继续见证巨大的增长,开放式运行越来越最近的势头。在O-RAN规范中,RAN智能控制器(RIC)用作自动化主机。本文介绍了对O-RAN堆栈相关的机器学习(ML)的原则,特别是加强学习(RL)。此外,我们审查无线网络的最先进的研究,并将其投入到RAN框架和O-RAN架构的层次结构上。我们在整个开发生命周期中提供ML / RL模型面临的挑战的分类:从系统规范到生产部署(数据采集,模型设计,测试和管理等)。为了解决挑战,我们将一组现有的MLOPS原理整合,当考虑RL代理时,具有独特的特性。本文讨论了系统的生命周期模型开发,测试和验证管道,称为:RLOPS。我们讨论了RLOP的所有基本部分,包括:模型规范,开发和蒸馏,生产环境服务,运营监控,安全/安全和数据工程平台。根据这些原则,我们提出了最佳实践,以实现自动化和可重复的模型开发过程。
translated by 谷歌翻译
Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels. Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policybased methods. Our survey will cover central algorithms in deep reinforcement learning, including the deep Q-network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning. To conclude, we describe several current areas of research within the field.
translated by 谷歌翻译
在过去的十年中,由于分散控制应用程序的趋势和网络物理系统应用的出现,网络控制系统在过去十年中引起了广泛的关注。但是,由于无线网络的复杂性质,现实世界中无线网络控制系统的通信带宽,可靠性问题以及对网络动态的认识不足。将机器学习和事件触发的控制结合起来有可能减轻其中一些问题。例如,可以使用机器学习来克服缺乏网络模型的问题,通过学习系统行为或通过不断学习模型动态来适应动态变化的模型。事件触发的控制可以通过仅在必要时或可用资源时传输控制信息来帮助保护通信带宽。本文的目的是对有关机器学习的使用与事件触发的控制的使用进行综述。机器学习技术,例如统计学习,神经网络和基于强化的学习方法,例如深入强化学习,并结合事件触发的控制。我们讨论如何根据机器学习使用的目的将这些学习算法用于不同的应用程序。在对文献的审查和讨论之后,我们重点介绍了与基于机器学习的事件触发的控制并提出潜在解决方案相关的开放研究问题和挑战。
translated by 谷歌翻译
主动位置估计(APE)是使用一个或多个传感平台本地化一个或多个目标的任务。 APE是搜索和拯救任务,野生动物监测,源期限估计和协作移动机器人的关键任务。 APE的成功取决于传感平台的合作水平,他们的数量,他们的自由度和收集的信息的质量。 APE控制法通过满足纯粹剥削或纯粹探索性标准,可以实现主动感测。前者最大限度地减少了位置估计的不确定性;虽然后者驱动了更接近其任务完成的平台。在本文中,我们定义了系统地分类的主要元素,并批判地讨论该域中的最新状态。我们还提出了一个参考框架作为对截图相关的解决方案的形式主义。总体而言,本调查探讨了主要挑战,并设想了本地化任务的自主感知系统领域的主要研究方向。促进用于搜索和跟踪应用的强大主动感测方法的开发也有益。
translated by 谷歌翻译
The decarbonization of buildings presents new challenges for the reliability of the electrical grid as a result of the intermittency of renewable energy sources and increase in grid load brought about by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide flexibility services to the grid through demand response. Residential demand response programs are hindered by the need for manual intervention by customers. To maximize the energy flexibility potential of residential buildings, an advanced control architecture is needed. Reinforcement learning is well-suited for the control of flexible resources as it is able to adapt to unique building characteristics compared to expert systems. Yet, factors hindering the adoption of RL in real-world applications include its large data requirements for training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework and using a digital twin of a real-world 17-building grid-interactive residential community in CityLearn. We show that 1) independent RL-controllers for batteries improve building and district level KPIs compared to a reference RBC by tailoring their policies to individual buildings, 2) despite unique occupant behaviours, transferring the RL policy of any one of the buildings to other buildings provides comparable performance while reducing the cost of training, 3) training RL-controllers on limited temporal data that does not capture full seasonality in occupant behaviour has little effect on performance. Although, the zero-net-energy (ZNE) condition of the buildings could be maintained or worsened as a result of controlled batteries, KPIs that are typically improved by ZNE condition (electricity price and carbon emissions) are further improved when the batteries are managed by an advanced controller.
translated by 谷歌翻译
数字化和远程连接扩大了攻击面,使网络系统更脆弱。由于攻击者变得越来越复杂和资源丰富,仅仅依赖传统网络保护,如入侵检测,防火墙和加密,不足以保护网络系统。网络弹性提供了一种新的安全范式,可以使用弹性机制来补充保护不足。一种网络弹性机制(CRM)适应了已知的或零日威胁和实际威胁和不确定性,并对他们进行战略性地响应,以便在成功攻击时保持网络系统的关键功能。反馈架构在启用CRM的在线感应,推理和致动过程中发挥关键作用。强化学习(RL)是一个重要的工具,对网络弹性的反馈架构构成。它允许CRM提供有限或没有事先知识和攻击者的有限攻击的顺序响应。在这项工作中,我们审查了Cyber​​恢复力的RL的文献,并讨论了对三种主要类型的漏洞,即姿势有关,与信息相关的脆弱性的网络恢复力。我们介绍了三个CRM的应用领域:移动目标防御,防守网络欺骗和辅助人类安全技术。 RL算法也有漏洞。我们解释了RL的三个漏洞和目前的攻击模型,其中攻击者针对环境与代理商之间交换的信息:奖励,国家观察和行动命令。我们展示攻击者可以通过最低攻击努力来欺骗RL代理商学习邪恶的政策。最后,我们讨论了RL为基于RL的CRM的网络安全和恢复力和新兴应用的未来挑战。
translated by 谷歌翻译
在现代环境和社会问题的背景下,人们对能够识别土木工程系统的管理策略的方法的需求越来越大,最大程度地降低了结构性故障风险,同时最好计划检查和维护(I&M)流程。由于与联合系统级状态描述下的全局优化方法相关的计算复杂性,大多数可用方法将I&M决策问题简化为组件级别。在本文中,我们提出了一个有效的算法框架,用于在暴露于恶化环境的工程系统下进行推理和决策制定,从而直接在系统级别提供最佳的管理策略。在我们的方法中,决策问题被提出为部分可观察到的马尔可夫决策过程,其动态是在贝叶斯网络条件结构中编码的。该方法可以通过高斯层次结构和动态贝叶斯网络在组件之间平等或一般,不平等的恶化相关性下处理环境。在政策优化方面,我们采用了深层分散的多代理参与者 - 批评(DDMAC)强化学习方法,其中政策由批评家网络指导的参与者神经网络近似。通过在模拟环境中包括劣化依赖性,并通过在系统级别制定成本模型,DDMAC策略本质上考虑了基本系统效应。通过对疲劳恶化下的9分和钢架进行的数值实验证明了这一点。结果表明,与最先进的启发式方法相比,DDMAC政策可提供可观的好处。 DDMAC策略对系统效应的固有考虑也可以根据学习的政策来解释。
translated by 谷歌翻译
互联网连接系统的指数增长产生了许多挑战,例如频谱短缺问题,需要有效的频谱共享(SS)解决方案。复杂和动态的SS系统可以接触不同的潜在安全性和隐私问题,需要保护机制是自适应,可靠和可扩展的。基于机器学习(ML)的方法经常提议解决这些问题。在本文中,我们对最近的基于ML的SS方法,最关键的安全问题和相应的防御机制提供了全面的调查。特别是,我们详细说明了用于提高SS通信系统的性能的最先进的方法,包括基于ML基于ML的基于的数据库辅助SS网络,ML基于基于的数据库辅助SS网络,包括基于ML的数据库辅助的SS网络,基于ML的LTE-U网络,基于ML的环境反向散射网络和其他基于ML的SS解决方案。我们还从物理层和基于ML算法的相应防御策略的安全问题,包括主要用户仿真(PUE)攻击,频谱感测数据伪造(SSDF)攻击,干扰攻击,窃听攻击和隐私问题。最后,还给出了对ML基于ML的开放挑战的广泛讨论。这种全面的审查旨在为探索新出现的ML的潜力提供越来越复杂的SS及其安全问题,提供基础和促进未来的研究。
translated by 谷歌翻译