We developed a simulator to quantify the effect of changes in environmental parameters on plant growth in precision farming. Our approach combines the processing of plant images with deep convolutional neural networks (CNN), growth curve modeling, and machine learning. As a result, our system is able to predict growth rates based on environmental variables, which opens the door for the development of versatile reinforcement learning agents.

我们展示了一种带有Openai健身房界面的作物仿真环境，并应用现代深度加强学习（DRL）算法以优化产量。我们经验表明，DRL算法可用于发现新的政策和方法，以帮助优化作物产量，同时最小化水和肥料使用等约束因素。我们提出这种混合厂建模和数据驱动的方法，用于发现新策略的优化作物产量可能有助于满足越来越多的全球粮食需求，由于人口扩张和气候变化。

在化学厂的运行过程中，必须始终保持产品质量，并应最大程度地降低规范产品的生产。因此，必须测量与产品质量相关的过程变量，例如工厂各个部分的材料的温度和组成，并且必须根据测量结果进行适当的操作（即控制）。一些过程变量（例如温度和流速）可以连续，即时测量。但是，其他变量（例如成分和粘度）只能通过从植物中抽样物质后进行耗时的分析来获得。已经提出了软传感器，用于估算从易于测量变量实时获得的过程变量。但是，在未记录的情况下（推断），传统统计软传感器的估计精度（由记录的测量值构成）可能非常差。在这项研究中，我们通过使用动态模拟器来估算植物的内部状态变量，该模拟器可以根据化学工程知识和人工智能（AI）技术估算和预测未记录的情况，称为增强学习，并建议使用使用估计植物的内部状态变量作为软传感器。此外，我们描述了使用此类软传感器的植物操作和控制的前景以及为拟议系统获得必要的预测模型（即模拟器）的方法。

通过加强学习解决现实世界的顺序决策问题（RL）通常始于使用模拟真实条件的模拟环境。我们为现实的农作物管理任务提供了一种新颖的开源RL环境。 Gym-DSSAT是高保真作物模拟器的农业技术转移决策支持系统（DSSAT）的健身房界面。在过去的30年中，DSSAT已发展，并被农学家广泛认可。 Gym-DSSAT带有基于现实世界玉米实验的预定义仿真。环境与任何健身房环境一样易于使用。我们使用基本RL算法提供性能基准。我们还简要概述了用Fortran编写的单片DSSAT模拟器如何变成Python RL环境。我们的方法是通用的，可以应用于类似的模拟器。我们报告了非常初步的实验结果，这表明RL可以帮助研究人员改善受精和灌溉实践的可持续性。

在收获前的作物产量的准确预测对于世界各地的作物物流，市场计划和食物分配至关重要。产量预测需要在延长的时间段内监测物候和气候特征，以模拟农作物发育中涉及的复杂关系。绕过世界各种卫星提供的遥感卫星图像是获取数据预测数据的廉价且可靠的方法。目前，收益率预测的领域由深度学习方法主导。尽管使用这些方法达到的精度是有希望的，但所需的数据量和``Black-Box''性质可以限制深度学习方法的应用。可以通过提出一条管道将遥感图像处理为基于特征的表示形式来克服局限性，该图像允许使用极端梯度提升（XGBoost）进行产量预测。与基于深度学习的最先进的收益率预测系统相比，对美国大豆产量预测的比较评估显示出了有希望的预测准确性。特征重要性将近红外光谱视为我们模型中的重要特征。报告的结果暗示了XGBoost进行产量预测的能力，并鼓励将来对XGBoost进行XGBoost的实验，以对世界各地的其他农作物进行产量预测。

农作物管理，包括氮（N）受精和灌溉管理，对农作物产量，经济利润和环境产生了重大影响。尽管存在管理指南，但要在特定的种植环境和农作物中找到最佳的管理实践是挑战。先前的工作使用加强学习（RL）和作物模拟器来解决该问题，但是训练有素的政策要么具有有限的性能，要么在现实世界中不可部署。在本文中，我们提出了一种智能作物管理系统，该系统通过RL，模仿学习（IL）同时优化N受精和灌溉，并使用农业技术决策系统（DSSAT）进行了作物模拟。我们首先使用Deep RL，尤其是Deep Q-Network来培训需要从模拟器中的所有状态信息作为观测值（表示为完整观察）的管理政策。然后，我们援引IL来培训管理政策，这些政策只需要有限的国家信息，这些信息可以通过模仿以前的RL训练有素的政策在全面观察中轻松获得的国家（表示为部分观察）。我们在佛罗里达州使用玉米的案例研究进行实验，并将受过训练的政策与玉米管理指南进行比较。我们在全面观察和部分观察中训练有素的政策取得了更好的结果，从而获得更高的利润或类似的利润，而环境影响较小。此外，部分观察管理政策在使用易于使用的信息时直接在现实世界中部署。

Deep reinforcement learning has considerable potential to improve irrigation scheduling in many cropping systems by applying adaptive amounts of water based on various measurements over time. The goal is to discover an intelligent decision rule that processes information available to growers and prescribes sensible irrigation amounts for the time steps considered. Due to the technical novelty, however, the research on the technique remains sparse and impractical. To accelerate the progress, the paper proposes a general framework and actionable procedure that allow researchers to formulate their own optimisation problems and implement solution algorithms based on deep reinforcement learning. The effectiveness of the framework was demonstrated using a case study of irrigated wheat grown in a productive region of Australia where profits were maximised. Specifically, the decision rule takes nine state variable inputs: crop phenological stage, leaf area index, extractable soil water for each of the five top layers, cumulative rainfall and cumulative irrigation. It returns a probabilistic prescription over five candidate irrigation amounts (0, 10, 20, 30 and 40 mm) every day. The production system was simulated at Goondiwindi using the APSIM-Wheat crop model. After training in the learning environment using 1981--2010 weather data, the learned decision rule was tested individually for each year of 2011--2020. The results were compared against the benchmark profits obtained using irrigation schedules optimised individually for each of the considered years. The discovered decision rule prescribed daily irrigation amounts that achieved more than 96% of the benchmark profits. The framework is general and applicable to a wide range of cropping systems with realistic optimisation problems.

截至2017年，鱼类产品约占全球人类饮食的16％。计数作用是生产和生产这些产品的重要组成部分。种植者必须准确计算鱼类，以便这样做技术解决方案。开发了两个计算机视觉系统，以自动计算在工业池塘中生长的甲壳类幼虫。第一个系统包括带有3024x4032分辨率的iPhone 11摄像头，该摄像头在室内条件下从工业池塘中获取图像。使用该系统进行了两次实验，第一部实验包括在一天的增长阶段，在9,10的一天中使用iPhone 11相机在特定照明条件下获得的200张图像。在第二个实验中，用两个设备iPhone 11和索尼DSCHX90V摄像机拍摄了一个幼虫工业池。使用第一个设备（iPhone 11）测试了两个照明条件。在每种情况下，都获得了110张图像。该系统的准确性为88.4％的图像检测。第二个系统包括DSLR Nikon D510相机，具有2000x2000分辨率，在工业池塘外进行了七次实验。在幼虫生长阶段的第1天获取图像，从而获得了总共700张图像。该系统的密度为50的精度为86％。一种基于Yolov5 CNN模型开发的算法，该算法自动计算两种情况的幼虫数量。此外，在这项研究中，开发了幼虫生长函数。每天，从工业池塘手动取几个幼虫，并在显微镜下进行分析。确定生长阶段后，就获得了幼虫的图像。每个幼虫的长度都是通过图像手动测量的。最合适的模型是Gompertz模型，其拟合指数的良好性r平方为0.983。

我们提出了一个混合工业冷却系统模型，该模型将分析解决方案嵌入多物理模拟中。该模型设计用于增强学习（RL）应用程序，并平衡简单性与模拟保真度和解释性。该模型的忠诚度根据大规模冷却系统的现实世界数据进行了评估。接下来是一个案例研究，说明如何将模型用于RL研究。为此，我们开发了一个工业任务套件，该套件允许指定不同的问题设置和复杂性水平，并使用它来评估不同RL算法的性能。

Fruit is a key crop in worldwide agriculture feeding millions of people. The standard supply chain of fruit products involves quality checks to guarantee freshness, taste, and, most of all, safety. An important factor that determines fruit quality is its stage of ripening. This is usually manually classified by experts in the field, which makes it a labor-intensive and error-prone process. Thus, there is an arising need for automation in the process of fruit ripeness classification. Many automatic methods have been proposed that employ a variety of feature descriptors for the food item to be graded. Machine learning and deep learning techniques dominate the top-performing methods. Furthermore, deep learning can operate on raw data and thus relieve the users from having to compute complex engineered features, which are often crop-specific. In this survey, we review the latest methods proposed in the literature to automatize fruit ripeness classification, highlighting the most common feature descriptors they operate on.

Agriculture is at the heart of the solution to achieve sustainability in feeding the world population, but advancing our understanding on how agricultural output responds to climatic variability is still needed. Precision Agriculture (PA), which is a management strategy that uses technology such as remote sensing, Geographical Information System (GIS), and machine learning for decision making in the field, has emerged as a promising approach to enhance crop production, increase yield, and reduce water and nutrient losses and environmental impacts. In this context, multiple models to predict agricultural phenotypes, such as crop yield, from genomics (G), environment (E), weather and soil, and field management practices (M) have been developed. These models have traditionally been based on mechanistic or statistical approaches. However, AI approaches are intrinsically well-suited to model complex interactions and have more recently been developed, outperforming classical methods. Here, we present a Natural Language Processing (NLP)-based neural network architecture to process the G, E and M inputs and their interactions. We show that by modeling DNA as natural language, our approach performs better than previous approaches when tested for new environments and similarly to other approaches for unseen seed varieties.

已经开发了增强学习（RL）技术来优化工业冷却系统，与传统的启发式政策相比，提供了可观的节能。工业控制中的一个主要挑战涉及由于机械限制而在现实世界中可行的学习行为。例如，某些操作只能每隔几个小时执行一次，而其他动作可以更频繁地采取。如果没有广泛的奖励工程和实验，RL代理可能无法学习机械的现实操作。为了解决这个问题，我们使用层次结构的增强学习与多种根据操作时间尺度控制动作子集的代理。我们的分层方法可以在现有基线上节省能源，同时在模拟的HVAC控制环境中保持在安全范围内的限制（例如操作冷却器）。

Energy consumption in buildings, both residential and commercial, accounts for approximately 40% of all energy usage in the U.S., and similar numbers are being reported from countries around the world. This significant amount of energy is used to maintain a comfortable, secure, and productive environment for the occupants. So, it is crucial that the energy consumption in buildings must be optimized, all the while maintaining satisfactory levels of occupant comfort, health, and safety. Recently, Machine Learning has been proven to be an invaluable tool in deriving important insights from data and optimizing various systems. In this work, we review the ways in which machine learning has been leveraged to make buildings smart and energy-efficient. For the convenience of readers, we provide a brief introduction of several machine learning paradigms and the components and functioning of each smart building system we cover. Finally, we discuss challenges faced while implementing machine learning algorithms in smart buildings and provide future avenues for research at the intersection of smart buildings and machine learning.

在空间显式的基于个别模型中捕获和模拟智能自适应行为仍然是研究人员持续的挑战。虽然收集了不断增长的现实行为数据，但存在很少的方法，可以量化和正式化关键的个人行为以及它们如何改变空间和时间。因此，通常使用的常用代理决策框架（例如事件条件 - 行动规则）通常只需要仅关注狭窄的行为范围。我们认为，这些行为框架通常不会反映现实世界的情景，并且未能捕捉如何以响应刺激而发展行为。对机器学习方法的兴趣增加了近年来模拟智能自适应行为的兴趣。在该区域中开始获得牵引的一种方法是增强学习（RL）。本文探讨了如何使用基于简单的捕食者 - 猎物代理的模型（ABM）来应用RL创建紧急代理行为。运行一系列模拟，我们证明了使用新型近端政策优化（PPO）算法培训的代理以展示现实世界智能自适应行为的性质，例如隐藏，逃避和觅食。

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.

本文展示了alphaRARDEN：一个自治的多种植花园，在1.5米×3.0米的物理测试平台中撒上和灌溉生物植物。alphanArden使用架空相机和传感器来跟踪植物分布和土壤水分。我们模拟个体植物生长和平面动态，以培训选择行动以最大化叶片覆盖和多样性的政策。对于自主修剪，alphanarden使用两个定制的修剪工具和训练有素的神经网络来检测紫杉角。我们为四个60天的花园周期提供了结果。结果表明，alphaRARARDEN可以自主地实现0.96个归一化多样性，在循环峰值期间保持0.86的平均冠层覆盖率。可以在https://github.com/berkeleyautomation/alpharden找到代码，数据集和补充材料。

Machine learning frameworks such as Genetic Programming (GP) and Reinforcement Learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, bench-marking some of their most representative algorithms against global optimization techniques such as Bayesian Optimization (BO) and Lipschitz global optimization (LIPO). First, we review the general framework of the model-free control problem, bringing together all methods as black-box optimization problems. Then, we test the control algorithms on three test cases. These are (1) the stabilization of a nonlinear dynamical system featuring frequency cross-talk, (2) the wave cancellation from a Burgers' flow and (3) the drag reduction in a cylinder wake flow. We present a comprehensive comparison to illustrate their differences in exploration versus exploitation and their balance between `model capacity' in the control law definition versus `required complexity'. We believe that such a comparison paves the way toward the hybridization of the various methods, and we offer some perspective on their future development in the literature on flow control problems.

在这项研究中，我们开发了机器学习模型，以预测废物到燃料植物的未来传感器读数，这将积极控制工厂的运营。我们开发了可预测传感器读数30和60分钟的模型。使用历史数据对模型进行了培训，并根据在特定时间进行的传感器读数进行预测。我们比较了三种类型的模型：（a）仅考虑最后一个预测值的a n \“ aive预测，（b）基于过去的传感器数据进行预测的神经网络（我们考虑了不同的时间窗口尺寸以进行预测）和（c）由我们开发的一组功能创建的梯度增强树回收剂。我们在加拿大的一家废物燃料工厂上开发并测试了模型。我们发现提供的方法（c）提供了最佳结果，而方法（b）提供了不同的结果，并且无法始终如一地超越n \“ aive”。

In contrast to the rapid digitalization of several industries, agriculture suffers from low adoption of smart farming tools. While AI-driven digital agriculture tools can offer high-performing predictive functionalities, they lack tangible quantitative evidence on their benefits to the farmers. Field experiments can derive such evidence, but are often costly, time consuming and hence limited in scope and scale of application. To this end, we propose an observational causal inference framework for the empirical evaluation of the impact of digital tools on target farm performance indicators (e.g., yield in this case). This way, we can increase farmers' trust via enhancing the transparency of the digital agriculture market and accelerate the adoption of technologies that aim to secure farmer income resilience and global agricultural sustainability. As a case study, we designed and implemented a recommendation system for the optimal sowing time of cotton based on numerical weather predictions, which was used by a farmers' cooperative during the growing season of 2021. We then leverage agricultural knowledge, collected yield data, and environmental information to develop a causal graph of the farm system. Using the back-door criterion, we identify the impact of sowing recommendations on the yield and subsequently estimate it using linear regression, matching, inverse propensity score weighting and meta-learners. The results reveal that a field sown according to our recommendations exhibited a statistically significant yield increase that ranged from 12% to 17%, depending on the method. The effect estimates were robust, as indicated by the agreement among the estimation methods and four successful refutation tests. We argue that this approach can be implemented for decision support systems of other fields, extending their evaluation beyond a performance assessment of internal functionalities.

监测草原的健康和活力对于告知管理决策至关优化农业应用中的旋转放牧的态度至关重要。为了利用饲料资源，提高土地生产力，我们需要了解牧场的增长模式，这在最先进的状态下即可。在本文中，我们建议部署一个机器人团队来监测一个未知的牧场环境的演变，以实现上述目标。为了监测这种环境，通常会缓慢发展，我们需要设计一种以低成本在大面积上快速评估环境的策略。因此，我们提出了一种集成管道，包括数据综合，深度神经网络训练和预测以及一个间歇地监测牧场的多机器人部署算法。具体而言，使用与ROS Gazebo的新型数据综合耦合的专家知识的农业数据，我们首先提出了一种新的神经网络架构来学习环境的时空动态。这种预测有助于我们了解大规模上的牧场增长模式，并为未来做出适当的监测决策。基于我们的预测，我们设计了一个用于低成本监控的间歇多机器人部署策略。最后，我们将提议的管道与其他方法进行比较，从数据综合到预测和规划，以证实我们的管道的性能。