随着风能的渗透到电网，能够预测大型风电场的预期电力生产变得越来越重要。深度学习（DL）模型可以在数据中学习复杂的模式，并在预测唤醒损失和预期的电力生产方面找到了广泛的成功。本文提出了一种基于关注的图形神经网络（GNN）的模块化框架，其中可以应用于图形块的任何所需组件。结果表明，该模型显着优于多层的Perceptron（MLP）和双向LSTM（BLSTM）模型，同时通过Vanilla GNN模型提供性能。此外，我们认为，所提出的图表架构可以通过为要使用的所需注意操作提供灵活性来轻松适应不同的应用，这可能取决于特定应用。通过分析注意力的重量，据表明，采用基于关注的GNN可以提供洞察模型学习的内容。特别是，注意网络似乎意识到与唤醒损失的一些物理直觉对齐的涡轮机依赖性。

为了提高风能生产的安全性和可靠性，短期预测已成为最重要的。这项研究的重点是挪威大陆架的多步时时空风速预测。图形神经网络（GNN）体系结构用于提取空间依赖性，具有不同的更新功能以学习时间相关性。这些更新功能是使用不同的神经网络体系结构实现的。近年来，一种这样的架构，即变压器，在序列建模中变得越来越流行。已经提出了对原始体系结构的各种改动，以更好地促进时间序列预测，本研究的重点是告密者Logsparse Transformer和AutoFormer。这是第一次将logsparse变压器和自动形态应用于风预测，并且第一次以任何一种或告密者的形式在时空设置以进行风向预测。通过比较时空长的短期记忆（LSTM）和多层感知器（MLP）模型，该研究表明，使用改变的变压器体系结构作为GNN中更新功能的模型能够超越这些功能。此外，我们提出了快速的傅立叶变压器（FFTRANSFORMER），该变压器是基于信号分解的新型变压器体系结构，由两个单独的流组成，分别分析趋势和周期性成分。发现FFTRANSFORMER和自动成型器可在10分钟和1小时的预测中取得优异的结果，而FFTRANSFORMER显着优于所有其他模型的4小时预测。最后，通过改变图表表示的连通性程度，该研究明确说明了所有模型如何利用空间依赖性来改善局部短期风速预测。

对于电网操作，具有精细时间和空间分辨率的太阳能发电准确预测对于电网的操作至关重要。然而，与数值天气预报（NWP）结合机器学习的最先进方法具有粗略分辨率。在本文中，我们采用曲线图信号处理透视和型号的多网站光伏（PV）生产时间序列作为图表上的信号，以捕获它们的时空依赖性并实现更高的空间和时间分辨率预测。我们提出了两种新颖的图形神经网络模型，用于确定性多站点PV预测，被称为图形 - 卷积的长期内存（GCLSTM）和图形 - 卷积变压器（GCTRAFO）模型。这些方法仅依赖于生产数据并利用PV系统提供密集的虚拟气象站网络的直觉。所提出的方法是在整整一年的两组数据集中评估：1）来自304个真实光伏系统的生产数据，以及2）模拟生产1000个PV系统，包括瑞士分布。该拟议的模型优于最先进的多站点预测方法，用于预测前方6小时的预测视野。此外，所提出的模型以NWP优于最先进的单站点方法，如前方的视野上的输入。

Wind turbine wake modelling is of crucial importance to accurate resource assessment, to layout optimisation, and to the operational control of wind farms. This work proposes a surrogate model for the representation of wind turbine wakes based on a state-of-the-art graph representation learning method termed a graph neural network. The proposed end-to-end deep learning model operates directly on unstructured meshes and has been validated against high-fidelity data, demonstrating its ability to rapidly make accurate 3D flow field predictions for various inlet conditions and turbine yaw angles. The specific graph neural network model employed here is shown to generalise well to unseen data and is less sensitive to over-smoothing compared to common graph neural networks. A case study based upon a real world wind farm further demonstrates the capability of the proposed approach to predict farm scale power generation. Moreover, the proposed graph neural network framework is flexible and highly generic and as formulated here can be applied to any steady state computational fluid dynamics simulations on unstructured meshes.

研究人员通常会采用数值方法来理解和预测海洋动力学，这是掌握环境现象的关键任务。在地形图很复杂，有关基础过程的知识不完整或应用程序至关重要的情况下，此类方法可能不适合。另一方面，如果观察到海洋动力学，则可以通过最近的机器学习方法来利用它们。在本文中，我们描述了一种数据驱动的方法，可以预测环境变量，例如巴西东南海岸的Santos-Sao Vicente-Bertioga estuarine系统的当前速度和海面高度。我们的模型通过连接最新的序列模型（LSTM和Transformers）以及关系模型（图神经网络）来利用时间和空间归纳偏见，以学习时间特征和空间特征，观察站点之间共享的关系。我们将结果与桑托斯运营预测系统（SOFS）进行比较。实验表明，我们的模型可以实现更好的结果，同时保持灵活性和很少的领域知识依赖性。

Wind power forecasting helps with the planning for the power systems by contributing to having a higher level of certainty in decision-making. Due to the randomness inherent to meteorological events (e.g., wind speeds), making highly accurate long-term predictions for wind power can be extremely difficult. One approach to remedy this challenge is to utilize weather information from multiple points across a geographical grid to obtain a holistic view of the wind patterns, along with temporal information from the previous power outputs of the wind farms. Our proposed CNN-RNN architecture combines convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to extract spatial and temporal information from multi-dimensional input data to make day-ahead predictions. In this regard, our method incorporates an ultra-wide learning view, combining data from multiple numerical weather prediction models, wind farms, and geographical locations. Additionally, we experiment with global forecasting approaches to understand the impact of training the same model over the datasets obtained from multiple different wind farms, and we employ a method where spatial information extracted from convolutional layers is passed to a tree ensemble (e.g., Light Gradient Boosting Machine (LGBM)) instead of fully connected layers. The results show that our proposed CNN-RNN architecture outperforms other models such as LGBM, Extra Tree regressor and linear regression when trained globally, but fails to replicate such performance when trained individually on each farm. We also observe that passing the spatial information from CNN to LGBM improves its performance, providing further evidence of CNN's spatial feature extraction capabilities.

Traffic state prediction in a transportation network is paramount for effective traffic operations and management, as well as informed user and system-level decision-making. However, long-term traffic prediction (beyond 30 minutes into the future) remains challenging in current research. In this work, we integrate the spatio-temporal dependencies in the transportation network from network modeling, together with the graph convolutional network (GCN) and graph attention network (GAT). To further tackle the dramatic computation and memory cost caused by the giant model size (i.e., number of weights) caused by multiple cascaded layers, we propose sparse training to mitigate the training cost, while preserving the prediction accuracy. It is a process of training using a fixed number of nonzero weights in each layer in each iteration. We consider the problem of long-term traffic speed forecasting for a real large-scale transportation network data from the California Department of Transportation (Caltrans) Performance Measurement System (PeMS). Experimental results show that the proposed GCN-STGT and GAT-STGT models achieve low prediction errors on short-, mid- and long-term prediction horizons, of 15, 30 and 45 minutes in duration, respectively. Using our sparse training, we could train from scratch with high sparsity (e.g., up to 90%), equivalent to 10 times floating point operations per second (FLOPs) reduction on computational cost using the same epochs as dense training, and arrive at a model with very small accuracy loss compared with the original dense training

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

在各种下游机器学习任务中，多元时间序列的可靠和有效表示至关重要。在多元时间序列预测中，每个变量都取决于其历史值，并且变量之间也存在相互依存关系。必须设计模型以捕获时间序列之间的内部和相互关系。为了朝着这一目标迈进，我们提出了时间序列注意变压器（TSAT），以进行多元时间序列表示学习。使用TSAT，我们以边缘增强动态图来表示多元时间序列的时间信息和相互依赖性。在动态图中的节点表示，串行中的相关性表示。修改了一种自我注意力的机制，以使用超经验模式分解（SMD）模块捕获序列间的相关性。我们将嵌入式动态图应用于时代序列预测问题，包括两个现实世界数据集和两个基准数据集。广泛的实验表明，TSAT显然在各种预测范围内使用六种最先进的基线方法。我们进一步可视化嵌入式动态图，以说明TSAT的图形表示功能。我们在https://github.com/radiantresearch/tsat上共享代码。

Accurate short-term traffic prediction plays a pivotal role in various smart mobility operation and management systems. Currently, most of the state-of-the-art prediction models are based on graph neural networks (GNNs), and the required training samples are proportional to the size of the traffic network. In many cities, the available amount of traffic data is substantially below the minimum requirement due to the data collection expense. It is still an open question to develop traffic prediction models with a small size of training data on large-scale networks. We notice that the traffic states of a node for the near future only depend on the traffic states of its localized neighborhoods, which can be represented using the graph relational inductive biases. In view of this, this paper develops a graph network (GN)-based deep learning model LocaleGN that depicts the traffic dynamics using localized data aggregating and updating functions, as well as the node-wise recurrent neural networks. LocaleGN is a light-weighted model designed for training on few samples without over-fitting, and hence it can solve the problem of few-sample traffic prediction. The proposed model is examined on predicting both traffic speed and flow with six datasets, and the experimental results demonstrate that LocaleGN outperforms existing state-of-the-art baseline models. It is also demonstrated that the learned knowledge from LocaleGN can be transferred across cities. The research outcomes can help to develop light-weighted traffic prediction systems, especially for cities lacking historically archived traffic data.

标准的神经网络可以近似一般的非线性操作员，要么通过数学运算符的组合（例如，在对流 - 扩散反应部分微分方程中）的组合，要么仅仅是黑匣子，例如黑匣子，例如一个系统系统。第一个神经操作员是基于严格的近似理论于2019年提出的深层操作员网络（DeepOnet）。从那时起，已经发布了其他一些较少的一般操作员，例如，基于图神经网络或傅立叶变换。对于黑匣子系统，对神经操作员的培训仅是数据驱动的，但是如果知道管理方程式可以在培训期间将其纳入损失功能，以开发物理知识的神经操作员。神经操作员可以用作设计问题，不确定性量化，自主系统以及几乎任何需要实时推断的应用程序中的代替代物。此外，通过将它们与相对轻的训练耦合，可以将独立的预训练deponets用作复杂多物理系统的组成部分。在这里，我们介绍了Deponet，傅立叶神经操作员和图神经操作员的评论，以及适当的扩展功能扩展，并突出显示它们在计算机械师中的各种应用中的实用性，包括多孔媒体，流体力学和固体机制， 。

基于预测方法的深度学习已成为时间序列预测或预测的许多应用中的首选方法，通常通常优于其他方法。因此，在过去的几年中，这些方法现在在大规模的工业预测应用中无处不在，并且一直在预测竞赛（例如M4和M5）中排名最佳。这种实践上的成功进一步提高了学术兴趣，以理解和改善深厚的预测方法。在本文中，我们提供了该领域的介绍和概述：我们为深入预测的重要构建块提出了一定深度的深入预测；随后，我们使用这些构建块，调查了最近的深度预测文献的广度。

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-theart results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a proteinprotein interaction dataset (wherein test graphs remain unseen during training).

天气预报是一项有吸引力的挑战性任务，因为它对人类生活和大气运动的复杂性的影响。在大量历史观察到的时间序列数据的支持下，该任务适用于数据驱动的方法，尤其是深层神经网络。最近，基于图神经网络（GNN）方法在时空预测方面取得了出色的性能。但是，基于规范的GNNS方法仅分别对每个站的气象变量的局部图或整个车站的全局图进行建模，从而缺乏不同站点的气象变量之间的信息相互作用。在本文中，我们提出了一种新型的层次时空图形神经网络（Histgnn），以模拟多个站点气象变量之间的跨区域时空相关性。自适应图学习层和空间图卷积用于构建自学习图，并研究可变级别和站点级别图的节点之间的隐藏依赖性。为了捕获时间模式，扩张的成立为GATE时间卷积的主干旨在对长而各种气象趋势进行建模。此外，提出了动态的交互学习来构建在层次图中传递的双向信息。三个现实世界中的气象数据集的实验结果表明，史基元超过7个基准的卓越性能，并且将误差降低了4.2％至11.6％，尤其是与最先进的天气预测方法相比。

图表神经网络（GNN）基于故障诊断（FD）近年来收到了越来越多的关注，因为来自来自多个应用域的数据可以有利地表示为图。实际上，与传统的FD方法相比，这种特殊的代表性表格导致了卓越的性能。在本次审查中，给出了GNN，对故障诊断领域的潜在应用以及未来观点的简单介绍。首先，通过专注于它们的数据表示，即时间序列，图像和图形，回顾基于神经网络的FD方法。其次，引入了GNN的基本原则和主要架构，注意了图形卷积网络，图注意网络，图形样本和聚合，图形自动编码器和空间 - 时间图卷积网络。第三，通过详细实验验证基于GNN的最相关的故障诊断方法，结论是基于GNN的方法可以实现良好的故障诊断性能。最后，提供了讨论和未来的挑战。

Remaining Useful Life (RUL) estimation plays a critical role in Prognostics and Health Management (PHM). Traditional machine health maintenance systems are often costly, requiring sufficient prior expertise, and are difficult to fit into highly complex and changing industrial scenarios. With the widespread deployment of sensors on industrial equipment, building the Industrial Internet of Things (IIoT) to interconnect these devices has become an inexorable trend in the development of the digital factory. Using the device's real-time operational data collected by IIoT to get the estimated RUL through the RUL prediction algorithm, the PHM system can develop proactive maintenance measures for the device, thus, reducing maintenance costs and decreasing failure times during operation. This paper carries out research into the remaining useful life prediction model for multi-sensor devices in the IIoT scenario. We investigated the mainstream RUL prediction models and summarized the basic steps of RUL prediction modeling in this scenario. On this basis, a data-driven approach for RUL estimation is proposed in this paper. It employs a Multi-Head Attention Mechanism to fuse the multi-dimensional time-series data output from multiple sensors, in which the attention on features is used to capture the interactions between features and attention on sequences is used to learn the weights of time steps. Then, the Long Short-Term Memory Network is applied to learn the features of time series. We evaluate the proposed model on two benchmark datasets (C-MAPSS and PHM08), and the results demonstrate that it outperforms the state-of-art models. Moreover, through the interpretability of the multi-head attention mechanism, the proposed model can provide a preliminary explanation of engine degradation. Therefore, this approach is promising for predictive maintenance in IIoT scenarios.

Graphs are ubiquitous in nature and can therefore serve as models for many practical but also theoretical problems. For this purpose, they can be defined as many different types which suitably reflect the individual contexts of the represented problem. To address cutting-edge problems based on graph data, the research field of Graph Neural Networks (GNNs) has emerged. Despite the field's youth and the speed at which new models are developed, many recent surveys have been published to keep track of them. Nevertheless, it has not yet been gathered which GNN can process what kind of graph types. In this survey, we give a detailed overview of already existing GNNs and, unlike previous surveys, categorize them according to their ability to handle different graph types and properties. We consider GNNs operating on static and dynamic graphs of different structural constitutions, with or without node or edge attributes. Moreover, we distinguish between GNN models for discrete-time or continuous-time dynamic graphs and group the models according to their architecture. We find that there are still graph types that are not or only rarely covered by existing GNN models. We point out where models are missing and give potential reasons for their absence.

时空数据包含丰富的信息，近年来由于许多领域的相关应用程序的快速发展，近年来已广泛研究。例如，医疗机构经常使用与患者不同部位相关的电极来分析具有空间和时间特征富含脑的数据，以进行健康评估和疾病诊断。现有的研究主要使用了深度学习技术，例如卷积神经网络（CNN）或经常性神经网络（RNN）来提取隐藏的时空特征。然而，同时合并相互依存的空间信息和动态时间变化是一项挑战。实际上，对于利用这些时空特征来完成复杂预测任务的模型，它通常需要大量的培训数据才能获得令人满意的模型性能。考虑到上述挑战，我们提出了一个自适应的联合相关性框架，即Fedrel，用于在本文中为时空的图形学习。在将原始时空数据转换为高质量特征之后，框架中的核心动力学间图（DIIG）模块能够使用这些功能来生成能够捕获隐藏拓扑和长期的时空图这些图中的时间相关信息。为了提高模型的概括能力和性能，在保留本地数据隐私的同时，我们还设计了一个相关性驱动的联合学习模块，以利用其模型的细心聚合来利用来自不同参与者的各种数据分布。

人口级社会事件，如民事骚乱和犯罪，往往对我们的日常生活产生重大影响。预测此类事件对于决策和资源分配非常重要。由于缺乏关于事件发生的真实原因和潜在机制的知识，事件预测传统上具有挑战性。近年来，由于两个主要原因，研究事件预测研究取得了重大进展：（1）机器学习和深度学习算法的开发和（2）社交媒体，新闻来源，博客，经济等公共数据的可访问性指标和其他元数据源。软件/硬件技术中的数据的爆炸性增长导致了社会事件研究中的深度学习技巧的应用。本文致力于提供社会事件预测的深层学习技术的系统和全面概述。我们专注于两个社会事件的域名：\ Texit {Civil unrest}和\ texit {犯罪}。我们首先介绍事件预测问题如何作为机器学习预测任务制定。然后，我们总结了这些问题的数据资源，传统方法和最近的深度学习模型的发展。最后，我们讨论了社会事件预测中的挑战，并提出了一些有希望的未来研究方向。

时间图代表实体之间的动态关系，并发生在许多现实生活中的应用中，例如社交网络，电子商务，通信，道路网络，生物系统等。他们需要根据其生成建模和表示学习的研究超出与静态图有关的研究。在这项调查中，我们全面回顾了近期针对处理时间图提出的神经时间依赖图表的学习和生成建模方法。最后，我们确定了现有方法的弱点，并讨论了我们最近发表的论文提格的研究建议[24]。