Networks have become indispensable and ubiquitous structures in many fields to model the interactions among different entities, such as friendship in social networks or protein interactions in biological graphs. A major challenge is to understand the structure and dynamics of these systems. Although networks evolve through time, most existing graph representation learning methods target only static networks. Whereas approaches have been developed for the modeling of dynamic networks, there is a lack of efficient continuous time dynamic graph representation learning methods that can provide accurate network characterization and visualization in low dimensions while explicitly accounting for prominent network characteristics such as homophily and transitivity. In this paper, we propose the Piecewise-Velocity Model (PiVeM) for the representation of continuous-time dynamic networks. It learns dynamic embeddings in which the temporal evolution of nodes is approximated by piecewise linear interpolations based on a latent distance model with piecewise constant node-specific velocities. The model allows for analytically tractable expressions of the associated Poisson process likelihood with scalable inference invariant to the number of events. We further impose a scalable Kronecker structured Gaussian Process prior to the dynamics accounting for community structure, temporal smoothness, and disentangled (uncorrelated) latent embedding dimensions optimally learned to characterize the network dynamics. We show that PiVeM can successfully represent network structure and dynamics in ultra-low two-dimensional spaces. It outperforms relevant state-of-art methods in downstream tasks such as link prediction. In summary, PiVeM enables easily interpretable dynamic network visualizations and characterizations that can further improve our understanding of the intrinsic dynamics of time-evolving networks.

时间图代表实体之间的动态关系，并发生在许多现实生活中的应用中，例如社交网络，电子商务，通信，道路网络，生物系统等。他们需要根据其生成建模和表示学习的研究超出与静态图有关的研究。在这项调查中，我们全面回顾了近期针对处理时间图提出的神经时间依赖图表的学习和生成建模方法。最后，我们确定了现有方法的弱点，并讨论了我们最近发表的论文提格的研究建议[24]。

网络和时间点过程是建模各个领域中复杂动态关系数据的基本构件。我们建议使用节点的潜在空间表示形式，提出了潜在空间鹰队（LSH）模型，这是一种连续时间的关系网络的新型生成模型。我们使用共同令人兴奋的霍克斯工艺在节点之间建模关系事件，其基线强度取决于潜在空间中的节点与发件人和接收器特定效果之间的距离。我们证明，我们提出的LSH模型可以复制在包括互惠和传递性在内的真实时间网络中观察到的许多功能，同时还可以实现卓越的预测准确性并提供比现有模型更明显的拟合。

在低维空间中节点的学习表示是一项至关重要的任务，在网络分析中具有许多有趣的应用，包括链接预测，节点分类和可视化。解决此问题的两种流行方法是矩阵分解和基于步行的随机模型。在本文中，我们旨在将两全其美的最好的人融合在一起，以学习节点表示。特别是，我们提出了一个加权矩阵分解模型，该模型编码有关网络节点的随机步行信息。这种新颖的表述的好处是，它使我们能够利用内核函数，而无需意识到确切的接近矩阵，从而增强现有矩阵分解方法的表达性，并减轻其计算复杂性。我们通过多个内核学习公式扩展了方法，该公式提供了学习内核作为以数据驱动方式的词典的线性组合的灵活性。我们在现实世界网络上执行经验评估，表明所提出的模型优于基线节点嵌入下游机器学习任务中的算法。

网络表示学习（NRL）方法在过去几年中受到了重大关注，因此由于它们在几个图形分析问题中的成功，包括节点分类，链路预测和聚类。这种方法旨在以一种保留网络的结构信息的方式将网络的每个顶点映射到低维空间中。特别感兴趣的是基于随机行走的方法;这些方法将网络转换为节点序列的集合，旨在通过预测序列内每个节点的上下文来学习节点表示。在本文中，我们介绍了一种通用框架，以增强通过基于主题信息的随机行走方法获取的节点的嵌入。类似于自然语言处理中局部单词嵌入的概念，所提出的模型首先将每个节点分配给潜在社区，并有利于各种统计图模型和社区检测方法，然后了解增强的主题感知表示。我们在两个下游任务中评估我们的方法：节点分类和链路预测。实验结果表明，通过纳入节点和社区嵌入，我们能够以广泛的广泛的基线NRL模型表明。

Clustering is a fundamental problem in network analysis that finds closely connected groups of nodes and separates them from other nodes in the graph, while link prediction is to predict whether two nodes in a network are likely to have a link. The definition of both naturally determines that clustering must play a positive role in obtaining accurate link prediction tasks. Yet researchers have long ignored or used inappropriate ways to undermine this positive relationship. In this article, We construct a simple but efficient clustering-driven link prediction framework(ClusterLP), with the goal of directly exploiting the cluster structures to obtain connections between nodes as accurately as possible in both undirected graphs and directed graphs. Specifically, we propose that it is easier to establish links between nodes with similar representation vectors and cluster tendencies in undirected graphs, while nodes in a directed graphs can more easily point to nodes similar to their representation vectors and have greater influence in their own cluster. We customized the implementation of ClusterLP for undirected and directed graphs, respectively, and the experimental results using multiple real-world networks on the link prediction task showed that our models is highly competitive with existing baseline models. The code implementation of ClusterLP and baselines we use are available at https://github.com/ZINUX1998/ClusterLP.

最近，对从交互数据提取信息的大量兴趣。传统上，这是通过将其建模为动态网络中特定时间的配对交互来完成的。然而，真实世界的互动很少是对的;它们可以涉及超过两个节点。在文献中，这些类型的群组交互由HyperUredges /超链接建模。现有的HIFEBEGE建模工作仅关注静态网络，并且它们无法模拟节点的时间演变，因为它们与其他节点交互。此外，它们无法应答时间查询，如下一步以及发生交互时将发生的相互作用类型。为了解决这些限制，在本文中，我们开发了一种用于超链接预测的时间点过程模型。我们提出的模型使用用于节点的动态表示技术来模拟演化，并在神经点过程框架中使用该表示来制作推断。我们在五个现实世界交互数据上评估我们的模型，并显示我们的动态模型在静态模型上具有显着的性能增益。此外，我们还展示了我们在对双向交互建模技术上的技术的优势。

最近有一项激烈的活动在嵌入非常高维和非线性数据结构的嵌入中，其中大部分在数据科学和机器学习文献中。我们分四部分调查这项活动。在第一部分中，我们涵盖了非线性方法，例如主曲线，多维缩放，局部线性方法，ISOMAP，基于图形的方法和扩散映射，基于内核的方法和随机投影。第二部分与拓扑嵌入方法有关，特别是将拓扑特性映射到持久图和映射器算法中。具有巨大增长的另一种类型的数据集是非常高维网络数据。第三部分中考虑的任务是如何将此类数据嵌入中等维度的向量空间中，以使数据适合传统技术，例如群集和分类技术。可以说，这是算法机器学习方法与统计建模（所谓的随机块建模）之间的对比度。在论文中，我们讨论了两种方法的利弊。调查的最后一部分涉及嵌入$ \ mathbb {r}^ 2 $，即可视化中。提出了三种方法：基于第一部分，第二和第三部分中的方法，$ t $ -sne，UMAP和大节。在两个模拟数据集上进行了说明和比较。一个由嘈杂的ranunculoid曲线组成的三胞胎，另一个由随机块模型和两种类型的节点产生的复杂性的网络组成。

Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph (e.g., degree statistics or kernel functions). However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. Here we provide a conceptual review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph neural networks. We review methods to embed individual nodes as well as approaches to embed entire (sub)graphs. In doing so, we develop a unified framework to describe these recent approaches, and we highlight a number of important applications and directions for future work.

提出了一种新的动态网络模型，称为相互刺激的点处理图（MEG）。 MEG是一种可扩展的网络范围统计模型，用于多达数码标记的点进程，可用于评估未来事件的重要事件时，包括以前未观察到的连接的异常检测。该模型组合了互励磁点过程来估计事件和潜在空间模型之间的依赖性，以推断节点之间的关系。每个网络边缘的强度函数专用于节点特定参数参数，允许跨网络共享信息。这种结构甚至可以估计强度，即使对于未被观察的边缘，这在现实世界中尤其重要，例如网络安全中产生的计算机网络。获得了日志似然的递归形式，用于通过现代梯度上升算法推导快速推理过程。也导出了EM算法。该模型在模拟图和现实世界数据集上进行测试，展示出色的性能。

在过去十年中，图形内核引起了很多关注，并在结构化数据上发展成为一种快速发展的学习分支。在过去的20年中，该领域发生的相当大的研究活动导致开发数十个图形内核，每个图形内核都对焦于图形的特定结构性质。图形内核已成功地成功地在广泛的域中，从社交网络到生物信息学。本调查的目标是提供图形内核的文献的统一视图。特别是，我们概述了各种图形内核。此外，我们对公共数据集的几个内核进行了实验评估，并提供了比较研究。最后，我们讨论图形内核的关键应用，并概述了一些仍有待解决的挑战。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

Graphs are ubiquitous in nature and can therefore serve as models for many practical but also theoretical problems. For this purpose, they can be defined as many different types which suitably reflect the individual contexts of the represented problem. To address cutting-edge problems based on graph data, the research field of Graph Neural Networks (GNNs) has emerged. Despite the field's youth and the speed at which new models are developed, many recent surveys have been published to keep track of them. Nevertheless, it has not yet been gathered which GNN can process what kind of graph types. In this survey, we give a detailed overview of already existing GNNs and, unlike previous surveys, categorize them according to their ability to handle different graph types and properties. We consider GNNs operating on static and dynamic graphs of different structural constitutions, with or without node or edge attributes. Moreover, we distinguish between GNN models for discrete-time or continuous-time dynamic graphs and group the models according to their architecture. We find that there are still graph types that are not or only rarely covered by existing GNN models. We point out where models are missing and give potential reasons for their absence.

许多实际关系系统，如社交网络和生物系统，包含动态相互作用。在学习动态图形表示时，必须采用连续的时间信息和几何结构。主流工作通过消息传递网络（例如，GCN，GAT）实现拓扑嵌入。另一方面，时间演进通常通过在栅极机构中具有方便信息过滤的存储单元（例如，LSTM或GU）来表达。但是，由于过度复杂的编码，这种设计可以防止大规模的输入序列。这项工作从自我关注的哲学中学习，并提出了一种高效的基于频谱的神经单元，采用信息的远程时间交互。发达的频谱窗口单元（SWINIT）模型预测了具有保证效率的可扩展动态图形。该架构与一些构成随机SVD，MLP和图形帧卷积的一些简单的有效计算块组装。 SVD加MLP模块编码动态图事件的长期特征演进。帧卷积中的快速帧图形变换嵌入了结构动态。两种策略都提高了模型对可扩展分析的能力。特别地，迭代的SVD近似度将注意力的计算复杂性缩小到具有n个边缘和D边缘特征的动态图形的关注的计算复杂性，并且帧卷积的多尺度变换允许在网络训练中具有足够的可扩展性。我们的Swinit在各种在线连续时间动态图表学习任务中实现了最先进的性能，而与基线方法相比，可学习参数的数量可达七倍。

Temporal networks are an important type of network whose topological structure changes over time. Compared with methods on static networks, temporal network embedding (TNE) methods are facing three challenges: 1) it cannot describe the temporal dependence across network snapshots; 2) the node embedding in the latent space fails to indicate changes in the network topology; and 3) it cannot avoid a lot of redundant computation via parameter inheritance on a series of snapshots. To this end, we propose a novel temporal network embedding method named Dynamic Cluster Structure Constraint model (DyCSC), whose core idea is to capture the evolution of temporal networks by imposing a temporal constraint on the tendency of the nodes in the network to a given number of clusters. It not only generates low-dimensional embedding vectors for nodes but also preserves the dynamic nonlinear features of temporal networks. Experimental results on multiple realworld datasets have demonstrated the superiority of DyCSC for temporal graph embedding, as it consistently outperforms competing methods by significant margins in multiple temporal link prediction tasks. Moreover, the ablation study further validates the effectiveness of the proposed temporal constraint.

Graph AutoCododers（GAE）和变分图自动编码器（VGAE）作为链接预测的强大方法出现。他们的表现对社区探测问题的印象不那么令人印象深刻，根据最近和同意的实验评估，它们的表现通常超过了诸如louvain方法之类的简单替代方案。目前尚不清楚可以通过GAE和VGAE改善社区检测的程度，尤其是在没有节点功能的情况下。此外，不确定是否可以在链接预测上同时保留良好的性能。在本文中，我们表明，可以高精度地共同解决这两个任务。为此，我们介绍和理论上研究了一个社区保留的消息传递方案，通过在计算嵌入空间时考虑初始图形结构和基于模块化的先验社区来掺杂我们的GAE和VGAE编码器。我们还提出了新颖的培训和优化策略，包括引入一个模块化的正规器，以补充联合链路预测和社区检测的现有重建损失。我们通过对各种现实世界图的深入实验验证，证明了方法的经验有效性，称为模块化感知的GAE和VGAE。

Network data are ubiquitous in modern machine learning, with tasks of interest including node classification, node clustering and link prediction. A frequent approach begins by learning an Euclidean embedding of the network, to which algorithms developed for vector-valued data are applied. For large networks, embeddings are learned using stochastic gradient methods where the sub-sampling scheme can be freely chosen. Despite the strong empirical performance of such methods, they are not well understood theoretically. Our work encapsulates representation methods using a subsampling approach, such as node2vec, into a single unifying framework. We prove, under the assumption that the graph is exchangeable, that the distribution of the learned embedding vectors asymptotically decouples. Moreover, we characterize the asymptotic distribution and provided rates of convergence, in terms of the latent parameters, which includes the choice of loss function and the embedding dimension. This provides a theoretical foundation to understand what the embedding vectors represent and how well these methods perform on downstream tasks. Notably, we observe that typically used loss functions may lead to shortcomings, such as a lack of Fisher consistency.

复杂的网络是代表现实生活系统的图形，这些系统表现出独特的特征，这些特征在纯粹的常规或完全随机的图中未发现。由于基础过程的复杂性，对此类系统的研究至关重要，但具有挑战性。然而，由于大量网络数据的可用性，近几十年来，这项任务变得更加容易。复杂网络中的链接预测旨在估计网络中缺少两个节点之间的链接的可能性。由于数据收集的不完美或仅仅是因为它们尚未出现，因此可能会缺少链接。发现网络数据中实体之间的新关系吸引了研究人员在社会学，计算机科学，物理学和生物学等各个领域的关注。大多数现有研究的重点是无向复杂网络中的链接预测。但是，并非所有现实生活中的系统都可以忠实地表示为无向网络。当使用链接预测算法时，通常会做出这种简化的假设，但不可避免地会导致有关节点之间关系和预测性能中降解的信息的丢失。本文介绍了针对有向网络的明确设计的链接预测方法。它基于相似性范式，该范式最近已证明在无向网络中成功。提出的算法通过在相似性和受欢迎程度上将其建模为不对称性来处理节点关系中的不对称性。鉴于观察到的网络拓扑结构，该算法将隐藏的相似性近似为最短路径距离，并使用边缘权重捕获并取消链接的不对称性和节点的受欢迎程度。在现实生活中评估了所提出的方法，实验结果证明了其在预测各种网络数据类型和大小的丢失链接方面的有效性。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

时间点过程作为连续域的随机过程通常用于模拟具有发生时间戳的异步事件序列。由于深度神经网络的强烈表达性，在时间点过程的背景下，它们是捕获异步序列中的模式的有希望的选择。在本文中，我们首先审查了最近的研究强调和困难，在深处时间点过程建模异步事件序列，可以得出四个领域：历史序列的编码，条件强度函数的制定，事件的关系发现和学习方法优化。我们通过将其拆除进入四个部分来介绍最近提出的模型，并通过对公平实证评估的相同学习策略进行重新涂布前三个部分进行实验。此外，我们扩展了历史编码器和条件强度函数家族，并提出了一种GRANGER因果区发现框架，用于利用多种事件之间的关系。因为格兰杰因果关系可以由格兰杰因果关系图表示，所以采用分层推断框架中的离散图结构学习来揭示图的潜在结构。进一步的实验表明，具有潜在图表发现的提议框架可以捕获关系并实现改进的拟合和预测性能。