图表学习方法为解决图形所代表的复杂的现实世界问题打开了新的可能性。但是，这些应用程序中使用的许多图包括数百万节点和数十亿个边缘，并且超出了当前方法和软件实现的功能。我们提供葡萄，这是一种用于图形处理和表示学习的软件资源，能够通过使用专业和智能数据结构，算法和快速并行实现来通过大图扩展。与最先进的软件资源相比，葡萄显示出经验空间和时间复杂性的数量级的改善，以及边缘预测和节点标签预测性能的实质和统计学上的显着改善。此外，葡萄提供了来自文献和其他来源的80,000多种图，标准化界面允许直接整合第三方库，61个节点嵌入方法，25个推理模型和3个模块化管道，以允许公平且可重复的方法比较以及用于图形处理和嵌入的库。

图表表示学习是一种快速增长的领域，其中一个主要目标是在低维空间中产生有意义的图形表示。已经成功地应用了学习的嵌入式来执行各种预测任务，例如链路预测，节点分类，群集和可视化。图表社区的集体努力提供了数百种方法，但在所有评估指标下没有单一方法擅长，例如预测准确性，运行时间，可扩展性等。该调查旨在通过考虑算法来评估嵌入方法的所有主要类别的图表变体，参数选择，可伸缩性，硬件和软件平台，下游ML任务和多样化数据集。我们使用包含手动特征工程，矩阵分解，浅神经网络和深图卷积网络的分类法组织了图形嵌入技术。我们使用广泛使用的基准图表评估了节点分类，链路预测，群集和可视化任务的这些类别算法。我们在Pytorch几何和DGL库上设计了我们的实验，并在不同的多核CPU和GPU平台上运行实验。我们严格地审查了各种性能指标下嵌入方法的性能，并总结了结果。因此，本文可以作为比较指南，以帮助用户选择最适合其任务的方法。

在过去十年中，图形内核引起了很多关注，并在结构化数据上发展成为一种快速发展的学习分支。在过去的20年中，该领域发生的相当大的研究活动导致开发数十个图形内核，每个图形内核都对焦于图形的特定结构性质。图形内核已成功地成功地在广泛的域中，从社交网络到生物信息学。本调查的目标是提供图形内核的文献的统一视图。特别是，我们概述了各种图形内核。此外，我们对公共数据集的几个内核进行了实验评估，并提供了比较研究。最后，我们讨论图形内核的关键应用，并概述了一些仍有待解决的挑战。

在低维空间中节点的学习表示是一项至关重要的任务，在网络分析中具有许多有趣的应用，包括链接预测，节点分类和可视化。解决此问题的两种流行方法是矩阵分解和基于步行的随机模型。在本文中，我们旨在将两全其美的最好的人融合在一起，以学习节点表示。特别是，我们提出了一个加权矩阵分解模型，该模型编码有关网络节点的随机步行信息。这种新颖的表述的好处是，它使我们能够利用内核函数，而无需意识到确切的接近矩阵，从而增强现有矩阵分解方法的表达性，并减轻其计算复杂性。我们通过多个内核学习公式扩展了方法，该公式提供了学习内核作为以数据驱动方式的词典的线性组合的灵活性。我们在现实世界网络上执行经验评估，表明所提出的模型优于基线节点嵌入下游机器学习任务中的算法。

即使机器学习算法已经在数据科学中发挥了重要作用，但许多当前方法对输入数据提出了不现实的假设。由于不兼容的数据格式，或数据集中的异质，分层或完全缺少的数据片段，因此很难应用此类方法。作为解决方案，我们提出了一个用于样本表示，模型定义和培训的多功能，统一的框架，称为“ Hmill”。我们深入审查框架构建和扩展的机器学习的多个范围范式。从理论上讲，为HMILL的关键组件的设计合理，我们将通用近似定理的扩展显示到框架中实现的模型所实现的所有功能的集合。本文还包含有关我们实施中技术和绩效改进的详细讨论，该讨论将在MIT许可下发布供下载。该框架的主要资产是其灵活性，它可以通过相同的工具对不同的现实世界数据源进行建模。除了单独观察到每个对象的一组属性的标准设置外，我们解释了如何在框架中实现表示整个对象系统的图表中的消息推断。为了支持我们的主张，我们使用框架解决了网络安全域的三个不同问题。第一种用例涉及来自原始网络观察结果的IoT设备识别。在第二个问题中，我们研究了如何使用以有向图表示的操作系统的快照可以对恶意二进制文件进行分类。最后提供的示例是通过网络中实体之间建模域黑名单扩展的任务。在所有三个问题中，基于建议的框架的解决方案可实现与专业方法相当的性能。

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

Traditional network embedding primarily focuses on learning a continuous vector representation for each node, preserving network structure and/or node content information, such that off-the-shelf machine learning algorithms can be easily applied to the vector-format node representations for network analysis. However, the learned continuous vector representations are inefficient for large-scale similarity search, which often involves finding nearest neighbors measured by distance or similarity in a continuous vector space. In this paper, we propose a search efficient binary network embedding algorithm called BinaryNE to learn a binary code for each node, by simultaneously modeling node context relations and node attribute relations through a three-layer neural network. BinaryNE learns binary node representations through a stochastic gradient descent based online learning algorithm. The learned binary encoding not only reduces memory usage to represent each node, but also allows fast bit-wise comparisons to support faster node similarity search than using Euclidean distance or other distance measures. Extensive experiments and comparisons demonstrate that BinaryNE not only delivers more than 25 times faster search speed, but also provides comparable or better search quality than traditional continuous vector based network embedding methods. The binary codes learned by BinaryNE also render competitive performance on node classification and node clustering tasks. The source code of this paper is available at https://github.com/daokunzhang/BinaryNE.

最近有一项激烈的活动在嵌入非常高维和非线性数据结构的嵌入中，其中大部分在数据科学和机器学习文献中。我们分四部分调查这项活动。在第一部分中，我们涵盖了非线性方法，例如主曲线，多维缩放，局部线性方法，ISOMAP，基于图形的方法和扩散映射，基于内核的方法和随机投影。第二部分与拓扑嵌入方法有关，特别是将拓扑特性映射到持久图和映射器算法中。具有巨大增长的另一种类型的数据集是非常高维网络数据。第三部分中考虑的任务是如何将此类数据嵌入中等维度的向量空间中，以使数据适合传统技术，例如群集和分类技术。可以说，这是算法机器学习方法与统计建模（所谓的随机块建模）之间的对比度。在论文中，我们讨论了两种方法的利弊。调查的最后一部分涉及嵌入$ \ mathbb {r}^ 2 $，即可视化中。提出了三种方法：基于第一部分，第二和第三部分中的方法，$ t $ -sne，UMAP和大节。在两个模拟数据集上进行了说明和比较。一个由嘈杂的ranunculoid曲线组成的三胞胎，另一个由随机块模型和两种类型的节点产生的复杂性的网络组成。

数据处理的最新进展刺激了对非常大尺度的学习图的需求。众所周知，图形神经网络（GNN）是解决图形学习任务的一种新兴和有力的方法，很难扩大规模。大多数可扩展模型应用基于节点的技术来简化GNN的昂贵图形消息传播过程。但是，我们发现当应用于百万甚至数十亿尺度的图表时，这种加速度不足。在这项工作中，我们提出了Scara，这是一种可扩展的GNN，具有针对图形计算的特征优化。 Scara有效地计算出从节点功能中嵌入的图形，并进一步选择和重用功能计算结果以减少开销。理论分析表明，我们的模型在传播过程以及GNN培训和推理中具有确保精度，实现了子线性时间的复杂性。我们在各种数据集上进行了广泛的实验，以评估圣aca的功效和效率。与基线的性能比较表明，与快速收敛和可比精度相比，与当前的最新方法相比，圣aca最高可达到100倍的图形传播加速度。最值得注意的是，在100秒内处理最大的十亿个GNN数据集纸100m（1.11亿节点，1.6B边缘）上的预先计算是有效的。

Prediction tasks over nodes and edges in networks require careful effort in engineering features used by learning algorithms. Recent research in the broader field of representation learning has led to significant progress in automating prediction by learning the features themselves. However, present feature learning approaches are not expressive enough to capture the diversity of connectivity patterns observed in networks.Here we propose node2vec, an algorithmic framework for learning continuous feature representations for nodes in networks. In node2vec, we learn a mapping of nodes to a low-dimensional space of features that maximizes the likelihood of preserving network neighborhoods of nodes. We define a flexible notion of a node's network neighborhood and design a biased random walk procedure, which efficiently explores diverse neighborhoods. Our algorithm generalizes prior work which is based on rigid notions of network neighborhoods, and we argue that the added flexibility in exploring neighborhoods is the key to learning richer representations.We demonstrate the efficacy of node2vec over existing state-ofthe-art techniques on multi-label classification and link prediction in several real-world networks from diverse domains. Taken together, our work represents a new way for efficiently learning stateof-the-art task-independent representations in complex networks.

Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph (e.g., degree statistics or kernel functions). However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. Here we provide a conceptual review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph neural networks. We review methods to embed individual nodes as well as approaches to embed entire (sub)graphs. In doing so, we develop a unified framework to describe these recent approaches, and we highlight a number of important applications and directions for future work.

图表上的表示学习（也称为图形嵌入）显示了其对一系列机器学习应用程序（例如分类，预测和建议）的重大影响。但是，现有的工作在很大程度上忽略了现代应用程序中图和边缘的属性（或属性）中包含的丰富信息，例如，属性图表示的节点和边缘。迄今为止，大多数现有的图形嵌入方法要么仅关注具有图形拓扑的普通图，要么仅考虑节点上的属性。我们提出了PGE，这是一个图形表示学习框架，该框架将节点和边缘属性都包含到图形嵌入过程中。 PGE使用节点聚类来分配偏差来区分节点的邻居，并利用多个数据驱动的矩阵来汇总基于偏置策略采样的邻居的属性信息。 PGE采用了流行的邻里聚合归纳模型。我们通过显示PGE如何实现更好的嵌入结果的详细分析，并验证PGE的性能，而不是最新的嵌入方法嵌入方法在基准应用程序上的嵌入方法，例如节点分类和对现实世界中的链接预测数据集。

最近提出了基于子图的图表学习（SGRL）来应对规范图神经网络（GNNS）遇到的一些基本挑战，并在许多重要的数据科学应用（例如链接，关系和主题预测）中证明了优势。但是，当前的SGRL方法遇到了可伸缩性问题，因为它们需要为每个培训或测试查询提取子图。扩大规范GNN的最新解决方案可能不适用于SGRL。在这里，我们通过共同设计学习算法及其系统支持，为可扩展的SGRL提出了一种新颖的框架Surel。 Surel采用基于步行的子图表分解，并将步行重新形成子图，从而大大降低了子图提取的冗余并支持并行计算。具有数百万个节点和边缘的六个同质，异质和高阶图的实验证明了Surel的有效性和可扩展性。特别是，与SGRL基线相比，Surel可以实现10 $ \ times $ Quad-Up，具有可比甚至更好的预测性能；与规范GNN相比，Surel可实现50％的预测准确性。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

图表神经网络（GNNS）在各种机器学习任务中获得了表示学习的提高。然而，应用邻域聚合的大多数现有GNN通常在图中的图表上执行不良，其中相邻的节点属于不同的类。在本文中，我们示出了在典型的异界图中，边缘可以被引导，以及是否像是处理边缘，也可以使它们过度地影响到GNN模型的性能。此外，由于异常的限制，节点对来自本地邻域之外的类似节点的消息非常有益。这些激励我们开发一个自适应地学习图表的方向性的模型，并利用潜在的长距离相关性节点之间。我们首先将图拉普拉斯概括为基于所提出的特征感知PageRank算法向数字化，该算法同时考虑节点之间的图形方向性和长距离特征相似性。然后，Digraph Laplacian定义了一个图形传播矩阵，导致一个名为{\ em diglaciangcn}的模型。基于此，我们进一步利用节点之间的通勤时间测量的节点接近度，以便在拓扑级别上保留节点的远距离相关性。具有不同级别的10个数据集的广泛实验，同意级别展示了我们在节点分类任务任务中对现有解决方案的有效性。

图形嵌入是图形节点到一组向量的转换。良好的嵌入应捕获图形拓扑，节点与节点的关系以及有关图，其子图和节点的其他相关信息。如果实现了这些目标，则嵌入是网络中有意义的，可理解的，可理解的压缩表示形式，可用于其他机器学习工具，例如节点分类，社区检测或链接预测。主要的挑战是，需要确保嵌入很好地描述图形的属性。结果，选择最佳嵌入是一项具有挑战性的任务，并且通常需要领域专家。在本文中，我们在现实世界网络和人为生成的网络上进行了一系列广泛的实验，并使用选定的图嵌入算法进行了一系列的实验。根据这些实验，我们制定了两个一般结论。首先，如果需要在运行实验之前选择一种嵌入算法，则Node2Vec是最佳选择，因为它在我们的测试中表现最好。话虽如此，在所有测试中都没有单一的赢家，此外，大多数嵌入算法都具有应该调整并随机分配的超参数。因此，如果可能的话，我们对从业者的主要建议是生成几个问题的嵌入，然后使用一个通用框架，该框架为无监督的图形嵌入比较提供了工具。该框架（最近在文献中引入并在GitHub存储库中很容易获得）将分歧分数分配给嵌入，以帮助区分好的分数和不良的分数。

Graph is an important data representation which appears in a wide diversity of real-world scenarios. Effective graph analytics provides users a deeper understanding of what is behind the data, and thus can benefit a lot of useful applications such as node classification, node recommendation, link prediction, etc. However, most graph analytics methods suffer the high computation and space cost. Graph embedding is an effective yet efficient way to solve the graph analytics problem. It converts the graph data into a low dimensional space in which the graph structural information and graph properties are maximumly preserved. In this survey, we conduct a comprehensive review of the literature in graph embedding. We first introduce the formal definition of graph embedding as well as the related concepts. After that, we propose two taxonomies of graph embedding which correspond to what challenges exist in different graph embedding problem settings and how the existing work address these challenges in their solutions. Finally, we summarize the applications that graph embedding enables and suggest four promising future research directions in terms of computation efficiency, problem settings, techniques and application scenarios.

一组广泛建立的无监督节点嵌入方法可以解释为由两个独特的步骤组成：i）基于兴趣图的相似性矩阵的定义，然后是II）ii）该矩阵的明确或隐式因素化。受这个观点的启发，我们提出了框架的两个步骤的改进。一方面，我们建议根据自由能距离编码节点相似性，该自由能距离在最短路径和通勤时间距离之间进行了插值，从而提供了额外的灵活性。另一方面，我们根据损耗函数提出了一种基质分解方法，该方法将Skip-Gram模型的损失函数推广到任意相似性矩阵。与基于广泛使用的$ \ ell_2 $损失的因素化相比，该方法可以更好地保留与较高相似性分数相关的节点对。此外，它可以使用高级自动分化工具包轻松实现，并通过利用GPU资源进行有效计算。在现实世界数据集上的节点聚类，节点分类和链接预测实验证明了与最先进的替代方案相比，合并基于自由能的相似性以及所提出的矩阵分解的有效性。

Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions. * The two first authors made equal contributions. 1 While it is common to refer to these data structures as social or biological networks, we use the term graph to avoid ambiguity with neural network terminology.