This paper studies the problem of embedding very large information networks into low-dimensional vector spaces, which is useful in many tasks such as visualization, node classification, and link prediction. Most existing graph embedding methods do not scale for real world information networks which usually contain millions of nodes. In this paper, we propose a novel network embedding method called the "LINE," which is suitable for arbitrary types of information networks: undirected, directed, and/or weighted. The method optimizes a carefully designed objective function that preserves both the local and global network structures. An edge-sampling algorithm is proposed that addresses the limitation of the classical stochastic gradient descent and improves both the effectiveness and the efficiency of the inference. Empirical experiments prove the effectiveness of the LINE on a variety of real-world information networks, including language networks, social networks, and citation networks. The algorithm is very efficient, which is able to learn the embedding of a network with millions of vertices and billions of edges in a few hours on a typical single machine. The source code of the LINE is available online. 1

图表表示学习是一种快速增长的领域，其中一个主要目标是在低维空间中产生有意义的图形表示。已经成功地应用了学习的嵌入式来执行各种预测任务，例如链路预测，节点分类，群集和可视化。图表社区的集体努力提供了数百种方法，但在所有评估指标下没有单一方法擅长，例如预测准确性，运行时间，可扩展性等。该调查旨在通过考虑算法来评估嵌入方法的所有主要类别的图表变体，参数选择，可伸缩性，硬件和软件平台，下游ML任务和多样化数据集。我们使用包含手动特征工程，矩阵分解，浅神经网络和深图卷积网络的分类法组织了图形嵌入技术。我们使用广泛使用的基准图表评估了节点分类，链路预测，群集和可视化任务的这些类别算法。我们在Pytorch几何和DGL库上设计了我们的实验，并在不同的多核CPU和GPU平台上运行实验。我们严格地审查了各种性能指标下嵌入方法的性能，并总结了结果。因此，本文可以作为比较指南，以帮助用户选择最适合其任务的方法。

Network embedding is an important method to learn low-dimensional representations of vertexes in networks, aiming to capture and preserve the network structure. Almost all the existing network embedding methods adopt shallow models. However, since the underlying network structure is complex, shallow models cannot capture the highly non-linear network structure, resulting in sub-optimal network representations. Therefore, how to find a method that is able to effectively capture the highly non-linear network structure and preserve the global and local structure is an open yet important problem. To solve this problem, in this paper we propose a Structural Deep Network Embedding method, namely SDNE. More specifically, we first propose a semi-supervised deep model, which has multiple layers of non-linear functions, thereby being able to capture the highly non-linear network structure. Then we propose to exploit the first-order and second-order proximity jointly to preserve the network structure. The second-order proximity is used by the unsupervised component to capture the global network structure. While the first-order proximity is used as the supervised information in the supervised component to preserve the local network structure. By jointly optimizing them in the semi-supervised deep model, our method can preserve both the local and global network structure and is robust to sparse networks. Empirically, we conduct the experiments on five real-world networks, including a language network, a citation network and three social networks. The results show that compared to the baselines, our method can reconstruct the original network significantly better and achieves substantial gains in three applications, i.e. multi-label classification, link prediction and visualization.

Clustering is a fundamental problem in network analysis that finds closely connected groups of nodes and separates them from other nodes in the graph, while link prediction is to predict whether two nodes in a network are likely to have a link. The definition of both naturally determines that clustering must play a positive role in obtaining accurate link prediction tasks. Yet researchers have long ignored or used inappropriate ways to undermine this positive relationship. In this article, We construct a simple but efficient clustering-driven link prediction framework(ClusterLP), with the goal of directly exploiting the cluster structures to obtain connections between nodes as accurately as possible in both undirected graphs and directed graphs. Specifically, we propose that it is easier to establish links between nodes with similar representation vectors and cluster tendencies in undirected graphs, while nodes in a directed graphs can more easily point to nodes similar to their representation vectors and have greater influence in their own cluster. We customized the implementation of ClusterLP for undirected and directed graphs, respectively, and the experimental results using multiple real-world networks on the link prediction task showed that our models is highly competitive with existing baseline models. The code implementation of ClusterLP and baselines we use are available at https://github.com/ZINUX1998/ClusterLP.

网络表示学习（NRL）方法在过去几年中受到了重大关注，因此由于它们在几个图形分析问题中的成功，包括节点分类，链路预测和聚类。这种方法旨在以一种保留网络的结构信息的方式将网络的每个顶点映射到低维空间中。特别感兴趣的是基于随机行走的方法;这些方法将网络转换为节点序列的集合，旨在通过预测序列内每个节点的上下文来学习节点表示。在本文中，我们介绍了一种通用框架，以增强通过基于主题信息的随机行走方法获取的节点的嵌入。类似于自然语言处理中局部单词嵌入的概念，所提出的模型首先将每个节点分配给潜在社区，并有利于各种统计图模型和社区检测方法，然后了解增强的主题感知表示。我们在两个下游任务中评估我们的方法：节点分类和链路预测。实验结果表明，通过纳入节点和社区嵌入，我们能够以广泛的广泛的基线NRL模型表明。

Traditional network embedding primarily focuses on learning a continuous vector representation for each node, preserving network structure and/or node content information, such that off-the-shelf machine learning algorithms can be easily applied to the vector-format node representations for network analysis. However, the learned continuous vector representations are inefficient for large-scale similarity search, which often involves finding nearest neighbors measured by distance or similarity in a continuous vector space. In this paper, we propose a search efficient binary network embedding algorithm called BinaryNE to learn a binary code for each node, by simultaneously modeling node context relations and node attribute relations through a three-layer neural network. BinaryNE learns binary node representations through a stochastic gradient descent based online learning algorithm. The learned binary encoding not only reduces memory usage to represent each node, but also allows fast bit-wise comparisons to support faster node similarity search than using Euclidean distance or other distance measures. Extensive experiments and comparisons demonstrate that BinaryNE not only delivers more than 25 times faster search speed, but also provides comparable or better search quality than traditional continuous vector based network embedding methods. The binary codes learned by BinaryNE also render competitive performance on node classification and node clustering tasks. The source code of this paper is available at https://github.com/daokunzhang/BinaryNE.

Graph is an important data representation which appears in a wide diversity of real-world scenarios. Effective graph analytics provides users a deeper understanding of what is behind the data, and thus can benefit a lot of useful applications such as node classification, node recommendation, link prediction, etc. However, most graph analytics methods suffer the high computation and space cost. Graph embedding is an effective yet efficient way to solve the graph analytics problem. It converts the graph data into a low dimensional space in which the graph structural information and graph properties are maximumly preserved. In this survey, we conduct a comprehensive review of the literature in graph embedding. We first introduce the formal definition of graph embedding as well as the related concepts. After that, we propose two taxonomies of graph embedding which correspond to what challenges exist in different graph embedding problem settings and how the existing work address these challenges in their solutions. Finally, we summarize the applications that graph embedding enables and suggest four promising future research directions in terms of computation efficiency, problem settings, techniques and application scenarios.

We present DeepWalk, a novel approach for learning latent representations of vertices in a network. These latent representations encode social relations in a continuous vector space, which is easily exploited by statistical models. Deep-Walk generalizes recent advancements in language modeling and unsupervised feature learning (or deep learning) from sequences of words to graphs.DeepWalk uses local information obtained from truncated random walks to learn latent representations by treating walks as the equivalent of sentences. We demonstrate DeepWalk's latent representations on several multi-label network classification tasks for social networks such as Blog-Catalog, Flickr, and YouTube. Our results show that Deep-Walk outperforms challenging baselines which are allowed a global view of the network, especially in the presence of missing information. DeepWalk's representations can provide F1 scores up to 10% higher than competing methods when labeled data is sparse. In some experiments, Deep-Walk's representations are able to outperform all baseline methods while using 60% less training data.DeepWalk is also scalable. It is an online learning algorithm which builds useful incremental results, and is trivially parallelizable. These qualities make it suitable for a broad class of real world applications such as network classification, and anomaly detection.

Prediction tasks over nodes and edges in networks require careful effort in engineering features used by learning algorithms. Recent research in the broader field of representation learning has led to significant progress in automating prediction by learning the features themselves. However, present feature learning approaches are not expressive enough to capture the diversity of connectivity patterns observed in networks.Here we propose node2vec, an algorithmic framework for learning continuous feature representations for nodes in networks. In node2vec, we learn a mapping of nodes to a low-dimensional space of features that maximizes the likelihood of preserving network neighborhoods of nodes. We define a flexible notion of a node's network neighborhood and design a biased random walk procedure, which efficiently explores diverse neighborhoods. Our algorithm generalizes prior work which is based on rigid notions of network neighborhoods, and we argue that the added flexibility in exploring neighborhoods is the key to learning richer representations.We demonstrate the efficacy of node2vec over existing state-ofthe-art techniques on multi-label classification and link prediction in several real-world networks from diverse domains. Taken together, our work represents a new way for efficiently learning stateof-the-art task-independent representations in complex networks.

我们研究大规模网络嵌入问题，旨在学习网络挖掘应用的低维潜在表示。网络嵌入领域的最新研究导致了大型进展，如深散，线，NetMF，NetSMF。然而，许多真实网络的巨大尺寸使得从整个网络学习网络嵌入的网络昂贵。在这项工作中，我们提出了一种新的网络嵌入方法，称为“NES”，其学习来自小型代表性子图的网络嵌入。 NES利用图表采样的理论，以有效地构建具有较小尺寸的代表性子图，该子图尺寸可用于对完整网络进行推断，使得能够显着提高嵌入学习的效率。然后，NES有效地计算从该代表子图嵌入的网络。与众所周知的方法相比，对各种规模和类型网络的广泛实验表明NES实现了可比性和显着的效率优势。

在低维空间中节点的学习表示是一项至关重要的任务，在网络分析中具有许多有趣的应用，包括链接预测，节点分类和可视化。解决此问题的两种流行方法是矩阵分解和基于步行的随机模型。在本文中，我们旨在将两全其美的最好的人融合在一起，以学习节点表示。特别是，我们提出了一个加权矩阵分解模型，该模型编码有关网络节点的随机步行信息。这种新颖的表述的好处是，它使我们能够利用内核函数，而无需意识到确切的接近矩阵，从而增强现有矩阵分解方法的表达性，并减轻其计算复杂性。我们通过多个内核学习公式扩展了方法，该公式提供了学习内核作为以数据驱动方式的词典的线性组合的灵活性。我们在现实世界网络上执行经验评估，表明所提出的模型优于基线节点嵌入下游机器学习任务中的算法。

Graph embedding algorithms embed a graph into a vector space where the structure and the inherent properties of the graph are preserved. The existing graph embedding methods cannot preserve the asymmetric transitivity well, which is a critical property of directed graphs. Asymmetric transitivity depicts the correlation among directed edges, that is, if there is a directed path from u to v, then there is likely a directed edge from u to v. Asymmetric transitivity can help in capturing structures of graphs and recovering from partially observed graphs. To tackle this challenge, we propose the idea of preserving asymmetric transitivity by approximating high-order proximity which are based on asymmetric transitivity. In particular, we develop a novel graph embedding algorithm, High-Order Proximity preserved Embedding (HOPE for short), which is scalable to preserve high-order proximities of large scale graphs and capable of capturing the asymmetric transitivity. More specifically, we first derive a general formulation that cover multiple popular highorder proximity measurements, then propose a scalable embedding algorithm to approximate the high-order proximity measurements based on their general formulation. Moreover, we provide a theoretical upper bound on the RMSE (Root Mean Squared Error) of the approximation. Our empirical experiments on a synthetic dataset and three realworld datasets demonstrate that HOPE can approximate the high-order proximities significantly better than the state-ofart algorithms and outperform the state-of-art algorithms in tasks of reconstruction, link prediction and vertex recommendation.

图表是一个宇宙数据结构，广泛用于组织现实世界中的数据。像交通网络，社交和学术网络这样的各种实际网络网络可以由图表代表。近年来，目睹了在网络中代表顶点的快速发展，进入低维矢量空间，称为网络表示学习。表示学习可以促进图形数据上的新算法的设计。在本调查中，我们对网络代表学习的当前文献进行了全面审查。现有算法可以分为三组：浅埋模型，异构网络嵌入模型，图形神经网络的模型。我们为每个类别审查最先进的算法，并讨论这些算法之间的基本差异。调查的一个优点是，我们系统地研究了不同类别的算法底层的理论基础，这提供了深入的见解，以更好地了解网络表示学习领域的发展。

Since the invention of word2vec [28,29], the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk [31] empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE [37], in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE [36] can be viewed as the joint factorization of multiple networks' Laplacians; (4) node2vec [16] is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method 1 as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.

一组广泛建立的无监督节点嵌入方法可以解释为由两个独特的步骤组成：i）基于兴趣图的相似性矩阵的定义，然后是II）ii）该矩阵的明确或隐式因素化。受这个观点的启发，我们提出了框架的两个步骤的改进。一方面，我们建议根据自由能距离编码节点相似性，该自由能距离在最短路径和通勤时间距离之间进行了插值，从而提供了额外的灵活性。另一方面，我们根据损耗函数提出了一种基质分解方法，该方法将Skip-Gram模型的损失函数推广到任意相似性矩阵。与基于广泛使用的$ \ ell_2 $损失的因素化相比，该方法可以更好地保留与较高相似性分数相关的节点对。此外，它可以使用高级自动分化工具包轻松实现，并通过利用GPU资源进行有效计算。在现实世界数据集上的节点聚类，节点分类和链接预测实验证明了与最先进的替代方案相比，合并基于自由能的相似性以及所提出的矩阵分解的有效性。

近年来，图形嵌入技术导致了重大进展。然而，目前的技术不足以捕获网络的模式。本文提出了邻居2VEC，一种基于邻居的采样策略使用算法来学习节点的邻域表示，通过节点与其邻居之间的特征传播来收集结构信息的框架。我们声称，邻居2VEC是一种简单有效的方法来提高可扩展性以及图形嵌入的平等，并且它破坏了现有最先进的无监督技术的限制。我们对诸如OGBN-ARXIV，OGBN-产品，OGBN-蛋白，OGBL-PPA，OGBL-COLLAB和OGBL-CTITE2等网络的多个节点分类和链路预测任务进行实验。结果表明，邻居2VEC的表示提供了比节点分类任务中的竞争方法高达6.8％的平均精度，以及链路预测任务中的3.0％。邻居2VEC的表示能够在所有六个实验中优于所有基线方法和两个古典GNN模型。

学习在动态环境中网络的低维拓扑表示由于许多真实网络的时间不断发展而引起了很多关注。动态网络嵌入（DNE）的主要和共同目标是有效更新节点嵌入品，同时在每次步骤保留网络拓扑时。大多数现有DNE方法的想法是捕获受影响的节点（而不是所有节点）的拓扑变化，并因此更新节点嵌入。遗憾的是，这种近似虽然可以提高效率，但是在每次步骤中不能有效地保留动态网络的全局拓扑，因为没有考虑通过高阶接近传播的累积拓扑变化的非活动子网。为了解决这一挑战，我们提出了一种新颖的节点选择策略，以在网络上多移地选择代表节点，这与基于Skip-gram的嵌入方法的新增量学习范例协调。广泛的实验显示Glodyne，较小的节点部分被选中，可以实现优越或相当的性能W.R.T.在三个典型的下游任务中最先进的DNE方法。特别是，Glodyne显着优于图形重建任务中的其他方法，这表明了其全球拓扑保存能力。源代码可在https://github.com/houchengbin/glodyne获得

最近有一项激烈的活动在嵌入非常高维和非线性数据结构的嵌入中，其中大部分在数据科学和机器学习文献中。我们分四部分调查这项活动。在第一部分中，我们涵盖了非线性方法，例如主曲线，多维缩放，局部线性方法，ISOMAP，基于图形的方法和扩散映射，基于内核的方法和随机投影。第二部分与拓扑嵌入方法有关，特别是将拓扑特性映射到持久图和映射器算法中。具有巨大增长的另一种类型的数据集是非常高维网络数据。第三部分中考虑的任务是如何将此类数据嵌入中等维度的向量空间中，以使数据适合传统技术，例如群集和分类技术。可以说，这是算法机器学习方法与统计建模（所谓的随机块建模）之间的对比度。在论文中，我们讨论了两种方法的利弊。调查的最后一部分涉及嵌入$ \ mathbb {r}^ 2 $，即可视化中。提出了三种方法：基于第一部分，第二和第三部分中的方法，$ t $ -sne，UMAP和大节。在两个模拟数据集上进行了说明和比较。一个由嘈杂的ranunculoid曲线组成的三胞胎，另一个由随机块模型和两种类型的节点产生的复杂性的网络组成。

图形嵌入是图形节点到一组向量的转换。良好的嵌入应捕获图形拓扑，节点与节点的关系以及有关图，其子图和节点的其他相关信息。如果实现了这些目标，则嵌入是网络中有意义的，可理解的，可理解的压缩表示形式，可用于其他机器学习工具，例如节点分类，社区检测或链接预测。主要的挑战是，需要确保嵌入很好地描述图形的属性。结果，选择最佳嵌入是一项具有挑战性的任务，并且通常需要领域专家。在本文中，我们在现实世界网络和人为生成的网络上进行了一系列广泛的实验，并使用选定的图嵌入算法进行了一系列的实验。根据这些实验，我们制定了两个一般结论。首先，如果需要在运行实验之前选择一种嵌入算法，则Node2Vec是最佳选择，因为它在我们的测试中表现最好。话虽如此，在所有测试中都没有单一的赢家，此外，大多数嵌入算法都具有应该调整并随机分配的超参数。因此，如果可能的话，我们对从业者的主要建议是生成几个问题的嵌入，然后使用一个通用框架，该框架为无监督的图形嵌入比较提供了工具。该框架（最近在文献中引入并在GitHub存储库中很容易获得）将分歧分数分配给嵌入，以帮助区分好的分数和不良的分数。

Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph (e.g., degree statistics or kernel functions). However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. Here we provide a conceptual review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph neural networks. We review methods to embed individual nodes as well as approaches to embed entire (sub)graphs. In doing so, we develop a unified framework to describe these recent approaches, and we highlight a number of important applications and directions for future work.