Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.
translated by 谷歌翻译
Contrastive deep graph clustering, which aims to divide nodes into disjoint groups via contrastive mechanisms, is a challenging research spot. Among the recent works, hard sample mining-based algorithms have achieved great attention for their promising performance. However, we find that the existing hard sample mining methods have two problems as follows. 1) In the hardness measurement, the important structural information is overlooked for similarity calculation, degrading the representativeness of the selected hard negative samples. 2) Previous works merely focus on the hard negative sample pairs while neglecting the hard positive sample pairs. Nevertheless, samples within the same cluster but with low similarity should also be carefully learned. To solve the problems, we propose a novel contrastive deep graph clustering method dubbed Hard Sample Aware Network (HSAN) by introducing a comprehensive similarity measure criterion and a general dynamic sample weighing strategy. Concretely, in our algorithm, the similarities between samples are calculated by considering both the attribute embeddings and the structure embeddings, better revealing sample relationships and assisting hardness measurement. Moreover, under the guidance of the carefully collected high-confidence clustering information, our proposed weight modulating function will first recognize the positive and negative samples and then dynamically up-weight the hard sample pairs while down-weighting the easy ones. In this way, our method can mine not only the hard negative samples but also the hard positive sample, thus improving the discriminative capability of the samples further. Extensive experiments and analyses demonstrate the superiority and effectiveness of our proposed method.
translated by 谷歌翻译
对比度学习最近引起了深度群集的充满希望的表现。但是,复杂的数据增强和耗时的图卷积操作破坏了这些方法的效率。为了解决此问题,我们提出了一种简单的对比度图聚类(SCGC)算法,以从网络体系结构,数据增强和目标函数的角度改进现有方法。至于架构,我们的网络包括两个主要部分,即预处理和网络骨干。一个简单的低通denoising操作将邻居信息聚合作为独立的预处理,仅包括两个多层感知器(MLP)作为骨干。对于数据增强,我们没有通过图形引入复杂操作,而是通过设计参数UNSHARED SIAMESE编码并直接损坏节点嵌入的参数来构造同一顶点的两个增强视图。最后,关于目标函数,为了进一步提高聚类性能,新型的跨视图结构一致性目标函数旨在增强学习网络的判别能力。七个基准数据集的广泛实验结果验证了我们提出的算法的有效性和优势。值得注意的是,我们的算法的表现超过了最近的对比群集竞争对手,平均速度至少七倍。
translated by 谷歌翻译
Graph contrastive learning is an important method for deep graph clustering. The existing methods first generate the graph views with stochastic augmentations and then train the network with a cross-view consistency principle. Although good performance has been achieved, we observe that the existing augmentation methods are usually random and rely on pre-defined augmentations, which is insufficient and lacks negotiation between the final clustering task. To solve the problem, we propose a novel Graph Contrastive Clustering method with the Learnable graph Data Augmentation (GCC-LDA), which is optimized completely by the neural networks. An adversarial learning mechanism is designed to keep cross-view consistency in the latent space while ensuring the diversity of augmented views. In our framework, a structure augmentor and an attribute augmentor are constructed for augmentation learning in both structure level and attribute level. To improve the reliability of the learned affinity matrix, clustering is introduced to the learning procedure and the learned affinity matrix is refined with both the high-confidence pseudo-label matrix and the cross-view sample similarity matrix. During the training procedure, to provide persistent optimization for the learned view, we design a two-stage training strategy to obtain more reliable clustering information. Extensive experimental results demonstrate the effectiveness of GCC-LDA on six benchmark datasets.
translated by 谷歌翻译
深图形聚类,旨在揭示底层的图形结构并将节点划分为不同的群体,近年来引起了密集的关注。然而,我们观察到,在节点编码的过程中,现有方法遭受表示崩溃,这倾向于将所有数据映射到相同的表示中。因此,节点表示的鉴别能力是有限的,导致不满足的聚类性能。为了解决这个问题,我们提出了一种新颖的自我监督的深图聚类方法,通过以双向还原信息相关性来称呼双重关联减少网络(DCRN)。具体而言,在我们的方法中,我们首先将暹罗网络设计为编码样本。然后通过强制跨视图样本相关矩阵和跨视图特征相关矩阵分别近似两个标识矩阵,我们减少了双级的信息相关性,从而提高了所得特征的判别能力。此外,为了减轻通过在GCN中过度平滑引起的表示崩溃,我们引入了传播正规化术语,使网络能够利用浅网络结构获得远程信息。六个基准数据集的广泛实验结果证明了提出的DCRN对现有最先进方法的有效性。
translated by 谷歌翻译
近年来,图形神经网络(GNNS)在半监督节点分类中实现了有希望的性能。但是,监督不足的问题以及代表性崩溃,在很大程度上限制了GNN在该领域的性能。为了减轻半监督场景中节点表示的崩溃,我们提出了一种新型的图形对比学习方法,称为混合图对比度网络(MGCN)。在我们的方法中,我们通过扩大决策边界的边距并提高潜在表示的跨视图一致性来提高潜在特征的歧视能力。具体而言,我们首先采用了基于插值的策略来在潜在空间中进行数据增强,然后迫使预测模型在样本之间进行线性更改。其次,我们使学习的网络能够通过强迫跨视图的相关矩阵近似身份矩阵来分开两个插值扰动视图的样品。通过结合两个设置,我们从丰富的未标记节点和罕见但有价值的标记节点中提取丰富的监督信息,以进行判别表示学习。六个数据集的广泛实验结果证明了与现有最​​新方法相比,MGCN的有效性和普遍性。
translated by 谷歌翻译
长期以来,半监督学习(SSL)已被证明是一种有限的标签模型的有效技术。在现有的文献中,基于一致性的基于正则化的方法,这些方法迫使扰动样本具有类似的预测,而原始的样本则引起了极大的关注。但是,我们观察到,当标签变得极为有限时,例如,每个类别的2或3标签时,此类方法的性能会大大降低。我们的实证研究发现,主要问题在于语义信息在数据增强过程中的漂移。当提供足够的监督时,可以缓解问题。但是,如果几乎没有指导,错误的正则化将误导网络并破坏算法的性能。为了解决该问题,我们(1)提出了一种基于插值的方法来构建更可靠的正样品对; (2)设计一种新颖的对比损失,以指导学习网络的嵌入以在样品之间进行线性更改,从而通过扩大保证金决策边界来提高网络的歧视能力。由于未引入破坏性正则化,因此我们提出的算法的性能在很大程度上得到了改善。具体而言,所提出的算法的表现优于第二好算法(COMATT),而当CIFAR-10数据集中的每个类只有两个标签可用时,可以实现88.73%的分类精度,占5.3%。此外,我们通过通过我们提出的策略大大改善现有最新算法的性能,进一步证明了所提出的方法的普遍性。
translated by 谷歌翻译
图形相似性学习是指计算两个图之间的相似性得分,这在许多现实的应用程序(例如视觉跟踪,图形分类和协作过滤)中需要。由于大多数现有的图形神经网络产生了单个图的有效图表,因此几乎没有努力共同学习两个图表并计算其相似性得分。此外,现有的无监督图相似性学习方法主要基于聚类,它忽略了图对中体现的有价值的信息。为此,我们提出了一个对比度图匹配网络(CGMN),以进行自我监督的图形相似性学习,以计算任何两个输入图对象之间的相似性。具体而言,我们分别在一对中为每个图生成两个增强视图。然后,我们采用两种策略,即跨视图相互作用和跨刻画相互作用,以实现有效的节点表示学习。前者求助于两种观点中节点表示的一致性。后者用于识别不同图之间的节点差异。最后,我们通过汇总操作进行图形相似性计算将节点表示形式转换为图形表示。我们已经在八个现实世界数据集上评估了CGMN,实验结果表明,所提出的新方法优于图形相似性学习下游任务的最新方法。
translated by 谷歌翻译
图形表示学习(GRL)属性缺失的图表,这是一个常见的难以具有挑战性的问题,最近引起了相当大的关注。我们观察到现有文献:1)隔离属性和结构嵌入的学习因此未能采取两种类型的信息的充分优势; 2)对潜伏空间变量的分布假设施加过于严格的分布假设,从而导致差异较少的特征表示。在本文中,基于在两个信息源之间引入亲密信息交互的想法,我们提出了我们的暹罗属性丢失的图形自动编码器(SAGA)。具体而言,已经进行了三种策略。首先,我们通过引入暹罗网络结构来共享两个进程学习的参数来纠缠嵌入属性嵌入和结构嵌入,这允许网络培训从更丰富和不同的信息中受益。其次,我们介绍了一个K到最近的邻居(knn)和结构约束,增强了学习机制,通过过滤不可靠的连接来提高缺失属性的潜在特征的质量。第三,我们手动掩盖多个相邻矩阵上的连接,并强力嵌入子网恢复真正的相邻矩阵,从而强制实现所得到的网络能够选择性地利用更高级别的判别特征来进行数据完成。六个基准数据集上的广泛实验表明了我们传奇的优越性,反对最先进的方法。
translated by 谷歌翻译
尽管有关超图的机器学习吸引了很大的关注,但大多数作品都集中在(半)监督的学习上,这可能会导致繁重的标签成本和不良的概括。最近,对比学习已成为一种成功的无监督表示学习方法。尽管其他领域中对比度学习的发展繁荣,但对超图的对比学习仍然很少探索。在本文中,我们提出了Tricon(三个方向对比度学习),这是对超图的对比度学习的一般框架。它的主要思想是三个方向对比度,具体来说,它旨在在两个增强视图中最大化同一节点之间的协议(a),(b)在同一节点之间以及(c)之间,每个组之间的成员及其成员之间的协议(b) 。加上简单但令人惊讶的有效数据增强和负抽样方案,这三种形式的对比使Tricon能够在节点嵌入中捕获显微镜和介观结构信息。我们使用13种基线方法,5个数据集和两个任务进行了广泛的实验,这证明了Tricon的有效性,最明显的是,Tricon始终优于无监督的竞争对手,而且(半)受监督的竞争对手,大多数是由大量的节点分类的大量差额。
translated by 谷歌翻译
Graph Contrastive Learning (GCL) has recently drawn much research interest for learning generalizable node representations in a self-supervised manner. In general, the contrastive learning process in GCL is performed on top of the representations learned by a graph neural network (GNN) backbone, which transforms and propagates the node contextual information based on its local neighborhoods. However, nodes sharing similar characteristics may not always be geographically close, which poses a great challenge for unsupervised GCL efforts due to their inherent limitations in capturing such global graph knowledge. In this work, we address their inherent limitations by proposing a simple yet effective framework -- Simple Neural Networks with Structural and Semantic Contrastive Learning} (S^3-CL). Notably, by virtue of the proposed structural and semantic contrastive learning algorithms, even a simple neural network can learn expressive node representations that preserve valuable global structural and semantic patterns. Our experiments demonstrate that the node representations learned by S^3-CL achieve superior performance on different downstream tasks compared with the state-of-the-art unsupervised GCL methods. Implementation and more experimental details are publicly available at \url{https://github.com/kaize0409/S-3-CL.}
translated by 谷歌翻译
Contrastive learning (CL), which can extract the information shared between different contrastive views, has become a popular paradigm for vision representation learning. Inspired by the success in computer vision, recent work introduces CL into graph modeling, dubbed as graph contrastive learning (GCL). However, generating contrastive views in graphs is more challenging than that in images, since we have little prior knowledge on how to significantly augment a graph without changing its labels. We argue that typical data augmentation techniques (e.g., edge dropping) in GCL cannot generate diverse enough contrastive views to filter out noises. Moreover, previous GCL methods employ two view encoders with exactly the same neural architecture and tied parameters, which further harms the diversity of augmented views. To address this limitation, we propose a novel paradigm named model augmented GCL (MA-GCL), which will focus on manipulating the architectures of view encoders instead of perturbing graph inputs. Specifically, we present three easy-to-implement model augmentation tricks for GCL, namely asymmetric, random and shuffling, which can respectively help alleviate high- frequency noises, enrich training instances and bring safer augmentations. All three tricks are compatible with typical data augmentations. Experimental results show that MA-GCL can achieve state-of-the-art performance on node classification benchmarks by applying the three tricks on a simple base model. Extensive studies also validate our motivation and the effectiveness of each trick. (Code, data and appendix are available at https://github.com/GXM1141/MA-GCL. )
translated by 谷歌翻译
图级表示在各种现实世界中至关重要,例如预测分子的特性。但是实际上,精确的图表注释通常非常昂贵且耗时。为了解决这个问题,图形对比学习构造实例歧视任务,将正面对(同一图的增强对)汇总在一起,并将负面对(不同图的增强对)推开,以进行无监督的表示。但是,由于为了查询,其负面因素是从所有图中均匀抽样的,因此现有方法遭受关键采样偏置问题的损失,即,否定物可能与查询具有相同的语义结构,从而导致性能降解。为了减轻这种采样偏见问题,在本文中,我们提出了一种典型的图形对比度学习(PGCL)方法。具体而言,PGCL通过将语义相似的图形群群归为同一组的群集数据的基础语义结构,并同时鼓励聚类的一致性,以实现同一图的不同增强。然后给出查询,它通过从与查询群集不同的群集中绘制图形进行负采样,从而确保查询及其阴性样本之间的语义差异。此外,对于查询,PGCL根据其原型(集群质心)和查询原型之间的距离进一步重新重新重新重新重新享受其负样本,从而使那些具有中等原型距离的负面因素具有相对较大的重量。事实证明,这种重新加权策略比统一抽样更有效。各种图基准的实验结果证明了我们的PGCL比最新方法的优势。代码可在https://github.com/ha-lins/pgcl上公开获取。
translated by 谷歌翻译
归因图群集是图形分析字段中最重要的任务之一,其目的是将具有相似表示的节点分组到没有手动指导的情况下。基于图形对比度学习的最新研究在处理图形结构数据方面取得了令人印象深刻的结果。但是,现有的基于图形学习的方法1)不要直接解决聚类任务,因为表示和聚类过程是分开的; 2)过多地取决于图数据扩展,这极大地限制了对比度学习的能力; 3)忽略子空间聚类的对比度消息。为了适应上述问题,我们提出了一个通用框架,称为双重对比归因于图形聚类网络(DCAGC)。在DCAGC中,通过利用邻里对比模块,将最大化邻居节点的相似性,并提高节点表示的质量。同时,对比度自我表达模块是通过在自我表达层重建之前和之后最小化节点表示形式来构建的,以获得用于光谱群集的区分性自我表达矩阵。 DCAGC的所有模块均在统一框架中训练和优化,因此学习的节点表示包含面向群集的消息。与16种最先进的聚类方法相比,四个属性图数据集的大量实验结果显示了DCAGC的优势。本文的代码可在https://github.com/wangtong627/dual-contrastive-attributed-graph-cluster-clustering-network上获得。
translated by 谷歌翻译
Knowledge graph embedding (KGE) aims to learn powerful representations to benefit various artificial intelligence applications, such as question answering and recommendations. Meanwhile, contrastive learning (CL), as an effective mechanism to enhance the discriminative capacity of the learned representations, has been leveraged in different fields, especially graph-based models. However, since the structures of knowledge graphs (KGs) are usually more complicated compared to homogeneous graphs, it is hard to construct appropriate contrastive sample pairs. In this paper, we find that the entities within a symmetrical structure are usually more similar and correlated. This key property can be utilized to construct contrastive positive pairs for contrastive learning. Following the ideas above, we propose a relational symmetrical structure based knowledge graph contrastive learning framework, termed KGE-SymCL, which leverages the symmetrical structure information in KGs to enhance the discriminative ability of KGE models. Concretely, a plug-and-play approach is designed by taking the entities in the relational symmetrical positions as the positive samples. Besides, a self-supervised alignment loss is used to pull together the constructed positive sample pairs for contrastive learning. Extensive experimental results on benchmark datasets have verified the good generalization and superiority of the proposed framework.
translated by 谷歌翻译
灵感来自最近应用于图像上的自我监督方法的成功,图形结构数据的自我监督学习已经看到迅速增长,特别是基于增强的对比方法。但是,我们认为没有精心设计的增强技术,图形上的增强可能是任意行为的,因为图形的底层语义可以急剧地改变。因此,现有增强的方法的性能高度依赖于增强方案的选择,即与增强相关联的超级参数。在本文中,我们提出了一种名为AFGRL的图表的一种新的增强自我监督学习框架。具体地,我们通过发现与图形共享本地结构信息和全局语义的节点来生成图表的替代视图。各种数据集的各种节点级任务,即节点分类,群集和相似性搜索的广泛实验证明了AFGRL的优越性。 AFGRL的源代码可在https://github.com/namkyeong/afgrl中获得。
translated by 谷歌翻译
Inspired by the impressive success of contrastive learning (CL), a variety of graph augmentation strategies have been employed to learn node representations in a self-supervised manner. Existing methods construct the contrastive samples by adding perturbations to the graph structure or node attributes. Although impressive results are achieved, it is rather blind to the wealth of prior information assumed: with the increase of the perturbation degree applied on the original graph, 1) the similarity between the original graph and the generated augmented graph gradually decreases; 2) the discrimination between all nodes within each augmented view gradually increases. In this paper, we argue that both such prior information can be incorporated (differently) into the contrastive learning paradigm following our general ranking framework. In particular, we first interpret CL as a special case of learning to rank (L2R), which inspires us to leverage the ranking order among positive augmented views. Meanwhile, we introduce a self-ranking paradigm to ensure that the discriminative information among different nodes can be maintained and also be less altered to the perturbations of different degrees. Experiment results on various benchmark datasets verify the effectiveness of our algorithm compared with the supervised and unsupervised models.
translated by 谷歌翻译
由于在建模相互依存系统中,由于其高效用,多层图已经在许多领域获得了大量的研究。然而,多层图的聚类,其旨在将图形节点划分为类别或社区,仍处于新生阶段。现有方法通常限于利用MultiView属性或多个网络,并忽略更复杂和更丰富的网络框架。为此,我们向多层图形聚类提出了一种名为Multidayer agal对比聚类网络(MGCCN)的多层图形聚类的通用和有效的AutoEncoder框架。 MGCCN由三个模块组成:(1)应用机制以更好地捕获节点与邻居之间的相关性以获得更好的节点嵌入。 (2)更好地探索不同网络中的一致信息,引入了对比融合策略。 (3)MGCCN采用自我监督的组件,可迭代地增强节点嵌入和聚类。对不同类型的真实图数据数据的广泛实验表明我们所提出的方法优于最先进的技术。
translated by 谷歌翻译
生成的自我监督学习(SSL),尤其是蒙面自动编码器,已成为最令人兴奋的学习范式之一,并且在处理图形数据方面表现出了巨大的潜力。但是,现实世界图总是异质的,它提出了现有方法忽略的三个关键挑战:1)如何捕获复杂的图形结构? 2)如何合并各种节点属性? 3)如何编码不同的节点位置?鉴于此,我们研究了异质图上生成SSL的问题,并提出了HGMAE,这是一种新型的异质图掩盖自动编码器模型,以应对这些挑战。 HGMAE通过两种创新的掩蔽技术和三种独特的培训策略捕获了全面的图形信息。特别是,我们首先使用动态掩模速率开发Metapath掩盖和自适应属性掩蔽,以实现在异质图上有效和稳定的学习。然后,我们设计了几种培训策略,包括基于Metapath的边缘重建,以采用复杂的结构信息,目标属性恢复以结合各种节点属性,以及位置特征预测以编码节点位置信息。广泛的实验表明,HGMAE在多个数据集上的几个任务上均优于对比度和生成的最新基准。
translated by 谷歌翻译
Contrastive learning methods based on InfoNCE loss are popular in node representation learning tasks on graph-structured data. However, its reliance on data augmentation and its quadratic computational complexity might lead to inconsistency and inefficiency problems. To mitigate these limitations, in this paper, we introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL in short). Local-GCL consists of two key designs: 1) We fabricate the positive examples for each node directly using its first-order neighbors, which frees our method from the reliance on carefully-designed graph augmentations; 2) To improve the efficiency of contrastive learning on graphs, we devise a kernelized contrastive loss, which could be approximately computed in linear time and space complexity with respect to the graph size. We provide theoretical analysis to justify the effectiveness and rationality of the proposed methods. Experiments on various datasets with different scales and properties demonstrate that in spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
translated by 谷歌翻译