机器学习（ML）已经证明了用于准确和结晶材料的准确性能预测的承诺。为了化学结构的高度精确的ML型号的化学结构属性预测，需要具有足够样品的数据集。然而，获得昂贵的化学性质的获得和充分数据可以是昂贵的令人昂贵的，这大大限制了ML模型的性能。通过计算机视觉和黑暗语言处理中数据增强的成功，我们开发了奥古里希姆：数据八级化图书馆化学结构。引入了弃头晶系统和分子的增强方法，其可以对基于指纹的ML模型和图形神经网络（GNNS）进行脱颖而出。我们表明，使用我们的增强策略意义地提高了ML模型的性能，特别是在使用GNNS时，我们开发的增强件在训练期间可以用作广告插件模块，并在用不同的GNN实施时证明了有效性。模型通过Theauglichem图书馆。基于Python的封装我们实现了EugliChem：用于化学结构的数据增强库，可公开获取：https：//github.com/baratilab/auglichem.1

Ionic Liquids (ILs) provide a promising solution for CO$_2$ capture and storage to mitigate global warming. However, identifying and designing the high-capacity IL from the giant chemical space requires expensive, and exhaustive simulations and experiments. Machine learning (ML) can accelerate the process of searching for desirable ionic molecules through accurate and efficient property predictions in a data-driven manner. But existing descriptors and ML models for the ionic molecule suffer from the inefficient adaptation of molecular graph structure. Besides, few works have investigated the explainability of ML models to help understand the learned features that can guide the design of efficient ionic molecules. In this work, we develop both fingerprint-based ML models and Graph Neural Networks (GNNs) to predict the CO$_2$ absorption in ILs. Fingerprint works on graph structure at the feature extraction stage, while GNNs directly handle molecule structure in both the feature extraction and model prediction stage. We show that our method outperforms previous ML models by reaching a high accuracy (MAE of 0.0137, $R^2$ of 0.9884). Furthermore, we take the advantage of GNNs feature representation and develop a substructure-based explanation method that provides insight into how each chemical fragments within IL molecules contribute to the CO$_2$ absorption prediction of ML models. We also show that our explanation result agrees with some ground truth from the theoretical reaction mechanism of CO$_2$ absorption in ILs, which can advise on the design of novel and efficient functional ILs in the future.

Models that accurately predict properties based on chemical structure are valuable tools in drug discovery. However, for many properties, public and private training sets are typically small, and it is difficult for the models to generalize well outside of the training data. Recently, large language models have addressed this problem by using self-supervised pretraining on large unlabeled datasets, followed by fine-tuning on smaller, labeled datasets. In this paper, we report MolE, a molecular foundation model that adapts the DeBERTa architecture to be used on molecular graphs together with a two-step pretraining strategy. The first step of pretraining is a self-supervised approach focused on learning chemical structures, and the second step is a massive multi-task approach to learn biological information. We show that fine-tuning pretrained MolE achieves state-of-the-art results on 9 of the 22 ADMET tasks included in the Therapeutic Data Commons.

阐明并准确预测分子的吸毒性和生物活性在药物设计和发现中起关键作用，并且仍然是一个开放的挑战。最近，图神经网络（GNN）在基于图的分子属性预测方面取得了显着进步。但是，当前基于图的深度学习方法忽略了分子的分层信息以及特征通道之间的关系。在这项研究中，我们提出了一个精心设计的分层信息图神经网络框架（称为hignn），用于通过利用分子图和化学合成的可见的无限元素片段来预测分子特性。此外，首先在Hignn体系结构中设计了一个插件功能的注意块，以适应消息传递阶段后自适应重新校准原子特征。广泛的实验表明，Hignn在许多具有挑战性的药物发现相关基准数据集上实现了最先进的预测性能。此外，我们设计了一种分子碎片的相似性机制，以全面研究Hignn模型在子图水平上的解释性，表明Hignn作为强大的深度学习工具可以帮助化学家和药剂师识别出设计更好分子的关键分子，以设计更好的分子，以设计出所需的更好分子。属性或功能。源代码可在https://github.com/idruglab/hignn上公开获得。

学习表达性分子表示对于促进分子特性的准确预测至关重要。尽管图形神经网络（GNNS）在分子表示学习中取得了显着进步，但它们通常面临诸如邻居探索，不足，过度光滑和过度阵列之类的局限性。同样，由于参数数量大，GNN通常具有较高的计算复杂性。通常，当面对相对大尺寸的图形或使用更深的GNN模型体系结构时，这种限制会出现或增加。克服这些问题的一个想法是将分子图简化为小型，丰富且有益的信息，这更有效，更具挑战性的培训GNN。为此，我们提出了一个新颖的分子图粗化框架，名为FUNQG利用函数组，作为分子的有影响力的构件来确定其性质，基于称为商图的图理论概念。通过实验，我们表明所产生的信息图比分子图小得多，因此是训练GNN的良好候选者。我们将FUNQG应用于流行的分子属性预测基准，然后比较所获得的数据集上的GNN体系结构的性能与原始数据集上的几个最先进的基线。通过实验，除了其参数数量和低计算复杂性的急剧减少之外，该方法除了其急剧减少之外，在各种数据集上的表现显着优于先前的基准。因此，FUNQG可以用作解决分子表示学习问题的简单，成本效益且可靠的方法。

Molecular machine learning has been maturing rapidly over the last few years.Improved methods and the presence of larger datasets have enabled machine learning algorithms to make increasingly accurate predictions about molecular properties. However, algorithmic progress has been limited due to the lack of a standard benchmark to compare the efficacy of proposed methods; most new algorithms are benchmarked on different datasets making it challenging to gauge the quality of proposed methods. This work introduces MoleculeNet, a large scale benchmark for molecular machine learning. MoleculeNet curates multiple public datasets, establishes metrics for evaluation, and offers high quality open-source implementations of multiple previously proposed molecular featurization and learning algorithms (released as part of the DeepChem

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

The accurate prediction of physicochemical properties of chemical compounds in mixtures (such as the activity coefficient at infinite dilution $\gamma_{ij}^\infty$) is essential for developing novel and more sustainable chemical processes. In this work, we analyze the performance of previously-proposed GNN-based models for the prediction of $\gamma_{ij}^\infty$, and compare them with several mechanistic models in a series of 9 isothermal studies. Moreover, we develop the Gibbs-Helmholtz Graph Neural Network (GH-GNN) model for predicting $\ln \gamma_{ij}^\infty$ of molecular systems at different temperatures. Our method combines the simplicity of a Gibbs-Helmholtz-derived expression with a series of graph neural networks that incorporate explicit molecular and intermolecular descriptors for capturing dispersion and hydrogen bonding effects. We have trained this model using experimentally determined $\ln \gamma_{ij}^\infty$ data of 40,219 binary-systems involving 1032 solutes and 866 solvents, overall showing superior performance compared to the popular UNIFAC-Dortmund model. We analyze the performance of GH-GNN for continuous and discrete inter/extrapolation and give indications for the model's applicability domain and expected accuracy. In general, GH-GNN is able to produce accurate predictions for extrapolated binary-systems if at least 25 systems with the same combination of solute-solvent chemical classes are contained in the training set and a similarity indicator above 0.35 is also present. This model and its applicability domain recommendations have been made open-source at https://github.com/edgarsmdn/GH-GNN.

图形神经网络（GNN）正在化学工程中出现，以基于分子图的物理化学特性端到端学习。 GNNS的一个关键要素是合并函数，将原子矢量结合到分子指纹中。大多数以前的作品都使用标准池功能来预测各种属性。但是，不合适的合并功能会导致概括不佳的非物理GNN。我们根据有关学习特性的物理知识比较并选择有意义的GNN合并方法。通过量子机械计算计算出的分子特性证明了物理池函数的影响。我们还将结果与最近的SET2Set合并方法进行了比较。我们建议使用总和池来预测取决于分子大小的性能并比较分子大小无关的属性的池函数。总体而言，我们表明物理池功能的使用显着增强了概括。

自我监督学习（SSL）是一种通过利用数据中固有的监督来学习数据表示的方法。这种学习方法是药物领域的焦点，由于耗时且昂贵的实验，缺乏带注释的数据。使用巨大未标记数据的SSL显示出在分子属性预测方面表现出色的性能，但存在一些问题。 （1）现有的SSL模型是大规模的；在计算资源不足的情况下实现SSL有限制。 （2）在大多数情况下，它们不利用3D结构信息进行分子表示学习。药物的活性与药物分子的结构密切相关。但是，大多数当前模型不使用3D信息或部分使用它。 （3）以前对分子进行对比学习的模型使用置换原子和键的增强。因此，具有不同特征的分子可以在相同的阳性样品中。我们提出了一个新颖的对比学习框架，用于分子属性预测的小规模3D图对比度学习（3DGCL），以解决上述问题。 3DGCL通过不改变药物语义的预训练过程来反映分子的结构来学习分子表示。仅使用1,128个样本用于预训练数据和100万个模型参数，我们在四个回归基准数据集中实现了最先进或可比性的性能。广泛的实验表明，基于化学知识的3D结构信息对于用于财产预测的分子表示学习至关重要。

离子液体（ILS）是可持续过程的重要溶剂，并且需要预测IL中溶质的活性系数（AC）。最近，矩阵完成方法（MCM），变压器和图神经网络（GNN）在预测二元混合物的AC方面表现出很高的精度，例如宇宙RS和UNIFAC优于公认的模型。 GNN在这里特别有希望，因为他们学习了分子图到特性的关系，而无需预处理，通常是变压器所需的，并且与MCMS不同，适用于不包括训练中不包括的分子。但是，对于ILS，目前缺少GNN应用程序。在此，我们提出了一个GNN，以预测IL中溶质的温度依赖性无限稀释液。我们在包括40,000多个AC值的数据库上训练GNN，并将其与最先进的MCM进行比较。 GNN和MCM实现了类似的高预测性能，GNN还可以对培训期间未考虑的IL和溶质的AC进行高质量的预测。

事实证明，分子机器学习（ML）对于解决各种分子问题很重要，包括预测蛋白质 - 药物相互作用和血液脑性渗透性。自最近以来，已经为分子ML实施了所谓的图神经网络（GNN），显示出与基于描述符的方法相当或出色的性能。尽管存在各种工具和包装用于将GNN用于分子ML，但新的GNN包装，名为Molgraph（https://github.com/akensert/molgraph），在这项工作中开发了，以创建GNNS与TensorFlow高度兼容的动力和KERAS应用程序编程接口（API）。由于Molgraph专门关注分子ML，因此实施了化学模块，以适应分子图的产生$ \ unicode {x2014} $，然后可以将其输入到GNNS中以用于分子ML。为了验证GNN，它们针对分子数据集以及三个色谱保留时间数据集进行了基准测试。这些基准测试的结果表明，GNN按预期进行。此外，GNN被证明可用于分子识别和改善色谱保留数据的可解释性。

分子表示学习有助于多个下游任务，例如分子性质预测和药物设计。为了适当地代表分子，图形对比学习是一个有前途的范式，因为它利用自我监督信号并没有人类注释要求。但是，先前的作品未能将基本域名知识纳入图表语义，因此忽略了具有共同属性的原子之间的相关性，但不通过键连接连接。为了解决这些问题，我们构建化学元素知识图（KG），总结元素之间的微观关联，并提出了一种用于分子代表学习的新颖知识增强的对比学习（KCL）框架。 KCL框架由三个模块组成。第一个模块，知识引导的图形增强，基于化学元素kg增强原始分子图。第二模块，知识意识的图形表示，利用用于原始分子图的公共曲线图编码器和通过神经网络（KMPNN）的知识感知消息来提取分子表示来编码增强分子图中的复杂信息。最终模块是一种对比目标，在那里我们在分子图的这两个视图之间最大化协议。广泛的实验表明，KCL获得了八个分子数据集上的最先进基线的优异性能。可视化实验适当地解释了在增强分子图中从原子和属性中了解的KCL。我们的代码和数据可用于补充材料。

使用图神经网络（GNN）提取分子的信息表示，对于AI驱动的药物发现至关重要。最近，图形研究界一直在试图复制自然语言处理预处理的成功，并获得了一些成功。但是，我们发现在许多情况下，自我监督预审计对分子数据的益处可以忽略不计。我们对GNN预处理的关键组成部分进行了彻底的消融研究，包括预处理目标，数据拆分方法，输入特征，预处理数据集量表和GNN体系结构，以决定下游任务的准确性。我们的第一个重要发现是，在许多情况下，自我监督的图表预处理没有统计学上的显着优势。其次，尽管可以通过额外的监督预处理可以观察到改进，但通过更丰富或更平衡的数据拆分，改进可能会减少。第三，实验性超参数对下游任务的准确性具有更大的影响，而不是训练训练的任务。我们假设对分子进行预训练的复杂性不足，从而导致下游任务的可转移知识较低。

人工智能（AI）已被广泛应用于药物发现中，其主要任务是分子财产预测。尽管分子表示学习中AI技术的繁荣，但尚未仔细检查分子性质预测的一些关键方面。在这项研究中，我们对三个代表性模型，即随机森林，莫尔伯特和格罗弗进行了系统比较，该模型分别利用了三个主要的分子表示，扩展连接的指纹，微笑的字符串和分子图。值得注意的是，莫尔伯特（Molbert）和格罗弗（Grover）以自我监督的方式在大规模的无标记分子库中进行了预定。除了常用的分子基准数据集外，我们还组装了一套与阿片类药物相关的数据集进行下游预测评估。我们首先对标签分布和结构分析进行了数据集分析；我们还检查了阿片类药物相关数据集中的活动悬崖问题。然后，我们培训了4,320个预测模型，并评估了学习表示的有用性。此外，我们通过研究统计测试，评估指标和任务设置的效果来探索模型评估。最后，我们将化学空间的概括分解为施加间和支柱内的概括，并测量了预测性能，以评估两种设置下模型的普遍性。通过采取这种喘息，我们反映了分子财产预测的基本关键方面，希望在该领域带来更好的AI技术的意识。

Although substantial efforts have been made using graph neural networks (GNNs) for AI-driven drug discovery (AIDD), effective molecular representation learning remains an open challenge, especially in the case of insufficient labeled molecules. Recent studies suggest that big GNN models pre-trained by self-supervised learning on unlabeled datasets enable better transfer performance in downstream molecular property prediction tasks. However, they often require large-scale datasets and considerable computational resources, which is time-consuming, computationally expensive, and environmentally unfriendly. To alleviate these limitations, we propose a novel pre-training model for molecular representation learning, Bi-branch Masked Graph Transformer Autoencoder (BatmanNet). BatmanNet features two tailored and complementary graph autoencoders to reconstruct the missing nodes and edges from a masked molecular graph. To our surprise, BatmanNet discovered that the highly masked proportion (60%) of the atoms and bonds achieved the best performance. We further propose an asymmetric graph-based encoder-decoder architecture for either nodes and edges, where a transformer-based encoder only takes the visible subset of nodes or edges, and a lightweight decoder reconstructs the original molecule from the latent representation and mask tokens. With this simple yet effective asymmetrical design, our BatmanNet can learn efficiently even from a much smaller-scale unlabeled molecular dataset to capture the underlying structural and semantic information, overcoming a major limitation of current deep neural networks for molecular representation learning. For instance, using only 250K unlabelled molecules as pre-training data, our BatmanNet with 2.575M parameters achieves a 0.5% improvement on the average AUC compared with the current state-of-the-art method with 100M parameters pre-trained on 11M molecules.

在三维分子结构上运行的计算方法有可能解决生物学和化学的重要问题。特别地，深度神经网络的重视，但它们在生物分子结构域中的广泛采用受到缺乏系统性能基准或统一工具包的限制，用于与分子数据相互作用。为了解决这个问题，我们呈现Atom3D，这是一个新颖的和现有的基准数据集的集合，跨越几个密钥的生物分子。我们为这些任务中的每一个实施多种三维分子学习方法，并表明它们始终如一地提高了基于单维和二维表示的方法的性能。结构的具体选择对于性能至关重要，具有涉及复杂几何形状的任务的三维卷积网络，在需要详细位置信息的系统中表现出良好的图形网络，以及最近开发的设备越多的网络显示出显着承诺。我们的结果表明，许多分子问题符合三维分子学习的增益，并且有可能改善许多仍然过分曝光的任务。为了降低进入并促进现场进一步发展的障碍，我们还提供了一套全面的DataSet处理，模型培训和在我们的开源ATOM3D Python包中的评估工具套件。所有数据集都可以从https://www.atom3d.ai下载。

Many applications of machine learning require a model to make accurate predictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and then fine-tune it on a downstream task of interest. While pre-training has been effective in many language and vision domains, it remains an open question how to effectively use pre-training on graph datasets. In this paper, we develop a new strategy and self-supervised methods for pre-training Graph Neural Networks (GNNs). The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction.However, pre-training on graph datasets remains a hard challenge. Several key studies (

Models based on machine learning can enable accurate and fast molecular property predictions, which is of interest in drug discovery and material design. Various supervised machine learning models have demonstrated promising performance, but the vast chemical space and the limited availability of property labels make supervised learning challenging. Recently, unsupervised transformer-based language models pretrained on a large unlabelled corpus have produced state-of-the-art results in many downstream natural language processing tasks. Inspired by this development, we present molecular embeddings obtained by training an efficient transformer encoder model, MoLFormer, which uses rotary positional embeddings. This model employs a linear attention mechanism, coupled with highly distributed training, on SMILES sequences of 1.1 billion unlabelled molecules from the PubChem and ZINC datasets. We show that the learned molecular representation outperforms existing baselines, including supervised and self-supervised graph neural networks and language models, on several downstream tasks from ten benchmark datasets. They perform competitively on two others. Further analyses, specifically through the lens of attention, demonstrate that MoLFormer trained on chemical SMILES indeed learns the spatial relationships between atoms within a molecule. These results provide encouraging evidence that large-scale molecular language models can capture sufficient chemical and structural information to predict various distinct molecular properties, including quantum-chemical properties.

Advancements in neural machinery have led to a wide range of algorithmic solutions for molecular property prediction. Two classes of models in particular have yielded promising results: neural networks applied to computed molecular fingerprints or expert-crafted descriptors, and graph convolutional neural networks that construct a learned molecular representation by operating on the graph structure of the molecule.However, recent literature has yet to clearly determine which of these two methods is superior when generalizing to new chemical space. Furthermore, prior research has