成功的药物开发的主要障碍是临床试验的复杂性，成本和规模。临床试验数据的详细内部结构可以使常规优化难以实现。最近的机器学习进步，具体说明性结构的数据分析，有可能在改善临床试验设计方面取得重大进展。 TrimeGraph旨在应用这些方法，为开发模型的概念证明框架，可以帮助药物开发和益处患者。在这项工作中，我们首先介绍从CT.Gov，AACT和FISTTROVE数据库编译的策划临床试验数据集（n = 1191试验;代表一百万名患者）并将该数据转换为图形结构格式。然后，我们详细介绍了一系列图形机学习算法的数学依据和实现，其通常在嵌入在低维特征空间中的图形数据上使用标准机器分类器。我们培训了这些模型，以预测临床试验的副作用信息给出关于疾病，现有的医疗病症和治疗的信息。 Metapath2Vec算法表现良好，具有标准的逻辑回归，决策树，随机森林，支持向量和神经网络分类器，以及分别显示0.85,0.68,0.86,0.80和0.77的典型Roc-Auc谱分别。值得注意的是，当在等效的阵列结构数据上训练时，最好的执行分类器只能产生0.70的典型的Roc-Auc得分。我们的工作表明，图形建模可以显着提高适当的数据集上的预测准确性。改进建模假设和更多数据类型的项目的连续版本可以产生具有现实世界的药物开发应用的优秀预测因子。

Multimodal deep learning has been used to predict clinical endpoints and diagnoses from clinical routine data. However, these models suffer from scaling issues: they have to learn pairwise interactions between each piece of information in each data type, thereby escalating model complexity beyond manageable scales. This has so far precluded a widespread use of multimodal deep learning. Here, we present a new technical approach of "learnable synergies", in which the model only selects relevant interactions between data modalities and keeps an "internal memory" of relevant data. Our approach is easily scalable and naturally adapts to multimodal data inputs from clinical routine. We demonstrate this approach on three large multimodal datasets from radiology and ophthalmology and show that it outperforms state-of-the-art models in clinically relevant diagnosis tasks. Our new approach is transferable and will allow the application of multimodal deep learning to a broad set of clinically relevant problems.

The success of Deep Learning applications critically depends on the quality and scale of the underlying training data. Generative adversarial networks (GANs) can generate arbitrary large datasets, but diversity and fidelity are limited, which has recently been addressed by denoising diffusion probabilistic models (DDPMs) whose superiority has been demonstrated on natural images. In this study, we propose Medfusion, a conditional latent DDPM for medical images. We compare our DDPM-based model against GAN-based models, which constitute the current state-of-the-art in the medical domain. Medfusion was trained and compared with (i) StyleGan-3 on n=101,442 images from the AIROGS challenge dataset to generate fundoscopies with and without glaucoma, (ii) ProGAN on n=191,027 from the CheXpert dataset to generate radiographs with and without cardiomegaly and (iii) wGAN on n=19,557 images from the CRCMS dataset to generate histopathological images with and without microsatellite stability. In the AIROGS, CRMCS, and CheXpert datasets, Medfusion achieved lower (=better) FID than the GANs (11.63 versus 20.43, 30.03 versus 49.26, and 17.28 versus 84.31). Also, fidelity (precision) and diversity (recall) were higher (=better) for Medfusion in all three datasets. Our study shows that DDPM are a superior alternative to GANs for image synthesis in the medical domain.

Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models in particular have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen and Stable Diffusion. However, their use in medicine, where image data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play a crucial role in privacy preserving artificial intelligence and can also be used to augment small datasets. Here we show that diffusion probabilistic models can synthesize high quality medical imaging data, which we show for Magnetic Resonance Images (MRI) and Computed Tomography (CT) images. We provide quantitative measurements of their performance through a reader study with two medical experts who rated the quality of the synthesized images in three categories: Realistic image appearance, anatomical correctness and consistency between slices. Furthermore, we demonstrate that synthetic images can be used in a self-supervised pre-training and improve the performance of breast segmentation models when data is scarce (dice score 0.91 vs. 0.95 without vs. with synthetic data).