Generative Modelling of the Ageing Heart with Cross-Sectional Imaging and Clinical Data

The field of generative modelling has made tremendous progress recently, driven by deep learning methods such as variational autoencoders (VAEs) [10, 16], generative adversarial networks (GANs) [12], cycle-consistent GAN (CycleGAN) [23]. Generative models have been widely used in medical imaging. For example, Wang et al. proposed a CycleGAN-based model for cross-domain image generation, which generates pseudo-CT for PET-MR attenuation correction [17]. Yurt et al. proposed a multi-stream GAN architecture for multi-contrast MRI synthesis [21]. Pawlowski et al. formulated a structural causal model with deep learning components for synthesising and counterfactual inference of MNIST and brain MR images [15].

Most ageing synthesis works focused on images of human face while some works explored the synthesis of brain MR images. These works investigated different ways of incorporating age information into the generating process. One way is to concatenate age vector with image feature vector to learn a joint distribution of age and image appearance in face ageing [2, 22] or brain ageing [20]. Another way is to use a pre-trained age regression network, which provides guidance in age-related latent code generation [1, 19]. Some works introduced an age estimation loss accounting for age distribution [7, 13]. In [11], high-order interactions between the given identity and target age were explored to learn personalized age features. Although these methods are not designed for cardiac imaging, they provide valuable insights for modelling the ageing heart.

We incorporate two major clinical conditions, age and gender, into the generative model. The age space $A$ is represented using a $m\times 1$ categorical vector, where $m$ denotes the number of age groups. For age group $i$, an age vector $a_i\in A$ is generated:

where $A_i$ is a one-hot encoded vector that contains one at the $i$th-element and zeros elsewhere and $\epsilon$ is random noise sampled from a prior distribution. The age vector $a_i$ is concatenated with a one-hot gender vector $g$ to form the clinical condition $c$.

The input to the generator $G$ includes the source anatomy $I_s$ and the condition code $z_c$ generated from the condition mapping network $M(a,g)$. The generator follows an encoder-decoder structure, where the encoder $E$ maps the input $I_s$ into a subject-specific latent code $z$, the decoder concatenates $z$ with the clinical condition code $z_c$ and generates an output anatomy $I_t^{\prime }$. The generation is described by,

We assume that the encoder $E\left(I_s\right)$ preserves the high-level subject-specific feature of the input anatomy $I_s$ and the decoder $D$ utilises this information as well as the clinical information to generate the anatomy $I_t^{\prime }$. Adopting a cyclic design [23], we also generate an image $I_s^{\prime \prime }=G(I_t^{\prime },a_s,g)$ that maps the generated the image $I_t^{\prime }$ back to the source age $a_s$. This cycle-consistent generation is only applied in training process.

Two discriminator networks are imposed on the latent code $z$ and the generated anatomy. $D_z$ is designed to discriminate the latent code $z=E\left(I\right)$ by training $z$ to be uniformly distributed. Simultaneously, $E$ will be trained to compute $z$ to fool $D_z$. Such an adversarial process forces the distribution of $z$ to gradually approach the prior, which is the uniform distribution. Another discriminator $D_{img}$ forces the generator to generate realistic cardiac anatomies.

The encoder $E$ consisted of five 3D convolutional layers and one flatten layer, outputting the latent code $z$. The decoder $D$ consisted of one flatten layer and five 3D transposed convolution layer. The transposed convolution in encoder and decoder used a kernel size of $4\times 4\times 4$. All intermediate layers of each block use the ReLU activation function. The dimension of the latent variables $z$ and $z_c$ are both 32. The anatomy segmentations are transferred into one-hot map, and the output of $E$($I$) is restricted to $[-1,1]$ using the hyperbolic tangent activation function. For optimisation, the Adam optimizer [9] is used with learning rate of $2\cdot {10}^{-4}$ and weigh decay of $1\cdot {10}^{-5}$. We set $\sigma =0.02$ in Eq. 1, ${\lambda }_0=1$, ${\lambda }_1=0.1$, ${\lambda }_2=0.01$, ${\lambda }_3=0.1$ and ${\lambda }_4=1$ in Eq. 8. The model was implemented using PyTorch [14]. At the inference stage, only the generator $G$ is active, containing $E$, $D$ and $M$ as described in Eq. 2, while the other parts are not used. Our code will be made publicly available.

Two ageing synthesis methods, CAAE [22] and a modified version of Lifespan [13], are used as baselines. Since the original codes were developed for 2D face image synthesis, we re-implemented all the codes for 3D cardiac data synthesis. For Lifespan, we replaced the modulated convolution layers with 3D convolution layers to save GPU memory for 3D data.

For each subject in the UK Biobank test set, we synthesise a series of anatomies for the same heart at age groups from 40 to 80 with interval of 5 years old. For example, in Fig. 2, the cardiac anatomy of a 50-55 years old female is taken as input and the anatomies of this heart at other ages are predicted using the proposed generative model. From the generated anatomies, we also derive clinical measures, including the left ventricular myocardial mass (LVM), LV end-diastolic volume (LVEDV), LV end-systolic volume (LVESV), RV end-diastolic volume (RVEDV) and RV end-systolic volume (RVESV). The bottom of Fig. 2 illustrates the trends of these clinical measures during heart ageing synthesis. Consistent with the literature [6], we observe a decreasing trend for LV or RV volumes. It demonstrates our model captures the relation between cardiac anatomical structure and age.

Based the synthetic anatomies, we calculate the probability distribution $P_c\left(a\right)$ of each clinical measure $c$ against age $a$ and compare it to the probability distribution of the real data $Q_c\left(a\right)$. Here, $c$ denotes one of the five clinical measures, e.g. LVM. The distance between $P_c\left(a\right)$ and $Q_c\left(a\right)$ is evaluated in terms of the Kullback–Leibler divergence (KL) and Wasserstein distance (WD). Table 1 compares the distribution similarities using different generative models. It shows the proposed method achieves a higher distribution similarity, compared to state-of-the-art ageing synthesis models.

Using the repeated imaging scans from UK Biobank longitudinal dataset, we evaluate the predictive performance of the model. Given the anatomy at the first time point, the anatomy at the second time point is predicted and compared to the ground truth in terms of Dice metric, Hausdorff distance (HD) and average symmetric surface distance (ASSD), reported in Table 2. It shows that the proposed method achieves a good performance in prediction comparable to or better than other competing methods.