We present a principled approach to incorporating labels in VAEs that captures the rich characteristic information associated with those labels. While prior work has typically conflated these by learning latent variables that directly correspond to label values, we argue this is contrary to the intended effect of supervision in VAEs-capturing rich label characteristics with the latents. For example, we may want to capture the characteristics of a face that make it look young, rather than just the age of the person. To this end, we develop the CCVAE, a novel VAE model and concomitant variational objective which captures label characteristics explicitly in the latent space, eschewing direct correspondences between label values and latents. Through judicious structuring of mappings between such characteristic latents and labels, we show that the CCVAE can effectively learn meaningful representations of the characteristics of interest across a variety of supervision schemes. In particular, we show that the CCVAE allows for more effective and more general interventions to be performed, such as smooth traversals within the characteristics for a given label, diverse conditional generation, and transferring characteristics across datapoints.

Multimodal VAEs seek to model the joint distribution over heterogeneous data (e.g.\ vision, language), whilst also capturing a shared representation across such modalities. Prior work has typically combined information from the modalities by reconciling idiosyncratic representations directly in the recognition model through explicit products, mixtures, or other such factorisations. Here we introduce a novel alternative, the MEME, that avoids such explicit combinations by repurposing semi-supervised VAEs to combine information between modalities implicitly through mutual supervision. This formulation naturally allows learning from partially-observed data where some modalities can be entirely missing -- something that most existing approaches either cannot handle, or do so to a limited extent. We demonstrate that MEME outperforms baselines on standard metrics across both partial and complete observation schemes on the MNIST-SVHN (image-image) and CUB (image-text) datasets. We also contrast the quality of the representations learnt by mutual supervision against standard approaches and observe interesting trends in its ability to capture relatedness between data.

变异推理（VI）的核心原理是将计算复杂后概率密度计算的统计推断问题转换为可拖动的优化问题。该属性使VI比几种基于采样的技术更快。但是，传统的VI算法无法扩展到大型数据集，并且无法轻易推断出越野数据点，而无需重新运行优化过程。该领域的最新发展，例如随机，黑框和摊销VI，已帮助解决了这些问题。如今，生成的建模任务广泛利用摊销VI来实现其效率和可扩展性，因为它利用参数化函数来学习近似的后验密度参数。在本文中，我们回顾了各种VI技术的数学基础，以构成理解摊销VI的基础。此外，我们还概述了最近解决摊销VI问题的趋势，例如摊销差距，泛化问题，不一致的表示学习和后验崩溃。最后，我们分析了改善VI优化的替代差异度量。

变异因素之间的相关性在现实数据中普遍存在。机器学习算法可能会受益于利用这种相关性，因为它们可以提高噪声数据的预测性能。然而，通常这种相关性不稳定（例如，它们可能在域，数据集或应用程序之间发生变化），我们希望避免利用它们。解剖学方法旨在学习捕获潜伏子空间变化不同因素的表示。常用方法涉及最小化潜伏子空间之间的相互信息，使得每个潜在的底层属性。但是，当属性相关时，这会失败。我们通过强制执行可用属性上的子空间之间的独立性来解决此问题，这允许我们仅删除不导致的依赖性，这些依赖性是由于训练数据中存在的相关结构。我们通过普发的方法实现这一目标，以最小化关于分类变量的子空间之间的条件互信息（CMI）。我们首先在理论上展示了CMI最小化是对高斯数据线性问题的稳健性解剖的良好目标。然后，我们基于MNIST和Celeba在现实世界数据集上应用我们的方法，并表明它会在相关偏移下产生脱屑和强大的模型，包括弱监督设置。

本文提出了在适当的监督信息下进行分解的生成因果代表（亲爱的）学习方法。与实施潜在变量独立性的现有分解方法不同，我们考虑了一种基本利益因素可以因果关系相关的一般情况。我们表明，即使在监督下，先前具有独立先验的方法也无法解散因果关系。在这一发现的激励下，我们提出了一种称为DEAR的新的解开学习方法，该方法可以使因果可控的产生和因果代表学习。这种新公式的关键要素是使用结构性因果模型（SCM）作为双向生成模型的先验分布。然后，使用合适的GAN算法与发电机和编码器共同训练了先验，并与有关地面真相因子及其基本因果结构的监督信息合并。我们提供了有关该方法的可识别性和渐近收敛性的理论理由。我们对合成和真实数据集进行了广泛的实验，以证明DEAR在因果可控生成中的有效性，以及在样本效率和分布鲁棒性方面，学到的表示表示对下游任务的好处。

最近，在深度生成模型中，不可能是非线性ICA的可识别性的文艺复兴。对于i.I.D.数据，先前的作品已经假定访问足够丰富的辅助观察集，表示$ \ mathbf {u} $。我们在这里展示了在没有这种侧面信息的情况下可以获得可识别性。以前的方法必须制定强烈的假设，以获得可识别的模型。在这里，我们在一组宽松的约束集中获得了经验识别的模型。特别是，我们专注于在其潜在空间中执行聚类的生成模型 - 一种匹配以前可识别模型的模型结构，而是使用学习群集提供辅助信息的合成形式。我们评估我们的提案，包括通过统计测试，并发现学习群集有效功能：具有潜在群集的深度生成模型是经验识别的，与依赖侧面信息的模型相同。

The combination of machine learning models with physical models is a recent research path to learn robust data representations. In this paper, we introduce p$^3$VAE, a generative model that integrates a perfect physical model which partially explains the true underlying factors of variation in the data. To fully leverage our hybrid design, we propose a semi-supervised optimization procedure and an inference scheme that comes along meaningful uncertainty estimates. We apply p$^3$VAE to the semantic segmentation of high-resolution hyperspectral remote sensing images. Our experiments on a simulated data set demonstrated the benefits of our hybrid model against conventional machine learning models in terms of extrapolation capabilities and interpretability. In particular, we show that p$^3$VAE naturally has high disentanglement capabilities. Our code and data have been made publicly available at https://github.com/Romain3Ch216/p3VAE.

Deep neural networks are susceptible to shortcut learning, using simple features to achieve low training loss without discovering essential semantic structure. Contrary to prior belief, we show that generative models alone are not sufficient to prevent shortcut learning, despite an incentive to recover a more comprehensive representation of the data than discriminative approaches. However, we observe that shortcuts are preferentially encoded with minimal information, a fact that generative models can exploit to mitigate shortcut learning. In particular, we propose Chroma-VAE, a two-pronged approach where a VAE classifier is initially trained to isolate the shortcut in a small latent subspace, allowing a secondary classifier to be trained on the complementary, shortcut-free latent subspace. In addition to demonstrating the efficacy of Chroma-VAE on benchmark and real-world shortcut learning tasks, our work highlights the potential for manipulating the latent space of generative classifiers to isolate or interpret specific correlations.

现代生成型号在包括图像或文本生成和化学分子建模的各种任务中获得优异的品质。然而，现有方法往往缺乏通过所要求的属性产生实例的基本能力，例如照片中的人的年龄或产生的分子的重量。包含此类额外的调节因子将需要重建整个架构并从头开始优化参数。此外，难以解除选定的属性，以便仅在将其他属性中执行不变的同时执行编辑。为了克服这些限制，我们提出插件（插件生成网络），这是一种简单而有效的生成技术，可以用作预先训练的生成模型的插件。我们的方法背后的想法是使用基于流的模块将纠缠潜在的潜在表示转换为多维空间，其中每个属性的值被建模为独立的一维分布。因此，插件可以生成具有所需属性的新样本，以及操作现有示例的标记属性。由于潜在代表的解散，我们甚至能够在数据集中的稀有或看不见的属性组合生成样本，例如具有灰色头发的年轻人，有妆容的男性或胡须的女性。我们将插入与GaN和VAE模型组合并将其应用于图像和化学分子建模的条件生成和操纵。实验表明，插件保留了骨干型号的质量，同时添加控制标记属性值的能力。

变异自动编码器（VAE）是最常用的无监督机器学习模型之一。但是，尽管对先前和后验的高斯分布的默认选择通常代表了数学方便的分布通常会导致竞争结果，但我们表明该参数化无法用潜在的超球体结构对数据进行建模。为了解决这个问题，我们建议使用von Mises-fisher（VMF）分布，从而导致超级潜在空间。通过一系列实验，我们展示了这种超球vae或$ \ mathcal {s} $ - vae如何更适合于用超球形结构捕获数据，同时胜过正常的，$ \ mathcal {n} $ - vae-，在其他数据类型的低维度中。http://github.com/nicola-decao/s-vae-tf和https://github.com/nicola-decao/nicola-decao/s-vae-pytorch

The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. In this paper, we provide a sober look at recent progress in the field and challenge some common assumptions. We first theoretically show that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data. Then, we train more than 12 000 models covering most prominent methods and evaluation metrics in a reproducible large-scale experimental study on seven different data sets. We observe that while the different methods successfully enforce properties "encouraged" by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision. Furthermore, increased disentanglement does not seem to lead to a decreased sample complexity of learning for downstream tasks. Our results suggest that future work on disentanglement learning should be explicit about the role of inductive biases and (implicit) supervision, investigate concrete benefits of enforcing disentanglement of the learned representations, and consider a reproducible experimental setup covering several data sets.

改变特定特征但不是其他特性的输入扰动的反事实示例 - 已经显示用于评估机器学习模型的偏差，例如，对特定的人口组。然而，由于图像的各种特征上的底层的因果结构，生成用于图像的反事实示例是非琐碎的。为了有意义，生成的扰动需要满足因果模型所暗示的约束。我们通过在前瞻性学习推断（ALI）的改进变型中结合结构因果模型（SCM）来提出一种方法，该方法是根据图像的属性之间的因果关系生成反事实。基于所生成的反事实，我们展示了如何解释预先训练的机器学习分类器，评估其偏置，并使用反事实程序缓解偏差。在Morpho-Mnist DataSet上，我们的方法会在质量上产生与基于SCM的Factficuls（DeepScm）的质量相当的反功能，而在更复杂的Celeba DataSet上，我们的方法优于DeepScm在产生高质量的有效反应性时。此外，生成的反事件难以从人类评估实验中的重建图像中无法区分，并且随后使用它们来评估在Celeba数据上培训的标准分类器的公平性。我们表明分类器是偏见的w.r.t.皮肤和头发颜色，以及反事实规则化如何消除这些偏差。

神经影像学技术产生高维的随机数据，从中提取有关感兴趣现象的高级知识可能是具有挑战性的。我们通过将生成建模框架应用于1）分类多种病理和2）以数据驱动方式来解决这些挑战。我们的框架学习了与病理有关的数据的生成因素。我们提供了一种算法来进一步解码这些因素，并观察不同的病理如何影响观察到的数据。我们说明了拟议方法在识别精神分裂症的适用性，然后是听觉语言幻觉。我们进一步证明了该框架学习与当前领域知识一致的疾病相关机制的能力。我们还将所提出的框架与几种基准方法进行比较，并表明其分类性能和解释性优势。

Latent variable models such as the Variational Auto-Encoder (VAE) have become a go-to tool for analyzing biological data, especially in the field of single-cell genomics. One remaining challenge is the interpretability of latent variables as biological processes that define a cell's identity. Outside of biological applications, this problem is commonly referred to as learning disentangled representations. Although several disentanglement-promoting variants of the VAE were introduced, and applied to single-cell genomics data, this task has been shown to be infeasible from independent and identically distributed measurements, without additional structure. Instead, recent methods propose to leverage non-stationary data, as well as the sparse mechanism shift assumption in order to learn disentangled representations with a causal semantic. Here, we extend the application of these methodological advances to the analysis of single-cell genomics data with genetic or chemical perturbations. More precisely, we propose a deep generative model of single-cell gene expression data for which each perturbation is treated as a stochastic intervention targeting an unknown, but sparse, subset of latent variables. We benchmark these methods on simulated single-cell data to evaluate their performance at latent units recovery, causal target identification and out-of-domain generalization. Finally, we apply those approaches to two real-world large-scale gene perturbation data sets and find that models that exploit the sparse mechanism shift hypothesis surpass contemporary methods on a transfer learning task. We implement our new model and benchmarks using the scvi-tools library, and release it as open-source software at \url{https://github.com/Genentech/sVAE}.

仅观察一组有限的示例，应该能够从新分布中生成数据。在几次学习中，该模型经过了来自分布的许多集合的数据培训，这些分布共享了一些基本属性，例如来自不同字母的字符集或来自不同类别的对象。我们将当前的潜在变量模型扩展到具有基于注意力级聚合的基于注意力的点的完全层次结构方法，并将我们的方法scha-vae称为set-context层次层次结构 - 构造变异自动编码器。我们探索基于似然的模型比较，迭代数据采样和无适应性分布概括。我们的结果表明，分层公式可以更好地捕获小型数据制度中集合中的内在变异性。这项工作将深层可变方法推广到几乎没有学习的方法，迈出了一步，朝着大规模的几杆生成迈出了一步。

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning.

现实世界数据库很复杂，它们通常会呈现冗余，并在同一数据的异质和多个表示之间共享相关性。因此，在视图之间利用和解开共享信息至关重要。为此，最近的研究经常将所有观点融合到共享的非线性复杂潜在空间中，但它们失去了解释性。为了克服这一局限性，我们在这里提出了一种新的方法，将多个变异自动编码器（VAE）结构与因子分析潜在空间（FA-VAE）相结合。具体而言，我们使用VAE在连续的潜在空间中学习每个异质观点的私人表示。然后，我们通过使用线性投影矩阵将每个私有变量投影到低维的潜在空间来对共享潜在空间进行建模。因此，我们在私人信息和共享信息之间创建了可解释的层次依赖性。这样，新型模型可以同时：（i）从多种异质观点中学习，（ii）获得可解释的层次共享空间，以及（iii）在生成模型之间执行传输学习。

We define and address the problem of unsupervised learning of disentangled representations on data generated from independent factors of variation. We propose FactorVAE, a method that disentangles by encouraging the distribution of representations to be factorial and hence independent across the dimensions. We show that it improves upon β-VAE by providing a better trade-off between disentanglement and reconstruction quality. Moreover, we highlight the problems of a commonly used disentanglement metric and introduce a new metric that does not suffer from them.

数据清洁通常包括离群检测和数据修复。系统错误是由于数据反复发生的几乎确定性转换而导致的，例如特定的图像像素设置为默认值或水印。因此，容量足够的模型很容易地超出这些错误，从而使检测和修复变得困难。作为系统的离群值是干净实例和系统误差模式的模式的组合，我们的主要见解是，嵌入者可以通过模型中的较小的表示形式（子空间）来建模，而不是离群值。通过利用这一点，我们提出了清洁子空间变量自动编码器（CLSVAE），这是一种新型的半监督模型，用于检测和自动修复系统误差。主要思想是分别分别分区潜在的空间和模型模型。与以前的相关模型相比，CLSVAE的有效数据少得多，通常不到2％的数据。我们在具有不同级别的损坏和标记的集合大小的方案中使用三个图像数据集提供实验，与相关基线相比。 CLSVAE提供了无人干预的优质维修，例如与最接近的基线相比，只有标记数据的0.25％的相对误差下降了58％。

The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.