粒子过滤是针对多种顺序推断任务的标准蒙特卡洛方法。粒子过滤器的关键成分是一组具有重要性权重的粒子，它们可以作为某些随机过程的真实后验分布的代理。在这项工作中，我们提出了连续的潜在粒子过滤器，该方法将粒子过滤扩展到连续时域。我们证明了如何将连续的潜在粒子过滤器用作依赖于学到的变异后验的推理技术的通用插件替换。我们对基于潜在神经随机微分方程的不同模型家族进行的实验表明，在推理任务中，连续时间粒子滤波在推理任务中的卓越性能，例如似然估计和各种随机过程的顺序预测。

非线性状态空间模型是一种强大的工具，可以在复杂时间序列中描述动态结构。在一个流的媒体设置中，当一次处理一个样本的情况下，状态的同时推断及其非线性动力学在实践中提出了重大挑战。我们开发了一个小说在线学习框架，利用变分推理和顺序蒙特卡罗，这使得灵活和准确的贝叶斯联合过滤。我们的方法提供了滤波后的近似，这可以任意地接近针对广泛的动态模型和观察模型的真正滤波分布。具体地，所提出的框架可以使用稀疏高斯过程有效地近似于动态的后验，允许潜在动力学的可解释模型。每个样本的恒定时间复杂性使我们的方法能够适用于在线学习场景，适用于实时应用。

变分推理（VI）与贝叶斯非线性滤波相结合，为潜在时间序列建模产生最先进的结果。最近的工作中的一个身体专注于序贯蒙特卡罗（SMC）及其变体，例如，前向滤波后仿真（FFBSI）。虽然这些研究成功了，但严重的问题仍然存在于粒子退化和偏见的渐变估计中。在本文中，我们提出了Enemble Kalman变分目标（ENKO），VI的混合方法和集合卡尔曼滤波器（ENKF），以推断出状态空间模型（SSMS）。我们所提出的方法可以有效地识别潜在动力学，因为其粒子多样性和无偏梯度估计值。我们展示了我们的ENKO在三个基准非线性系统识别任务的预测能力和粒子效率方面优于基于SMC的方法。

顺序蒙特卡洛（SMC）是状态空间模型的推理算法，通过从一系列中间目标分布进行采样来近似后验。目标分布通常被选择为过滤分布，但是这些忽略了未来观察结果的信息，从而导致推理和模型学习的实际和理论局限性。我们介绍了SIXO，这种方法将学习近似平滑分布的目标，并结合了所有观测值的信息。关键思想是使用密度比估计来拟合将过滤分布扭曲到平滑分布中的功能。然后，我们将SMC与这些学习的目标一起使用，以定义模型和建议学习的变异目标。六体的产量可证明更紧密的对数边缘下限，并在各种域中提供了更准确的后验推断和参数估计。

最近的机器学习进展已直接从数据中直接提出了对未知连续时间系统动力学的黑盒估计。但是，较早的作品基于近似ODE解决方案或点估计。我们提出了一种新型的贝叶斯非参数模型，该模型使用高斯工艺直接从数据中直接从数据中推断出未知ODE系统的后代。我们通过脱钩的功能采样得出稀疏的变异推断，以表示矢量场后代。我们还引入了一种概率的射击增强，以从任意长的轨迹中有效推断。该方法证明了计算矢量场后代的好处，预测不确定性得分优于多个ODE学习任务的替代方法。

The purpose of this paper is to explore the use of deep learning for the solution of the nonlinear filtering problem. This is achieved by solving the Zakai equation by a deep splitting method, previously developed for approximate solution of (stochastic) partial differential equations. This is combined with an energy-based model for the approximation of functions by a deep neural network. This results in a computationally fast filter that takes observations as input and that does not require re-training when new observations are received. The method is tested on four examples, two linear in one and twenty dimensions and two nonlinear in one dimension. The method shows promising performance when benchmarked against the Kalman filter and the bootstrap particle filter.

本文研究了使用神经跳跃（NJ-ODE）框架扩展的一般随机过程的问题。虽然NJ-ODE是为预测不规则观察到的时间序列而建立收敛保证的第一个框架，但这些结果仅限于从中\^o-diffusions的数据，特别是Markov过程，特别是在其中同时观察到所有坐标。。在这项工作中，我们通过利用签名变换的重建属性，将这些结果推广到具有不完整观察结果的通用，可能是非马克维亚或不连续的随机过程。这些理论结果得到了经验研究的支持，在该研究中，在非马克维亚数据的情况下，依赖路径依赖性的NJ-ode优于原始的NJ-ode框架。

We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a blackbox differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.

我们呈现路径积分采样器〜（PIS），一种新型算法，用于从非正规化概率密度函数中绘制样本。 PIS建立在SCHR \“odinger桥问题上，旨在恢复鉴于其初始分布和终端分布的扩散过程的最可能演变。PIS从初始分布中抽取样品，然后通过SCHR \”传播样本“少剂桥到达终端分布。应用Girsanov定理，通过简单的先前扩散，我们将PIS制定为随机最佳控制问题，其运行成本是根据目标分布选择控制能量和终端成本。通过将控件建模为神经网络，我们建立了一种可以训练结束到底的采样算法。在使用子最优控制时，我们在Wassersein距离方面提供了PIS的采样质量的理论典范。此外，路径积分理论用于计算样本的重要性权重，以补偿由控制器的次级最优性和时间离散化引起的偏差。我们通过关于各种任务的其他启动采样方法进行了实验证明了PIS的优势。

尽管存在扩散模型的各种变化，但将线性扩散扩散到非线性扩散过程中仅由几项作品研究。非线性效应几乎没有被理解，但是直觉上，将有更多有希望的扩散模式来最佳地训练生成分布向数据分布。本文介绍了基于分数扩散模型的数据自适应和非线性扩散过程。提出的隐式非线性扩散模型（INDM）通过结合归一化流量和扩散过程来学习非线性扩散过程。具体而言，INDM通过通过流网络利用\ textIt {litex {litex {littent Space}的线性扩散来隐式构建\ textIt {data Space}的非线性扩散。由于非线性完全取决于流网络，因此该流网络是形成非线性扩散的关键。这种灵活的非线性是针对DDPM ++的非MLE训练，将INDM的学习曲线提高到了几乎最大的似然估计（MLE）训练，事实证明，这是具有身份流量的INDM的特殊情况。同样，训练非线性扩散可以通过离散的步骤大小产生采样鲁棒性。在实验中，INDM实现了Celeba的最新FID。

Methods based on ordinary differential equations (ODEs) are widely used to build generative models of time-series. In addition to high computational overhead due to explicitly computing hidden states recurrence, existing ODE-based models fall short in learning sequence data with sharp transitions - common in many real-world systems - due to numerical challenges during optimization. In this work, we propose LS4, a generative model for sequences with latent variables evolving according to a state space ODE to increase modeling capacity. Inspired by recent deep state space models (S4), we achieve speedups by leveraging a convolutional representation of LS4 which bypasses the explicit evaluation of hidden states. We show that LS4 significantly outperforms previous continuous-time generative models in terms of marginal distribution, classification, and prediction scores on real-world datasets in the Monash Forecasting Repository, and is capable of modeling highly stochastic data with sharp temporal transitions. LS4 sets state-of-the-art for continuous-time latent generative models, with significant improvement of mean squared error and tighter variational lower bounds on irregularly-sampled datasets, while also being x100 faster than other baselines on long sequences.

退火重要性采样（AIS）是一种流行的算法，用于估计深层生成模型的棘手边际可能性。尽管AIS可以保证为任何一组超参数提供无偏估计，但共同的实现依赖于简单的启发式方法，例如初始和目标分布之间的几何平均桥接分布，这些分布在计算预算有限时会影响估计性性能。由于使用Markov过渡中的大都市磨碎（MH）校正步骤，因此对完全参数AI的优化仍然具有挑战性。我们提出一个具有灵活中间分布的参数AIS过程，并优化桥接分布以使用较少数量的采样步骤。一种重新聚集方法，它允许我们优化分布序列和Markov转换的参数，该参数适用于具有MH校正的大型Markov内核。我们评估了优化AIS的性能，以进行深层生成模型的边际可能性估计，并将其与其他估计器进行比较。

本论文主要涉及解决深层（时间）高斯过程（DGP）回归问题的状态空间方法。更具体地，我们代表DGP作为分层组合的随机微分方程（SDES），并且我们通过使用状态空间过滤和平滑方法来解决DGP回归问题。由此产生的状态空间DGP（SS-DGP）模型生成丰富的电视等级，与建模许多不规则信号/功能兼容。此外，由于他们的马尔可道结构，通过使用贝叶斯滤波和平滑方法可以有效地解决SS-DGPS回归问题。本论文的第二次贡献是我们通过使用泰勒力矩膨胀（TME）方法来解决连续离散高斯滤波和平滑问题。这诱导了一类滤波器和SmooThers，其可以渐近地精确地预测随机微分方程（SDES）解决方案的平均值和协方差。此外，TME方法和TME过滤器和SmoOthers兼容模拟SS-DGP并解决其回归问题。最后，本文具有多种状态 - 空间（深）GPS的应用。这些应用主要包括（i）来自部分观察到的轨迹的SDES的未知漂移功能和信号的光谱 - 时间特征估计。

本文使用最佳运输理论介绍了贝叶法律的各种表示。差异表示是根据（状态，观察）及其独立耦合之间的最佳运输。通过将某些结构施加在传输图上，用于变异问题的解决方案用于构建一个将先前分布传输到观测信号的任何值的Brenier型图。新的公式用于用于离散时间过滤问题的集合卡尔曼滤波器（ENKF）的最佳传输形式，并使用输入凸神经网络提出了ENKF向非高斯设置的新型扩展。最后，所提出的方法用于在连续时限内得出反馈粒子填充物（FPF）的最佳运输形式，该形式构成了其第一个变化构建，而无需明确使用非线性滤波方程或贝叶斯定律。

Temporal data like time series are often observed at irregular intervals which is a challenging setting for existing machine learning methods. To tackle this problem, we view such data as samples from some underlying continuous function. We then define a diffusion-based generative model that adds noise from a predefined stochastic process while preserving the continuity of the resulting underlying function. A neural network is trained to reverse this process which allows us to sample new realizations from the learned distribution. We define suitable stochastic processes as noise sources and introduce novel denoising and score-matching models on processes. Further, we show how to apply this approach to the multivariate probabilistic forecasting and imputation tasks. Through our extensive experiments, we demonstrate that our method outperforms previous models on synthetic and real-world datasets.

本文与离散观察到的非线性扩散过程的在线过滤有关。我们的方法基于完全适应的辅助粒子滤波器，该滤芯涉及DOOB的$ h $转换通常是棘手的。我们提出了一个计算框架，通过使用非线性FEYNMAN-KAC公式和神经网络求解基础的落后Kolmogorov方程来近似这些$ H $转换。该方法允许在数据鉴别过程之前训练本地最佳的粒子过滤器。数值实验表明，在高度信息观察结果的制度中，当观测值在模型下极端，如果状态维度很大时，所提出的方法可以比引导粒子滤波器更有效的数量级。

Non-linear state-space models, also known as general hidden Markov models, are ubiquitous in statistical machine learning, being the most classical generative models for serial data and sequences in general. The particle-based, rapid incremental smoother PaRIS is a sequential Monte Carlo (SMC) technique allowing for efficient online approximation of expectations of additive functionals under the smoothing distribution in these models. Such expectations appear naturally in several learning contexts, such as likelihood estimation (MLE) and Markov score climbing (MSC). PARIS has linear computational complexity, limited memory requirements and comes with non-asymptotic bounds, convergence results and stability guarantees. Still, being based on self-normalised importance sampling, the PaRIS estimator is biased. Our first contribution is to design a novel additive smoothing algorithm, the Parisian particle Gibbs PPG sampler, which can be viewed as a PaRIS algorithm driven by conditional SMC moves, resulting in bias-reduced estimates of the targeted quantities. We substantiate the PPG algorithm with theoretical results, including new bounds on bias and variance as well as deviation inequalities. Our second contribution is to apply PPG in a learning framework, covering MLE and MSC as special examples. In this context, we establish, under standard assumptions, non-asymptotic bounds highlighting the value of bias reduction and the implicit Rao--Blackwellization of PPG. These are the first non-asymptotic results of this kind in this setting. We illustrate our theoretical results with numerical experiments supporting our claims.

Neural compression offers a domain-agnostic approach to creating codecs for lossy or lossless compression via deep generative models. For sequence compression, however, most deep sequence models have costs that scale with the sequence length rather than the sequence complexity. In this work, we instead treat data sequences as observations from an underlying continuous-time process and learn how to efficiently discretize while retaining information about the full sequence. As a consequence of decoupling sequential information from its temporal discretization, our approach allows for greater compression rates and smaller computational complexity. Moreover, the continuous-time approach naturally allows us to decode at different time intervals. We empirically verify our approach on multiple domains involving compression of video and motion capture sequences, showing that our approaches can automatically achieve reductions in bit rates by learning how to discretize.

去噪扩散概率模型（DDPMS）在没有对抗性训练的情况下实现了高质量的图像生成，但它们需要模拟Markov链以产生样品的许多步骤。为了加速采样，我们呈现去噪扩散隐式模型（DDIM），更有效的迭代类隐式概率模型，具有与DDPM相同的培训过程。在DDPMS中，生成过程被定义为Markovian扩散过程的反向。我们构建一类导致相同的训练目标的非马尔可瓦夫扩散过程，但其反向过程可能会更快地采样。我们经验证明，与DDPM相比，DDIM可以生产高质量的样本10倍以上$ 50 \时间$ 50 \倍。允许我们缩小对样本质量的计算，并可以直接执行语义有意义的图像插值潜在的空间。

Using historical data to predict future events has many applications in the real world, such as stock price prediction; the robot localization. In the past decades, the Convolutional long short-term memory (LSTM) networks have achieved extraordinary success with sequential data in the related field. However, traditional recurrent neural networks (RNNs) keep the hidden states in a deterministic way. In this paper, we use the particles to approximate the distribution of the latent state and show how it can extend into a more complex form, i.e., the Encoder-Decoder mechanism. With the proposed continuous differentiable scheme, our model is capable of adaptively extracting valuable information and updating the latent state according to the Bayes rule. Our empirical studies demonstrate the effectiveness of our method in the prediction tasks.