尽管量子至高无上尚未到来，但最近在实用量子计算的迫在眉睫的时代，人们对​​确定量子机学习的潜力（QML）的兴趣越来越大。由此激励，在本文中，我们基于具有两个单独的可训练参数的单独维度的量子神经网络（QNN）的独特特征重新设计多代理增强学习（MARL）：影响输出Qubit状态和极点参数的角度参数：与输出测量基础相关。我们提出了将这种二元训练性作为元学习能力，我们提出了量子元marl（QM2ARL），该量子元MARL（QM2ARL）首先应用角度训练进行元学习，然后进行极点训练，以进行几次射击或局部QNN培训。为了避免过度拟合，我们在角度训练期间开发了一种将噪声注入到极域中的角度正则化技术。此外，通过将极点作为每个受过训练的QNN的内存地址利用，我们介绍了极点内存的概念，允许仅使用两参数极点值保存和加载经过训练的QNN。从理论上讲，我们证明了角度到极正则化下的角度训练的收敛性，并通过模拟证实了QM2ARL在获得高奖励和快速收敛方面的有效性，以及在快速适应时间变化环境中的极点记忆。

Quantum Computing在古典计算机上解决困难的计算任务的显着改进承诺。然而，为实际使用设计量子电路不是琐碎的目标，并且需要专家级知识。为了帮助这一努力，提出了一种基于机器学习的方法来构建量子电路架构。以前的作品已经证明，经典的深度加强学习（DRL）算法可以成功构建量子电路架构而没有编码的物理知识。但是，这些基于DRL的作品不完全在更换设备噪声中的设置，从而需要大量的培训资源来保持RL模型最新。考虑到这一点，我们持续学习，以提高算法的性能。在本文中，我们介绍了深度Q-Learning（PPR-DQL）框架的概率策略重用来解决这个电路设计挑战。通过通过各种噪声模式进行数值模拟，我们证明了具有PPR的RL代理能够找到量子栅极序列，以比从划痕训练的代理更快地生成双量标铃声状态。所提出的框架是一般的，可以应用于其他量子栅极合成或控制问题 - 包括量子器件的自动校准。

随着真实世界量子计算的出现，参数化量子计算可以用作量子古典机器学习系统中的假设家庭的想法正在增加牵引力的增加。这种混合系统已经表现出潜力在监督和生成学习中解决现实世界任务，最近的作品已经在特殊的人工任务中建立了他们可提供的优势。然而，在加强学习的情况下，可以说是最具挑战性的，并且学习提升将是极为有价值的，在解决甚至标准的基准测试方面没有成功地取得了成功，也没有在典型算法上表达理论上的学习优势。在这项工作中，我们均达到两者。我们提出了一种使用很少的Qubits的混合量子古典强化学习模型，我们展示了可以有效地培训，以解决若干标准基准环境。此外，我们展示和正式证明，参数化量子电路解决了用于古典模型的棘手的某些学习任务的能力，包括当前最先进的深神经网络，在离散对数问题的广泛的经典硬度下。

Quantum computing (QC) promises significant advantages on certain hard computational tasks over classical computers. However, current quantum hardware, also known as noisy intermediate-scale quantum computers (NISQ), are still unable to carry out computations faithfully mainly because of the lack of quantum error correction (QEC) capability. A significant amount of theoretical studies have provided various types of QEC codes; one of the notable topological codes is the surface code, and its features, such as the requirement of only nearest-neighboring two-qubit control gates and a large error threshold, make it a leading candidate for scalable quantum computation. Recent developments of machine learning (ML)-based techniques especially the reinforcement learning (RL) methods have been applied to the decoding problem and have already made certain progress. Nevertheless, the device noise pattern may change over time, making trained decoder models ineffective. In this paper, we propose a continual reinforcement learning method to address these decoding challenges. Specifically, we implement double deep Q-learning with probabilistic policy reuse (DDQN-PPR) model to learn surface code decoding strategies for quantum environments with varying noise patterns. Through numerical simulations, we show that the proposed DDQN-PPR model can significantly reduce the computational complexity. Moreover, increasing the number of trained policies can further improve the agent's performance. Our results open a way to build more capable RL agents which can leverage previously gained knowledge to tackle QEC challenges.

在过去的十年中，深入的强化学习（RL）已经取得了长足的进步。同时，最先进的RL算法在培训时间融合方面需要大量的计算预算。最近的工作已经开始通过量子计算的角度来解决这个问题，这有望为几项传统上的艰巨任务做出理论上的速度。在这项工作中，我们研究了一类混合量子古典RL算法，我们共同称为变异量子Q-NETWORKS（VQ-DQN）。我们表明，VQ-DQN方法受到导致学习政策分歧的不稳定性的约束，研究了基于经典模拟的既定结果的重复性，并执行系统的实验以识别观察到的不稳定性的潜在解释。此外，与大多数现有的量子增强学习中现有工作相反，我们在实际量子处理单元（IBM量子设备）上执行RL算法，并研究模拟和物理量子系统之间因实施不足而进行的行为差异。我们的实验表明，与文献中相反的主张相反，与经典方法相比，即使在没有物理缺陷的情况下进行模拟，也不能最终决定是否已知量子方法，也可以提供优势。最后，我们提供了VQ-DQN作为可再现的测试床的强大，通用且经过充分测试的实现，以实现未来的实验。

在这项工作中，我们利用量子深的增强学习作为方法，以在三个模拟的复杂性的模拟环境中为简单的，轮式机器人学习导航任务。我们显示了与经典基线相比，在混合量子古典设置中训练有良好建立的深钢筋学习技术的参数化量子电路的相似性能。据我们所知，这是用于机器人行为的量子机学习（QML）的首次演示。因此，我们将机器人技术建立为QML算法的可行研究领域，此后量子计算和量子机学习是自治机器人技术未来进步的潜在技术。除此之外，我们讨论了当前的方法的限制以及自动机器人量子机学习领域的未来研究方向。

While witnessing the noisy intermediate-scale quantum (NISQ) era and beyond, quantum federated learning (QFL) has recently become an emerging field of study. In QFL, each quantum computer or device locally trains its quantum neural network (QNN) with trainable gates, and communicates only these gate parameters over classical channels, without costly quantum communications. Towards enabling QFL under various channel conditions, in this article we develop a depth-controllable architecture of entangled slimmable quantum neural networks (eSQNNs), and propose an entangled slimmable QFL (eSQFL) that communicates the superposition-coded parameters of eS-QNNs. Compared to the existing depth-fixed QNNs, training the depth-controllable eSQNN architecture is more challenging due to high entanglement entropy and inter-depth interference, which are mitigated by introducing entanglement controlled universal (CU) gates and an inplace fidelity distillation (IPFD) regularizer penalizing inter-depth quantum state differences, respectively. Furthermore, we optimize the superposition coding power allocation by deriving and minimizing the convergence bound of eSQFL. In an image classification task, extensive simulations corroborate the effectiveness of eSQFL in terms of prediction accuracy, fidelity, and entropy compared to Vanilla QFL as well as under different channel conditions and various data distributions.

量子机学习（QML）被认为是近术语量子设备最有前途的应用之一。然而，量子机器学习模型的优化呈现出众多挑战，从硬件的缺陷和导航指数上缩放的希尔伯特空间中的缺陷产生了巨大的挑战。在这项工作中，我们评估了深度增强学习中的当代方法的潜力，以增加量子变分电路中的增强基于梯度的优化例程。我们发现强化学习增强了优化器，始终突出噪声环境中的渐变血统。所有代码和备用重量都可用于复制结果或在https://github.com/lockwo/rl_qvc_opt上部署模型。

Deep Reinforcement Learning is emerging as a promising approach for the continuous control task of robotic arm movement. However, the challenges of learning robust and versatile control capabilities are still far from being resolved for real-world applications, mainly because of two common issues of this learning paradigm: the exploration strategy and the slow learning speed, sometimes known as "the curse of dimensionality". This work aims at exploring and assessing the advantages of the application of Quantum Computing to one of the state-of-art Reinforcement Learning techniques for continuous control - namely Soft Actor-Critic. Specifically, the performance of a Variational Quantum Soft Actor-Critic on the movement of a virtual robotic arm has been investigated by means of digital simulations of quantum circuits. A quantum advantage over the classical algorithm has been found in terms of a significant decrease in the amount of required parameters for satisfactory model training, paving the way for further promising developments.

Variational quantum circuits have been widely employed in quantum simulation and quantum machine learning in recent years. However, quantum circuits with random structures have poor trainability due to the exponentially vanishing gradient with respect to the circuit depth and the qubit number. This result leads to a general standpoint that deep quantum circuits would not be feasible for practical tasks. In this work, we propose an initialization strategy with theoretical guarantees for the vanishing gradient problem in general deep quantum circuits. Specifically, we prove that under proper Gaussian initialized parameters, the norm of the gradient decays at most polynomially when the qubit number and the circuit depth increase. Our theoretical results hold for both the local and the global observable cases, where the latter was believed to have vanishing gradients even for very shallow circuits. Experimental results verify our theoretical findings in the quantum simulation and quantum chemistry.

量子计算的最新进展已显示出许多问题领域的有希望的计算优势。作为越来越关注的领域之一，混合量子古典机器学习系统已经证明了解决各种数据驱动的学习任务的能力。最近的作品表明，参数化的量子电路（PQC）可用于以可证明的学习优势来解决具有挑战性的强化学习（RL）任务。尽管现有的作品产生了基于PQC的方法的潜力，但PQC体系结构的设计选择及其对学习任务的影响通常没有得到充实。在这项工作中，我们介绍了基于PQC的模型EQAS-PQC，这是一种进化的量子体系结构搜索框架，该模型使用基于人群的遗传算法来通过探索量子操作的搜索空间来发展PQC体系结构。实验结果表明，我们的方法可以显着改善混合量子古典模型在解决基准增强问题方面的性能。我们还对量子操作的概率分布进行建模，以表现出色的体系结构，以识别对性能至关重要的基本设计选择。

量子联合学习（QFL）最近受到了越来越多的关注，其中量子神经网络（QNN）集成到联邦学习（FL）中。与现有的静态QFL方法相反，我们在本文中提出了可靠的QFL（SLIMQFL），这是一个动态QFL框架，可以应对时变的通信通道和计算能量限制。通过利用QNN的独特性质，可以分别训练并动态利用其角度参数，从而使其可行。模拟结果证实了SLIMQFL比香草QFL更高的分类精度，尤其是在较差的通道条件下。

我们提出了一种普遍和系统的策略来编制任意量子信道而不使用辅助额度的辅助额度 - 一种强大的深度加强学习算法。我们严格证明，与编译酉栅极的情况鲜明对比，不管分解序列的长度如何，不可能将任意信道与任意精度编译成任意精度。但是，对于固定精度$ \ epsilon $一个可以用恒定数量的$ \ epsilon $ -dependent基本通道构建通用集，使得任意量子通道可以分解成这些基本通道的序列，然后是酉门，序列长度有$ o（\ frac {1} {\ epsilon} \ log \ frac {1} {\ epsilon}）$。通过一个关于Majorana Fermions的拓扑编译的具体例子，我们表明我们所提出的算法可以通过将加权成本添加到近端政策优化的奖励功能中方便和有效地减少昂贵的基本栅极的使用。

强化学习目睹了最近在量子编程中的各种任务中的应用。基本的假设是这些任务可以建模为马尔可夫决策过程（MDP）。在这里，我们通过探索量子编程中的两个基本任务的后果来研究该假设的可行性：状态制备和门编译。通过形成离散的MDP，专门针对单量的情况（无论有没有噪声），我们可以通过策略迭代准确地为最佳策略求解。我们找到与最短门序列相对应的最佳路径，以准备状态或编译门，直至某些目标精度。例如，我们发现$ h $和$ t $门的序列长达$ 11 $生产$ \ sim 99 \％$ $ fidelity表格$（ht）^{n} | 0 \ rangle $值高达$ n = 10^{10} $。在存在门噪声的情况下，我们演示了最佳政策如何适应嘈杂的门的影响，以实现更高的状态忠诚度。我们的工作表明，人们可以将离散，随机和马尔可夫的性质强加于连续，确定性和非马克维亚量子演化，并提供理论上的洞察力，以了解为什么可以成功地使用强化学习来找到量子编程中的最佳短门序列。

在当前的嘈杂中间尺度量子（NISQ）时代，量子机学习正在成为基于程序门的量子计算机的主要范式。在量子机学习中，对量子电路的门进行了参数化，并且参数是根据数据和电路输出的测量来通过经典优化来调整的。参数化的量子电路（PQC）可以有效地解决组合优化问题，实施概率生成模型并进行推理（分类和回归）。该专着为具有概率和线性代数背景的工程师的观众提供了量子机学习的独立介绍。它首先描述了描述量子操作和测量所必需的必要背景，概念和工具。然后，它涵盖了参数化的量子电路，变异量子本质层以及无监督和监督的量子机学习公式。

The emergence of variational quantum applications has led to the development of automatic differentiation techniques in quantum computing. Recently, Zhu et al. (PLDI 2020) have formulated differentiable quantum programming with bounded loops, providing a framework for scalable gradient calculation by quantum means for training quantum variational applications. However, promising parameterized quantum applications, e.g., quantum walk and unitary implementation, cannot be trained in the existing framework due to the natural involvement of unbounded loops. To fill in the gap, we provide the first differentiable quantum programming framework with unbounded loops, including a newly designed differentiation rule, code transformation, and their correctness proof. Technically, we introduce a randomized estimator for derivatives to deal with the infinite sum in the differentiation of unbounded loops, whose applicability in classical and probabilistic programming is also discussed. We implement our framework with Python and Q#, and demonstrate a reasonable sample efficiency. Through extensive case studies, we showcase an exciting application of our framework in automatically identifying close-to-optimal parameters for several parameterized quantum applications.

With the development of experimental quantum technology, quantum control has attracted increasing attention due to the realization of controllable artificial quantum systems. However, because quantum-mechanical systems are often too difficult to analytically deal with, heuristic strategies and numerical algorithms which search for proper control protocols are adopted, and, deep learning, especially deep reinforcement learning (RL), is a promising generic candidate solution for the control problems. Although there have been a few successful applications of deep RL to quantum control problems, most of the existing RL algorithms suffer from instabilities and unsatisfactory reproducibility, and require a large amount of fine-tuning and a large computational budget, both of which limit their applicability. To resolve the issue of instabilities, in this dissertation, we investigate the non-convergence issue of Q-learning. Then, we investigate the weakness of existing convergent approaches that have been proposed, and we develop a new convergent Q-learning algorithm, which we call the convergent deep Q network (C-DQN) algorithm, as an alternative to the conventional deep Q network (DQN) algorithm. We prove the convergence of C-DQN and apply it to the Atari 2600 benchmark. We show that when DQN fail, C-DQN still learns successfully. Then, we apply the algorithm to the measurement-feedback cooling problems of a quantum quartic oscillator and a trapped quantum rigid body. We establish the physical models and analyse their properties, and we show that although both C-DQN and DQN can learn to cool the systems, C-DQN tends to behave more stably, and when DQN suffers from instabilities, C-DQN can achieve a better performance. As the performance of DQN can have a large variance and lack consistency, C-DQN can be a better choice for researches on complicated control problems.

我们提出了新型量子加固学习（RL）方法的完整实现和模拟，并在数学上证明了量子优势。我们的方法详细说明了如何将振幅估计和Grover搜索结合到政策评估和改进方案中。我们首先开发量子策略评估（QPE），与类似的经典蒙特卡洛估计相比，它在四四方面更有效，并且基于有限马尔可夫决策过程（MDP）的量子机械实现。在QPE的基础上，我们得出了一种量子策略迭代，该迭代迭代可以反复使用Grover搜索来改善初始策略，直到达到最佳。最后，我们为两臂强盗MDP提供了算法的实现，然后我们进行了模拟。结果证实QPE在RL问题中提供了量子优势。

量子计算有可能彻底改变和改变我们的生活和理解世界的方式。该审查旨在提供对量子计算的可访问介绍，重点是统计和数据分析中的应用。我们从介绍了了解量子计算所需的基本概念以及量子和经典计算之间的差异。我们描述了用作量子算法的构建块的核心量子子程序。然后，我们审查了一系列预期的量子算法，以便在统计和机器学习中提供计算优势。我们突出了将量子计算应用于统计问题的挑战和机遇，并讨论潜在的未来研究方向。

Understanding the power and limitations of quantum access to data in machine learning tasks is primordial to assess the potential of quantum computing in artificial intelligence. Previous works have already shown that speed-ups in learning are possible when given quantum access to reinforcement learning environments. Yet, the applicability of quantum algorithms in this setting remains very limited, notably in environments with large state and action spaces. In this work, we design quantum algorithms to train state-of-the-art reinforcement learning policies by exploiting quantum interactions with an environment. However, these algorithms only offer full quadratic speed-ups in sample complexity over their classical analogs when the trained policies satisfy some regularity conditions. Interestingly, we find that reinforcement learning policies derived from parametrized quantum circuits are well-behaved with respect to these conditions, which showcases the benefit of a fully-quantum reinforcement learning framework.