集中式培训（CT）是许多受欢迎的多代理增强学习（MARL）方法的基础，因为它允许代理商快速学习高性能的政策。但是，CT依靠代理人从对特定州对其他代理商的行为的一次性观察中学习。由于MARL代理商在培训期间探索和更新其政策，因此这些观察结果通常会为其他代理商的行为和预期的给定行动回报提供不良的预测。因此，CT方法患有较高的差异和容易出错的估计，从而损害了学习。除非施加了强大的分解限制，否则CT方法还遭受了复杂性爆炸性增长（例如，QMIX的单调奖励函数）。我们通过一个新的半居中的MAL框架来应对这些挑战，该框架执行政策安装的培训和分散的执行。我们的方法是嵌入式增强学习算法（PERLA），是参与者批评的MARL算法的增强工具，它利用了一种新型参数共享协议和策略嵌入方法来维持对其他代理商的行为的估计。我们的理论证明，佩拉大大降低了价值估计的差异。与各种CT方法不同，Perla无缝地采用MARL算法，它可以轻松地与代理数量缩放，而无需限制性分解假设。我们展示了Perla在基准环境中的出色经验表现和有效的缩放，包括Starcraft Micromagement II和Multi-Agent Mujoco

几乎可以肯定（或使用概率）满足安全限制对于在现实生活中的增强学习（RL）的部署至关重要。例如，理想情况下，平面降落和起飞应以概率为单位发生。我们通过引入安全增强（SAUTE）马尔可夫决策过程（MDP）来解决该问题，在该过程中，通过将其扩大到州空间并重塑目标来消除安全限制。我们表明，Saute MDP满足了Bellman方程，并使我们更加接近解决安全的RL，几乎可以肯定地满足。我们认为，Saute MDP允许从不同的角度查看安全的RL问题，从而实现新功能。例如，我们的方法具有插件的性质，即任何RL算法都可以“炒”。此外，国家扩展允许跨安全限制进行政策概括。我们最终表明，当约束满意度非常重要时，SAUTE RL算法的表现可以胜过其最先进的对应物。

强化学习（RL）涉及在未知系统中执行探索性动作。这可以将学习代理放在危险且潜在的灾难性系统中。当前在RL中解决安全学习的方法同时权衡了安全探索和任务实现。在本文中，我们介绍了新一代的RL求解器，这些求解器学会最大程度地减少安全性违规行为，同时在安全政策可以容忍的范围内最大化任务奖励。我们的方法引入了一个新型的两人框架，用于安全RL，称为分配探索安全培训算法（DESTA）。 DESTA的核心是两种自适应代理之间的游戏：安全代理，其任务是最大程度地减少安全违规行为和任务代理，其目标是最大程度地提高环境奖励。具体而言，安全代理可以在任何给定点有选择地控制系统，以防止任务代理在任何其他州自由执行其策略时违反安全性。该框架使安全代理能够学会在培训和测试时间中最大程度地减少未来安全违规行为的某些行动，而任务代理人执行的动作可以最大程度地提高其他任何地方的任务绩效。从理论上讲，我们证明DESTA会汇合到稳定的点，从而最大程度地违反了对预验证的政策的行为。从经验上讲，我们表明了DESTA提高现有政策安全性的能力，其次，当对任务代理和安全代理人同时培训时，构建安全的RL政策。我们展示了DESTA在Lunar Lander和Openai Gym的Frozen Lake中的领先RL方法的出色表现。

奖励成型（RS）是克服稀疏或不信息奖励问题的强大方法（RL）。但是，RS通常依赖于手动设计的成型奖励功能，其构造耗时且容易出错。它还需要与自主学习目标相反的领域知识。我们介绍了增强学习优化塑造算法（ROSA）的增强型，这是一个自动化的RS框架，其中塑造奖励函数是在两个代理之间的新型马尔可夫游戏中构建的。奖励塑料代理（Shaper）使用切换控件来确定在其他代理（控制器）使用这些形状奖励的任务中学习任务的最佳策略，以确定要添加形状奖励及其最佳值的状态。我们证明，Rosa很容易采用现有的RL算法，学会了构建针对任务的塑造奖励功能，从而确保有效地收敛到高性能策略。我们在三个经过精心设计的实验中展示了罗莎（Rosa）在挑战稀疏奖励环境中对最先进的RS算法的优越性能。

In this paper, we propose a novel technique, namely INVALIDATOR, to automatically assess the correctness of APR-generated patches via semantic and syntactic reasoning. INVALIDATOR reasons about program semantic via program invariants while it also captures program syntax via language semantic learned from large code corpus using the pre-trained language model. Given a buggy program and the developer-patched program, INVALIDATOR infers likely invariants on both programs. Then, INVALIDATOR determines that a APR-generated patch overfits if: (1) it violates correct specifications or (2) maintains errors behaviors of the original buggy program. In case our approach fails to determine an overfitting patch based on invariants, INVALIDATOR utilizes a trained model from labeled patches to assess patch correctness based on program syntax. The benefit of INVALIDATOR is three-fold. First, INVALIDATOR is able to leverage both semantic and syntactic reasoning to enhance its discriminant capability. Second, INVALIDATOR does not require new test cases to be generated but instead only relies on the current test suite and uses invariant inference to generalize the behaviors of a program. Third, INVALIDATOR is fully automated. We have conducted our experiments on a dataset of 885 patches generated on real-world programs in Defects4J. Experiment results show that INVALIDATOR correctly classified 79% overfitting patches, accounting for 23% more overfitting patches being detected by the best baseline. INVALIDATOR also substantially outperforms the best baselines by 14% and 19% in terms of Accuracy and F-Measure, respectively.

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.

Cashews are grown by over 3 million smallholders in more than 40 countries worldwide as a principal source of income. As the third largest cashew producer in Africa, Benin has nearly 200,000 smallholder cashew growers contributing 15% of the country's national export earnings. However, a lack of information on where and how cashew trees grow across the country hinders decision-making that could support increased cashew production and poverty alleviation. By leveraging 2.4-m Planet Basemaps and 0.5-m aerial imagery, newly developed deep learning algorithms, and large-scale ground truth datasets, we successfully produced the first national map of cashew in Benin and characterized the expansion of cashew plantations between 2015 and 2021. In particular, we developed a SpatioTemporal Classification with Attention (STCA) model to map the distribution of cashew plantations, which can fully capture texture information from discriminative time steps during a growing season. We further developed a Clustering Augmented Self-supervised Temporal Classification (CASTC) model to distinguish high-density versus low-density cashew plantations by automatic feature extraction and optimized clustering. Results show that the STCA model has an overall accuracy of 80% and the CASTC model achieved an overall accuracy of 77.9%. We found that the cashew area in Benin has doubled from 2015 to 2021 with 60% of new plantation development coming from cropland or fallow land, while encroachment of cashew plantations into protected areas has increased by 70%. Only half of cashew plantations were high-density in 2021, suggesting high potential for intensification. Our study illustrates the power of combining high-resolution remote sensing imagery and state-of-the-art deep learning algorithms to better understand tree crops in the heterogeneous smallholder landscape.

Coronary Computed Tomography Angiography (CCTA) provides information on the presence, extent, and severity of obstructive coronary artery disease. Large-scale clinical studies analyzing CCTA-derived metrics typically require ground-truth validation in the form of high-fidelity 3D intravascular imaging. However, manual rigid alignment of intravascular images to corresponding CCTA images is both time consuming and user-dependent. Moreover, intravascular modalities suffer from several non-rigid motion-induced distortions arising from distortions in the imaging catheter path. To address these issues, we here present a semi-automatic segmentation-based framework for both rigid and non-rigid matching of intravascular images to CCTA images. We formulate the problem in terms of finding the optimal \emph{virtual catheter path} that samples the CCTA data to recapitulate the coronary artery morphology found in the intravascular image. We validate our co-registration framework on a cohort of $n=40$ patients using bifurcation landmarks as ground truth for longitudinal and rotational registration. Our results indicate that our non-rigid registration significantly outperforms other co-registration approaches for luminal bifurcation alignment in both longitudinal (mean mismatch: 3.3 frames) and rotational directions (mean mismatch: 28.6 degrees). By providing a differentiable framework for automatic multi-modal intravascular data fusion, our developed co-registration modules significantly reduces the manual effort required to conduct large-scale multi-modal clinical studies while also providing a solid foundation for the development of machine learning-based co-registration approaches.

Springs are efficient in storing and returning elastic potential energy but are unable to hold the energy they store in the absence of an external load. Lockable springs use clutches to hold elastic potential energy in the absence of an external load but have not yet been widely adopted in applications, partly because clutches introduce design complexity, reduce energy efficiency, and typically do not afford high-fidelity control over the energy stored by the spring. Here, we present the design of a novel lockable compression spring that uses a small capstan clutch to passively lock a mechanical spring. The capstan clutch can lock up to 1000 N force at any arbitrary deflection, unlock the spring in less than 10 ms with a control force less than 1 % of the maximal spring force, and provide an 80 % energy storage and return efficiency (comparable to a highly efficient electric motor operated at constant nominal speed). By retaining the form factor of a regular spring while providing high-fidelity locking capability even under large spring forces, the proposed design could facilitate the development of energy-efficient spring-based actuators and robots.