This paper introduces the use of evolutionary algorithms for solving differential equations. The solution is obtained by optimizing a deep neural network whose loss function is defined by the residual terms from the differential equations. Recent studies have used stochastic gradient descent (SGD) variants to train these physics-informed neural networks (PINNs), but these methods can struggle to find accurate solutions due to optimization challenges. When solving differential equations, it is important to find the globally optimum parameters of the network, rather than just finding a solution that works well during training. SGD only searches along a single gradient direction, so it may not be the best approach for training PINNs with their accompanying complex optimization landscapes. In contrast, evolutionary algorithms perform a parallel exploration of different solutions in order to avoid getting stuck in local optima and can potentially find more accurate solutions. However, evolutionary algorithms can be slow, which can make them difficult to use in practice. To address this, we provide a set of five benchmark problems with associated performance metrics and baseline results to support the development of evolutionary algorithms for enhanced PINN training. As a baseline, we evaluate the performance and speed of using the widely adopted Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for solving PINNs. We provide the loss and training time for CMA-ES run on TensorFlow, and CMA-ES and SGD run on JAX (with GPU acceleration) for the five benchmark problems. Our results show that JAX-accelerated evolutionary algorithms, particularly CMA-ES, can be a useful approach for solving differential equations. We hope that our work will support the exploration and development of alternative optimization algorithms for the complex task of optimizing PINNs.

最近，在推荐系统领域中，一个关键问题隐约可见 - 没有进行严格评估的有效基准 - 因此，这会导致不可再生的评估和不公平的比较。因此，我们从实践理论和实验的角度进行研究，目的是为严格的评估做出基准建议。关于理论研究，一系列影响整个评估链中建议性能的超级因素通过对2017 - 2020年在八个顶级会议上发表的141篇论文进行的详尽评价进行了系统的总结和分析。然后，我们将它们分类为独立于模型和模型依赖性的超因子，并相应地定义和讨论了不同的严格评估模式。在实验研究中，我们通过将这些超级因子整合以进行严格的评估来发布DaisyREC 2.0文库，从而进行了整体经验研究，以揭示不同超级效应器对建议性能的影响。在理论和实验研究的支持下，我们最终通过提出标准化程序并在六个数据集上的六个评估指标中提供10个最先进的方法来创建严格评估的基准，以作为以后研究的参考。总体而言，我们的工作阐明了建议评估中的问题，为严格的评估提供了潜在的解决方案，并为进一步调查提供了基础。

对于深度学习，大小就是力量。经过广泛数据训练的大量神经网是人工智能的最前沿。这些基础模型或“所有行业的千斤顶”（JATS）（JATS）进行了微调，以进行下游任务，在推动深度学习进步方面变得重要。但是，具有严格资源限制的环境，目标和意图不断变化或任务要求各异，可能会限制单数JAT的实际实用程序。因此，本文与当前建立越来越大的Jats的趋势同时进行了对概念的初步探索，该概念是创建各种紧凑的机器学习模型集的基础。由许多较小和专业的模型组成，我们制定了一组集合，以同时满足许多任务设置和环境条件。首次提出了在神经进化多任务算法的一次传球中进行此类设置的一种手段，这使我们更接近了“所有行业的大师”的模型。

研究表明，进化策略（ES）是具有深层神经网络的强化学习（RL）的有前途的方法。但是，高样本复杂性的问题仍然存在于ES对深度RL的应用中。本文是第一个通过新颖的神经进化多任务处理（NUEMT）算法解决当今方法的缺点，该算法旨在将信息从一组（短情节长度）转移到目标（全长）的RL任务。从目标中提取的辅助任务允许代理更新并快速评估较短时间范围的策略。然后转移进化的技能，以指导更长，更艰巨的任务实现最佳政策。我们证明了NUEMT算法达到了数据叶进化RL，从而减少了昂贵的代理环境相互作用数据要求。在这种情况下，我们的主要算法贡献是首次基于统计重要性抽样技术引入多任务技能转移机制。此外，利用自适应资源分配策略将计算资源分配给基于其收集的实用性的辅助任务。关于OpenAI体育馆的一系列连续控制任务的实验证实，与最近的ES基线相比，我们提出的算法有效。

多任务高斯流程（MTGP）是一种众所周知的非参数贝叶斯模型，用于通过跨任务传输知识来有效地学习相关任务。但是当前的MTGP通常仅限于在同一输入域中定义的多任务场景，没有留出空间来解决异质案例，即输入域的特征在任务上有所不同。为此，本文提出了一个新型的异质随机变化线性模型（\ texttt {hsvlmc}）模型，用于同时学习具有不同输入域的任务。特别是，我们通过贝叶斯校准开发了随机变化框架，该框架（i）考虑了域映射提高的尺寸降低的影响，以实现有效的输入对准； （ii）采用残差建模策略来利用先前域映射带来的电感偏差来获得更好的模型推断。最后，对现有LMC模型的优势在各种异质的多任务案例和实用的多保真蒸汽轮机排气问题上进行了广泛的验证。

在本文中，我们提出了联合关注（CAS），一类新颖的学习与参加图形神经网络（GNN）的策略。除了考虑GNN内传播的层面节点特征，CAS可以另外包含各种结构干预，例如节点集群嵌入，以及在计算注意力分数时可以在GNN之外学习的高阶结构相关性。因此，由联合标准视为重要的节点特征，因此更可能在GNN中传播。鉴于新颖的联合注意力策略，我们提出了可以学习嵌入嵌入的表示的表演的图表关注网络（CAT），这些网络嵌入具有与联合关注所认为的显着潜在的特征。此外，我们理论上验证了猫的辨别能力。利用所提出的联合注意力策略的猫已经在建立了完善的基准数据集中广泛测试，并与最先进的基线进行了全面的基础测试。所获得的显着性能证明了所提出的联合关注的有效性。

预处理的基于变压器的语言模型（LMS）显示出显着的自然语言生成能力。凭借其巨大的潜力，控制这种LM的文本生成引起了人们的关注。尽管有一些研究试图控制生成的文本的高级属性（例如情感和主题），但仍然缺乏对其在单词和短语级别上的内容的更精确的控制。在这里，我们建议内容调节器（COCON）以细粒度的水平控制LM的输出文本。在我们的自我监督方法中，Cocon Block学会了通过调节从LM中扣留的内容输入来帮助LM完成部分观察到的文本序列。通过实验，我们表明Cocon可以自然地将目标内容纳入生成的文本中，并以零拍的方式控制高级文本属性。

Graph neural networks (GNNs) have been shown to be highly sensitive to the choice of aggregation function. While summing over a node's neighbours can approximate any permutation-invariant function over discrete inputs, Cohen-Karlik et al. [2020] proved there are set-aggregation problems for which summing cannot generalise to unbounded inputs, proposing recurrent neural networks regularised towards permutation-invariance as a more expressive aggregator. We show that these results carry over to the graph domain: GNNs equipped with recurrent aggregators are competitive with state-of-the-art permutation-invariant aggregators, on both synthetic benchmarks and real-world problems. However, despite the benefits of recurrent aggregators, their $O(V)$ depth makes them both difficult to parallelise and harder to train on large graphs. Inspired by the observation that a well-behaved aggregator for a GNN is a commutative monoid over its latent space, we propose a framework for constructing learnable, commutative, associative binary operators. And with this, we construct an aggregator of $O(\log V)$ depth, yielding exponential improvements for both parallelism and dependency length while achieving performance competitive with recurrent aggregators. Based on our empirical observations, our proposed learnable commutative monoid (LCM) aggregator represents a favourable tradeoff between efficient and expressive aggregators.

Purpose: The aim of this study was to demonstrate the utility of unsupervised domain adaptation (UDA) in automated knee osteoarthritis (OA) phenotype classification using a small dataset (n=50). Materials and Methods: For this retrospective study, we collected 3,166 three-dimensional (3D) double-echo steady-state magnetic resonance (MR) images from the Osteoarthritis Initiative dataset and 50 3D turbo/fast spin-echo MR images from our institute (in 2020 and 2021) as the source and target datasets, respectively. For each patient, the degree of knee OA was initially graded according to the MRI Osteoarthritis Knee Score (MOAKS) before being converted to binary OA phenotype labels. The proposed UDA pipeline included (a) pre-processing, which involved automatic segmentation and region-of-interest cropping; (b) source classifier training, which involved pre-training phenotype classifiers on the source dataset; (c) target encoder adaptation, which involved unsupervised adaption of the source encoder to the target encoder and (d) target classifier validation, which involved statistical analysis of the target classification performance evaluated by the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity and accuracy. Additionally, a classifier was trained without UDA for comparison. Results: The target classifier trained with UDA achieved improved AUROC, sensitivity, specificity and accuracy for both knee OA phenotypes compared with the classifier trained without UDA. Conclusion: The proposed UDA approach improves the performance of automated knee OA phenotype classification for small target datasets by utilising a large, high-quality source dataset for training. The results successfully demonstrated the advantages of the UDA approach in classification on small datasets.

Generalized Labeled Multi-Bernoulli (GLMB) densities arise in a host of multi-object system applications analogous to Gaussians in single-object filtering. However, computing the GLMB filtering density requires solving NP-hard problems. To alleviate this computational bottleneck, we develop a linear complexity Gibbs sampling framework for GLMB density computation. Specifically, we propose a tempered Gibbs sampler that exploits the structure of the GLMB filtering density to achieve an $\mathcal{O}(T(P+M))$ complexity, where $T$ is the number of iterations of the algorithm, $P$ and $M$ are the number hypothesized objects and measurements. This innovation enables an $\mathcal{O}(T(P+M+\log(T))+PM)$ complexity implementation of the GLMB filter. Convergence of the proposed Gibbs sampler is established and numerical studies are presented to validate the proposed GLMB filter implementation.