Symmetry arises in many optimization and decision-making problems, and has attracted considerable attention from the optimization community: By utilizing the existence of such symmetries, the process of searching for optimal solutions can be improved significantly. Despite its success in (offline) optimization, the utilization of symmetries has not been well examined within the online optimization settings, especially in the bandit literature. As such, in this paper we study the invariant Lipschitz bandit setting, a subclass of the Lipschitz bandits where the reward function and the set of arms are preserved under a group of transformations. We introduce an algorithm named \texttt{UniformMesh-N}, which naturally integrates side observations using group orbits into the \texttt{UniformMesh} algorithm (\cite{Kleinberg2005_UniformMesh}), which uniformly discretizes the set of arms. Using the side-observation approach, we prove an improved regret upper bound, which depends on the cardinality of the group, given that the group is finite. We also prove a matching regret's lower bound for the invariant Lipschitz bandit class (up to logarithmic factors). We hope that our work will ignite further investigation of symmetry in bandit theory and sequential decision-making theory in general.

在现实世界应用中，联合学习（FL）遇到了两个挑战：（1）可伸缩性，尤其是应用于大型物联网网络时； （2）如何使用异质数据对环境进行健全。意识到第一个问题，我们旨在设计一个名为Full-Stack FL（F2L）的新型FL框架。更具体地说，F2L使用层次结构架构，使扩展FL网络可以访问而无需重建整个网络系统。此外，利用层次网络设计的优势，我们在全球服务器上提出了一种新的标签驱动知识蒸馏（LKD）技术来解决第二个问题。与当前的知识蒸馏技术相反，LKD能够训练学生模型，该模型由所有教师模型的良好知识组成。因此，我们提出的算法可以有效地提取区域数据分布（即区域汇总模型）的知识，以减少客户在使用非独立分布数据的FL系统下操作时客户模型之间的差异。广泛的实验结果表明：（i）我们的F2L方法可以显着提高所有全球蒸馏的总体FL效率，并且（ii）F2L随着全球蒸馏阶段的发生而迅速达到收敛性，而不是在每个通信周期中提高。

为了了解强化学习的安全威胁（RL）算法，本文研究中毒攻击以操纵\ emph {any}订单 - 最佳学习算法对偶发性RL中有针对性的政策，并研究了两种自然中毒攻击的潜在损害，即，\ emph {Reward}和\ Emph {Action}的操作。我们发现攻击的影响至关重要地取决于奖励是有界还是无限的。在有限的奖励设置中，我们表明只有奖励操纵或只有动作操纵不能保证成功的攻击。但是，通过结合奖励和行动操纵，对手可以操纵任何订单最佳学习算法，以遵循任何有针对性的策略，并使用$ \ tilde {\ theta}（\ sqrt {t}）$总攻击成本，这是订单 - 优越，最佳的攻击成本不知道基础MDP。相反，在无限的奖励设置中，我们表明奖励操纵攻击足以使对手成功操纵任何订单最佳学习算法，以使用$ \ tilde {o}（\ sqrt {t}）遵循任何有针对性的策略污染。我们的结果揭示了有关中毒攻击无法获得或无法实现的有用见解，并将刺激有关强大RL算法设计的更多作品。

图形神经网络已被证明可以为各种软件工程任务产生令人印象深刻的结果。但是，现有技术仍然有两个问题：（1）长期依赖性和（2）不同的代码组件在不应该的情况下被视为平等。为了解决这些问题，我们提出了一种表示代码为层次结构（代码层次结构）的方法，其中不同的代码组件在各个粒度级别分别表示。然后，为了处理每个表示级别的表示，我们设计了一个新颖的网络体系结构Echelon，它结合了异质图形变压器网络和基于树的卷积神经网络的优势，以学习具有代码依赖性信息丰富的抽象语法树。我们还提出了一个新颖的预处理目标，称为缺失子树预测以补充我们的代码层次结构。评估结果表明，我们的方法在三个任务中大大优于其他基准：任何代码完成，代码分类和代码克隆检测。

在本文中，我们介绍了时间多解决图形神经网络（TMGNN），这是两个学会构建多尺度和多分辨率图结构的第一个体系结构，并结合了时间序列信号以捕获动态图的时间变化。我们已经将我们提出的模型应用于预测流行病和大流行病的任务，该模型是根据几个欧洲国家从实际的covid-19-19-19大流行病和水痘流行中收集的历史时间序列数据，并获得了与其他竞争性的结果相比，与其他竞争性的结果相比先前的最新时间架构和图形学习算法。我们已经表明，捕获图的多尺度和多分辨率结构对于提取本地或全球信息很重要，这些信息在理解全球流行病（例如covid-9）的动态中起着至关重要世界。我们的工作为预测和减轻未来的流行病和流行病带来了有希望的研究方向。

Process monitoring and control are essential in modern industries for ensuring high quality standards and optimizing production performance. These technologies have a long history of application in production and have had numerous positive impacts, but also hold great potential when integrated with Industry 4.0 and advanced machine learning, particularly deep learning, solutions. However, in order to implement these solutions in production and enable widespread adoption, the scalability and transferability of deep learning methods have become a focus of research. While transfer learning has proven successful in many cases, particularly with computer vision and homogenous data inputs, it can be challenging to apply to heterogeneous data. Motivated by the need to transfer and standardize established processes to different, non-identical environments and by the challenge of adapting to heterogeneous data representations, this work introduces the Domain Adaptation Neural Network with Cyclic Supervision (DBACS) approach. DBACS addresses the issue of model generalization through domain adaptation, specifically for heterogeneous data, and enables the transfer and scalability of deep learning-based statistical control methods in a general manner. Additionally, the cyclic interactions between the different parts of the model enable DBACS to not only adapt to the domains, but also match them. To the best of our knowledge, DBACS is the first deep learning approach to combine adaptation and matching for heterogeneous data settings. For comparison, this work also includes subspace alignment and a multi-view learning that deals with heterogeneous representations by mapping data into correlated latent feature spaces. Finally, DBACS with its ability to adapt and match, is applied to a virtual metrology use case for an etching process run on different machine types in semiconductor manufacturing.

Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q-learning algorithm, which integrates principal component analysis (PCA) with deep Q-learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.

Searching long egocentric videos with natural language queries (NLQ) has compelling applications in augmented reality and robotics, where a fluid index into everything that a person (agent) has seen before could augment human memory and surface relevant information on demand. However, the structured nature of the learning problem (free-form text query inputs, localized video temporal window outputs) and its needle-in-a-haystack nature makes it both technically challenging and expensive to supervise. We introduce Narrations-as-Queries (NaQ), a data augmentation strategy that transforms standard video-text narrations into training data for a video query localization model. Validating our idea on the Ego4D benchmark, we find it has tremendous impact in practice. NaQ improves multiple top models by substantial margins (even doubling their accuracy), and yields the very best results to date on the Ego4D NLQ challenge, soundly outperforming all challenge winners in the CVPR and ECCV 2022 competitions and topping the current public leaderboard. Beyond achieving the state-of-the-art for NLQ, we also demonstrate unique properties of our approach such as gains on long-tail object queries, and the ability to perform zero-shot and few-shot NLQ.

Energy storage resources must consider both price uncertainties and their physical operating characteristics when participating in wholesale electricity markets. This is a challenging problem as electricity prices are highly volatile, and energy storage has efficiency losses, power, and energy constraints. This paper presents a novel, versatile, and transferable approach combining model-based optimization with a convolutional long short-term memory network for energy storage to respond to or bid into wholesale electricity markets. We apply transfer learning to the ConvLSTM network to quickly adapt the trained bidding model to new market environments. We test our proposed approach using historical prices from New York State, showing it achieves state-of-the-art results, achieving between 70% to near 90% profit ratio compared to perfect foresight cases, in both price response and wholesale market bidding setting with various energy storage durations. We also test a transfer learning approach by pre-training the bidding model using New York data and applying it to arbitrage in Queensland, Australia. The result shows transfer learning achieves exceptional arbitrage profitability with as little as three days of local training data, demonstrating its significant advantage over training from scratch in scenarios with very limited data availability.

Surgical robot automation has attracted increasing research interest over the past decade, expecting its huge potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied AI has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant researchers. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how human demonstrations would affect policy learning. In this paper, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. With these, we further propose to collect human demonstrations and imitate the action patterns to achieve more effective policy learning. We showcase the improvement of our simulation environment with the designed new features and tasks, and validate state-of-the-art reinforcement learning algorithms using the interactive environment. Promising results are obtained, with which we hope to pave the way for future research on surgical embodied intelligence. Our platform is released and will be continuously updated in the website: https://med-air.github.io/SurRoL/