尽管在许多控制任务中进行了大量的应用和深入的强化学习的成功，但它仍然存在许多关键问题和局限性，包括具有稀疏奖励的时间信用分配，缺乏有效的探索以及对对超级参数的脆弱融合，这对超级参与者非常敏感问题。持续控制中深厚的强化学习的问题以及进化算法在面对其中一些问题方面的成功，已经出现了进化增强学习的想法，这引起了许多争议。尽管在该领域的一些研究中取得了成功的结果，但针对这些问题及其局限性的适当解决方案尚待提出。本研究旨在研究进一步加强强化学习和进化计算的两个领域的效率，并朝着改善方法和现有挑战迈出一步。 “使用精英缓冲液的进化深度强化学习”算法通过互动学习能力和人脑中的假设结果的灵感引入了一种新的机制。在这种方法中，精英缓冲液的利用（这是受到人类思想的经验概括的启发），以及跨界和突变操作员的存在，以及连续一代的交互式学习，具有提高的效率，收敛性和收敛性，收敛性和在连续控制领域的正确进步。根据实验的结果，所提出的方法超过了具有高复杂性和维度的环境中的其他知名方法，并且在解决上述问题和局限性方面表现出色。

背景和客观的高医学多样性一直是处方的重大挑战，在医师的决策过程中引起混乱或怀疑。本文旨在通过提供有关其他医生开处方的药物的信息，并弄清楚除所讨论的一种药物外还推荐了哪些其他药物，以开发一种称为推荐的药物推荐系统，以帮助医生进行高血压处方。方法有两个步骤开发的方法：首先，采用了协会规则挖掘算法来查找医学协会规则。第二步需要图形挖掘和聚类，以通过ATC代码提供丰富的建议，该建议本身包括多个步骤。首先，初始图是根据历史处方数据构建的。然后，在第二步中进行数据修剪，此后，由普通医生酌情裁定具有高重复率的药物。接下来，将药物与称为ATC代码的著名医学分类系统相匹配，以提供丰富的建议。最后，DBSCAN和Louvain算法在最后一步中群集药物。结果作为系统的输出，提供了推荐药物的清单，医生可以根据患者的临床症状选择一种或多种药物。仅使用与高血压药物有关的2级药物用于评估系统的性能。从该系统获得的结果已由该领域的专家进行了审查和确认。

本文介绍了一种“混合自我注意整洁”方法，以改善高维输入中增强拓扑（整洁）算法的原始神经发展。虽然整洁的算法显示出在不同具有挑战性的任务中的显着结果，但由于输入表示是高维度，但它无法创建一个良好的调谐网络。我们的研究通过使用自我关注作为间接编码方法来解决此限制，以选择输入的最重要部分。此外，我们在混合方法的帮助下提高了整体性能，以发展最终网络权重。主要结论是混合自我关注整洁可以消除原始整洁的限制。结果表明，与进化算法相比，我们的模型可以在ATARI游戏中获得与原始像素输入的可比分数，其中参数数量较少。

In large-scale machine learning, recent works have studied the effects of compressing gradients in stochastic optimization in order to alleviate the communication bottleneck. These works have collectively revealed that stochastic gradient descent (SGD) is robust to structured perturbations such as quantization, sparsification, and delays. Perhaps surprisingly, despite the surge of interest in large-scale, multi-agent reinforcement learning, almost nothing is known about the analogous question: Are common reinforcement learning (RL) algorithms also robust to similar perturbations? In this paper, we investigate this question by studying a variant of the classical temporal difference (TD) learning algorithm with a perturbed update direction, where a general compression operator is used to model the perturbation. Our main technical contribution is to show that compressed TD algorithms, coupled with an error-feedback mechanism used widely in optimization, exhibit the same non-asymptotic theoretical guarantees as their SGD counterparts. We then extend our results significantly to nonlinear stochastic approximation algorithms and multi-agent settings. In particular, we prove that for multi-agent TD learning, one can achieve linear convergence speedups in the number of agents while communicating just $\tilde{O}(1)$ bits per agent at each time step. Our work is the first to provide finite-time results in RL that account for general compression operators and error-feedback in tandem with linear function approximation and Markovian sampling. Our analysis hinges on studying the drift of a novel Lyapunov function that captures the dynamics of a memory variable introduced by error feedback.

With Twitter's growth and popularity, a huge number of views are shared by users on various topics, making this platform a valuable information source on various political, social, and economic issues. This paper investigates English tweets on the Russia-Ukraine war to analyze trends reflecting users' opinions and sentiments regarding the conflict. The tweets' positive and negative sentiments are analyzed using a BERT-based model, and the time series associated with the frequency of positive and negative tweets for various countries is calculated. Then, we propose a method based on the neighborhood average for modeling and clustering the time series of countries. The clustering results provide valuable insight into public opinion regarding this conflict. Among other things, we can mention the similar thoughts of users from the United States, Canada, the United Kingdom, and most Western European countries versus the shared views of Eastern European, Scandinavian, Asian, and South American nations toward the conflict.

Solving portfolio management problems using deep reinforcement learning has been getting much attention in finance for a few years. We have proposed a new method using experts signals and historical price data to feed into our reinforcement learning framework. Although experts signals have been used in previous works in the field of finance, as far as we know, it is the first time this method, in tandem with deep RL, is used to solve the financial portfolio management problem. Our proposed framework consists of a convolutional network for aggregating signals, another convolutional network for historical price data, and a vanilla network. We used the Proximal Policy Optimization algorithm as the agent to process the reward and take action in the environment. The results suggested that, on average, our framework could gain 90 percent of the profit earned by the best expert.

Autoencoders are a popular model in many branches of machine learning and lossy data compression. However, their fundamental limits, the performance of gradient methods and the features learnt during optimization remain poorly understood, even in the two-layer setting. In fact, earlier work has considered either linear autoencoders or specific training regimes (leading to vanishing or diverging compression rates). Our paper addresses this gap by focusing on non-linear two-layer autoencoders trained in the challenging proportional regime in which the input dimension scales linearly with the size of the representation. Our results characterize the minimizers of the population risk, and show that such minimizers are achieved by gradient methods; their structure is also unveiled, thus leading to a concise description of the features obtained via training. For the special case of a sign activation function, our analysis establishes the fundamental limits for the lossy compression of Gaussian sources via (shallow) autoencoders. Finally, while the results are proved for Gaussian data, numerical simulations on standard datasets display the universality of the theoretical predictions.

Recently, Smart Video Surveillance (SVS) systems have been receiving more attention among scholars and developers as a substitute for the current passive surveillance systems. These systems are used to make the policing and monitoring systems more efficient and improve public safety. However, the nature of these systems in monitoring the public's daily activities brings different ethical challenges. There are different approaches for addressing privacy issues in implementing the SVS. In this paper, we are focusing on the role of design considering ethical and privacy challenges in SVS. Reviewing four policy protection regulations that generate an overview of best practices for privacy protection, we argue that ethical and privacy concerns could be addressed through four lenses: algorithm, system, model, and data. As an case study, we describe our proposed system and illustrate how our system can create a baseline for designing a privacy perseverance system to deliver safety to society. We used several Artificial Intelligence algorithms, such as object detection, single and multi camera re-identification, action recognition, and anomaly detection, to provide a basic functional system. We also use cloud-native services to implement a smartphone application in order to deliver the outputs to the end users.

Domain adaptation aims to transfer the knowledge acquired by models trained on (data-rich) source domains to (low-resource) target domains, for which a popular method is invariant representation learning. While they have been studied extensively for classification and regression problems, how they apply to ranking problems, where the data and metrics have a list structure, is not well understood. Theoretically, we establish a domain adaptation generalization bound for ranking under listwise metrics such as MRR and NDCG. The bound suggests an adaptation method via learning list-level domain-invariant feature representations, whose benefits are empirically demonstrated by unsupervised domain adaptation experiments on real-world ranking tasks, including passage reranking. A key message is that for domain adaptation, the representations should be analyzed at the same level at which the metric is computed, as we show that learning invariant representations at the list level is most effective for adaptation on ranking problems.

In recent years, we have seen a significant interest in data-driven deep learning approaches for video anomaly detection, where an algorithm must determine if specific frames of a video contain abnormal behaviors. However, video anomaly detection is particularly context-specific, and the availability of representative datasets heavily limits real-world accuracy. Additionally, the metrics currently reported by most state-of-the-art methods often do not reflect how well the model will perform in real-world scenarios. In this article, we present the Charlotte Anomaly Dataset (CHAD). CHAD is a high-resolution, multi-camera anomaly dataset in a commercial parking lot setting. In addition to frame-level anomaly labels, CHAD is the first anomaly dataset to include bounding box, identity, and pose annotations for each actor. This is especially beneficial for skeleton-based anomaly detection, which is useful for its lower computational demand in real-world settings. CHAD is also the first anomaly dataset to contain multiple views of the same scene. With four camera views and over 1.15 million frames, CHAD is the largest fully annotated anomaly detection dataset including person annotations, collected from continuous video streams from stationary cameras for smart video surveillance applications. To demonstrate the efficacy of CHAD for training and evaluation, we benchmark two state-of-the-art skeleton-based anomaly detection algorithms on CHAD and provide comprehensive analysis, including both quantitative results and qualitative examination.