预计到2023年，物联网设备的数量将达到1,250亿。物联网设备的增长将加剧设备之间的碰撞，从而降低通信性能。选择适当的传输参数，例如通道和扩展因子（SF），可以有效地减少远程（LORA）设备之间的碰撞。但是，当前文献中提出的大多数方案在具有有限的计算复杂性和内存的物联网设备上都不容易实现。为了解决此问题，我们提出了一种轻巧的传输参数选择方案，即使用用于低功率大区域网络（Lorawan）的增强学习的联合通道和SF选择方案。在拟议的方案中，可以仅使用确认（ACK）信息来选择适当的传输参数。此外，我们从理论上分析了我们提出的方案的计算复杂性和记忆要求，该方案验证了我们所提出的方案可以选择具有极低计算复杂性和内存要求的传输参数。此外，在现实世界中的洛拉设备上实施了大量实验，以评估我们提出的计划的有效性。实验结果证明了以下主要现象。 （1）与其他轻型传输参数选择方案相比，我们在Lorawan中提出的方案可以有效避免Lora设备之间的碰撞，而与可用通道的变化无关。 （2）可以通过选择访问通道和使用SFS而不是仅选择访问渠道来提高帧成功率（FSR）。 （3）由于相邻通道之间存在干扰，因此可以通过增加相邻可用通道的间隔来改善FSR和公平性。

未来几年物联网设备计数的预期增加促使有效算法的开发，可以帮助其有效管理，同时保持功耗低。在本文中，我们提出了一种智能多通道资源分配算法，用于Loradrl的密集Lora网络，并提供详细的性能评估。我们的结果表明，所提出的算法不仅显着提高了Lorawan的分组传递比（PDR），而且还能够支持移动终端设备（EDS），同时确保较低的功耗，因此增加了网络的寿命和容量。}大多数之前作品侧重于提出改进网络容量的不同MAC协议，即Lorawan，传输前的延迟等。我们展示通过使用Loradrl，我们可以通过Aloha \ TextColor {Black}与Lorasim相比，我们可以实现相同的效率LORA-MAB在将复杂性从EDS移动到网关的同时，因此使EDS更简单和更便宜。此外，我们在大规模的频率干扰攻击下测试Loradrl的性能，并显示其对环境变化的适应性。我们表明，与基于学习的技术相比，Loradrl的输出改善了最先进的技术的性能，从而提高了PR的500多种\％。

未来的互联网涉及几种新兴技术，例如5G和5G网络，车辆网络，无人机（UAV）网络和物联网（IOT）。此外，未来的互联网变得异质并分散了许多相关网络实体。每个实体可能需要做出本地决定，以在动态和不确定的网络环境下改善网络性能。最近使用标准学习算法，例如单药强化学习（RL）或深入强化学习（DRL），以使每个网络实体作为代理人通过与未知环境进行互动来自适应地学习最佳决策策略。但是，这种算法未能对网络实体之间的合作或竞争进行建模，而只是将其他实体视为可能导致非平稳性问题的环境的一部分。多机构增强学习（MARL）允许每个网络实体不仅观察环境，还可以观察其他实体的政策来学习其最佳政策。结果，MAL可以显着提高网络实体的学习效率，并且最近已用于解决新兴网络中的各种问题。在本文中，我们因此回顾了MAL在新兴网络中的应用。特别是，我们提供了MARL的教程，以及对MARL在下一代互联网中的应用进行全面调查。特别是，我们首先介绍单代机Agent RL和MARL。然后，我们回顾了MAL在未来互联网中解决新兴问题的许多应用程序。这些问题包括网络访问，传输电源控制，计算卸载，内容缓存，数据包路由，无人机网络的轨迹设计以及网络安全问题。

互联网连接系统的指数增长产生了许多挑战，例如频谱短缺问题，需要有效的频谱共享（SS）解决方案。复杂和动态的SS系统可以接触不同的潜在安全性和隐私问题，需要保护机制是自适应，可靠和可扩展的。基于机器学习（ML）的方法经常提议解决这些问题。在本文中，我们对最近的基于ML的SS方法，最关键的安全问题和相应的防御机制提供了全面的调查。特别是，我们详细说明了用于提高SS通信系统的性能的最先进的方法，包括基于ML基于ML的基于的数据库辅助SS网络，ML基于基于的数据库辅助SS网络，包括基于ML的数据库辅助的SS网络，基于ML的LTE-U网络，基于ML的环境反向散射网络和其他基于ML的SS解决方案。我们还从物理层和基于ML算法的相应防御策略的安全问题，包括主要用户仿真（PUE）攻击，频谱感测数据伪造（SSDF）攻击，干扰攻击，窃听攻击和隐私问题。最后，还给出了对ML基于ML的开放挑战的广泛讨论。这种全面的审查旨在为探索新出现的ML的潜力提供越来越复杂的SS及其安全问题，提供基础和促进未来的研究。

随着数据生成越来越多地在没有连接连接的设备上进行，因此与机器学习（ML）相关的流量将在无线网络中无处不在。许多研究表明，传统的无线协议高效或不可持续以支持ML，这创造了对新的无线通信方法的需求。在这项调查中，我们对最先进的无线方法进行了详尽的审查，这些方法是专门设计用于支持分布式数据集的ML服务的。当前，文献中有两个明确的主题，模拟的无线计算和针对ML优化的数字无线电资源管理。这项调查对这些方法进行了全面的介绍，回顾了最重要的作品，突出了开放问题并讨论了应用程序方案。

本文着重于根据数据包输送比率（PDR）（即，在远程广阔的区域（Lorawan）中通过End Devices（EDS）发送）的数据包数量来改善资源分配算法。设置传输参数会显着影响PDR。我们采用强化学习（RL）提出了一种资源分配算法，该算法使ED可以以分布式方式配置其传输参数。我们将资源分配问题建模为多臂强盗（MAB），然后通过提出一种名为Mix-MAB的两相算法来解决它，该算法由探索和开发（EXP3）和连续消除（SE）组成，该算法由指数重量组成（SE）算法。我们通过仿真结果评估混合MAB性能，并将其与其他现有方法进行比较。数值结果表明，就收敛时间和PDR而言，所提出的解决方案的性能优于现有方案。

Unmanned aerial vehicle (UAV) swarms are considered as a promising technique for next-generation communication networks due to their flexibility, mobility, low cost, and the ability to collaboratively and autonomously provide services. Distributed learning (DL) enables UAV swarms to intelligently provide communication services, multi-directional remote surveillance, and target tracking. In this survey, we first introduce several popular DL algorithms such as federated learning (FL), multi-agent Reinforcement Learning (MARL), distributed inference, and split learning, and present a comprehensive overview of their applications for UAV swarms, such as trajectory design, power control, wireless resource allocation, user assignment, perception, and satellite communications. Then, we present several state-of-the-art applications of UAV swarms in wireless communication systems, such us reconfigurable intelligent surface (RIS), virtual reality (VR), semantic communications, and discuss the problems and challenges that DL-enabled UAV swarms can solve in these applications. Finally, we describe open problems of using DL in UAV swarms and future research directions of DL enabled UAV swarms. In summary, this survey provides a comprehensive survey of various DL applications for UAV swarms in extensive scenarios.

智能物联网环境（iiote）由可以协作执行半自动的IOT应用的异构装置，其示例包括高度自动化的制造单元或自主交互收获机器。能量效率是这种边缘环境中的关键，因为它们通常基于由无线和电池运行设备组成的基础设施，例如电子拖拉机，无人机，自动引导车辆（AGV）S和机器人。总能源消耗从多种技术技术汲取贡献，使得能够实现边缘计算和通信，分布式学习以及分布式分区和智能合同。本文提供了本技术的最先进的概述，并说明了它们的功能和性能，特别关注资源，延迟，隐私和能源消耗之间的权衡。最后，本文提供了一种在节能IIOTE和路线图中集成这些能力技术的愿景，以解决开放的研究挑战

Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D physical world images into 3D virtual world scenes is computationally intensive and requires computation offloading. The disparity in transmitted scene dimension (2D as opposed to 3D) leads to asymmetric data sizes in uplink (UL) and downlink (DL). To ensure the reliability and low latency of the system, we consider an asynchronous joint UL-DL scenario where in the UL stage, the smaller data size of the physical world scenes captured by multiple extended reality users (XUs) will be uploaded to the Metaverse Console (MC) to be construed and rendered. In the DL stage, the larger-size 3D virtual world scenes need to be transmitted back to the XUs. The decisions pertaining to computation offloading and channel assignment are optimized in the UL stage, and the MC will optimize power allocation for users assigned with a channel in the UL transmission stage. Some problems arise therefrom: (i) interactive multi-process chain, specifically Asynchronous Markov Decision Process (AMDP), (ii) joint optimization in multiple processes, and (iii) high-dimensional objective functions, or hybrid reward scenarios. To ensure the reliability and low latency of the system, we design a novel multi-agent reinforcement learning algorithm structure, namely Asynchronous Actors Hybrid Critic (AAHC). Extensive experiments demonstrate that compared to proposed baselines, AAHC obtains better solutions with preferable training time.

为了满足下一代无线通信网络的极其异构要求，研究界越来越依赖于使用机器学习解决方案进行实时决策和无线电资源管理。传统的机器学习采用完全集中的架构，其中整个培训数据在一个节点上收集，即云服务器，显着提高了通信开销，并提高了严重的隐私问题。迄今为止，最近提出了作为联合学习（FL）称为联合学习的分布式机器学习范式。在FL中，每个参与边缘设备通过使用自己的培训数据列举其本地模型。然后，通过无线信道，本地训练模型的权重或参数被发送到中央ps，聚合它们并更新全局模型。一方面，FL对优化无线通信网络的资源起着重要作用，另一方面，无线通信对于FL至关重要。因此，FL和无线通信之间存在“双向”关系。虽然FL是一个新兴的概念，但许多出版物已经在FL的领域发表了发布及其对下一代无线网络的应用。尽管如此，我们注意到没有任何作品突出了FL和无线通信之间的双向关系。因此，本调查纸的目的是通过提供关于FL和无线通信之间的相互依存性的及时和全面的讨论来弥合文学中的这种差距。

雇用无人驾驶航空公司（无人机）吸引了日益增长的兴趣，并成为互联网（物联网）网络中的数据收集技术的最先进技术。在本文中，目的是最大限度地减少UAV-IOT系统的总能耗，我们制定了联合设计了UAV的轨迹和选择IOT网络中的群集头作为受约束的组合优化问题的问题，该问题被归类为NP-努力解决。我们提出了一种新的深度加强学习（DRL），其具有顺序模型策略，可以通过无监督方式有效地学习由UAV的轨迹设计来实现由序列到序列神经网络表示的策略。通过广泛的模拟，所获得的结果表明，与其他基线算法相比，所提出的DRL方法可以找到无人机的轨迹，这些轨迹需要更少的能量消耗，并实现近乎最佳性能。此外，仿真结果表明，我们所提出的DRL算法的训练模型具有出色的概括能力，对更大的问题尺寸而没有必要恢复模型。

Recent technological advancements in space, air and ground components have made possible a new network paradigm called "space-air-ground integrated network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs. However, due to UAVs' high dynamics and complexity, the real-world deployment of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the space and terrestrial components, UAVs are expected to meet performance requirements with high flexibility and dynamics using limited resources. Therefore, employing UAVs in various usage scenarios requires well-designed planning in algorithmic approaches. In this paper, we provide a comprehensive review of recent learning-based algorithmic approaches. We consider possible reward functions and discuss the state-of-the-art algorithms for optimizing the reward functions, including Q-learning, deep Q-learning, multi-armed bandit (MAB), particle swarm optimization (PSO) and satisfaction-based learning algorithms. Unlike other survey papers, we focus on the methodological perspective of the optimization problem, which can be applicable to various UAV-assisted missions on a SAGIN using these algorithms. We simulate users and environments according to real-world scenarios and compare the learning-based and PSO-based methods in terms of throughput, load, fairness, computation time, etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional (3D) variations of these algorithms to reflect different deployment cases. Our simulation suggests that the $3$D satisfaction-based learning algorithm outperforms the other approaches for various metrics in most cases. We discuss some open challenges at the end and our findings aim to provide design guidelines for algorithm selections while optimizing the deployment of UAV-assisted SAGINs.

联合学习（FL）使移动设备能够在保留本地数据的同时协作学习共享的预测模型。但是，实际上在移动设备上部署FL存在两个主要的研究挑战：（i）频繁的无线梯度更新v.s.频谱资源有限，以及（ii）培训期间渴望的FL通信和本地计算V.S.电池约束的移动设备。为了应对这些挑战，在本文中，我们提出了一种新型的多位空天空计算（MAIRCOMP）方法，用于FL中本地模型更新的频谱有效聚合，并进一步介绍用于移动的能源有效的FL设计设备。具体而言，高精度数字调制方案是在MAIRCOMP中设计和合并的，允许移动设备同时在多访问通道中同时在所选位置上传模型更新。此外，我们理论上分析了FL算法的收敛性。在FL收敛分析的指导下，我们制定了联合传输概率和局部计算控制优化，旨在最大程度地减少FL移动设备的总体能源消耗（即迭代局部计算 +多轮通信）。广泛的仿真结果表明，我们提出的方案在频谱利用率，能源效率和学习准确性方面优于现有计划。

In recent years, the exponential proliferation of smart devices with their intelligent applications poses severe challenges on conventional cellular networks. Such challenges can be potentially overcome by integrating communication, computing, caching, and control (i4C) technologies. In this survey, we first give a snapshot of different aspects of the i4C, comprising background, motivation, leading technological enablers, potential applications, and use cases. Next, we describe different models of communication, computing, caching, and control (4C) to lay the foundation of the integration approach. We review current state-of-the-art research efforts related to the i4C, focusing on recent trends of both conventional and artificial intelligence (AI)-based integration approaches. We also highlight the need for intelligence in resources integration. Then, we discuss integration of sensing and communication (ISAC) and classify the integration approaches into various classes. Finally, we propose open challenges and present future research directions for beyond 5G networks, such as 6G.

本文调查了大师无人机（MUAV） - 互联网（IOT）网络，我们建议使用配备有智能反射表面（IRS）的可充电辅助UAV（AUAV）来增强来自MUAV的通信信号并将MUAG作为充电电源利用。在拟议的模型下，我们研究了这些能量有限的无人机的最佳协作策略，以最大限度地提高物联网网络的累计吞吐量。根据两个无人机之间是否有收费，配制了两个优化问题。为了解决这些问题，提出了两个多代理深度强化学习（DRL）方法，这些方法是集中培训多师深度确定性政策梯度（CT-MADDPG）和多代理深度确定性政策选项评论仪（MADDPOC）。结果表明，CT-MADDPG可以大大减少对UAV硬件的计算能力的要求，拟议的MADDPOC能够在连续动作域中支持低水平的多代理合作学习，其优于优势基于选项的分层DRL，只支持单代理学习和离散操作。

With the increasing growth of information through smart devices, increasing the quality level of human life requires various computational paradigms presentation including the Internet of Things, fog, and cloud. Between these three paradigms, the cloud computing paradigm as an emerging technology adds cloud layer services to the edge of the network so that resource allocation operations occur close to the end-user to reduce resource processing time and network traffic overhead. Hence, the resource allocation problem for its providers in terms of presenting a suitable platform, by using computational paradigms is considered a challenge. In general, resource allocation approaches are divided into two methods, including auction-based methods(goal, increase profits for service providers-increase user satisfaction and usability) and optimization-based methods(energy, cost, network exploitation, Runtime, reduction of time delay). In this paper, according to the latest scientific achievements, a comprehensive literature study (CLS) on artificial intelligence methods based on resource allocation optimization without considering auction-based methods in various computing environments are provided such as cloud computing, Vehicular Fog Computing, wireless, IoT, vehicular networks, 5G networks, vehicular cloud architecture,machine-to-machine communication(M2M),Train-to-Train(T2T) communication network, Peer-to-Peer(P2P) network. Since deep learning methods based on artificial intelligence are used as the most important methods in resource allocation problems; Therefore, in this paper, resource allocation approaches based on deep learning are also used in the mentioned computational environments such as deep reinforcement learning, Q-learning technique, reinforcement learning, online learning, and also Classical learning methods such as Bayesian learning, Cummins clustering, Markov decision process.

In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications, e.g., for medical purposes and in vehicular networks. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL.

Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.

车辆到车辆（V2V）通信的性能在很大程度上取决于使用的调度方法。虽然集中式网络调度程序提供高V2V通信可靠性，但它们的操作通常仅限于具有完整的蜂窝网络覆盖范围的区域。相比之下，在细胞外覆盖区域中，使用了相对效率低下的分布式无线电资源管理。为了利用集中式方法的好处来增强V2V通信在缺乏蜂窝覆盖的道路上的可靠性，我们建议使用VRLS（车辆加固学习调度程序），这是一种集中的调度程序，该调度程序主动为覆盖外的V2V Communications主动分配资源，以前}车辆离开蜂窝网络覆盖范围。通过在模拟的车辆环境中进行培训，VRL可以学习一项适应环境变化的调度策略，从而消除了在复杂的现实生活环境中对有针对性（重新）培训的需求。我们评估了在不同的移动性，网络负载，无线通道和资源配置下VRL的性能。 VRL的表现优于最新的区域中最新分布式调度算法，而无需蜂窝网络覆盖，通过在高负载条件下将数据包错误率降低了一半，并在低负载方案中实现了接近最大的可靠性。

联邦学习（FL）变得流行，并在训练大型机器学习（ML）模型的情况下表现出很大的潜力，而不会使所有者的原始数据曝光。在FL中，数据所有者可以根据其本地数据培训ML模型，并且仅将模型更新发送到模型更新，而不是原始数据到模型所有者进行聚合。为了提高模型准确性和培训完成时间的学习绩效，招募足够的参与者至关重要。同时，数据所有者是理性的，可能不愿意由于资源消耗而参与协作学习过程。为了解决这些问题，最近有各种作品旨在激励数据业主贡献其资源。在本文中，我们为文献中提出的经济和游戏理论方法提供了全面的审查，以设计刺激数据业主参加流程培训过程的各种计划。特别是，我们首先在激励机制设计中常用的佛罗里达州的基础和背景，经济理论。然后，我们审查博弈理论和经济方法应用于FL的激励机制的应用。最后，我们突出了一些开放的问题和未来关于FL激励机制设计的研究方向。