连接和自动驾驶汽车（CAVS）正在越来越广泛地部署，但是目前尚不清楚如何最好地部署智能基础架构以最大程度地发挥其功能。一个关键的挑战是确保骑士能够可靠地感知其他代理，尤其是被阻塞的药物。另一个挑战是，智能基础架构的渴望是自主的，并且很容易扩展到与现代交通信号灯相似的广阔部署。目前的工作提出了自我监督的交通顾问（SSTA），这是一种基础架构边缘设备概念，该概念利用与通信和共同培训框架共同利用自我监督的视频预测，以启用整个智能城市的自动预测流量。 SSTA是一款静态安装的摄像头，可俯瞰复杂的交通流量的交点或区域，可预测流量流作为未来的视频帧，并学会与相邻的SSTA进行通信，以在视野（FOV）中出现在视野中之前预测流量。拟议的框架旨在达到三个目标：（1）设备间的通信以实现高质量的预测，（2）对任意数量的设备的可伸缩性，以及（3）终身在线学习以确保对不断变化的环境的适应性。最后，SSTA可以直接广播其未来预测的视频框架，以供骑士进行自己的后期处理以进行控制。

We are introducing a multi-scale predictive model for video prediction here, whose design is inspired by the "Predictive Coding" theories and "Coarse to Fine" approach. As a predictive coding model, it is updated by a combination of bottom-up and top-down information flows, which is different from traditional bottom-up training style. Its advantage is to reduce the dependence on input information and improve its ability to predict and generate images. Importantly, we achieve with a multi-scale approach -- higher level neurons generate coarser predictions (lower resolution), while the lower level generate finer predictions (higher resolution). This is different from the traditional predictive coding framework in which higher level predict the activity of neurons in lower level. To improve the predictive ability, we integrate an encoder-decoder network in the LSTM architecture and share the final encoded high-level semantic information between different levels. Additionally, since the output of each network level is an RGB image, a smaller LSTM hidden state can be used to retain and update the only necessary hidden information, avoiding being mapped to an overly discrete and complex space. In this way, we can reduce the difficulty of prediction and the computational overhead. Finally, we further explore the training strategies, to address the instability in adversarial training and mismatch between training and testing in long-term prediction. Code is available at https://github.com/Ling-CF/MSPN.

估计路径的旅行时间是智能运输系统的重要主题。它是现实世界应用的基础，例如交通监控，路线计划和出租车派遣。但是，为这样的数据驱动任务构建模型需要大量用户的旅行信息，这与其隐私直接相关，因此不太可能共享。数据所有者之间的非独立和相同分布的（非IID）轨迹数据也使一个预测模型变得极具挑战性，如果我们直接应用联合学习。最后，以前关于旅行时间估算的工作并未考虑道路的实时交通状态，我们认为这可以极大地影响预测。为了应对上述挑战，我们为移动用户组引入GOF-TTE，生成的在线联合学习框架以进行旅行时间估计，这是我）使用联合学习方法，允许在培训时将私人数据保存在客户端设备上，并设计设计和设计。所有客户共享的全球模型作为在线生成模型推断实时道路交通状态。 ii）除了在服务器上共享基本模型外，还针对每个客户调整了一个微调的个性化模型来研究其个人驾驶习惯，从而弥补了本地化全球模型预测的残余错误。 ％iii）将全球模型设计为所有客户共享的在线生成模型，以推断实时道路交通状态。我们还对我们的框架采用了简单的隐私攻击，并实施了差异隐私机制，以进一步保证隐私安全。最后，我们对Didi Chengdu和Xi'an的两个现实世界公共出租车数据集进行了实验。实验结果证明了我们提出的框架的有效性。

Computer vision applications in intelligent transportation systems (ITS) and autonomous driving (AD) have gravitated towards deep neural network architectures in recent years. While performance seems to be improving on benchmark datasets, many real-world challenges are yet to be adequately considered in research. This paper conducted an extensive literature review on the applications of computer vision in ITS and AD, and discusses challenges related to data, models, and complex urban environments. The data challenges are associated with the collection and labeling of training data and its relevance to real world conditions, bias inherent in datasets, the high volume of data needed to be processed, and privacy concerns. Deep learning (DL) models are commonly too complex for real-time processing on embedded hardware, lack explainability and generalizability, and are hard to test in real-world settings. Complex urban traffic environments have irregular lighting and occlusions, and surveillance cameras can be mounted at a variety of angles, gather dirt, shake in the wind, while the traffic conditions are highly heterogeneous, with violation of rules and complex interactions in crowded scenarios. Some representative applications that suffer from these problems are traffic flow estimation, congestion detection, autonomous driving perception, vehicle interaction, and edge computing for practical deployment. The possible ways of dealing with the challenges are also explored while prioritizing practical deployment.

检测，预测和减轻交通拥堵是针对改善运输网络的服务水平的目标。随着对更高分辨率的更大数据集的访问，深度学习对这种任务的相关性正在增加。近年来几篇综合调查论文总结了运输领域的深度学习应用。然而，运输网络的系统动态在非拥挤状态和拥塞状态之间变化大大变化 - 从而需要清楚地了解对拥堵预测特异性特异性的挑战。在这项调查中，我们在与检测，预测和缓解拥堵相关的任务中，介绍了深度学习应用的当前状态。重复和非经常性充血是单独讨论的。我们的调查导致我们揭示了当前研究状态的固有挑战和差距。最后，我们向未来的研究方向提出了一些建议，因为所确定的挑战的答案。

可以通过定期预测未来的框架以增强虚拟现实应用程序中的用户体验，从而解决了低计算设备上图形渲染高帧速率视频的挑战。这是通过时间视图合成（TVS）的问题来研究的，该问题的目标是预测给定上一个帧的视频的下一个帧以及上一个和下一个帧的头部姿势。在这项工作中，我们考虑了用户和对象正在移动的动态场景的电视。我们设计了一个将运动解散到用户和对象运动中的框架，以在预测下一帧的同时有效地使用可用的用户运动。我们通过隔离和估计过去框架的3D对象运动，然后推断它来预测对象的运动。我们使用多平面图像（MPI）作为场景的3D表示，并将对象运动作为MPI表示中相应点之间的3D位移建模。为了在估计运动时处理MPI中的稀疏性，我们将部分卷积和掩盖的相关层纳入了相应的点。然后将预测的对象运动与给定的用户或相机运动集成在一起，以生成下一帧。使用不合格的填充模块，我们合成由于相机和对象运动而发现的区域。我们为动态场景的电视开发了一个新的合成数据集，该数据集由800个以全高清分辨率组成的视频组成。我们通过数据集和MPI Sintel数据集上的实验表明我们的模型优于文献中的所有竞争方法。

时空预测学习旨在通过从历史框架中学习来产生未来的帧。在本文中，我们研究了现有方法，并提出了时空预测学习的一般框架，其中空间编码器和解码器捕获框架内特征和中间时间模块捕获框架间相关性。尽管主流方法采用经常性单元来捕获长期的时间依赖性，但由于无法可行的架构，它们的计算效率低。为了使时间模块并行，我们提出了时间注意单元（TAU），该单元将时间关注分解为框内静态注意力和框架间动力学注意力。此外，虽然平方误差损失侧重于框架内错误，但我们引入了一种新颖的差异差异正则化，以考虑框架间的变化。广泛的实验表明，所提出的方法使派生模型能够在各种时空预测基准上实现竞争性能。

新颖的视图合成（NVS）和视频预测（VP）通常被视为计算机视觉中的不相交任务。但是，它们都可以看作是观察空间时代世界的方法：NVS的目的是从新的角度综合一个场景，而副总裁则旨在从新的时间点观看场景。这两个任务提供了互补的信号以获得场景表示形式，因为观点从空间观察中变化为深度的变化，并且时间观察为相机和单个对象的运动提供了信息。受这些观察的启发，我们建议研究时空（背心）中视频外推的问题。我们提出了一个模型，该模型利用了两项任务的自学和互补线索，而现有方法只能解决其中之一。实验表明，我们的方法比室内和室外现实世界数据集上的几种最先进的NVS和VP方法更好地实现了性能。

本文旨在统一非欧几里得空间中的空间依赖性和时间依赖性，同时捕获流量数据的内部空间依赖性。对于具有拓扑结构的时空属性实体，时空是连续的和统一的，而每个节点的当前状态都受到每个邻居的变异时期的邻居的过去状态的影响。大多数用于流量预测研究的空间依赖性和时间相关性的空间神经网络在处理中分别损害了时空完整性，而忽略了邻居节点的时间依赖期可以延迟和动态的事实。为了建模这种实际条件，我们提出了一种新型的空间 - 周期性图神经网络，将空间和时间视为不可分割的整体，以挖掘时空图，同时通过消息传播机制利用每个节点的发展时空依赖性。进行消融和参数研究的实验已经验证了拟议的遍及术的有效性，并且可以从https://github.com/nnzhan/traversenet中找到详细的实现。

对于由硬件和软件组件组成的复杂分布式系统而言，异常检测是一个重要的问题。对此类系统的异常检测的要求和挑战的透彻理解对于系统的安全性至关重要，尤其是对于现实世界的部署。尽管有许多解决问题的研究领域和应用领域，但很少有人试图对这种系统进行深入研究。大多数异常检测技术是针对某些应用域的专门开发的，而其他检测技术则更为通用。在这项调查中，我们探讨了基于图的算法在复杂分布式异质系统中识别和减轻不同类型异常的重要潜力。我们的主要重点是在分布在复杂分布式系统上的异质计算设备上应用时，可深入了解图。这项研究分析，比较和对比该领域的最新研究文章。首先，我们描述了现实世界分布式系统的特征及其在复杂网络中的异常检测的特定挑战，例如数据和评估，异常的性质以及现实世界的要求。稍后，我们讨论了为什么可以在此类系统中利用图形以及使用图的好处。然后，我们将恰当地深入研究最先进的方法，并突出它们的优势和劣势。最后，我们评估和比较这些方法，并指出可能改进的领域。

基于5G的车辆互联网（IOV）网络中机器学习（ML）的集成使智能运输和智能流量管理。尽管如此，抵抗对抗中毒攻击的安全也越来越成为一项艰巨的任务。具体而言，深钢筋学习（DRL）是IOV应用中广泛使用的ML设计之一。标准的ML安全技术在DRL中无效，该算法学会通过与环境的持续互动来解决顺序决策，并且环境是随时间变化的，动态的和移动的。在本文中，我们提出了一个基于IOV中基于SYBIL的数据中毒攻击的封闭式复发单元（GRU）的联邦持续学习（GFCL）异常检测框架。目的是提出一个轻巧且可扩展的框架，该框架在不包含由攻击样本组成的A-Priori培训数据集的情况下学习和检测非法行为。我们使用GRU预测未来的数据顺序，以基于联合学习的分布方式分析和检测车辆的非法行为。我们使用现实世界的车辆移动轨迹研究了框架的性能。结果证明了我们提出的解决方案在不同的性能指标方面的有效性。

不确定性在未来预测中起关键作用。未来是不确定的。这意味着可能有很多可能的未来。未来的预测方法应涵盖坚固的全部可能性。在自动驾驶中，涵盖预测部分中的多种模式对于做出安全至关重要的决策至关重要。尽管近年来计算机视觉系统已大大提高，但如今的未来预测仍然很困难。几个示例是未来的不确定性，全面理解的要求以及嘈杂的输出空间。在本论文中，我们通过以随机方式明确地对运动进行建模并学习潜在空间中的时间动态，从而提出了解决这些挑战的解决方案。

Along with the springing up of semantics-empowered communication (SemCom) researches, it is now witnessing an unprecedentedly growing interest towards a wide range of aspects (e.g., theories, applications, metrics and implementations) in both academia and industry. In this work, we primarily aim to provide a comprehensive survey on both the background and research taxonomy, as well as a detailed technical tutorial. Specifically, we start by reviewing the literature and answering the "what" and "why" questions in semantic transmissions. Afterwards, we present corresponding ecosystems, including theories, metrics, datasets and toolkits, on top of which the taxonomy for research directions is presented. Furthermore, we propose to categorize the critical enabling techniques by explicit and implicit reasoning-based methods, and elaborate on how they evolve and contribute to modern content \& channel semantics-empowered communications. Besides reviewing and summarizing the latest efforts in SemCom, we discuss the relations with other communication levels (e.g., reliable and goal-oriented communications) from a holistic and unified viewpoint. Subsequently, in order to facilitate the future developments and industrial applications, we also highlight advanced practical techniques for boosting semantic accuracy, robustness, and large-scale scalability, just to mention a few. Finally, we discuss the technical challenges that shed light on future research opportunities.

We introduce a Deep Stochastic IOC 1 RNN Encoderdecoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes. DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i.e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents. DESIRE achieves these in a single end-to-end trainable neural network model, while being computationally efficient. The model first obtains a diverse set of hypothetical future prediction samples employing a conditional variational autoencoder, which are ranked and refined by the following RNN scoring-regression module. Samples are scored by accounting for accumulated future rewards, which enables better long-term strategic decisions similar to IOC frameworks. An RNN scene context fusion module jointly captures past motion histories, the semantic scene context and interactions among multiple agents. A feedback mechanism iterates over the ranking and refinement to further boost the prediction accuracy. We evaluate our model on two publicly available datasets: KITTI and Stanford Drone Dataset. Our experiments show that the proposed model significantly improves the prediction accuracy compared to other baseline methods.

机器学习的最新进展使其在不同领域的广泛应用程序，最令人兴奋的应用程序之一是自动驾驶汽车（AV），这鼓励了从感知到预测到计划的许多ML算法的开发。但是，培训AV通常需要从不同驾驶环境（例如城市）以及不同类型的个人信息（例如工作时间和路线）收集的大量培训数据。这种收集的大数据被视为以数据为中心的AI时代的ML新油，通常包含大量对隐私敏感的信息，这些信息很难删除甚至审核。尽管现有的隐私保护方法已经取得了某些理论和经验成功，但将它们应用于自动驾驶汽车等现实世界应用时仍存在差距。例如，当培训AVS时，不仅可以单独识别的信息揭示对隐私敏感的信息，还可以揭示人口级别的信息，例如城市内的道路建设以及AVS的专有商业秘密。因此，重新审视AV中隐私风险和相应保护方法的前沿以弥合这一差距至关重要。遵循这一目标，在这项工作中，我们为AVS中的隐私风险和保护方法提供了新的分类法，并将AV中的隐私分为三个层面：个人，人口和专有。我们明确列出了保护每个级别的隐私级别，总结这些挑战的现有解决方案，讨论课程和结论，并为研究人员和从业者提供潜在的未来方向和机会。我们认为，这项工作将有助于塑造AV中的隐私研究，并指导隐私保护技术设计。

Graph learning is a popular approach for performing machine learning on graph-structured data. It has revolutionized the machine learning ability to model graph data to address downstream tasks. Its application is wide due to the availability of graph data ranging from all types of networks to information systems. Most graph learning methods assume that the graph is static and its complete structure is known during training. This limits their applicability since they cannot be applied to problems where the underlying graph grows over time and/or new tasks emerge incrementally. Such applications require a lifelong learning approach that can learn the graph continuously and accommodate new information whilst retaining previously learned knowledge. Lifelong learning methods that enable continuous learning in regular domains like images and text cannot be directly applied to continuously evolving graph data, due to its irregular structure. As a result, graph lifelong learning is gaining attention from the research community. This survey paper provides a comprehensive overview of recent advancements in graph lifelong learning, including the categorization of existing methods, and the discussions of potential applications and open research problems.

预测环境的未来占用状态对于实现自动驾驶汽车的明智决定很重要。占用预测中的常见挑战包括消失的动态对象和模糊的预测，尤其是对于长期预测范围。在这项工作中，我们提出了一个双独沟的神经网络体系结构，以预测占用状态的时空演化。一个插脚致力于预测移动的自我车辆将如何观察到静态环境。另一个插脚预测环境中的动态对象将如何移动。在现实Waymo开放数据集上进行的实验表明，两个插脚的融合输出能够保留动态对象并减少预测中比基线模型更长的预测时间范围。

Path prediction is an essential task for many real-world Cyber-Physical Systems (CPS) applications, from autonomous driving and traffic monitoring/management to pedestrian/worker safety. These real-world CPS applications need a robust, lightweight path prediction that can provide a universal network architecture for multiple subjects (e.g., pedestrians and vehicles) from different perspectives. However, most existing algorithms are tailor-made for a unique subject with a specific camera perspective and scenario. This article presents Pishgu, a universal lightweight network architecture, as a robust and holistic solution for path prediction. Pishgu's architecture can adapt to multiple path prediction domains with different subjects (vehicles, pedestrians), perspectives (bird's-eye, high-angle), and scenes (sidewalk, highway). Our proposed architecture captures the inter-dependencies within the subjects in each frame by taking advantage of Graph Isomorphism Networks and the attention module. We separately train and evaluate the efficacy of our architecture on three different CPS domains across multiple perspectives (vehicle bird's-eye view, pedestrian bird's-eye view, and human high-angle view). Pishgu outperforms state-of-the-art solutions in the vehicle bird's-eye view domain by 42% and 61% and pedestrian high-angle view domain by 23% and 22% in terms of ADE and FDE, respectively. Additionally, we analyze the domain-specific details for various datasets to understand their effect on path prediction and model interpretation. Finally, we report the latency and throughput for all three domains on multiple embedded platforms showcasing the robustness and adaptability of Pishgu for real-world integration into CPS applications.

“轨迹”是指由地理空间中的移动物体产生的迹线，通常由一系列按时间顺序排列的点表示，其中每个点由地理空间坐标集和时间戳组成。位置感应和无线通信技术的快速进步使我们能够收集和存储大量的轨迹数据。因此，许多研究人员使用轨迹数据来分析各种移动物体的移动性。在本文中，我们专注于“城市车辆轨迹”，这是指城市交通网络中车辆的轨迹，我们专注于“城市车辆轨迹分析”。城市车辆轨迹分析提供了前所未有的机会，可以了解城市交通网络中的车辆运动模式，包括以用户为中心的旅行经验和系统范围的时空模式。城市车辆轨迹数据的时空特征在结构上相互关联，因此，许多先前的研究人员使用了各种方法来理解这种结构。特别是，由于其强大的函数近似和特征表示能力，深度学习模型是由于许多研究人员的注意。因此，本文的目的是开发基于深度学习的城市车辆轨迹分析模型，以更好地了解城市交通网络的移动模式。特别是，本文重点介绍了两项研究主题，具有很高的必要性，重要性和适用性：下一个位置预测，以及合成轨迹生成。在这项研究中，我们向城市车辆轨迹分析提供了各种新型模型，使用深度学习。

Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation-learning abilities of deep neural networks. However, large centralized approaches quickly become infeasible as the number of agents scale, and fully decentralized approaches can miss important opportunities for information sharing and coordination. Furthermore, not all agents are equal -- in some cases, individual agents may not even have the ability to send communication to other agents or explicitly model other agents. This paper considers the case where there is a single, powerful, \emph{central agent} that can observe the entire observation space, and there are multiple, low-powered \emph{local agents} that can only receive local observations and are not able to communicate with each other. The central agent's job is to learn what message needs to be sent to different local agents based on the global observations, not by centrally solving the entire problem and sending action commands, but by determining what additional information an individual agent should receive so that it can make a better decision. In this work we present our MARL algorithm \algo, describe where it would be most applicable, and implement it in the cooperative navigation and multi-agent walker domains. Empirical results show that 1) learned communication does indeed improve system performance, 2) results generalize to heterogeneous local agents, and 3) results generalize to different reward structures.