深度神经网络（DNN）由于其高度的感知，决策和控制而被广泛用于自主驾驶中。在诸如自动驾驶之类的安全至关重要系统中，实时执行感测和感知等任务对于车辆的安全至关重要，这需要应用程序的执行时间才能预测。但是，在DNN推断中观察到不可忽略的时间变化。当前的DNN推理研究要么忽略时间变化问题，要么依靠调度程序来处理它。当前的工作都没有解释DNN推理时间变化的根本原因。了解DNN推理的时间变化成为自动驾驶实时计划的基本挑战。在这项工作中，我们从六个角度分析了DNN推断的时间变化：数据，I/O，模型，运行时，硬件和端到端感知系统。在理解DNN推断的时间变化方面得出了六个见解。

Video, as a key driver in the global explosion of digital information, can create tremendous benefits for human society. Governments and enterprises are deploying innumerable cameras for a variety of applications, e.g., law enforcement, emergency management, traffic control, and security surveillance, all facilitated by video analytics (VA). This trend is spurred by the rapid advancement of deep learning (DL), which enables more precise models for object classification, detection, and tracking. Meanwhile, with the proliferation of Internet-connected devices, massive amounts of data are generated daily, overwhelming the cloud. Edge computing, an emerging paradigm that moves workloads and services from the network core to the network edge, has been widely recognized as a promising solution. The resulting new intersection, edge video analytics (EVA), begins to attract widespread attention. Nevertheless, only a few loosely-related surveys exist on this topic. A dedicated venue for collecting and summarizing the latest advances of EVA is highly desired by the community. Besides, the basic concepts of EVA (e.g., definition, architectures, etc.) are ambiguous and neglected by these surveys due to the rapid development of this domain. A thorough clarification is needed to facilitate a consensus on these concepts. To fill in these gaps, we conduct a comprehensive survey of the recent efforts on EVA. In this paper, we first review the fundamentals of edge computing, followed by an overview of VA. The EVA system and its enabling techniques are discussed next. In addition, we introduce prevalent frameworks and datasets to aid future researchers in the development of EVA systems. Finally, we discuss existing challenges and foresee future research directions. We believe this survey will help readers comprehend the relationship between VA and edge computing, and spark new ideas on EVA.

Computer vision applications in intelligent transportation systems (ITS) and autonomous driving (AD) have gravitated towards deep neural network architectures in recent years. While performance seems to be improving on benchmark datasets, many real-world challenges are yet to be adequately considered in research. This paper conducted an extensive literature review on the applications of computer vision in ITS and AD, and discusses challenges related to data, models, and complex urban environments. The data challenges are associated with the collection and labeling of training data and its relevance to real world conditions, bias inherent in datasets, the high volume of data needed to be processed, and privacy concerns. Deep learning (DL) models are commonly too complex for real-time processing on embedded hardware, lack explainability and generalizability, and are hard to test in real-world settings. Complex urban traffic environments have irregular lighting and occlusions, and surveillance cameras can be mounted at a variety of angles, gather dirt, shake in the wind, while the traffic conditions are highly heterogeneous, with violation of rules and complex interactions in crowded scenarios. Some representative applications that suffer from these problems are traffic flow estimation, congestion detection, autonomous driving perception, vehicle interaction, and edge computing for practical deployment. The possible ways of dealing with the challenges are also explored while prioritizing practical deployment.

自主机时代的一个主要技术挑战是自动驾驶机器的编程，它要求跨多个领域的协同作用，包括基本的计算机科学，计算机架构和机器人技术，并且需要学术界和行业的专业知识。本文讨论了与生产现实生活自动驾驶机器相关的编程理论和实践，并在特定功能要求，性能期望和自主机的实施约束的背景下涵盖了从高级概念到低级代码生成的各个方面。

由于自动驾驶应用程序的高性能和安全要求，现代自动驾驶系统（AD）的复杂性一直在增长，刺激了对更复杂的硬件的需求，这可能会增加广告平台的能量足迹。在解决此问题时，Edge Computing有望包含自动驾驶应用程序，从而使计算密集型的自治任务能够在计算能力的边缘服务器下进行处理。但是，除了严格的鲁棒性需求外，ADS平台的复杂硬件体系结构还阐明了自动驾驶独有的任务卸载并发症。因此，我们提出了$ romanus $，这是一种具有多传感器处理管道的模块化广告平台的可靠和高效任务的方法。我们的方法论需要两个阶段：（i）沿相关深度学习模型的执行路径引入有效的卸载点，以及（ii）基于深度强化学习的运行时解决方案的实现，以根据在操作模式下根据变化的变化来调整操作模式。感知到的道路场景复杂性，网络连接和服务器负载。对象检测用例的实验表明，我们的方法比纯局部执行高14.99％，同时从强大的不稳定卸载基线中降低了危险行为的77.06％。

已经提出了高效和自适应计算机视觉系统以使计算机视觉任务，例如图像分类和对象检测，针对嵌入或移动设备进行了优化。这些解决方案最近的起源，专注于通过设计具有近似旋钮的自适应系统来优化模型（深神经网络，DNN）或系统。尽管最近的几项努力，但我们表明现有解决方案遭受了两个主要缺点。首先，系统不考虑模型的能量消耗，同时在制定要运行的模型的决定时。其次，由于其他共同居民工作负载，评估不考虑设备上的争用的实际情况。在这项工作中，我们提出了一种高效和自适应的视频对象检测系统，这是联合优化的精度，能量效率和延迟。底层Virtuoso是一个多分支执行内核，它能够在精度 - 能量 - 延迟轴上的不同运行点处运行，以及轻量级运行时调度程序，以选择最佳的执行分支以满足用户要求。要与Virtuoso相当比较，我们基准于15件最先进的或广泛使用的协议，包括更快的R-CNN（FRCNN），YOLO V3，SSD，培训台，SELSA，MEGA，REPP，FastAdapt和我们的内部FRCNN +，YOLO +，SSD +和高效+（我们的变体具有增强的手机效率）的自适应变体。通过这种全面的基准，Virtuoso对所有上述协议显示出优势，在NVIDIA Jetson Mobile GPU上的每一项效率水平上引领精度边界。具体而言，Virtuoso的准确性为63.9％，比一些流行的物体检测模型高于10％，51.1％，yolo为49.5％。

由于高系统复杂性和动态环境，测试和调试已成为机器人软件开发的主要障碍。标准，基于中间件的数据记录不提供有关内部计算和性能瓶颈的足够信息。其他现有方法还针对非常特定的问题，因此不能用于多用途分析。此外，它们不适合实时应用。在本文中，我们呈现ROS2_TRACING，一个灵活的跟踪工具和ROS 2的多功能仪器集合。它允许使用低开销LTTNG示踪器收集实时分布式系统的运行时执行信息。工具还将跟踪集成到无价的ROS 2 Orchestration系统和其他可用性工具中。消息延迟实验表明，当所有ROS 2仪器启用时，端到端消息延迟开销低于0.0055毫秒，我们认为适用于生产实时系统。使用ROS2_TRACING获得的ROS 2执行信息可以与操作系统的跟踪数据组合，从而实现更广泛的精确分析，有助于了解应用程序执行，以找到性能瓶颈和其他问题的原因。源代码可用于：https://gitlab.com/ros-tracing/ros2_tracing。

Semantic segmentation works on the computer vision algorithm for assigning each pixel of an image into a class. The task of semantic segmentation should be performed with both accuracy and efficiency. Most of the existing deep FCNs yield to heavy computations and these networks are very power hungry, unsuitable for real-time applications on portable devices. This project analyzes current semantic segmentation models to explore the feasibility of applying these models for emergency response during catastrophic events. We compare the performance of real-time semantic segmentation models with non-real-time counterparts constrained by aerial images under oppositional settings. Furthermore, we train several models on the Flood-Net dataset, containing UAV images captured after Hurricane Harvey, and benchmark their execution on special classes such as flooded buildings vs. non-flooded buildings or flooded roads vs. non-flooded roads. In this project, we developed a real-time UNet based model and deployed that network on Jetson AGX Xavier module.

The last decade witnessed increasingly rapid progress in self-driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence. The objective of this paper is to survey the current state-of-the-art on deep learning technologies used in autonomous driving. We start by presenting AI-based self-driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration and motion control algorithms. We investigate both the modular perception-planning-action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources and computational hardware. The comparison presented in this survey helps to gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices. 1

自动化驾驶系统（广告）开辟了汽车行业的新领域，为未来的运输提供了更高的效率和舒适体验的新可能性。然而，在恶劣天气条件下的自主驾驶已经存在，使自动车辆（AVS）长时间保持自主车辆（AVS）或更高的自主权。本文评估了天气在分析和统计方式中为广告传感器带来的影响和挑战，并对恶劣天气条件进行了解决方案。彻底报道了关于对每种天气的感知增强的最先进技术。外部辅助解决方案如V2X技术，当前可用的数据集，模拟器和天气腔室的实验设施中的天气条件覆盖范围明显。通过指出各种主要天气问题，自主驾驶场目前正在面临，近年来审查硬件和计算机科学解决方案，这项调查概述了在不利的天气驾驶条件方面的障碍和方向的障碍和方向。

自治机器人目前是最受欢迎的人工智能问题之一，在过去十年中，从自动驾驶汽车和人形系统到交付机器人和无人机，这是一项最受欢迎的智能问题。部分问题是获得一个机器人，以模仿人类的感知，我们的视觉感，用诸如神经网络等数学模型用相机和大脑的眼睛替换眼睛。开发一个能够在没有人为干预的情况下驾驶汽车的AI和一个小型机器人在城市中递送包裹可能看起来像不同的问题，因此来自感知和视觉的观点来看，这两个问题都有几种相似之处。我们目前的主要解决方案通过使用计算机视觉技术，机器学习和各种算法来实现对环境感知的关注，使机器人理解环境或场景，移动，调整其轨迹并执行其任务（维护，探索，等。）无需人为干预。在这项工作中，我们从头开始开发一个小型自动车辆，能够仅使用视觉信息理解场景，通过工业环境导航，检测人员和障碍，或执行简单的维护任务。我们审查了基本问题的最先进问题，并证明了小规模采用的许多方法类似于来自特斯拉或Lyft等公司的真正自动驾驶汽车中使用的方法。最后，我们讨论了当前的机器人和自主驾驶状态以及我们在这一领域找到的技术和道德限制。

分布式机器人系统在很大程度上依赖于支持它的Publish-Subscriber通信范式和中间件框架，例如机器人操作系统（ROS），以有效地实现模块化计算图。 ROS 2执行程序是一个处理ROS 2消息的高级任务调度程序，是性能瓶颈。我们扩展了ROS2_Tracing，这是一个带有仪器和用于实时跟踪ROS 2的工具的框架，并在分布式ROS 2系统中分析和可视化消息流的分析和可视化。我们的方法检测输入和输出消息之间的一对多因果关系，包括通过简单的用户级注释，包括间接因果链接。我们在合成和真实机器人系统上验证了我们的方法，并证明了其低运行时开销。此外，可以进一步利用基本的中间执行表示数据库来提取其他指标和高级结果。这可以提供有价值的时机和调度信息，以进一步研究和改善ROS 2执行者，并优化任何ROS 2系统。源代码可在以下网址获得：https：//github.com/christophebedard/ros2-message-flow-analysis。

在新兴应用中，自主机器人对日常生活的潜在影响是明显的，如精密农业，搜救，救援和基础设施检查。然而，这种应用需要在不明和复杂的一组目标中具有广泛而非结构化的环境，所有这些应用都在严格的计算和功率限制下。因此，我们认为必须安排和优化支持机器人自主权的计算内核，以保证及时和正确的行为，同时允许在运行时重新配置调度参数。在本文中，我们考虑了一个必要的第一步，迈出了自主机器人的计算意识的目标：从资源管理角度来看，基础计算内核的实证研究。具体地，我们对三个嵌入式计算平台进行了用于定位和映射，路径规划，任务分配，深度估计和光流的核的定时，电源和内存性能的数据驱动的研究。我们配置文件并分析这些内核，为计算感知自治机器人提供了解调度和动态资源管理的洞察。值得注意的是，我们的结果表明，内核性能与机器人的运营环境有关，证明了计算感知机器人的概念以及为什么我们的作品对这一目标的关键步骤。

对将AI功能从云上的数据中心转移到边缘或最终设备的需求越来越大，这是由在智能手机，AR/VR设备，自动驾驶汽车和各种汽车上运行的快速实时AI的应用程序举例说明的。物联网设备。然而，由于DNN计算需求与边缘或最终设备上的计算能力之间的较大增长差距，这种转变受到了严重的阻碍。本文介绍了XGEN的设计，这是DNN的优化框架，旨在弥合差距。 XGEN将横切共同设计作为其一阶考虑。它的全栈AI面向AI的优化包括在DNN软件堆栈的各个层的许多创新优化，所有这些优化都以合作的方式设计。独特的技术使XGEN能够优化各种DNN，包括具有极高深度的DNN（例如Bert，GPT，其他变形金刚），并生成代码比现有DNN框架中的代码快几倍，同时提供相同的准确性水平。

Automated Driving Systems (ADS) have rapidly evolved in recent years and their architecture becomes sophisticated. Ensuring robustness, reliability and safety of performance is particularly important. The main challenge in building an ADS is the ability to meet certain stringent performance requirements in terms of both making safe operational decisions and finishing processing in real-time. Middlewares play a crucial role to handle these requirements in ADS. The way middlewares share data between the different system components has a direct impact on the overall performance, particularly the latency overhead. To this end, this paper presents FastCycle as a lightweight multi-threaded zero-copy messaging broker to meet the requirements of a high fidelity ADS in terms of modularity, real-time performance and security. We discuss the architecture and the main features of the proposed framework. Evaluation of the proposed framework based on standard metrics in comparison with popular middlewares used in robotics and automated driving shows the improved performance of our framework. The implementation of FastCycle and the associated comparisons with other frameworks are open sourced.

感知环境是实现合作驾驶自动化（CDA）的最基本关键之一，该关键被认为是解决当代运输系统的安全性，流动性和可持续性问题的革命性解决方案。尽管目前在计算机视觉的物体感知领域正在发生前所未有的进化，但由于不可避免的物理遮挡和单辆车的接受程度有限，最先进的感知方法仍在与复杂的现实世界流量环境中挣扎系统。基于多个空间分离的感知节点，合作感知（CP）诞生是为了解锁驱动自动化的感知瓶颈。在本文中，我们全面审查和分析了CP的研究进度，据我们所知，这是第一次提出统一的CP框架。审查了基于不同类型的传感器的CP系统的体系结构和分类学，以显示对CP系统的工作流程和不同结构的高级描述。对节点结构，传感器模式和融合方案进行了审查和分析，并使用全面的文献进行了详细的解释。提出了分层CP框架，然后对现有数据集和模拟器进行审查，以勾勒出CP的整体景观。讨论重点介绍了当前的机会，开放挑战和预期的未来趋势。

商业自主机器是一个蓬勃发展的扇区，它可能是下一个无处不在的计算平台，它是在个人计算机（PC），云计算和移动计算之后的。然而，缺少适用于自动机器的合适计算基板，许多公司被迫开发既不原则也不可扩展的临时计算解决方案。通过分析自动机器计算的需求，本文提出了数据流加速器体系结构（DAA），这是经典数据流原理的现代实例化，与自动机器软件的特性相匹配。

机器人正在集成更大尺寸的模型以丰富功能并提高准确性，从而导致控制力计算压力。因此，机器人在计算功率和电池容量中遇到瓶颈。雾或云机器人技术是解决这些问题的最期待的理论之一。云机器人技术的方法已从系统级到节点级别开发。但是，当前的节点级系统不够灵活，无法动态适应变化的条件。为了解决这个问题，我们提出了Elasticros，该Elasticros将当前的节点级系统演变为算法级别。 Elasticros基于ROS和ROS2。对于FOG和Cloud Robotics，它是第一个具有算法级协作计算的机器人操作系统。 Elasticros开发弹性协作计算，以实现对动态条件的适应性。协作计算算法是Elasticros的核心和挑战。我们抽象问题，然后提出一种称为Elasaction的算法以解决。这是一种基于在线学习的动态行动决策算法，它决定了机器人和服务器的合作方式。该算法会动态更新参数，以适应机器人当前所在的条件的变化。它根据配置将计算任务的弹性分配到机器人和服务器上。此外，我们证明了弹性的遗憾上限是sublinear，它保证了其收敛性，因此使Elasticros在其弹性上保持稳定。最后，我们对机器人技术的常见任务进行了Elasticros进行实验，包括SLAM，GRASPING和HUMAN-OBOT对话，然后在延迟，CPU使用和功耗中测量其性能。算法级弹性弹性的性能明显优于当前的节点级系统。

Multi-modal fusion is a basic task of autonomous driving system perception, which has attracted many scholars' interest in recent years. The current multi-modal fusion methods mainly focus on camera data and LiDAR data, but pay little attention to the kinematic information provided by the bottom sensors of the vehicle, such as acceleration, vehicle speed, angle of rotation. These information are not affected by complex external scenes, so it is more robust and reliable. In this paper, we introduce the existing application fields of vehicle bottom information and the research progress of related methods, as well as the multi-modal fusion methods based on bottom information. We also introduced the relevant information of the vehicle bottom information data set in detail to facilitate the research as soon as possible. In addition, new future ideas of multi-modal fusion technology for autonomous driving tasks are proposed to promote the further utilization of vehicle bottom information.

In this tutorial paper, we look into the evolution and prospect of network architecture and propose a novel conceptual architecture for the 6th generation (6G) networks. The proposed architecture has two key elements, i.e., holistic network virtualization and pervasive artificial intelligence (AI). The holistic network virtualization consists of network slicing and digital twin, from the aspects of service provision and service demand, respectively, to incorporate service-centric and user-centric networking. The pervasive network intelligence integrates AI into future networks from the perspectives of networking for AI and AI for networking, respectively. Building on holistic network virtualization and pervasive network intelligence, the proposed architecture can facilitate three types of interplay, i.e., the interplay between digital twin and network slicing paradigms, between model-driven and data-driven methods for network management, and between virtualization and AI, to maximize the flexibility, scalability, adaptivity, and intelligence for 6G networks. We also identify challenges and open issues related to the proposed architecture. By providing our vision, we aim to inspire further discussions and developments on the potential architecture of 6G.