移动设备通过深神经网络(DNN)越来越依赖对象检测(OD)来执行关键任务。由于它们的复杂性高,这些DNN的执行需要过度的时间和能量。低复杂性对象跟踪(OT)可以与OD一起使用,后者定期应用后,以生成“新鲜”的跟踪参考。然而,使用OD处理的帧产生大的延迟,这可以使参考延迟过时并降低跟踪质量。这里,我们建议在这种情况下使用边缘计算,并在对大OD延迟中建立并行OT(在移动设备上)和OD(处于边缘服务器)的进程。我们提出Katch-Up,一种新型跟踪机制,可提高系统弹性过度OD延迟。但是,虽然Katch-up显着提高了性能,但它也增加了移动设备的计算负荷。因此,我们设计SmartDet,基于深度加强学习(DRL)的低复杂性控制器,了解资源利用率和OD性能之间的权衡。 SmartDet作为输入上下文相关信息与当前视频内容相关的信息和当前网络条件,以优化OD卸载的频率和类型,以及Katch-Up利用率。我们在通过Wi-Fi链路连接的GTX 980 TI为移动设备和GTX 980 TI,广泛地评估SmartDet。实验结果表明,SmartDET在跟踪性能 - 平均召回(MAR)和资源使用之间实现了最佳平衡。关于具有完全Katch-Upusage和最大渠道使用的基线,我们仍然将MAR增加4%,同时使用50%的通道和与Katch-Up相关的30%电力资源。对于使用最小资源的固定策略,我们在使用katch-up在框架的1/3上时,我们将MAR增加20%。
translated by 谷歌翻译
Video, as a key driver in the global explosion of digital information, can create tremendous benefits for human society. Governments and enterprises are deploying innumerable cameras for a variety of applications, e.g., law enforcement, emergency management, traffic control, and security surveillance, all facilitated by video analytics (VA). This trend is spurred by the rapid advancement of deep learning (DL), which enables more precise models for object classification, detection, and tracking. Meanwhile, with the proliferation of Internet-connected devices, massive amounts of data are generated daily, overwhelming the cloud. Edge computing, an emerging paradigm that moves workloads and services from the network core to the network edge, has been widely recognized as a promising solution. The resulting new intersection, edge video analytics (EVA), begins to attract widespread attention. Nevertheless, only a few loosely-related surveys exist on this topic. A dedicated venue for collecting and summarizing the latest advances of EVA is highly desired by the community. Besides, the basic concepts of EVA (e.g., definition, architectures, etc.) are ambiguous and neglected by these surveys due to the rapid development of this domain. A thorough clarification is needed to facilitate a consensus on these concepts. To fill in these gaps, we conduct a comprehensive survey of the recent efforts on EVA. In this paper, we first review the fundamentals of edge computing, followed by an overview of VA. The EVA system and its enabling techniques are discussed next. In addition, we introduce prevalent frameworks and datasets to aid future researchers in the development of EVA systems. Finally, we discuss existing challenges and foresee future research directions. We believe this survey will help readers comprehend the relationship between VA and edge computing, and spark new ideas on EVA.
translated by 谷歌翻译
由于自动驾驶应用程序的高性能和安全要求,现代自动驾驶系统(AD)的复杂性一直在增长,刺激了对更复杂的硬件的需求,这可能会增加广告平台的能量足迹。在解决此问题时,Edge Computing有望包含自动驾驶应用程序,从而使计算密集型的自治任务能够在计算能力的边缘服务器下进行处理。但是,除了严格的鲁棒性需求外,ADS平台的复杂硬件体系结构还阐明了自动驾驶独有的任务卸载并发症。因此,我们提出了$ romanus $,这是一种具有多传感器处理管道的模块化广告平台的可靠和高效任务的方法。我们的方法论需要两个阶段:(i)沿相关深度学习模型的执行路径引入有效的卸载点,以及(ii)基于深度强化学习的运行时解决方案的实现,以根据在操作模式下根据变化的变化来调整操作模式。感知到的道路场景复杂性,网络连接和服务器负载。对象检测用例的实验表明,我们的方法比纯局部执行高14.99%,同时从强大的不稳定卸载基线中降低了危险行为的77.06%。
translated by 谷歌翻译
在监控和搜索和救援应用程序中,重要的是在低端设备上实时执行多目标跟踪(MOT)。今天的MOT解决方案采用深度神经网络,往往具有高计算复杂性。识别帧大小对跟踪性能的影响,我们提出了深度,一种模型不可知框架尺寸选择方法,可在现有的全卷积网络基跟踪器之上进行操作,以加速跟踪吞吐量。在培训阶段,我们将可检测性分数纳入单次跟踪器架构,使得DeepScale以自我监督的方式学习不同帧大小的表示估计。在推理期间,它可以根据基于用户控制参数根据视觉内容的复杂性来调整帧大小。为了利用边缘服务器上的计算资源,我们提出了两个计算分区模式,即仅使用自适应帧大小传输和边缘服务器辅助跟踪仅适用于MOT,即边缘服务器。 MOT数据集的广泛实验和基准测试证明了深度的有效性和灵活性。与最先进的追踪器相比,DeepScale ++,DeepScale的变种实现1.57倍加速,仅在一个配置中的MOT15数据集上跟踪准确性。我们已经实现和评估了DeepScale ++,以及由NVIDIA JETSON TX2板和GPU服务器组成的小型测试平台上所提出的计算分区方案。实验显示与仅服务器或智能相机的解决方案相比跟踪性能和延迟之间的非琐碎权衡。
translated by 谷歌翻译
本文考虑了使用嵌入式设备来获取和分类图像的设置。由于计算能力有限,嵌入式设备依赖于具有不平衡精度的简约分类模型。当认为本地分类不准确时,设备可以决定使用更准确但资源密集型的模型将图像卸载到边缘服务器。但是,资源限制(例如,网络带宽)需要调节这种传输,以避免交通拥堵和高延迟。当传输调节是通过令牌桶时,该论文调查了此卸载问题,该机制通常用于此类目的。目的是设计一种轻巧的在线卸载策略,该策略在令牌存储桶的限制下优化了特定于应用程序的指标(例如,分类精度)。该论文制定了基于深Q网络(DQN)的政策,并证明了其功效和在嵌入式设备上部署的可行性。值得注意的是,该策略可以处理复杂的输入模式,包括图像到达中的相关性和分类精度。评估是通过使用来自Imagenet图像分类基准生成的合成痕迹对局部测试床进行图像分类进行的。这项工作的实施可在https://github.com/qiujiaming315/edgeml-dqn上获得。
translated by 谷歌翻译
已经提出了高效和自适应计算机视觉系统以使计算机视觉任务,例如图像分类和对象检测,针对嵌入或移动设备进行了优化。这些解决方案最近的起源,专注于通过设计具有近似旋钮的自适应系统来优化模型(深神经网络,DNN)或系统。尽管最近的几项努力,但我们表明现有解决方案遭受了两个主要缺点。首先,系统不考虑模型的能量消耗,同时在制定要运行的模型的决定时。其次,由于其他共同居民工作负载,评估不考虑设备上的争用的实际情况。在这项工作中,我们提出了一种高效和自适应的视频对象检测系统,这是联合优化的精度,能量效率和延迟。底层Virtuoso是一个多分支执行内核,它能够在精度 - 能量 - 延迟轴上的不同运行点处运行,以及轻量级运行时调度程序,以选择最佳的执行分支以满足用户要求。要与Virtuoso相当比较,我们基准于15件最先进的或广泛使用的协议,包括更快的R-CNN(FRCNN),YOLO V3,SSD,培训台,SELSA,MEGA,REPP,FastAdapt和我们的内部FRCNN +,YOLO +,SSD +和高效+(我们的变体具有增强的手机效率)的自适应变体。通过这种全面的基准,Virtuoso对所有上述协议显示出优势,在NVIDIA Jetson Mobile GPU上的每一项效率水平上引领精度边界。具体而言,Virtuoso的准确性为63.9%,比一些流行的物体检测模型高于10%,51.1%,yolo为49.5%。
translated by 谷歌翻译
In recent years, the exponential proliferation of smart devices with their intelligent applications poses severe challenges on conventional cellular networks. Such challenges can be potentially overcome by integrating communication, computing, caching, and control (i4C) technologies. In this survey, we first give a snapshot of different aspects of the i4C, comprising background, motivation, leading technological enablers, potential applications, and use cases. Next, we describe different models of communication, computing, caching, and control (4C) to lay the foundation of the integration approach. We review current state-of-the-art research efforts related to the i4C, focusing on recent trends of both conventional and artificial intelligence (AI)-based integration approaches. We also highlight the need for intelligence in resources integration. Then, we discuss integration of sensing and communication (ISAC) and classify the integration approaches into various classes. Finally, we propose open challenges and present future research directions for beyond 5G networks, such as 6G.
translated by 谷歌翻译
The deployment flexibility and maneuverability of Unmanned Aerial Vehicles (UAVs) increased their adoption in various applications, such as wildfire tracking, border monitoring, etc. In many critical applications, UAVs capture images and other sensory data and then send the captured data to remote servers for inference and data processing tasks. However, this approach is not always practical in real-time applications due to the connection instability, limited bandwidth, and end-to-end latency. One promising solution is to divide the inference requests into multiple parts (layers or segments), with each part being executed in a different UAV based on the available resources. Furthermore, some applications require the UAVs to traverse certain areas and capture incidents; thus, planning their paths becomes critical particularly, to reduce the latency of making the collaborative inference process. Specifically, planning the UAVs trajectory can reduce the data transmission latency by communicating with devices in the same proximity while mitigating the transmission interference. This work aims to design a model for distributed collaborative inference requests and path planning in a UAV swarm while respecting the resource constraints due to the computational load and memory usage of the inference requests. The model is formulated as an optimization problem and aims to minimize latency. The formulated problem is NP-hard so finding the optimal solution is quite complex; thus, this paper introduces a real-time and dynamic solution for online applications using deep reinforcement learning. We conduct extensive simulations and compare our results to the-state-of-the-art studies demonstrating that our model outperforms the competing models.
translated by 谷歌翻译
我们提出了一种新的四管齐下的方法,在文献中首次建立消防员的情境意识。我们构建了一系列深度学习框架,彼此之叠,以提高消防员在紧急首次响应设置中进行的救援任务的安全性,效率和成功完成。首先,我们使用深度卷积神经网络(CNN)系统,以实时地分类和识别来自热图像的感兴趣对象。接下来,我们将此CNN框架扩展了对象检测,跟踪,分割与掩码RCNN框架,以及具有多模级自然语言处理(NLP)框架的场景描述。第三,我们建立了一个深入的Q学习的代理,免受压力引起的迷失方向和焦虑,能够根据现场消防环境中观察和存储的事实来制定明确的导航决策。最后,我们使用了一种低计算无监督的学习技术,称为张量分解,在实时对异常检测进行有意义的特征提取。通过这些临时深度学习结构,我们建立了人工智能系统的骨干,用于消防员的情境意识。要将设计的系统带入消防员的使用,我们设计了一种物理结构,其中处理后的结果被用作创建增强现实的投入,这是一个能够建议他们所在地的消防员和周围的关键特征,这对救援操作至关重要在手头,以及路径规划功能,充当虚拟指南,以帮助迷彩的第一个响应者恢复安全。当组合时,这四种方法呈现了一种新颖的信息理解,转移和综合方法,这可能会大大提高消防员响应和功效,并降低寿命损失。
translated by 谷歌翻译
我们考虑一个用于边缘计算应用程序的智能传感器网络,该网络采样了感兴趣的信号,并将更新发送到基站进行远程全局监视。传感器配备了传感和计算,并且可以在传输前在板载上发送原始数据或处理它们。边缘的有限硬件资源产生基本的潜伏期 - 准确性权衡:原始测量值不准确,但及时,而计算延迟后准确的处理更新可用。同样,如果传感器在板载处理需要数据压缩,则无线通信引起的延迟可能会更高。因此,需要决定何时传感器应传输原始测量或依靠本地处理以最大程度地提高整体网络性能。为了解决这个传感设计问题,我们对一个嵌入计算和通信延迟的估计理论优化框架进行建模,并提出一种基于强化学习的方法,以在每个传感器上动态分配计算资源。我们提出的方法的有效性是通过数值模拟的验证,该案例研究是由无人机和自动驾驶车辆驱动的案例研究。
translated by 谷歌翻译
先进的可穿戴设备越来越多地利用高分辨率多摄像头系统。作为用于处理所得到的图像数据的最先进的神经网络是计算要求的,对于利用第五代(5G)无线连接和移动边缘计算,已经越来越感兴趣,以将该处理卸载到云。为了评估这种可能性,本文提出了一个详细的仿真和评估,用于5G无线卸载,用于对象检测,在一个名为Vis4ion的强大新型智能可穿戴物中,用于盲目损害(BVI)。目前的Vis4ion系统是一种具有高分辨率摄像机,视觉处理和触觉和音频反馈的仪表簿。本文认为将相机数据上载到移动边缘云以执行实时对象检测并将检测结果传输回可穿戴。为了确定视频要求,纸张评估视频比特率和分辨率对物体检测精度和范围的影响。利用与BVI导航相关的标记对象的新街道场景数据集进行分析。视觉评估与详细的全堆栈无线网络仿真结合,以确定吞吐量的分布和延迟,具有来自城市环境中的新高分辨率3D模型的实际导航路径和射线跟踪。为了比较,无线仿真考虑了标准的4G长期演进(LTE)载波和高速度5G毫米波(MMWAVE)载波。因此,该工作提供了对具有高带宽和低延迟要求的应用中的MMWAVE连接的边缘计算的彻底和现实评估。
translated by 谷歌翻译
未来的互联网涉及几种新兴技术,例如5G和5G网络,车辆网络,无人机(UAV)网络和物联网(IOT)。此外,未来的互联网变得异质并分散了许多相关网络实体。每个实体可能需要做出本地决定,以在动态和不确定的网络环境下改善网络性能。最近使用标准学习算法,例如单药强化学习(RL)或深入强化学习(DRL),以使每个网络实体作为代理人通过与未知环境进行互动来自适应地学习最佳决策策略。但是,这种算法未能对网络实体之间的合作或竞争进行建模,而只是将其他实体视为可能导致非平稳性问题的环境的一部分。多机构增强学习(MARL)允许每个网络实体不仅观察环境,还可以观察其他实体的政策来学习其最佳政策。结果,MAL可以显着提高网络实体的学习效率,并且最近已用于解决新兴网络中的各种问题。在本文中,我们因此回顾了MAL在新兴网络中的应用。特别是,我们提供了MARL的教程,以及对MARL在下一代互联网中的应用进行全面调查。特别是,我们首先介绍单代机Agent RL和MARL。然后,我们回顾了MAL在未来互联网中解决新兴问题的许多应用程序。这些问题包括网络访问,传输电源控制,计算卸载,内容缓存,数据包路由,无人机网络的轨迹设计以及网络安全问题。
translated by 谷歌翻译
In this tutorial paper, we look into the evolution and prospect of network architecture and propose a novel conceptual architecture for the 6th generation (6G) networks. The proposed architecture has two key elements, i.e., holistic network virtualization and pervasive artificial intelligence (AI). The holistic network virtualization consists of network slicing and digital twin, from the aspects of service provision and service demand, respectively, to incorporate service-centric and user-centric networking. The pervasive network intelligence integrates AI into future networks from the perspectives of networking for AI and AI for networking, respectively. Building on holistic network virtualization and pervasive network intelligence, the proposed architecture can facilitate three types of interplay, i.e., the interplay between digital twin and network slicing paradigms, between model-driven and data-driven methods for network management, and between virtualization and AI, to maximize the flexibility, scalability, adaptivity, and intelligence for 6G networks. We also identify challenges and open issues related to the proposed architecture. By providing our vision, we aim to inspire further discussions and developments on the potential architecture of 6G.
translated by 谷歌翻译
Unmanned aerial vehicle (UAV) swarms are considered as a promising technique for next-generation communication networks due to their flexibility, mobility, low cost, and the ability to collaboratively and autonomously provide services. Distributed learning (DL) enables UAV swarms to intelligently provide communication services, multi-directional remote surveillance, and target tracking. In this survey, we first introduce several popular DL algorithms such as federated learning (FL), multi-agent Reinforcement Learning (MARL), distributed inference, and split learning, and present a comprehensive overview of their applications for UAV swarms, such as trajectory design, power control, wireless resource allocation, user assignment, perception, and satellite communications. Then, we present several state-of-the-art applications of UAV swarms in wireless communication systems, such us reconfigurable intelligent surface (RIS), virtual reality (VR), semantic communications, and discuss the problems and challenges that DL-enabled UAV swarms can solve in these applications. Finally, we describe open problems of using DL in UAV swarms and future research directions of DL enabled UAV swarms. In summary, this survey provides a comprehensive survey of various DL applications for UAV swarms in extensive scenarios.
translated by 谷歌翻译
最近,使用卷积神经网络(CNNS)存在移动和嵌入式应用的爆炸性增长。为了减轻其过度的计算需求,开发人员传统上揭示了云卸载,突出了高基础设施成本以及对网络条件的强烈依赖。另一方面,强大的SOC的出现逐渐启用设备执行。尽管如此,低端和中层平台仍然努力充分运行最先进的CNN。在本文中,我们展示了Dyno,一种分布式推断框架,将两全其人的最佳框架结合起来解决了几个挑战,例如设备异质性,不同的带宽和多目标要求。启用这是其新的CNN特定数据包装方法,其在onloading计算时利用CNN的不同部分的精度需求的可变性以及其新颖的调度器,该调度器共同调谐分区点并在运行时传输数据精度适应其执行环境的推理。定量评估表明,Dyno优于当前最先进的,通过竞争对手的CNN卸载系统,在竞争对手的CNN卸载系统上提高吞吐量超过一个数量级,最高可达60倍的数据。
translated by 谷歌翻译
The ubiquity of camera-embedded devices and the advances in deep learning have stimulated various intelligent mobile video applications. These applications often demand on-device processing of video streams to deliver real-time, high-quality services for privacy and robustness concerns. However, the performance of these applications is constrained by the raw video streams, which tend to be taken with small-aperture cameras of ubiquitous mobile platforms in dim light. Despite extensive low-light video enhancement solutions, they are unfit for deployment to mobile devices due to their complex models and and ignorance of system dynamics like energy budgets. In this paper, we propose AdaEnlight, an energy-aware low-light video stream enhancement system on mobile devices. It achieves real-time video enhancement with competitive visual quality while allowing runtime behavior adaptation to the platform-imposed dynamic energy budgets. We report extensive experiments on diverse datasets, scenarios, and platforms and demonstrate the superiority of AdaEnlight compared with state-of-the-art low-light image and video enhancement solutions.
translated by 谷歌翻译
通过在图像传感器设计中加入可编程的兴趣区域(ROI)读数来提高嵌入式视觉系统的能量效率的巨大范围。在这项工作中,我们研究如何利用ROI可编程性,以便通过预期ROI将位于未来帧中的位置并在该区域之外切换像素来进行跟踪应用程序。我们将ROI预测的该过程和对应的传感器配置称为自适应限制。我们的自适应数据采样算法包括对象检测器和ROI预测器(卡尔曼滤波器),其结合地操作以优化视觉管道的能量效率,其结束任务是对象跟踪。为了进一步促进现实生活中的自适应算法的实施,我们选择候选算法并将其映射到FPGA上。利用Xilinx血管AI工具,我们设计并加速了基于YOLO对象探测器的自适应数据采样算法。为了进一步改进算法的部署后,我们在OTB100和LASOT数据集中评估了几个竞争的基线。我们发现将ECO跟踪器与Kalman滤波器耦合,在OTB100和Lasot Datasets上具有0.4568和0.3471的竞争性AUC分数。此外,该算法的功率效率与另一个基线优于相同的情况,并且在几个外部的情况下。基于ECO的算法在两个数据集上发生大约4W的功耗,而基于YOLO的方法需要大约6 W的功耗(根据我们的功耗模型)。在精度延迟权衡方面,基于ECO的算法在管理达到竞争跟踪精度的同时提供近实时性能(19.23 FPS)。
translated by 谷歌翻译
最近,通过协作推断部署深神经网络(DNN)模型,该推断将预训练的模型分为两个部分,并分别在用户设备(UE)和Edge Server上执行它们,从而变得有吸引力。但是,DNN的大型中间特征会阻碍灵活的脱钩,现有方法要么集中在单个UE方案上,要么只是在考虑所需的CPU周期的情况下定义任务,但忽略了单个DNN层的不可分割性。在本文中,我们研究了多代理协作推理方案,其中单个边缘服务器协调了多个UES的推理。我们的目标是为所有UES实现快速和节能的推断。为了实现这一目标,我们首先设计了一种基于自动编码器的轻型方法,以压缩大型中间功能。然后,我们根据DNN的推理开销定义任务,并将问题作为马尔可夫决策过程(MDP)。最后,我们提出了一种多代理混合近端策略优化(MAHPPO)算法,以解决混合动作空间的优化问题。我们对不同类型的网络进行了广泛的实验,结果表明,我们的方法可以降低56%的推理潜伏期,并节省多达72 \%的能源消耗。
translated by 谷歌翻译
尽管深度神经网络(DNN)已成为多个无处不在的应用程序的骨干技术,但它们在资源受限的机器中的部署,例如物联网(IoT)设备,仍然具有挑战性。为了满足这种范式的资源要求,引入了与IoT协同作用的深入推断。但是,DNN网络的分布遭受严重的数据泄漏。已经提出了各种威胁,包括黑盒攻击,恶意参与者可以恢复送入其设备的任意输入。尽管许多对策旨在实现隐私的DNN,但其中大多数会导致额外的计算和较低的准确性。在本文中,我们提出了一种方法,该方法通过重新考虑分配策略而无需牺牲模型性能来针对协作深度推断的安全性。特别是,我们检查了使该模型容易受到黑盒威胁的不同DNN分区,并得出了应分配每个设备的数据量以隐藏原始输入的所有权。我们将这种方法制定为一种优化,在该方法中,我们在共同推导的延迟与数据级别的数据级别之间建立了权衡。接下来,为了放大最佳解决方案,我们将方法塑造为支持异质设备以及多个DNN/数据集的增强学习(RL)设计。
translated by 谷歌翻译
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems.This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry.The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.
translated by 谷歌翻译