本研究专注于评估智能和安全车辆系统的热对象检测的实时性能，通过在GPU和单板边缘GPU计算平台上部署训练有素的网络进行车载汽车传感器套件测试。在充满挑战的天气和环境场景中，获取，加工和开放，包括具有> 35,000个不同框架的新型大规模热数据集。 DataSet是从丢失的成本且有效的未加工的LWIR热敏摄像机，安装独立和电动车辆中的记录，以最大限度地减少机械振动。最先进的YOLO-V5网络变体使用四个不同的公共数据集进行培训，也可以通过采用SGD优化器来实现DNN的最佳通用的本地数据集。培训网络的有效性在广泛的测试数据上使用了各种定量度量来验证，包括精度，召回曲线，平均精度和每秒帧。使用规特相关推理加速器进一步优化YOLO的较小网络变体，明确提高每秒速率的帧。在低功率边缘设备上测试时，优化的网络引擎在低功耗边缘设备上测试时，每秒速率增加3.5倍。在NVIDIA Jetson Nano和60 fps上的NVIDIA Xavier NX Development Landls上实现了11个FPS。

Neuromorphic vision or event vision is an advanced vision technology, where in contrast to the visible camera that outputs pixels, the event vision generates neuromorphic events every time there is a brightness change which exceeds a specific threshold in the field of view (FOV). This study focuses on leveraging neuromorphic event data for roadside object detection. This is a proof of concept towards building artificial intelligence (AI) based pipelines which can be used for forward perception systems for advanced vehicular applications. The focus is on building efficient state-of-the-art object detection networks with better inference results for fast-moving forward perception using an event camera. In this article, the event-simulated A2D2 dataset is manually annotated and trained on two different YOLOv5 networks (small and large variants). To further assess its robustness, single model testing and ensemble model testing are carried out.

自动化驾驶系统（广告）开辟了汽车行业的新领域，为未来的运输提供了更高的效率和舒适体验的新可能性。然而，在恶劣天气条件下的自主驾驶已经存在，使自动车辆（AVS）长时间保持自主车辆（AVS）或更高的自主权。本文评估了天气在分析和统计方式中为广告传感器带来的影响和挑战，并对恶劣天气条件进行了解决方案。彻底报道了关于对每种天气的感知增强的最先进技术。外部辅助解决方案如V2X技术，当前可用的数据集，模拟器和天气腔室的实验设施中的天气条件覆盖范围明显。通过指出各种主要天气问题，自主驾驶场目前正在面临，近年来审查硬件和计算机科学解决方案，这项调查概述了在不利的天气驾驶条件方面的障碍和方向的障碍和方向。

自动检测飞行无人机是一个关键问题，其存在（特别是未经授权）可以造成风险的情况或损害安全性。在这里，我们设计和评估了多传感器无人机检测系统。结合常见的摄像机和麦克风传感器，我们探索了热红外摄像机的使用，指出是一种可行且有希望的解决方案，在相关文献中几乎没有解决。我们的解决方案还集成了鱼眼相机，以监视天空的更大部分，并将其他摄像机转向感兴趣的对象。传感溶液与ADS-B接收器，GPS接收器和雷达模块相辅相成，尽管由于其有限的检测范围，后者未包含在我们的最终部署中。即使此处使用的摄像机的分辨率较低，热摄像机也被证明是与摄像机一样好的可行解决方案。我们作品的另外两个新颖性是创建一个新的公共数据集的多传感器注释数据，该数据与现有的类别相比扩大了类的数量，以及对探测器性能的研究作为传感器到传感器的函数的研究目标距离。还探索了传感器融合，表明可以以这种方式使系统更强大，从而减轻对单个传感器的虚假检测

Video, as a key driver in the global explosion of digital information, can create tremendous benefits for human society. Governments and enterprises are deploying innumerable cameras for a variety of applications, e.g., law enforcement, emergency management, traffic control, and security surveillance, all facilitated by video analytics (VA). This trend is spurred by the rapid advancement of deep learning (DL), which enables more precise models for object classification, detection, and tracking. Meanwhile, with the proliferation of Internet-connected devices, massive amounts of data are generated daily, overwhelming the cloud. Edge computing, an emerging paradigm that moves workloads and services from the network core to the network edge, has been widely recognized as a promising solution. The resulting new intersection, edge video analytics (EVA), begins to attract widespread attention. Nevertheless, only a few loosely-related surveys exist on this topic. A dedicated venue for collecting and summarizing the latest advances of EVA is highly desired by the community. Besides, the basic concepts of EVA (e.g., definition, architectures, etc.) are ambiguous and neglected by these surveys due to the rapid development of this domain. A thorough clarification is needed to facilitate a consensus on these concepts. To fill in these gaps, we conduct a comprehensive survey of the recent efforts on EVA. In this paper, we first review the fundamentals of edge computing, followed by an overview of VA. The EVA system and its enabling techniques are discussed next. In addition, we introduce prevalent frameworks and datasets to aid future researchers in the development of EVA systems. Finally, we discuss existing challenges and foresee future research directions. We believe this survey will help readers comprehend the relationship between VA and edge computing, and spark new ideas on EVA.

我们提出了一种新的四管齐下的方法，在文献中首次建立消防员的情境意识。我们构建了一系列深度学习框架，彼此之叠，以提高消防员在紧急首次响应设置中进行的救援任务的安全性，效率和成功完成。首先，我们使用深度卷积神经网络（CNN）系统，以实时地分类和识别来自热图像的感兴趣对象。接下来，我们将此CNN框架扩展了对象检测，跟踪，分割与掩码RCNN框架，以及具有多模级自然语言处理（NLP）框架的场景描述。第三，我们建立了一个深入的Q学习的代理，免受压力引起的迷失方向和焦虑，能够根据现场消防环境中观察和存储的事实来制定明确的导航决策。最后，我们使用了一种低计算无监督的学习技术，称为张量分解，在实时对异常检测进行有意义的特征提取。通过这些临时深度学习结构，我们建立了人工智能系统的骨干，用于消防员的情境意识。要将设计的系统带入消防员的使用，我们设计了一种物理结构，其中处理后的结果被用作创建增强现实的投入，这是一个能够建议他们所在地的消防员和周围的关键特征，这对救援操作至关重要在手头，以及路径规划功能，充当虚拟指南，以帮助迷彩的第一个响应者恢复安全。当组合时，这四种方法呈现了一种新颖的信息理解，转移和综合方法，这可能会大大提高消防员响应和功效，并降低寿命损失。

The last decade witnessed increasingly rapid progress in self-driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence. The objective of this paper is to survey the current state-of-the-art on deep learning technologies used in autonomous driving. We start by presenting AI-based self-driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration and motion control algorithms. We investigate both the modular perception-planning-action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources and computational hardware. The comparison presented in this survey helps to gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices. 1

Object detection models commonly deployed on uncrewed aerial systems (UAS) focus on identifying objects in the visible spectrum using Red-Green-Blue (RGB) imagery. However, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) images to increase the performance of object detection machine learning (ML) models. Currently LWIR ML models have received less research attention, especially for both ground- and air-based platforms, leading to a lack of baseline performance metrics evaluating LWIR, RGB and LWIR-RGB fused object detection models. Therefore, this research contributes such quantitative metrics to the literature .The results found that the ground-based blended RGB-LWIR model exhibited superior performance compared to the RGB or LWIR approaches, achieving a mAP of 98.4%. Additionally, the blended RGB-LWIR model was also the only object detection model to work in both day and night conditions, providing superior operational capabilities. This research additionally contributes a novel labelled training dataset of 12,600 images for RGB, LWIR, and RGB-LWIR fused imagery, collected from ground-based and air-based platforms, enabling further multispectral machine-driven object detection research.

计算机视觉在智能运输系统（ITS）和交通监视中发挥了重要作用。除了快速增长的自动化车辆和拥挤的城市外，通过实施深层神经网络的实施，可以使用视频监视基础架构进行自动和高级交通管理系统（ATM）。在这项研究中，我们为实时交通监控提供了一个实用的平台，包括3D车辆/行人检测，速度检测，轨迹估算，拥塞检测以及监视车辆和行人的相互作用，都使用单个CCTV交通摄像头。我们适应了定制的Yolov5深神经网络模型，用于车辆/行人检测和增强的排序跟踪算法。还开发了基于混合卫星的基于混合卫星的逆透视图（SG-IPM）方法，用于摄像机自动校准，从而导致准确的3D对象检测和可视化。我们还根据短期和长期的时间视频数据流开发了层次结构的交通建模解决方案，以了解脆弱道路使用者的交通流量，瓶颈和危险景点。关于现实世界情景和与最先进的比较的几项实验是使用各种交通监控数据集进行的，包括从高速公路，交叉路口和城市地区收集的MIO-TCD，UA-DETRAC和GRAM-RTM，在不同的照明和城市地区天气状况。

由于其前所未有的优势，在规模，移动，部署和隐蔽观察能力方面，空中平台和成像传感器的快速出现是实现新的空中监测形式。本文从计算机视觉和模式识别的角度来看，全面概述了以人为本的空中监控任务。它旨在为读者提供使用无人机，无人机和其他空中平台的空中监测任务当前状态的深入系统审查和技术分析。感兴趣的主要对象是人类，其中要检测单个或多个受试者，识别，跟踪，重新识别并进行其行为。更具体地，对于这四项任务中的每一个，我们首先讨论与基于地面的设置相比在空中环境中执行这些任务的独特挑战。然后，我们审查和分析公共可用于每项任务的航空数据集，并深入了解航空文学中的方法，并调查他们目前如何应对鸟瞰挑战。我们在讨论缺失差距和开放研究问题的讨论中得出结论，告知未来的研究途径。

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

Unmanned air vehicles (UAVs) popularity is on the rise as it enables the services like traffic monitoring, emergency communications, deliveries, and surveillance. However, the unauthorized usage of UAVs (a.k.a drone) may violate security and privacy protocols for security-sensitive national and international institutions. The presented challenges require fast, efficient, and precise detection of UAVs irrespective of harsh weather conditions, the presence of different objects, and their size to enable SafeSpace. Recently, there has been significant progress in using the latest deep learning models, but those models have shortcomings in terms of computational complexity, precision, and non-scalability. To overcome these limitations, we propose a precise and efficient multiscale and multifeature UAV detection network for SafeSpace, i.e., \textit{MultiFeatureNet} (\textit{MFNet}), an improved version of the popular object detection algorithm YOLOv5s. In \textit{MFNet}, we perform multiple changes in the backbone and neck of the YOLOv5s network to focus on the various small and ignored features required for accurate and fast UAV detection. To further improve the accuracy and focus on the specific situation and multiscale UAVs, we classify the \textit{MFNet} into small (S), medium (M), and large (L): these are the combinations of various size filters in the convolution and the bottleneckCSP layers, reside in the backbone and neck of the architecture. This classification helps to overcome the computational cost by training the model on a specific feature map rather than all the features. The dataset and code are available as an open source: github.com/ZeeshanKaleem/MultiFeatureNet.

自治机器人目前是最受欢迎的人工智能问题之一，在过去十年中，从自动驾驶汽车和人形系统到交付机器人和无人机，这是一项最受欢迎的智能问题。部分问题是获得一个机器人，以模仿人类的感知，我们的视觉感，用诸如神经网络等数学模型用相机和大脑的眼睛替换眼睛。开发一个能够在没有人为干预的情况下驾驶汽车的AI和一个小型机器人在城市中递送包裹可能看起来像不同的问题，因此来自感知和视觉的观点来看，这两个问题都有几种相似之处。我们目前的主要解决方案通过使用计算机视觉技术，机器学习和各种算法来实现对环境感知的关注，使机器人理解环境或场景，移动，调整其轨迹并执行其任务（维护，探索，等。）无需人为干预。在这项工作中，我们从头开始开发一个小型自动车辆，能够仅使用视觉信息理解场景，通过工业环境导航，检测人员和障碍，或执行简单的维护任务。我们审查了基本问题的最先进问题，并证明了小规模采用的许多方法类似于来自特斯拉或Lyft等公司的真正自动驾驶汽车中使用的方法。最后，我们讨论了当前的机器人和自主驾驶状态以及我们在这一领域找到的技术和道德限制。

自主车辆的环境感知受其物理传感器范围和算法性能的限制，以及通过降低其对正在进行的交通状况的理解的闭塞。这不仅构成了对安全和限制驾驶速度的重大威胁，而且它也可能导致不方便的动作。智能基础设施系统可以帮助缓解这些问题。智能基础设施系统可以通过在当前交通情况的数字模型的形式提供关于其周围环境的额外详细信息，填补了车辆的感知中的差距并扩展了其视野。数字双胞胎。然而，这种系统的详细描述和工作原型表明其可行性稀缺。在本文中，我们提出了一种硬件和软件架构，可实现这样一个可靠的智能基础架构系统。我们在现实世界中实施了该系统，并展示了它能够创建一个准确的延伸高速公路延伸的数字双胞胎，从而提高了自主车辆超越其车载传感器的极限的感知。此外，我们通过使用空中图像和地球观测方法来评估数字双胞胎的准确性和可靠性，用于产生地面真理数据。

信号处理是几乎任何传感器系统的基本组件，具有不同科学学科的广泛应用。时间序列数据，图像和视频序列包括可以增强和分析信息提取和量化的代表性形式的信号。人工智能和机器学习的最近进步正在转向智能，数据驱动，信号处理的研究。该路线图呈现了最先进的方法和应用程序的关键概述，旨在突出未来的挑战和对下一代测量系统的研究机会。它涵盖了广泛的主题，从基础到工业研究，以简明的主题部分组织，反映了每个研究领域的当前和未来发展的趋势和影响。此外，它为研究人员和资助机构提供了识别新前景的指导。

对象检测管道包括一个捕获场景的相机和处理这些图像的对象检测器。图像的质量直接影响对象检测器的性能。如今，许多工作重点是改善图像质量或独立改善对象检测模型，但忽略了两个子系统联合优化的重要性。本文的目的是通过专注于优化针对对象检测器量身定制的输入图像来调整遥感方案中现有对象检测器的检测吞吐量和准确性。为了实现这一目标，我们经验分析了两个选择的摄像机校准参数（摄像机失真校正和伽马校正）和五个图像参数（量化，压缩，分辨率，颜色模型，其他通道）的影响。对于我们的实验，我们利用来自不同域中的三个无人机数据集，以及大小的最新对象检测器模型的混合物，可对管道参数的影响进行广泛的评估。最后，我们在嵌入式平台上实现了一个对象检测管道原型，以便根据我们的发现为构建对象检测管道提供最佳练习建议。我们表明，并非所有参数都对检测准确性和数据吞吐量都有平等的影响，并且通过在参数之间使用合适的折衷方案，我们能够为轻质对象检测模型实现更高的检测精度，同时保持相同的数据吞吐量。

通过在图像传感器设计中加入可编程的兴趣区域（ROI）读数来提高嵌入式视觉系统的能量效率的巨大范围。在这项工作中，我们研究如何利用ROI可编程性，以便通过预期ROI将位于未来帧中的位置并在该区域之外切换像素来进行跟踪应用程序。我们将ROI预测的该过程和对应的传感器配置称为自适应限制。我们的自适应数据采样算法包括对象检测器和ROI预测器（卡尔曼滤波器），其结合地操作以优化视觉管道的能量效率，其结束任务是对象跟踪。为了进一步促进现实生活中的自适应算法的实施，我们选择候选算法并将其映射到FPGA上。利用Xilinx血管AI工具，我们设计并加速了基于YOLO对象探测器的自适应数据采样算法。为了进一步改进算法的部署后，我们在OTB100和LASOT数据集中评估了几个竞争的基线。我们发现将ECO跟踪器与Kalman滤波器耦合，在OTB100和Lasot Datasets上具有0.4568和0.3471的竞争性AUC分数。此外，该算法的功率效率与另一个基线优于相同的情况，并且在几个外部的情况下。基于ECO的算法在两个数据集上发生大约4W的功耗，而基于YOLO的方法需要大约6 W的功耗（根据我们的功耗模型）。在精度延迟权衡方面，基于ECO的算法在管理达到竞争跟踪精度的同时提供近实时性能（19.23 FPS）。

已经提出了高效和自适应计算机视觉系统以使计算机视觉任务，例如图像分类和对象检测，针对嵌入或移动设备进行了优化。这些解决方案最近的起源，专注于通过设计具有近似旋钮的自适应系统来优化模型（深神经网络，DNN）或系统。尽管最近的几项努力，但我们表明现有解决方案遭受了两个主要缺点。首先，系统不考虑模型的能量消耗，同时在制定要运行的模型的决定时。其次，由于其他共同居民工作负载，评估不考虑设备上的争用的实际情况。在这项工作中，我们提出了一种高效和自适应的视频对象检测系统，这是联合优化的精度，能量效率和延迟。底层Virtuoso是一个多分支执行内核，它能够在精度 - 能量 - 延迟轴上的不同运行点处运行，以及轻量级运行时调度程序，以选择最佳的执行分支以满足用户要求。要与Virtuoso相当比较，我们基准于15件最先进的或广泛使用的协议，包括更快的R-CNN（FRCNN），YOLO V3，SSD，培训台，SELSA，MEGA，REPP，FastAdapt和我们的内部FRCNN +，YOLO +，SSD +和高效+（我们的变体具有增强的手机效率）的自适应变体。通过这种全面的基准，Virtuoso对所有上述协议显示出优势，在NVIDIA Jetson Mobile GPU上的每一项效率水平上引领精度边界。具体而言，Virtuoso的准确性为63.9％，比一些流行的物体检测模型高于10％，51.1％，yolo为49.5％。

Multi-modal fusion is a basic task of autonomous driving system perception, which has attracted many scholars' interest in recent years. The current multi-modal fusion methods mainly focus on camera data and LiDAR data, but pay little attention to the kinematic information provided by the bottom sensors of the vehicle, such as acceleration, vehicle speed, angle of rotation. These information are not affected by complex external scenes, so it is more robust and reliable. In this paper, we introduce the existing application fields of vehicle bottom information and the research progress of related methods, as well as the multi-modal fusion methods based on bottom information. We also introduced the relevant information of the vehicle bottom information data set in detail to facilitate the research as soon as possible. In addition, new future ideas of multi-modal fusion technology for autonomous driving tasks are proposed to promote the further utilization of vehicle bottom information.

Computer vision applications in intelligent transportation systems (ITS) and autonomous driving (AD) have gravitated towards deep neural network architectures in recent years. While performance seems to be improving on benchmark datasets, many real-world challenges are yet to be adequately considered in research. This paper conducted an extensive literature review on the applications of computer vision in ITS and AD, and discusses challenges related to data, models, and complex urban environments. The data challenges are associated with the collection and labeling of training data and its relevance to real world conditions, bias inherent in datasets, the high volume of data needed to be processed, and privacy concerns. Deep learning (DL) models are commonly too complex for real-time processing on embedded hardware, lack explainability and generalizability, and are hard to test in real-world settings. Complex urban traffic environments have irregular lighting and occlusions, and surveillance cameras can be mounted at a variety of angles, gather dirt, shake in the wind, while the traffic conditions are highly heterogeneous, with violation of rules and complex interactions in crowded scenarios. Some representative applications that suffer from these problems are traffic flow estimation, congestion detection, autonomous driving perception, vehicle interaction, and edge computing for practical deployment. The possible ways of dealing with the challenges are also explored while prioritizing practical deployment.