The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacy-protecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them.
translated by 谷歌翻译
People living with dementia often exhibit behavioural and psychological symptoms of dementia that can put their and others' safety at risk. Existing video surveillance systems in long-term care facilities can be used to monitor such behaviours of risk to alert the staff to prevent potential injuries or death in some cases. However, these behaviours of risk events are heterogeneous and infrequent in comparison to normal events. Moreover, analyzing raw videos can also raise privacy concerns. In this paper, we present two novel privacy-protecting video-based anomaly detection approaches to detect behaviours of risks in people with dementia. We either extracted body pose information as skeletons and use semantic segmentation masks to replace multiple humans in the scene with their semantic boundaries. Our work differs from most existing approaches for video anomaly detection that focus on appearance-based features, which can put the privacy of a person at risk and is also susceptible to pixel-based noise, including illumination and viewing direction. We used anonymized videos of normal activities to train customized spatio-temporal convolutional autoencoders and identify behaviours of risk as anomalies. We show our results on a real-world study conducted in a dementia care unit with patients with dementia, containing approximately 21 hours of normal activities data for training and 9 hours of data containing normal and behaviours of risk events for testing. We compared our approaches with the original RGB videos and obtained an equivalent area under the receiver operating characteristic curve performance of 0.807 for the skeleton-based approach and 0.823 for the segmentation mask-based approach. This is one of the first studies to incorporate privacy for the detection of behaviours of risks in people with dementia.
translated by 谷歌翻译
深度学习模型已广泛用于监控视频中的异常检测。典型模型配备了重建普通视频的能力,并评估异常视频的重建错误以指示异常的程度。然而,现有方法遭受了两个缺点。首先,它们只能独立地编码每个身份的运动,而不考虑身份之间的相互作用,这也可以指示异常。其次,他们利用了结构在不同场景下固定的粘合模型,这种配置禁止了对场景的理解。在本文中,我们提出了一个分层时空图卷积神经网络(HSTGCNN)来解决这些问题,HSTGCNN由对应于不同级别的图形表示的多个分支组成。高级图形表示编码人们的轨迹以及多个身份之间的交互,而低级图表表示编码每个人的本地身体姿势。此外,我们建议加权组合在不同场景中更好的多个分支。以这种方式实现了对单级图形表示的改进。实现了对场景的理解并提供异常检测。在低分辨率视频中为在低分辨率视频中编码低分辨率视频中的人员的移动速度和方向编码高级别的图表表示,而在高分辨率视频中将更高的权重分配更高的权重。实验结果表明,建议的HSTGCNN在四个基准数据集(UCSD Spistrian,Shanghaitech,Cuhk Aveance和IITB-Whent)上的当前最先进的模型显着优于最新的最先进模型。
translated by 谷歌翻译
视频异常检测是视觉中的核心问题。正确检测和识别视频数据中行人中的异常行为将使安全至关重要的应用,例如监视,活动监测和人类机器人的互动。在本文中,我们建议利用无监督的行人异常事件检测的轨迹定位和预测。与以前的基于重建的方法不同,我们提出的框架依赖于正常和异常行人轨迹的预测误差来在空间和时间上检测异常。我们介绍了有关不同时间尺度的现实基准数据集的实验结果,并表明我们提出的基于轨迹预言的异常检测管道在识别视频中行人的异常活动方面有效有效。代码将在https://github.com/akanuasiegbu/leveraging-trajectory-prediction-for-pedestrian-video-anomaly-detection上提供。
translated by 谷歌翻译
In recent years, we have seen a significant interest in data-driven deep learning approaches for video anomaly detection, where an algorithm must determine if specific frames of a video contain abnormal behaviors. However, video anomaly detection is particularly context-specific, and the availability of representative datasets heavily limits real-world accuracy. Additionally, the metrics currently reported by most state-of-the-art methods often do not reflect how well the model will perform in real-world scenarios. In this article, we present the Charlotte Anomaly Dataset (CHAD). CHAD is a high-resolution, multi-camera anomaly dataset in a commercial parking lot setting. In addition to frame-level anomaly labels, CHAD is the first anomaly dataset to include bounding box, identity, and pose annotations for each actor. This is especially beneficial for skeleton-based anomaly detection, which is useful for its lower computational demand in real-world settings. CHAD is also the first anomaly dataset to contain multiple views of the same scene. With four camera views and over 1.15 million frames, CHAD is the largest fully annotated anomaly detection dataset including person annotations, collected from continuous video streams from stationary cameras for smart video surveillance applications. To demonstrate the efficacy of CHAD for training and evaluation, we benchmark two state-of-the-art skeleton-based anomaly detection algorithms on CHAD and provide comprehensive analysis, including both quantitative results and qualitative examination.
translated by 谷歌翻译
视频异常检测是现在计算机视觉中的热门研究主题之一,因为异常事件包含大量信息。异常是监控系统中的主要检测目标之一,通常需要实时行动。关于培训的标签数据的可用性(即,没有足够的标记数据进行异常),半监督异常检测方法最近获得了利益。本文介绍了该领域的研究人员,以新的视角,并评论了最近的基于深度学习的半监督视频异常检测方法,基于他们用于异常检测的共同策略。我们的目标是帮助研究人员开发更有效的视频异常检测方法。由于选择右深神经网络的选择对于这项任务的几个部分起着重要作用,首先准备了对DNN的快速比较审查。与以前的调查不同,DNN是从时空特征提取观点审查的,用于视频异常检测。这部分审查可以帮助本领域的研究人员选择合适的网络,以获取其方法的不同部分。此外,基于其检测策略,一些最先进的异常检测方法受到严格调查。审查提供了一种新颖,深入了解现有方法,并导致陈述这些方法的缺点,这可能是未来作品的提示。
translated by 谷歌翻译
人类堕落是非常关键的健康问题之一,尤其是对于长老和残疾人而言。在全球范围内,老年人口的数量正在稳步增加。因此,人类的跌倒发现已成为为这些人辅助生活的有效技术。为了辅助生活,大量使用了深度学习和计算机视觉。在这篇评论文章中,我们讨论了基于深度学习(DL)的最先进的非侵入性(基于视觉的)秋季检测技术。我们还提出了有关秋季检测基准数据集的调查。为了清楚理解,我们简要讨论用于评估秋季检测系统性能的不同指标。本文还为基于视觉的人类跌落检测技术提供了未来的指导。
translated by 谷歌翻译
Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of deep anomaly detection in time series across various application domains in recent years. It finally summarises open issues in research and challenges faced while adopting deep anomaly detection models.
translated by 谷歌翻译
在当代社会中,监视异常检测,即在监视视频中发现异常事件,例如犯罪或事故,是一项关键任务。由于异常发生很少发生,大多数培训数据包括没有标记的视频,没有异常事件,这使得任务具有挑战性。大多数现有方法使用自动编码器(AE)学习重建普通视频;然后,他们根据未能重建异常场景的出现来检测异常。但是,由于异常是通过外观和运动来区分的,因此许多先前的方法使用预训练的光流模型明确分开了外观和运动信息,例如。这种明确的分离限制了两种类型的信息之间的相互表示功能。相比之下,我们提出了一个隐式的两路AE(ITAE),其中两个编码器隐含模型外观和运动特征以及一个将它们组合在一起以学习正常视频模式的结构。对于正常场景的复杂分布,我们建议通过归一化流量(NF)的生成模型对ITAE特征的正常密度估计,以学习可拖动的可能性,并使用无法分布的检测来识别异常。 NF模型通过隐式学习的功能通过学习正常性来增强ITAE性能。最后,我们在六个基准测试中演示了ITAE及其特征分布建模的有效性,包括在现实世界中包含各种异常的数据库。
translated by 谷歌翻译
我们提出了Bipoco,这是一种带有姿势限制的双向轨迹预测指标,用于检测视频中行人的异常活动。与基于特征重建的先前工作相反,我们的工作通过预测他们的未来轨迹并将预测与他们的期望进行比较来确定行人异常事件。我们引入了一组新型的基于姿势的损失,并通过我们的预测指标和利用每个身体关节的预测误差来进行行人异常检测。实验结果表明,我们的Bipoco方法可以检测具有高检测率的行人异常活动(高达87.0%),并且纳入姿势限制有助于区分预测中的正常和异常姿势。这项工作扩展了使用基于预测的方法进行异常检测的当前文献,并可以受益于安全至关重要的应用,例如自动驾驶和监视。代码可从https://github.com/akanuasiegbu/bipoco获得。
translated by 谷歌翻译
对于由硬件和软件组件组成的复杂分布式系统而言,异常检测是一个重要的问题。对此类系统的异常检测的要求和挑战的透彻理解对于系统的安全性至关重要,尤其是对于现实世界的部署。尽管有许多解决问题的研究领域和应用领域,但很少有人试图对这种系统进行深入研究。大多数异常检测技术是针对某些应用域的专门开发的,而其他检测技术则更为通用。在这项调查中,我们探讨了基于图的算法在复杂分布式异质系统中识别和减轻不同类型异常的重要潜力。我们的主要重点是在分布在复杂分布式系统上的异质计算设备上应用时,可深入了解图。这项研究分析,比较和对比该领域的最新研究文章。首先,我们描述了现实世界分布式系统的特征及其在复杂网络中的异常检测的特定挑战,例如数据和评估,异常的性质以及现实世界的要求。稍后,我们讨论了为什么可以在此类系统中利用图形以及使用图的好处。然后,我们将恰当地深入研究最先进的方法,并突出它们的优势和劣势。最后,我们评估和比较这些方法,并指出可能改进的领域。
translated by 谷歌翻译
Surveillance videos are able to capture a variety of realistic anomalies. In this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i.e. the training labels (anomalous or normal) are at videolevel instead of clip-level. In our approach, we consider normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL), and automatically learn a deep anomaly ranking model that predicts high anomaly scores for anomalous video segments. Furthermore, we introduce sparsity and temporal smoothness constraints in the ranking loss function to better localize anomaly during training.We also introduce a new large-scale first of its kind dataset of 128 hours of videos. It consists of 1900 long and untrimmed real-world surveillance videos, with 13 realistic anomalies such as fighting, road accident, burglary, robbery, etc. as well as normal activities. This dataset can be used for two tasks. First, general anomaly detection considering all anomalies in one group and all normal activities in another group. Second, for recognizing each of 13 anomalous activities. Our experimental results show that our MIL method for anomaly detection achieves significant improvement on anomaly detection performance as compared to the state-of-the-art approaches. We provide the results of several recent deep learning baselines on anomalous activity recognition. The low recognition performance of these baselines reveals that our dataset is very challenging and opens more opportunities for future work. The dataset is
translated by 谷歌翻译
由于价格合理的可穿戴摄像头和大型注释数据集的可用性,在过去几年中,Egintric Vision(又名第一人称视觉-FPV)的应用程序在过去几年中蓬勃发展。可穿戴摄像机的位置(通常安装在头部上)允许准确记录摄像头佩戴者在其前面的摄像头,尤其是手和操纵物体。这种内在的优势可以从多个角度研究手:将手及其部分定位在图像中;了解双手涉及哪些行动和活动;并开发依靠手势的人类计算机界面。在这项调查中,我们回顾了使用以自我为中心的愿景专注于手的文献,将现有方法分类为:本地化(其中的手或部分在哪里?);解释(手在做什么?);和应用程序(例如,使用以上为中心的手提示解决特定问题的系统)。此外,还提供了带有手基注释的最突出的数据集的列表。
translated by 谷歌翻译
视频异常分析是在计算机视觉领域积极执行的一项核心任务,其应用程序扩展到了监视录像中现实世界中的犯罪检测。在这项工作中,我们解决了与人有关的犯罪分类的任务。在我们提出的方法中,用作骨骼关节轨迹的视频框架中的人体被用作探索的主要来源。首先,我们介绍了扩展HR-Crime数据集的地面真相标签的意义,因此提出了一种监督和无监督的方法,以生成轨迹级别的地面真相标签。接下来,鉴于轨迹级的地面真相的可用性,我们引入了基于轨迹的犯罪分类框架。消融研究是通过各种体系结构和特征融合策略来代表人类轨迹进行的。进行的实验证明了任务的可行性,并为该领域的进一步研究铺平了道路。
translated by 谷歌翻译
异常识别高度取决于对象与场景之间的关系,因为相同/不同场景中的不同/相同对象动作可能导致各种程度的正态性和异常。因此,对象场景关系实际上在异常检测中起着至关重要的作用,但在以前的工作中探讨了不足。在本文中,我们提出了一个时空关系学习(STRL)框架来解决视频异常检测任务。首先,考虑到对象的动态特征以及场景区域,我们构建了一个时空自动编码器(STAE),以共同利用代表学习的空间和时间演化模式。为了获得更好的图案提取,在STAE模块中设计了两个解码分支,即通过直接预测下一个帧来捕获空间提示的外观分支,以及一个运动分支,重点是通过光流预测对动态进行建模。然后,为了很好地融合对象场所关系,设计了一个关系学习(RL)模块来通过引入知识图嵌入方法来分析和总结正常关系。在此过程中具体来说,通过共同建模对象/场景特征和优化的对象场所关系图来衡量对象场景关系的合理性。在三个公共数据集上进行了广泛的实验,而对最新方法的优越性能证明了我们方法的有效性。
translated by 谷歌翻译
单个异常行为因人群的大小,上下文和场景而异。当检测,跟踪和认可异常行为的人时,诸如部分阻塞,模糊,大数字异常行为和摄像机观看之类的挑战发生在大规模的人群中。在本文中,我们的贡献是双重的。首先,我们介绍了一个注释和标记的大规模人群异常行为hajj数据集(hajjv2)。其次,我们提出了两种混合卷积神经网络(CNN)和随机森林(RFS)的两种方法,以检测和识别小型和大型人群视频中的时空异常行为。在小型人群视频中,对Resnet-50预训练的CNN模型进行了微调,以验证空间域中的每个帧是正常还是异常。如果观察到异常行为,则使用基于运动的个体检测方法基于角链光流的大小和方向来定位和跟踪具有异常行为的个体。大规模人群视频中使用了Kalman过滤器,以预测和跟踪随后的帧中检测到的个体。然后,将均值,方差和标准偏差统计特征计算出来并馈送到RF,以对时间域中的行为异常行为进行分类。在大规模的人群中,我们使用Yolov2对象检测技术微调Resnet-50模型,以检测空间域中行为异常的个体。
translated by 谷歌翻译
Object movement identification is one of the most researched problems in the field of computer vision. In this task, we try to classify a pixel as foreground or background. Even though numerous traditional machine learning and deep learning methods already exist for this problem, the two major issues with most of them are the need for large amounts of ground truth data and their inferior performance on unseen videos. Since every pixel of every frame has to be labeled, acquiring large amounts of data for these techniques gets rather expensive. Recently, Zhao et al. [1] proposed one of a kind Arithmetic Distribution Neural Network (ADNN) for universal background subtraction which utilizes probability information from the histogram of temporal pixels and achieves promising results. Building onto this work, we developed an intelligent video surveillance system that uses ADNN architecture for motion detection, trims the video with parts only containing motion, and performs anomaly detection on the trimmed video.
translated by 谷歌翻译
Computer vision applications in intelligent transportation systems (ITS) and autonomous driving (AD) have gravitated towards deep neural network architectures in recent years. While performance seems to be improving on benchmark datasets, many real-world challenges are yet to be adequately considered in research. This paper conducted an extensive literature review on the applications of computer vision in ITS and AD, and discusses challenges related to data, models, and complex urban environments. The data challenges are associated with the collection and labeling of training data and its relevance to real world conditions, bias inherent in datasets, the high volume of data needed to be processed, and privacy concerns. Deep learning (DL) models are commonly too complex for real-time processing on embedded hardware, lack explainability and generalizability, and are hard to test in real-world settings. Complex urban traffic environments have irregular lighting and occlusions, and surveillance cameras can be mounted at a variety of angles, gather dirt, shake in the wind, while the traffic conditions are highly heterogeneous, with violation of rules and complex interactions in crowded scenarios. Some representative applications that suffer from these problems are traffic flow estimation, congestion detection, autonomous driving perception, vehicle interaction, and edge computing for practical deployment. The possible ways of dealing with the challenges are also explored while prioritizing practical deployment.
translated by 谷歌翻译
当今智能城市中产生的大型视频数据从其有目的的用法角度引起了人们的关注,其中监视摄像机等是最突出的资源,是为大量数据做出贡献的最突出的资源,使其自动化分析成为计算方面的艰巨任务。和精确。暴力检测(VD)在行动和活动识别域中广泛崩溃,用于分析大型视频数据,以了解由于人类而引起的异常动作。传统上,VD文献基于手动设计的功能,尽管开发了基于深度学习的独立模型的进步用于实时VD分析。本文重点介绍了深度序列学习方法以及检测到的暴力的本地化策略。该概述还介入了基于机器学习的初始图像处理和基于机器学习的文献及其可能具有的优势,例如针对当前复杂模型的效率。此外,讨论了数据集,以提供当前模型的分析,并用对先前方法的深入分析得出的VD域中的未来方向解释了他们的利弊。
translated by 谷歌翻译
多媒体异常数据集在自动监视中发挥着至关重要的作用。它们具有广泛的应用程序,从异常对象/情况检测到检测危及生命事件的检测。该字段正在接收大量的1.5多年的巨大研究兴趣,因此,已经创建了越来越多地专用于异常动作和对象检测的数据集。点击这些公共异常数据集使研究人员能够生成和比较具有相同输入数据的各种异常检测框架。本文介绍了各种视频,音频以及基于异常检测的应用的综合调查。该调查旨在解决基于异常检测的多媒体公共数据集缺乏全面的比较和分析。此外,它可以帮助研究人员选择最佳可用数据集,用于标记框架。此外,我们讨论了现有数据集和未来方向洞察中开发多峰异常检测数据集的差距。
translated by 谷歌翻译