在单光子激光雷达,光子效率成像捕捉所述3D场景的由每个像素只几个检测到的信号的光子结构。此任务的现有深度学习模型被训练在模拟数据集,当应用到现实的情景,这对域转移的挑战。在本文中,我们提出了一种时空以来网络(STIN)用于光子效率成像,这是能够通过充分利用空间和时间信息精确地预测从稀疏和高噪声光子计数直方图的深度。然后,域对抗性适应框架,包括域对抗性神经网络和对抗性判别域适应,被有效地应用于STIN缓解域移位问题对于实际应用。从NYU〜v2和所述数据集Middlebury的所产生的模拟数据综合实验证明STIN优于国家的最先进的模型在低信号 - 背景比为2:10至2:100。此外,在由该单光子成像原型显示,相比与域对抗性训练STIN取得了较好的推广性能捕捉到的真实世界的数据集实验结果的国家的最艺术以及由模拟数据训练基线STIN 。
translated by 谷歌翻译
单光子光检测和测距(LIDAR)已广泛应用于挑战性方案的3D成像。然而,在收集的数据中有限的信号光子计数和高噪声对预测深度图像精确地构成了巨大的挑战。在本文中,我们提出了一种用于从高噪声数据的光子有效成像的像素 - 方面的剩余收缩网络,其自适应地产生每个像素的最佳阈值,并通过软阈值处理来剥夺中间特征。此外,重新定义优化目标作为像素明智的分类,提供了与现有研究相比产生自信和准确的深度估计的急剧优势。在模拟和现实世界数据集中进行的综合实验表明,所提出的模型优于现有技术,并在不同的信噪比下保持鲁棒成像性能,包括1:100的极端情况。
translated by 谷歌翻译
雨是最常见的天气之一,可以完全降低图像质量并干扰许多计算机视觉任务的执行,尤其是在大雨条件下。我们观察到:(i)雨是雨水和雨淋的混合物; (ii)场景的深度决定了雨条的强度以及变成多雨的阴霾的强度; (iii)大多数现有的DERANE方法仅在合成雨图像上进行训练,因此对现实世界的场景概括不佳。在这些观察结果的激励下,我们提出了一种新的半监督,清除降雨生成的对抗网络(半密集),该混合物由四个关键模块组成:(i)新的注意力深度预测网络以提供精确的深度估计; (ii)上下文特征预测网络由几个精心设计的详细残留块组成,以产生详细的图像上下文特征; (iii)金字塔深度引导的非本地网络,以有效地将图像上下文与深度信息整合在一起,并产生最终的无雨图像; (iv)全面的半监督损失函数,使该模型不限于合成数据集,而是平稳地将其概括为现实世界中的大雨场景。广泛的实验表明,在合成和现实世界中,我们的二十多种代表性的最先进的方法对我们的方法进行了明显的改进。
translated by 谷歌翻译
深度完成旨在预测从深度传感器(例如Lidars)中捕获的极稀疏图的密集像素深度。它在各种应用中起着至关重要的作用,例如自动驾驶,3D重建,增强现实和机器人导航。基于深度学习的解决方案已经证明了这项任务的最新成功。在本文中,我们首次提供了全面的文献综述,可帮助读者更好地掌握研究趋势并清楚地了解当前的进步。我们通过通过对现有方法进行分类的新型分类法提出建议,研究网络体系结构,损失功能,基准数据集和学习策略的设计方面的相关研究。此外,我们在包括室内和室外数据集(包括室内和室外数据集)上进行了三个广泛使用基准测试的模型性能进行定量比较。最后,我们讨论了先前作品的挑战,并为读者提供一些有关未来研究方向的见解。
translated by 谷歌翻译
最近,面部生物识别是对传统认证系统的方便替代的巨大关注。因此,检测恶意尝试已经发现具有重要意义,导致面部抗欺骗〜(FAS),即面部呈现攻击检测。与手工制作的功能相反,深度特色学习和技术已经承诺急剧增加FAS系统的准确性,解决了实现这种系统的真实应用的关键挑战。因此,处理更广泛的发展以及准确的模型的新研究区越来越多地引起了研究界和行业的关注。在本文中,我们为自2017年以来对与基于深度特征的FAS方法相关的文献综合调查。在这一主题上阐明,基于各种特征和学习方法的语义分类。此外,我们以时间顺序排列,其进化进展和评估标准(数据集内集和数据集互联集合中集)覆盖了FAS的主要公共数据集。最后,我们讨论了开放的研究挑战和未来方向。
translated by 谷歌翻译
高光谱成像由于其在捕获丰富的空间和光谱信息的能力上提供了多功能应用,这对于识别物质至关重要。但是,获取高光谱图像的设备昂贵且复杂。因此,已经通过直接从低成本,更多可用的RGB图像重建高光谱信息来提出了许多替代光谱成像方法。我们详细研究了来自广泛的RGB图像的这些最先进的光谱重建方法。对25种方法的系统研究和比较表明,尽管速度较低,但大多数数据驱动的深度学习方法在重建精度和质量方面都优于先前的方法。这项全面的审查可以成为同伴研究人员的富有成果的参考来源,从而进一步启发了相关领域的未来发展方向。
translated by 谷歌翻译
作为许多自主驾驶和机器人活动的基本组成部分,如自我运动估计,障碍避免和场景理解,单眼深度估计(MDE)引起了计算机视觉和机器人社区的极大关注。在过去的几十年中,已经开发了大量方法。然而,据我们所知,对MDE没有全面调查。本文旨在通过审查1970年至2021年之间发布的197个相关条款来弥补这一差距。特别是,我们为涵盖各种方法的MDE提供了全面的调查,介绍了流行的绩效评估指标并汇总公开的数据集。我们还总结了一些代表方法的可用开源实现,并比较了他们的表演。此外,我们在一些重要的机器人任务中审查了MDE的应用。最后,我们通过展示一些有希望的未来研究方向来结束本文。预计本调查有助于读者浏览该研究领域。
translated by 谷歌翻译
深度学习已被广​​泛用于医学图像分割,并且录制了录制了该领域深度学习的成功的大量论文。在本文中,我们使用深层学习技术对医学图像分割的全面主题调查。本文进行了两个原创贡献。首先,与传统调查相比,直接将深度学习的文献分成医学图像分割的文学,并为每组详细介绍了文献,我们根据从粗略到精细的多级结构分类目前流行的文献。其次,本文侧重于监督和弱监督的学习方法,而不包括无监督的方法,因为它们在许多旧调查中引入而且他们目前不受欢迎。对于监督学习方法,我们分析了三个方面的文献:骨干网络的选择,网络块的设计,以及损耗功能的改进。对于虚弱的学习方法,我们根据数据增强,转移学习和交互式分割进行调查文献。与现有调查相比,本调查将文献分类为比例不同,更方便读者了解相关理由,并将引导他们基于深度学习方法思考医学图像分割的适当改进。
translated by 谷歌翻译
Domain adaptation (DA) approaches address domain shift and enable networks to be applied to different scenarios. Although various image DA approaches have been proposed in recent years, there is limited research towards video DA. This is partly due to the complexity in adapting the different modalities of features in videos, which includes the correlation features extracted as long-term dependencies of pixels across spatiotemporal dimensions. The correlation features are highly associated with action classes and proven their effectiveness in accurate video feature extraction through the supervised action recognition task. Yet correlation features of the same action would differ across domains due to domain shift. Therefore we propose a novel Adversarial Correlation Adaptation Network (ACAN) to align action videos by aligning pixel correlations. ACAN aims to minimize the distribution of correlation information, termed as Pixel Correlation Discrepancy (PCD). Additionally, video DA research is also limited by the lack of cross-domain video datasets with larger domain shifts. We, therefore, introduce a novel HMDB-ARID dataset with a larger domain shift caused by a larger statistical difference between domains. This dataset is built in an effort to leverage current datasets for dark video classification. Empirical results demonstrate the state-of-the-art performance of our proposed ACAN for both existing and the new video DA datasets.
translated by 谷歌翻译
Wearable sensor-based human activity recognition (HAR) has emerged as a principal research area and is utilized in a variety of applications. Recently, deep learning-based methods have achieved significant improvement in the HAR field with the development of human-computer interaction applications. However, they are limited to operating in a local neighborhood in the process of a standard convolution neural network, and correlations between different sensors on body positions are ignored. In addition, they still face significant challenging problems with performance degradation due to large gaps in the distribution of training and test data, and behavioral differences between subjects. In this work, we propose a novel Transformer-based Adversarial learning framework for human activity recognition using wearable sensors via Self-KnowledgE Distillation (TASKED), that accounts for individual sensor orientations and spatial and temporal features. The proposed method is capable of learning cross-domain embedding feature representations from multiple subjects datasets using adversarial learning and the maximum mean discrepancy (MMD) regularization to align the data distribution over multiple domains. In the proposed method, we adopt the teacher-free self-knowledge distillation to improve the stability of the training procedure and the performance of human activity recognition. Experimental results show that TASKED not only outperforms state-of-the-art methods on the four real-world public HAR datasets (alone or combined) but also improves the subject generalization effectively.
translated by 谷歌翻译
我们提出了Vologan,这是一个对抗域的适应网络,该网络将一个人的高质量3D模型的合成RGB-D图像转换为可以使用消费者深度传感器生成的RGB-D图像。该系统对于为单视3D重建算法生成大量训练数据特别有用,该算法复制了现实世界中的捕获条件,能够模仿相同的高端3D模型数据库的不同传感器类型的样式。该网络使用具有u-net体系结构的CycleGAN框架,以及受SIV-GAN启发的鉴别器。我们使用不同的优化者和学习率计划来训练发电机和鉴别器。我们进一步构建了一个单独考虑图像通道的损失函数,除其他指标外,还评估了结构相似性。我们证明,可以使用自行车来应用合成3D数据的对抗结构域适应,以训练只有少量训练样本的体积视频发电机模型。
translated by 谷歌翻译
With the development of convolutional neural networks, hundreds of deep learning based dehazing methods have been proposed. In this paper, we provide a comprehensive survey on supervised, semi-supervised, and unsupervised single image dehazing. We first discuss the physical model, datasets, network modules, loss functions, and evaluation metrics that are commonly used. Then, the main contributions of various dehazing algorithms are categorized and summarized. Further, quantitative and qualitative experiments of various baseline methods are carried out. Finally, the unsolved issues and challenges that can inspire the future research are pointed out. A collection of useful dehazing materials is available at \url{https://github.com/Xiaofeng-life/AwesomeDehazing}.
translated by 谷歌翻译
随着深度学习技术的快速发展和计算能力的提高,深度学习已广泛应用于高光谱图像(HSI)分类领域。通常,深度学习模型通常包含许多可训练参数,并且需要大量标记的样品来实现最佳性能。然而,关于HSI分类,由于手动标记的难度和耗时的性质,大量标记的样本通常难以获取。因此,许多研究工作侧重于建立一个少数标记样本的HSI分类的深层学习模型。在本文中,我们专注于这一主题,并对相关文献提供系统审查。具体而言,本文的贡献是双重的。首先,相关方法的研究进展根据学习范式分类,包括转移学习,积极学习和少量学习。其次,已经进行了许多具有各种最先进的方法的实验,总结了结果以揭示潜在的研究方向。更重要的是,虽然深度学习模型(通常需要足够的标记样本)和具有少量标记样本的HSI场景之间存在巨大差距,但是通过深度学习融合,可以很好地表征小样本集的问题方法和相关技术,如转移学习和轻量级模型。为了再现性,可以在HTTPS://github.com/shuguoj/hsi-classification中找到纸张中评估的方法的源代码.git。
translated by 谷歌翻译
睡眠分期在诊断和治疗睡眠障碍中非常重要。最近,已经提出了许多数据驱动的深度学习模型,用于自动睡眠分期。他们主要在一个大型公共标签的睡眠数据集上训练该模型,并在较小的主题上对其进行测试。但是,他们通常认为火车和测试数据是从相同的分布中绘制的,这可能在现实世界中不存在。最近已经开发了无监督的域适应性(UDA)来处理此域移位问题。但是,以前用于睡眠分期的UDA方法具有两个主要局限性。首先,他们依靠一个完全共享的模型来对齐,该模型可能会在功能提取过程中丢失特定于域的信息。其次,它们仅在全球范围内将源和目标分布对齐,而无需考虑目标域中的类信息,从而阻碍了测试时模型的分类性能。在这项工作中,我们提出了一个名为Adast的新型对抗性学习框架,以解决未标记的目标域中的域转移问题。首先,我们开发了一个未共享的注意机制,以保留两个领域中的域特异性特征。其次,我们设计了一种迭代自我训练策略,以通过目标域伪标签提高目标域上的分类性能。我们还建议双重分类器,以提高伪标签的鲁棒性和质量。在六个跨域场景上的实验结果验证了我们提出的框架的功效及其优于最先进的UDA方法。源代码可在https://github.com/emadeldeen24/adast上获得。
translated by 谷歌翻译
Convolutional neural network-based approaches for semantic segmentation rely on supervision with pixel-level ground truth, but may not generalize well to unseen image domains. As the labeling process is tedious and labor intensive, developing algorithms that can adapt source ground truth labels to the target domain is of great interest. In this paper, we propose an adversarial learning method for domain adaptation in the context of semantic segmentation. Considering semantic segmentations as structured outputs that contain spatial similarities between the source and target domains, we adopt adversarial learning in the output space. To further enhance the adapted model, we construct a multi-level adversarial network to effectively perform output space domain adaptation at different feature levels. Extensive experiments and ablation study are conducted under various domain adaptation settings, including synthetic-to-real and cross-city scenarios. We show that the proposed method performs favorably against the stateof-the-art methods in terms of accuracy and visual quality.
translated by 谷歌翻译
单图像人群计数是一个充满挑战的计算机视觉问题,在公共安全,城市规划,交通管理等方面进行了广泛的应用。随着深度学习技术的最新发展,近年来,人群的数量引起了很多关注并取得了巨大的成功。这项调查是为了通过系统审查和总结该地区的200多件作品来提供有关基于深度学习的人群计数技术的最新进展的全面摘要。我们的目标是提供最新的评论。在最近的方法中,并在该领域教育新研究人员的设计原理和权衡。在介绍了公开可用的数据集和评估指标之后,我们通过对三个主要的设计模块进行了详细比较来回顾最近的进展:深度神经网络设计,损失功能和监督信号。我们使用公共数据集和评估指标研究和比较方法。我们以一些未来的指示结束了调查。
translated by 谷歌翻译
现有的基于深度学习的变更检测方法试图精心设计具有功能强大特征表示的复杂神经网络,但忽略了随时间变化的土地覆盖变化引起的通用域转移,包括亮度波动和事件前和事后图像之间的季节变化,从而产生亚最佳结果。在本文中,我们提出了一个端到端监督域的适应框架,用于跨域变更检测,即SDACD,以有效地减轻双期颞图像之间的域移位,以更好地变更预测。具体而言,我们的SDACD通过有监督的学习从图像和特征角度介绍了合作改编。图像适应性利用了具有循环矛盾的限制来利用生成的对抗学习,以执行跨域样式转换,从而有效地以两边的方式缩小了域间隙。为了特征适应性,我们提取域不变特征以对齐特征空间中的不同特征分布,这可以进一步减少跨域图像的域间隙。为了进一步提高性能,我们结合了三种类型的双颞图像,以进行最终变化预测,包括初始输入双期图像和两个来自事件前和事后域的生成的双颞图像。对两个基准的广泛实验和分析证明了我们提出的框架的有效性和普遍性。值得注意的是,我们的框架将几个代表性的基线模型推向了新的最先进的记录,分别在CDD和WHU建筑数据集上分别达到97.34%和92.36%。源代码和模型可在https://github.com/perfect-you/sdacd上公开获得。
translated by 谷歌翻译
We address the problem of estimating a high quality dense depth map from a single RGB input image. We start out with a baseline encoder-decoder convolutional neural network architecture and pose the question of how the global processing of information can help improve overall depth estimation. To this end, we propose a transformerbased architecture block that divides the depth range into bins whose center value is estimated adaptively per image. The final depth values are estimated as linear combinations of the bin centers. We call our new building block AdaBins. Our results show a decisive improvement over the state-ofthe-art on several popular depth datasets across all metrics. We also validate the effectiveness of the proposed block with an ablation study and provide the code and corresponding pre-trained weights of the new state-of-the-art model 1 .
translated by 谷歌翻译
Deep learning-based methods have achieved significant performance for image defogging. However, existing methods are mainly developed for land scenes and perform poorly when dealing with overwater foggy images, since overwater scenes typically contain large expanses of sky and water. In this work, we propose a Prior map Guided CycleGAN (PG-CycleGAN) for defogging of images with overwater scenes. To promote the recovery of the objects on water in the image, two loss functions are exploited for the network where a prior map is designed to invert the dark channel and the min-max normalization is used to suppress the sky and emphasize objects. However, due to the unpaired training set, the network may learn an under-constrained domain mapping from foggy to fog-free image, leading to artifacts and loss of details. Thus, we propose an intuitive Upscaling Inception Module (UIM) and a Long-range Residual Coarse-to-fine framework (LRC) to mitigate this issue. Extensive experiments on qualitative and quantitative comparisons demonstrate that the proposed method outperforms the state-of-the-art supervised, semi-supervised, and unsupervised defogging approaches.
translated by 谷歌翻译
Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings. The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets. We first conduct a comprehensive overview with in-depth analysis for closed-world person Re-ID from three different perspectives, including deep feature representation learning, deep metric learning and ranking optimization. With the performance saturation under closed-world setting, the research focus for person Re-ID has recently shifted to the open-world setting, facing more challenging issues. This setting is closer to practical applications under specific scenarios. We summarize the open-world Re-ID in terms of five different aspects. By analyzing the advantages of existing methods, we design a powerful AGW baseline, achieving state-of-the-art or at least comparable performance on twelve datasets for FOUR different Re-ID tasks. Meanwhile, we introduce a new evaluation metric (mINP) for person Re-ID, indicating the cost for finding all the correct matches, which provides an additional criteria to evaluate the Re-ID system for real applications. Finally, some important yet under-investigated open issues are discussed.
translated by 谷歌翻译