在视频压缩中,通过运动和剩余补偿从先前解码的帧重复使用像素来提高编码效率。我们在视频帧中定义了两个层次冗余的两个级别:1)一阶:像素空间中的冗余,即跨相邻帧的像素值的相似性,该框架的相似性是通过运动和残差补偿有效捕获的,2)二阶:二阶:冗余:自然视频中的平稳运动引起的运动和残留地图。尽管大多数现有的神经视频编码文献都涉及一阶冗余,但我们解决了通过预测变量在神经视频编解码器中捕获二阶冗余的问题。我们引入了通用运动和残留预测因子,这些预测因素学会从先前解码的数据中推断出来。这些预测因子是轻量级的,可以使用大多数神经视频编解码器来提高其率延伸性能。此外,虽然RGB是神经视频编码文献中的主导色彩空间,但我们引入了神经视频编解码器的一般修改,以包含YUV420 Colorspace并报告YUV420的结果。我们的实验表明,使用众所周知的神经视频编解码器使用我们的预测因子可在UVG数据集中测得的RGB和YUV420 Colorspace中节省38%和34%的比特率。
translated by 谷歌翻译
由于深层网络的计算复杂性和功率约束的移动硬件的计算复杂性,因此在移动设备上实现神经视频编解码器的潜力是一项巨大的技术挑战。我们通过利用高通公司的技术和创新来证明可行性,从而弥合了从基于神经网络的编解码器模拟在壁式工作站运行的差距,再到由Snapdragon技术供电的移动设备上的实时操作。我们显示有史以来第一个在商用手机上运行的框架间神经视频解码器,实时解码高清视频,同时保持低比特率和高视觉质量。
translated by 谷歌翻译
我们引入基于实例自适应学习的视频压缩算法。在要传输的每个视频序列上,我们介绍了预训练的压缩模型。最佳参数与潜在代码一起发送到接收器。通过熵编码在合适的混合模型下的参数更新,我们确保可以有效地编码网络参数。该实例自适应压缩算法对于基础模型的选择是不可知的,并且具有改进任何神经视频编解码器的可能性。在UVG,HEVC和XIPH数据集上,我们的CODEC通过21%至26%的BD速率节省,提高了低延迟尺度空间流量模型的性能,以及最先进的B帧模型17至20%的BD速率储蓄。我们还证明了实例 - 自适应FineTuning改善了域移位的鲁棒性。最后,我们的方法降低了压缩模型的容量要求。我们表明它即使在将网络大小减少72%之后也能实现最先进的性能。
translated by 谷歌翻译
Compared to regular cameras, Dynamic Vision Sensors or Event Cameras can output compact visual data based on a change in the intensity in each pixel location asynchronously. In this paper, we study the application of current image-based SLAM techniques to these novel sensors. To this end, the information in adaptively selected event windows is processed to form motion-compensated images. These images are then used to reconstruct the scene and estimate the 6-DOF pose of the camera. We also propose an inertial version of the event-only pipeline to assess its capabilities. We compare the results of different configurations of the proposed algorithm against the ground truth for sequences of two publicly available event datasets. We also compare the results of the proposed event-inertial pipeline with the state-of-the-art and show it can produce comparable or more accurate results provided the map estimate is reliable.
translated by 谷歌翻译
The purpose of this work was to tackle practical issues which arise when using a tendon-driven robotic manipulator with a long, passive, flexible proximal section in medical applications. A separable robot which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. A control input which resolves the redundancy in the kinematics and a physical interpretation of this redundancy are provided. The effect of a static change in the proximal section angle on bending angle error was explored under four testing conditions for a sinusoidal input. Bending angle error increased for increasing proximal section angle for all testing conditions with an average error reduction of 41.48% for retension, 4.28% for hysteresis, and 52.35% for re-tension + hysteresis compensation relative to the baseline case. Two major sources of error in tracking the bending angle were identified: time delay from hysteresis and DC offset from the proximal section angle. Examination of these error sources revealed that the simple hysteresis compensation was most effective for removing time delay and re-tension compensation for removing DC offset, which was the primary source of increasing error. The re-tension compensation was also tested for dynamic changes in the proximal section and reduced error in the final configuration of the tip by 89.14% relative to the baseline case.
translated by 谷歌翻译
Transformers have recently gained attention in the computer vision domain due to their ability to model long-range dependencies. However, the self-attention mechanism, which is the core part of the Transformer model, usually suffers from quadratic computational complexity with respect to the number of tokens. Many architectures attempt to reduce model complexity by limiting the self-attention mechanism to local regions or by redesigning the tokenization process. In this paper, we propose DAE-Former, a novel method that seeks to provide an alternative perspective by efficiently designing the self-attention mechanism. More specifically, we reformulate the self-attention mechanism to capture both spatial and channel relations across the whole feature dimension while staying computationally efficient. Furthermore, we redesign the skip connection path by including the cross-attention module to ensure the feature reusability and enhance the localization power. Our method outperforms state-of-the-art methods on multi-organ cardiac and skin lesion segmentation datasets without requiring pre-training weights. The code is publicly available at https://github.com/mindflow-institue/DAEFormer.
translated by 谷歌翻译
A track-before-detect (TBD) particle filter-based method for detection and tracking of low observable objects based on a sequence of image frames in the presence of noise and clutter is studied. At each time instance after receiving a frame of image, first, some preprocessing approaches are applied to the image. Then, it is sent to the detection and tracking algorithm which is based on a particle filter. Performance of the approach is evaluated for detection and tracking of an object in different scenarios including noise and clutter.
translated by 谷歌翻译
Machine reading comprehension (MRC) is a long-standing topic in natural language processing (NLP). The MRC task aims to answer a question based on the given context. Recently studies focus on multi-hop MRC which is a more challenging extension of MRC, which to answer a question some disjoint pieces of information across the context are required. Due to the complexity and importance of multi-hop MRC, a large number of studies have been focused on this topic in recent years, therefore, it is necessary and worth reviewing the related literature. This study aims to investigate recent advances in the multi-hop MRC approaches based on 31 studies from 2018 to 2022. In this regard, first, the multi-hop MRC problem definition will be introduced, then 31 models will be reviewed in detail with a strong focus on their multi-hop aspects. They also will be categorized based on their main techniques. Finally, a fine-grain comprehensive comparison of the models and techniques will be presented.
translated by 谷歌翻译
Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.
translated by 谷歌翻译
Regular cameras and cell phones are able to capture limited luminosity. Thus, in terms of quality, most of the produced images from such devices are not similar to the real world. They are overly dark or too bright, and the details are not perfectly visible. Various methods, which fall under the name of High Dynamic Range (HDR) Imaging, can be utilised to cope with this problem. Their objective is to produce an image with more details. However, unfortunately, most methods for generating an HDR image from Multi-Exposure images only concentrate on how to combine different exposures and do not have any focus on choosing the best details of each image. Therefore, it is strived in this research to extract the most visible areas of each image with the help of image segmentation. Two methods of producing the Ground Truth were considered, as manual threshold and Otsu threshold, and a neural network will be used to train segment these areas. Finally, it will be shown that the neural network is able to segment the visible parts of pictures acceptably.
translated by 谷歌翻译