长期以来,Robotics一直是一个遍布复杂系统体系结构的领域,无论传统或基于学习的模块和联系都需要大量的人类专业知识和先验知识。受大型预训练语言模型的启发,这项工作引入了预先培训的通用表示范式,该范式可以作为给定机器人多个任务的起点。我们提出了感知性因果变压器(PACT),这是一种基于生成变压器的架构,旨在以自我监督的方式直接从机器人数据构建表示形式。通过对状态和行动的自动回归预测,我们的模型隐含地编码了特定机器人的动态和行为。我们的实验评估重点是移动药物的域,我们表明该机器人特定的表示可以作为单个起点,以实现不同的任务,例如安全导航,定位和映射。我们评估了两个形式:使用激光雷达传感器作为感知输入(MUSHR)的轮式机器人,以及使用第一人称RGB图像(栖息地)的模拟药物。我们表明,与训练单个模型的同时训练单个模型相比,对所有任务的单个模型进行训练,并且与独立培训单独的大型模型相当的性能,对每个任务的单个模型进行了可比的训练,则在较大的审计模型上进行了固定小型任务特异性网络,从而使性能明显提高。通过跨任务共享共同的优质表示,我们可以降低整体模型容量并加快此类系统的实时部署。
translated by 谷歌翻译
本文涉及由马尔可夫噪声驱动的随机近似的收敛和渐近统计:$$ \ theta_ {n + 1} = \ theta_n + \ alpha_ {n + 1} f(\ theta_n,\ phi_ {n + 1})\, ,\ quad n \ ge 0,$$,其中每个$ \ theta_n \ in \ re ^ d $,$ \ {\ phi_n \} $是一般状态空间x上的马尔可夫链,静止分配$ \ pi $和$ f:\ re ^ d \ times \ text {x} \ to \ re ^ d $。除了在$ f $的标准lipschitz边界,以及消失的步骤大小序列$ \ {\ alpha_n \ \} $的条件外,假设相关ode是全局渐近稳定的静止点表示$ \ theta ^ * $ ,其中$ \ bar f(\ theta)= e [f(\ theta,\ phi)] $ with $ \ phi \ sim \ pi $。而且,ode @ $ \ infty $ virect with advoore字段,$$ \ bar f_ \ idty(\ theta):= \ lim_ {r \ to \ infty} r ^ { - 1} \ bar f(r \ theta)\ ,, \ qquad \ theta \ in \ re ^ d,$$是渐近稳定的。主要贡献总结如下:(i)如果$ \ phi $是几何ergodic,则序列$ \ theta $是融合的,并且在$ f $兼容兼容的界限。剩余的结果是在马尔可夫链的更强大假设下建立:Donsker-varadhan Lyapunov漂移条件的稍微弱版本(DV3)。 (ii)为联合过程$ \ {\ theta_n,\ phi_n \} $构建Lyapunov函数,这意味着$ \ {\ theta_n \} $ in $ l_4 $的融合。 (iii)建立了功能性CLT,以及归一化误差$ z_n:=(\ theta_n- \ theta ^ *)/ \ sqrt {\ alpha_n} $的常规一维CLT。时刻界限结合了CLT暗示了归一化协方差的收敛,$$ \ lim_ {n \ to \ infty} e [z_n z_n ^ t] = \ sigma_ \ theta,$$在$ \ sigma_ \ theta $ where asbptotic协方差出现在CLT中。 (iv)提供了一个例子,其中马尔可夫链$ \ phi $是几何ergodic,但它不满足(dv3)。虽然算法收敛,但第二个时刻是无限的。
translated by 谷歌翻译
卡尔曼滤波器广泛用于对象跟踪,其中过程和测量噪声通常被认为是准确的已知和恒定的。然而,确切的已知和常量假设并不总是在实践中保持。例如,当LIDAR用于跟踪非合作目标时,在不同距离和天气条件下测量噪声不同。另外,过程噪声随对象的运动状态而变化,尤其是当跟踪对象是行人时,并且过程噪声更频繁地改变。本文提出了一种新的估计校正校正闭环估计方法,用于在线估算卡尔曼滤波器过程和测量噪声协方差矩阵。首先,我们将噪声协方差矩阵分解为元素分布矩阵和噪声强度,并改善Sage滤波器以估计元素分布矩阵。其次,我们提出了一种校准方法来准确地诊断噪声强度偏差。然后,我们提出了一种正确的方法来在线自适应地校正噪声强度。第三,在假设系统是可检测的情况下,在数学上证明了所提出的方法的无偏偏差和收敛。仿真结果证明了所提出的方法的有效性和可靠性。最后,我们将建议的方法应用于多对LIDAR的跟踪并在官方Kitti服务器上进行评估。在基提步行者多元object跟踪排行榜上提出的方法(http://www.cvlibs.net/datasets /kitti/eval_tracking.php)超越了使用激光雷达的所有现有方法,证明了在实际应用中的方法的可行性。这项工作提供了一种提高卡尔曼滤波器和多功能跟踪性能的新方法。
translated by 谷歌翻译
Model-based optimization methods and discriminative learning methods have been the two dominant strategies for solving various inverse problems in low-level vision. Typically, those two kinds of methods have their respective merits and drawbacks, e.g., model-based optimization methods are flexible for handling different inverse problems but are usually time-consuming with sophisticated priors for the purpose of good performance; in the meanwhile, discriminative learning methods have fast testing speed but their application range is greatly restricted by the specialized task. Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e.g., deblurring). Such an integration induces considerable advantage when the denoiser is obtained via discriminative learning. However, the study of integration with fast discriminative denoiser prior is still lacking. To this end, this paper aims to train a set of fast and effective CNN (convolutional neural network) denoisers and integrate them into model-based optimization method to solve other inverse problems. Experimental results demonstrate that the learned set of denoisers not only achieve promising Gaussian denoising results but also can be used as prior to deliver good performance for various low-level vision applications.
translated by 谷歌翻译
Existing language models (LMs) predict tokens with a softmax over a finite vocabulary, which can make it difficult to predict rare tokens or phrases. We introduce NPM, the first nonparametric masked language model that replaces this softmax with a nonparametric distribution over every phrase in a reference corpus. We show that NPM can be efficiently trained with a contrastive objective and an in-batch approximation to full corpus retrieval. Zero-shot evaluation on 9 closed-set tasks and 7 open-set tasks demonstrates that NPM outperforms significantly larger parametric models, either with or without a retrieve-and-generate approach. It is particularly better on dealing with rare patterns (word senses or facts), and predicting rare or nearly unseen words (e.g., non-Latin script). We release the model and code at github.com/facebookresearch/NPM.
translated by 谷歌翻译
Change detection (CD) aims to find the difference between two images at different times and outputs a change map to represent whether the region has changed or not. To achieve a better result in generating the change map, many State-of-The-Art (SoTA) methods design a deep learning model that has a powerful discriminative ability. However, these methods still get lower performance because they ignore spatial information and scaling changes between objects, giving rise to blurry or wrong boundaries. In addition to these, they also neglect the interactive information of two different images. To alleviate these problems, we propose our network, the Scale and Relation-Aware Siamese Network (SARAS-Net) to deal with this issue. In this paper, three modules are proposed that include relation-aware, scale-aware, and cross-transformer to tackle the problem of scene change detection more effectively. To verify our model, we tested three public datasets, including LEVIR-CD, WHU-CD, and DSFIN, and obtained SoTA accuracy. Our code is available at https://github.com/f64051041/SARAS-Net.
translated by 谷歌翻译
The space-air-ground integrated network (SAGIN), one of the key technologies for next-generation mobile communication systems, can facilitate data transmission for users all over the world, especially in some remote areas where vast amounts of informative data are collected by Internet of remote things (IoRT) devices to support various data-driven artificial intelligence (AI) services. However, training AI models centrally with the assistance of SAGIN faces the challenges of highly constrained network topology, inefficient data transmission, and privacy issues. To tackle these challenges, we first propose a novel topology-aware federated learning framework for the SAGIN, namely Olive Branch Learning (OBL). Specifically, the IoRT devices in the ground layer leverage their private data to perform model training locally, while the air nodes in the air layer and the ring-structured low earth orbit (LEO) satellite constellation in the space layer are in charge of model aggregation (synchronization) at different scales.To further enhance communication efficiency and inference performance of OBL, an efficient Communication and Non-IID-aware Air node-Satellite Assignment (CNASA) algorithm is designed by taking the data class distribution of the air nodes as well as their geographic locations into account. Furthermore, we extend our OBL framework and CNASA algorithm to adapt to more complex multi-orbit satellite networks. We analyze the convergence of our OBL framework and conclude that the CNASA algorithm contributes to the fast convergence of the global model. Extensive experiments based on realistic datasets corroborate the superior performance of our algorithm over the benchmark policies.
translated by 谷歌翻译
Time series forecasting is a long-standing challenge due to the real-world information is in various scenario (e.g., energy, weather, traffic, economics, earthquake warning). However some mainstream forecasting model forecasting result is derailed dramatically from ground truth. We believe it's the reason that model's lacking ability of capturing frequency information which richly contains in real world datasets. At present, the mainstream frequency information extraction methods are Fourier transform(FT) based. However, use of FT is problematic due to Gibbs phenomenon. If the values on both sides of sequences differ significantly, oscillatory approximations are observed around both sides and high frequency noise will be introduced. Therefore We propose a novel frequency enhanced channel attention that adaptively modelling frequency interdependencies between channels based on Discrete Cosine Transform which would intrinsically avoid high frequency noise caused by problematic periodity during Fourier Transform, which is defined as Gibbs Phenomenon. We show that this network generalize extremely effectively across six real-world datasets and achieve state-of-the-art performance, we further demonstrate that frequency enhanced channel attention mechanism module can be flexibly applied to different networks. This module can improve the prediction ability of existing mainstream networks, which reduces 35.99% MSE on LSTM, 10.01% on Reformer, 8.71% on Informer, 8.29% on Autoformer, 8.06% on Transformer, etc., at a slight computational cost ,with just a few line of code. Our codes and data are available at https://github.com/Zero-coder/FECAM.
translated by 谷歌翻译
Stairs are common building structures in urban environment, and stair detection is an important part of environment perception for autonomous mobile robots. Most existing algorithms have difficulty combining the visual information from binocular sensors effectively and ensuring reliable detection at night and in the case of extremely fuzzy visual clues. To solve these problems, we propose a neural network architecture with inputs of both RGB map and depth map. Specifically, we design the selective module which can make the network learn the complementary relationship between RGB map and depth map and effectively combine the information from RGB map and depth map in different scenes. In addition, we also design a line clustering algorithm for the post-processing of detection results, which can make full use of the detection results to obtain the geometric parameters of stairs. Experiments on our dataset show that our method can achieve better accuracy and recall compared with the previous state-of-the-art deep learning method, which are 5.64% and 7.97%, respectively. Our method also has extremely fast detection speed, and a lightweight version can achieve 300 + frames per second with the same resolution, which can meet the needs of most real-time detection scenes.
translated by 谷歌翻译
Self-similarity is valuable to the exploration of non-local textures in single image super-resolution (SISR). Researchers usually assume that the importance of non-local textures is positively related to their similarity scores. In this paper, we surprisingly found that when repairing severely damaged query textures, some non-local textures with low-similarity which are closer to the target can provide more accurate and richer details than the high-similarity ones. In these cases, low-similarity does not mean inferior but is usually caused by different scales or orientations. Utilizing this finding, we proposed a Global Learnable Attention (GLA) to adaptively modify similarity scores of non-local textures during training instead of only using a fixed similarity scoring function such as the dot product. The proposed GLA can explore non-local textures with low-similarity but more accurate details to repair severely damaged textures. Furthermore, we propose to adopt Super-Bit Locality-Sensitive Hashing (SB-LSH) as a preprocessing method for our GLA. With the SB-LSH, the computational complexity of our GLA is reduced from quadratic to asymptotic linear with respect to the image size. In addition, the proposed GLA can be integrated into existing deep SISR models as an efficient general building block. Based on the GLA, we constructed a Deep Learnable Similarity Network (DLSN), which achieves state-of-the-art performance for SISR tasks of different degradation types (e.g. blur and noise). Our code and a pre-trained DLSN have been uploaded to GitHub{\dag} for validation.
translated by 谷歌翻译