Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications. However, collaboration between these actors is difficult due to the heterogeneous nature of geospatial data modalities (e.g., multi-spectral images of various resolutions, timeseries, weather data) and diversity of tasks (e.g., regression of human activity indicators or detecting forest fires). In this work, we present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias, pre-trained on large amounts of unlabelled earth observation data in a self-supervised manner. We envision how such a model may facilitate cooperation between members of the community. We show preliminary results on the first step of the roadmap, where we instantiate an architecture that can process a wide variety of geospatial data modalities and demonstrate that it can achieve competitive performance with domain-specific architectures on tasks relating to the U.N.'s Sustainable Development Goals.
translated by 谷歌翻译
从视觉感觉数据中控制人造代理是一项艰巨的任务。强化学习(RL)算法可以在这方面取得成功,但需要代理与环境之间进行大量相互作用。为了减轻该问题,无监督的RL建议采用自我监督的互动和学习,以更快地适应未来的任务。但是,目前的无监督策略是否可以改善概括能力,尤其是在视觉控制设置中。在这项工作中,我们为数据有效的视觉控制设计了有效的无监督RL策略。首先,我们表明,使用无监督的RL收集的数据预先训练的世界模型可以促进适应未来的任务。然后,我们与我们的混合计划者分析了一些设计选择,以有效地适应了代理的预训练组件,并在想象中学习和计划,并与我们的混合计划者一起使用,我们将其dub dyna-mpc进行了。通过结合一项大规模实证研究的发现,我们建立了一种方法,该方法强烈改善了无监督的RL基准测试的性能,需要20美元$ \ times $ $ $ $ $ \少于数据以符合监督方法的性能。该方法还表明了在现实词的RL基准测试上的稳健性能,暗示该方法概括为嘈杂的环境。
translated by 谷歌翻译
最近的自我监督进展表明,预先训练大量无监督数据的大型神经网络可能导致下游任务的概括令人印象深刻。这些模型最近被作为基础模型,一直转变为自然语言处理领域。虽然类似的模型也在大型图像的核心训练中,但它们不适合遥感数据。为刺激地球监测基础模型的发展,我们建议开发由与气候变化相关的各种下游任务组成的新基准。我们认为,这可能导致许多现有应用程序的大量改进,并促进新应用的发展。该提案还可以提出合作,并提出更好的评估过程,以减轻地球监测的基础模型的潜在缺陷。
translated by 谷歌翻译
这项工作介绍了一种新颖的原则,我们通过机制稀疏正规调用解剖学,基于高级概念的动态往往稀疏的想法。我们提出了一种表示学习方法,可以通过同时学习与它们相关的潜在因子和稀疏因果图形模型来引起解剖学。我们开发了一个严谨的可识别性理论,建立在最近的非线性独立分量分析(ICA)结果中,结果是模拟这一原理,并展示了如何恢复潜在变量,如果一个规则大致潜在机制为稀疏,如果某些图形连接标准通过数据生成过程满足。作为我们框架的特殊情况,我们展示了如何利用未知目标的干预措施来解除潜在因子,从而借鉴ICA和因果关系之间的进一步联系。我们还提出了一种基于VAE的方法,其中通过二进制掩码来学习和正规化潜在机制,并通过表明它学会在模拟中的解散表示来验证我们的理论。
translated by 谷歌翻译
鉴于部署更可靠的机器学习系统的重要性,研究界内的机器学习模型的解释性得到了相当大的关注。在计算机视觉应用中,生成反事实方法表示如何扰乱模型的输入来改变其预测,提供有关模型决策的详细信息。目前的方法倾向于产生关于模型决策的琐碎的反事实,因为它们通常建议夸大或消除所分类的属性的存在。对于机器学习从业者,这些类型的反事件提供了很少的价值,因为它们没有提供有关不期望的模型或数据偏差的新信息。在这项工作中,我们确定了琐碎的反事实生成问题,我们建议潜水以缓解它。潜水在使用多样性强制损失限制的解除印章潜在空间中学习扰动,以发现关于模型预测的多个有价值的解释。此外,我们介绍一种机制,以防止模型产生微不足道的解释。 Celeba和Synbols的实验表明,与先前的最先进的方法相比,我们的模型提高了生产高质量有价值解释的成功率。代码可在https://github.com/elementai/beyond- trial-explanations获得。
translated by 谷歌翻译
Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms. Our analysis reveals that simple metric scaling completely changes the nature of few-shot algorithm parameter updates. Metric scaling provides improvements up to 14% in accuracy for certain metrics on the mini-Imagenet 5-way 5-shot classification task. We further propose a simple and effective way of conditioning a learner on the task sample set, resulting in learning a task-dependent metric space. Moreover, we propose and empirically test a practical end-to-end optimization procedure based on auxiliary task co-training to learn a task-dependent metric space. The resulting few-shot learning model based on the task-dependent scaled metric achieves state of the art on mini-Imagenet. We confirm these results on another few-shot dataset that we introduce in this paper based on CIFAR100. Our code is publicly available at https://github.com/ElementAI/TADAM.
translated by 谷歌翻译
Compared to conventional artificial neurons that produce dense and real-valued responses, biologically-inspired spiking neurons transmit sparse and binary information, which can also lead to energy-efficient implementations. Recent research has shown that spiking neural networks can be trained like standard recurrent neural networks using the surrogate gradient method. They have shown promising results on speech command recognition tasks. Using the same technique, we show that they are scalable to large vocabulary continuous speech recognition, where they are capable of replacing LSTMs in the encoder with only minor loss of performance. This suggests that they may be applicable to more involved sequence-to-sequence tasks. Moreover, in contrast to their recurrent non-spiking counterparts, they show robustness to exploding gradient problems without the need to use gates.
translated by 谷歌翻译
We propose a new method for estimating the minimizer $\boldsymbol{x}^*$ and the minimum value $f^*$ of a smooth and strongly convex regression function $f$ from the observations contaminated by random noise. Our estimator $\boldsymbol{z}_n$ of the minimizer $\boldsymbol{x}^*$ is based on a version of the projected gradient descent with the gradient estimated by a regularized local polynomial algorithm. Next, we propose a two-stage procedure for estimation of the minimum value $f^*$ of regression function $f$. At the first stage, we construct an accurate enough estimator of $\boldsymbol{x}^*$, which can be, for example, $\boldsymbol{z}_n$. At the second stage, we estimate the function value at the point obtained in the first stage using a rate optimal nonparametric procedure. We derive non-asymptotic upper bounds for the quadratic risk and optimization error of $\boldsymbol{z}_n$, and for the risk of estimating $f^*$. We establish minimax lower bounds showing that, under certain choice of parameters, the proposed algorithms achieve the minimax optimal rates of convergence on the class of smooth and strongly convex functions.
translated by 谷歌翻译
In Novel Class Discovery (NCD), the goal is to find new classes in an unlabeled set given a labeled set of known but different classes. While NCD has recently gained attention from the community, no framework has yet been proposed for heterogeneous tabular data, despite being a very common representation of data. In this paper, we propose TabularNCD, a new method for discovering novel classes in tabular data. We show a way to extract knowledge from already known classes to guide the discovery process of novel classes in the context of tabular data which contains heterogeneous variables. A part of this process is done by a new method for defining pseudo labels, and we follow recent findings in Multi-Task Learning to optimize a joint objective function. Our method demonstrates that NCD is not only applicable to images but also to heterogeneous tabular data.
translated by 谷歌翻译
We investigate the sample complexity of learning the optimal arm for multi-task bandit problems. Arms consist of two components: one that is shared across tasks (that we call representation) and one that is task-specific (that we call predictor). The objective is to learn the optimal (representation, predictor)-pair for each task, under the assumption that the optimal representation is common to all tasks. Within this framework, efficient learning algorithms should transfer knowledge across tasks. We consider the best-arm identification problem for a fixed confidence, where, in each round, the learner actively selects both a task, and an arm, and observes the corresponding reward. We derive instance-specific sample complexity lower bounds satisfied by any $(\delta_G,\delta_H)$-PAC algorithm (such an algorithm identifies the best representation with probability at least $1-\delta_G$, and the best predictor for a task with probability at least $1-\delta_H$). We devise an algorithm OSRL-SC whose sample complexity approaches the lower bound, and scales at most as $H(G\log(1/\delta_G)+ X\log(1/\delta_H))$, with $X,G,H$ being, respectively, the number of tasks, representations and predictors. By comparison, this scaling is significantly better than the classical best-arm identification algorithm that scales as $HGX\log(1/\delta)$.
translated by 谷歌翻译