随着社交媒体平台越来越多地采用了简短的视频,通过视频帖子减少错误信息的传播已成为社交媒体提供商的关键挑战。在本文中,我们开发了在社交媒体帖子中检测错误信息的方法,从而利用了视频和文本等方式。由于缺乏在多模式数据集中检测错误信息检测的大规模公共数据,因此我们从Twitter收集160,000个视频帖子,并利用自学学习的学习来学习联合视觉和文本数据的表达性表示。在这项工作中,我们提出了两种新方法,用于基于对比度学习和掩盖语言建模的短形式社交媒体视频帖子中的语义不一致。我们证明,我们的新方法在通过随机交汇正面样本和在野外的新手动标记测试集中,在野外生成的人工数据上的最新方法都超过了当前的最新方法,以进行语义错误信息。
translated by 谷歌翻译
Face Anti-spoofing (FAS) is essential to secure face recognition systems from various physical attacks. However, recent research generally focuses on short-distance applications (i.e., phone unlocking) while lacking consideration of long-distance scenes (i.e., surveillance security checks). In order to promote relevant research and fill this gap in the community, we collect a large-scale Surveillance High-Fidelity Mask (SuHiFiMask) dataset captured under 40 surveillance scenes, which has 101 subjects from different age groups with 232 3D attacks (high-fidelity masks), 200 2D attacks (posters, portraits, and screens), and 2 adversarial attacks. In this scene, low image resolution and noise interference are new challenges faced in surveillance FAS. Together with the SuHiFiMask dataset, we propose a Contrastive Quality-Invariance Learning (CQIL) network to alleviate the performance degradation caused by image quality from three aspects: (1) An Image Quality Variable module (IQV) is introduced to recover image information associated with discrimination by combining the super-resolution network. (2) Using generated sample pairs to simulate quality variance distributions to help contrastive learning strategies obtain robust feature representation under quality variation. (3) A Separate Quality Network (SQN) is designed to learn discriminative features independent of image quality. Finally, a large number of experiments verify the quality of the SuHiFiMask dataset and the superiority of the proposed CQIL.
translated by 谷歌翻译
Solving partial differential equations is difficult. Recently proposed neural resolution-invariant models, despite their effectiveness and efficiency, usually require equispaced spatial points of data. However, sampling in spatial domain is sometimes inevitably non-equispaced in real-world systems, limiting their applicability. In this paper, we propose a Non-equispaced Fourier PDE Solver (\textsc{NFS}) with adaptive interpolation on resampled equispaced points and a variant of Fourier Neural Operators as its components. Experimental results on complex PDEs demonstrate its advantages in accuracy and efficiency. Compared with the spatially-equispaced benchmark methods, it achieves superior performance with $42.85\%$ improvements on MAE, and is able to handle non-equispaced data with a tiny loss of accuracy. Besides, to our best knowledge, \textsc{NFS} is the first ML-based method with mesh invariant inference ability to successfully model turbulent flows in non-equispaced scenarios, with a minor deviation of the error on unseen spatial points.
translated by 谷歌翻译
Recent years have witnessed great success in handling graph-related tasks with Graph Neural Networks (GNNs). Despite their great academic success, Multi-Layer Perceptrons (MLPs) remain the primary workhorse for practical industrial applications. One reason for this academic-industrial gap is the neighborhood-fetching latency incurred by data dependency in GNNs, which make it hard to deploy for latency-sensitive applications that require fast inference. Conversely, without involving any feature aggregation, MLPs have no data dependency and infer much faster than GNNs, but their performance is less competitive. Motivated by these complementary strengths and weaknesses, we propose a Graph Self-Distillation on Neighborhood (GSDN) framework to reduce the gap between GNNs and MLPs. Specifically, the GSDN framework is based purely on MLPs, where structural information is only implicitly used as prior to guide knowledge self-distillation between the neighborhood and the target, substituting the explicit neighborhood information propagation as in GNNs. As a result, GSDN enjoys the benefits of graph topology-awareness in training but has no data dependency in inference. Extensive experiments have shown that the performance of vanilla MLPs can be greatly improved with self-distillation, e.g., GSDN improves over stand-alone MLPs by 15.54\% on average and outperforms the state-of-the-art GNNs on six datasets. Regarding inference speed, GSDN infers 75X-89X faster than existing GNNs and 16X-25X faster than other inference acceleration methods.
translated by 谷歌翻译
时间点过程(TPP)通常用于模拟具有出现时间戳的异步事件序列,并由以历史影响为条件的概率模型揭示。尽管以前的许多作品通过最大程度地提高了TPP模型的“合适性”,但它们的预测性能不令人满意,这意味着模型产生的时间戳与真实的观察相距甚远。最近,诸如DENOTO扩散和得分匹配模型之类的深层生成模型通过证明其生成高质量样本的能力,在图像生成任务方面取得了巨大进展。但是,在事件发生在TPP的情况下,尚无完整而统一的作品来探索和研究生成模型的潜力。在这项工作中,我们尝试通过设计一个unified \ textbf {g} \ textbf {n} eural \ textbf {t} emporal \ emporal \ textbf {p} oint \ textbf {p} rocess {p} rocess(\ textsc {\ textsc { GNTPP})模型探索其可行性和有效性,并进一步改善模型的预测性能。此外,在衡量历史影响方面,我们修改了细心的模型,这些模型总结了历史事件的影响,并以适应性的重新加权术语来考虑事件的类型关系和时间间隔。已经进行了广泛的实验,以说明\ textsc {gntpp}的预测能力的提高,并用一系列生成概率解码器,并从修订后的注意力中获得了绩效增长。据我们所知,这是第一批适应生成模型在完整的统一框架中并在TPP背景下研究其有效性的作品。我们的代码库包括第5.1.1节中给出的所有方法。5.1.1在\ url {https://github.com/bird-tao/gntpp}中打开。我们希望代码框架可以促进神经TPP的未来研究。
translated by 谷歌翻译
在这封信中,我们提出了一种多功能的层次离线计划算法,以及用于敏捷四足球运动的在线控制管道。我们的离线规划师在优化降低阶模型和全身轨迹优化的质心动力学之间进行交替,以实现动力学共识。我们使用等椭圆形参数化的新型动量惰性质地优化能够通过``惯性塑造''来产生高度的杂技运动。我们的全身优化方法可显着改善基于标准DDP的方法的质量从质心层中利用反馈。对于在线控制,我们通过完整的质心动力学的线性转换开发了一种新颖的凸模型预测控制方案。我们的控制器可以在单个优化中有效地对接触力和关节加速度有效地优化,从而实现更直接的加速度,从而实现更直接的优化与现有四倍体MPC控制器相比,跟踪动量丰富的动作。我们在四个不同的动态操作中证明了我们的轨迹计划者的能力和通用性。然后,我们在MIT MINI Cheetah平台上展示了​​一个硬件实验,以证明整个计划的性能和整个计划的性能和性能扭曲的控制管道跳动。
translated by 谷歌翻译
可靠的机器人抓握,特别是具有可变形物体(例如水果),由于与夹持器,未知的物体动态和可变物体几何形状的欠扰接触相互作用,仍然是一个具有挑战性的任务。在这项研究中,我们提出了一种用于刚性夹持器的基于变压器的机器人抓握框架,其利用触觉和可视信息来用于安全对象抓握。具体地,变压器模型通过执行两个预定义的探索动作(夹紧和滑动)来学习具有传感器反馈的物理特征嵌入,并通过多层的Perceptron(MLP)预测最终抓握结果,具有给定的抓握强度。使用这些预测,通过推断使用用于抓握任务的安全抓握强度的抓握器。与卷积复制网络相比,变压器模型可以在图像序列上捕获长期依赖性,并同时处理空间时间特征。我们首先在公共数据集上基准测试在公共数据集上进行滑动检测。在此之后,我们表明变压器模型在掌握精度和计算效率方面优于CNN + LSTM模型。我们还收集我们自己的水果掌握数据集,并使用所看到和看不见的果实的拟议框架进行在线掌握实验。我们的代码和数据集在Github上公开。
translated by 谷歌翻译
由于深度学习在许多人工智能应用中显示了革命性的性能,其升级的计算需求需要用于巨大并行性的硬件加速器和改进的吞吐量。光学神经网络(ONN)是下一代神经关键组成的有希望的候选者,由于其高并行,低延迟和低能量消耗。在这里,我们设计了一个硬件高效的光子子空间神经网络(PSNN)架构,其针对具有比具有可比任务性能的前一个ONN架构的光学元件使用,区域成本和能量消耗。此外,提供了一种硬件感知培训框架,以最小化所需的设备编程精度,减少芯片区域,并提高噪声鲁棒性。我们在实验上展示了我们的PSNN在蝴蝶式可编程硅光子集成电路上,并在实用的图像识别任务中显示其实用性。
translated by 谷歌翻译
强化学习(RL)原则上可以让机器人自动适应新任务,但是当前的RL方法需要大量的试验来实现这一目标。在本文中,我们通过元学习的框架来快速适应新任务,该框架利用过去的任务学习适应了对工业插入任务的特定关注。快速适应至关重要,因为大量的机器人试验可能会损害硬件件。另外,在不同的插入应用之间的经验中,有效的适应性也可以在很大程度上彼此利用。在这种情况下,我们在应用元学习时解决了两个具体的挑战。首先,传统的元元算法需要冗长的在线元训练。 We show that this can be replaced with appropriately chosen offline data, resulting in an offline meta-RL method that only requires demonstrations and trials from each of the prior tasks, without the need to run costly meta-RL procedures online.其次,元RL方法可能无法推广到与元训练时间时看到的新任务太大的任务,这在高成功率至关重要的工业应用中构成了特定的挑战。我们通过将上下文元学习与直接在线填充结合结合来解决这一问题:如果新任务与先前数据中看到的任务相似,则可以立即适应上下文的元学习者,如果它太不同,它会逐渐通过Finetuning适应。我们表明,我们的方法能够快速适应各种不同的插入任务,成功率为100%仅使用从头开始学习任务所需的样本的一小部分。实验视频和详细信息可从https://sites.google.com/view/offline-metarl-insertion获得。
translated by 谷歌翻译
Proteins are fundamental biological entities that play a key role in life activities. The amino acid sequences of proteins can be folded into stable 3D structures in the real physicochemical world, forming a special kind of sequence-structure data. With the development of Artificial Intelligence (AI) techniques, Protein Representation Learning (PRL) has recently emerged as a promising research topic for extracting informative knowledge from massive protein sequences or structures. To pave the way for AI researchers with little bioinformatics background, we present a timely and comprehensive review of PRL formulations and existing PRL methods from the perspective of model architectures, pretext tasks, and downstream applications. We first briefly introduce the motivations for protein representation learning and formulate it in a general and unified framework. Next, we divide existing PRL methods into three main categories: sequence-based, structure-based, and sequence-structure co-modeling. Finally, we discuss some technical challenges and potential directions for improving protein representation learning. The latest advances in PRL methods are summarized in a GitHub repository https://github.com/LirongWu/awesome-protein-representation-learning.
translated by 谷歌翻译