Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.
translated by 谷歌翻译
Data heterogeneity across clients in federated learning (FL) settings is a widely acknowledged challenge. In response, personalized federated learning (PFL) emerged as a framework to curate local models for clients' tasks. In PFL, a common strategy is to develop local and global models jointly - the global model (for generalization) informs the local models, and the local models (for personalization) are aggregated to update the global model. A key observation is that if we can improve the generalization ability of local models, then we can improve the generalization of global models, which in turn builds better personalized models. In this work, we consider class imbalance, an overlooked type of data heterogeneity, in the classification setting. We propose FedNH, a novel method that improves the local models' performance for both personalization and generalization by combining the uniformity and semantics of class prototypes. FedNH initially distributes class prototypes uniformly in the latent space and smoothly infuses the class semantics into class prototypes. We show that imposing uniformity helps to combat prototype collapse while infusing class semantics improves local models. Extensive experiments were conducted on popular classification datasets under the cross-device setting. Our results demonstrate the effectiveness and stability of our method over recent works.
translated by 谷歌翻译
Point cloud completion, as the upstream procedure of 3D recognition and segmentation, has become an essential part of many tasks such as navigation and scene understanding. While various point cloud completion models have demonstrated their powerful capabilities, their robustness against adversarial attacks, which have been proven to be fatally malicious towards deep neural networks, remains unknown. In addition, existing attack approaches towards point cloud classifiers cannot be applied to the completion models due to different output forms and attack purposes. In order to evaluate the robustness of the completion models, we propose PointCA, the first adversarial attack against 3D point cloud completion models. PointCA can generate adversarial point clouds that maintain high similarity with the original ones, while being completed as another object with totally different semantic information. Specifically, we minimize the representation discrepancy between the adversarial example and the target point set to jointly explore the adversarial point clouds in the geometry space and the feature space. Furthermore, to launch a stealthier attack, we innovatively employ the neighbourhood density information to tailor the perturbation constraint, leading to geometry-aware and distribution-adaptive modifications for each point. Extensive experiments against different premier point cloud completion networks show that PointCA can cause a performance degradation from 77.9% to 16.7%, with the structure chamfer distance kept below 0.01. We conclude that existing completion models are severely vulnerable to adversarial examples, and state-of-the-art defenses for point cloud classification will be partially invalid when applied to incomplete and uneven point cloud data.
translated by 谷歌翻译
域的适应性旨在使标记的源域和未标记的目标域对齐,并且大多数现有方法都认为源数据是可访问的。不幸的是,这种范式引起了数据隐私和安全性的关注。最近的研究试图通过无源设置来消除这些问题,该设置将源训练的模型适应目标域而不暴露源数据。但是,由于对源模型的对抗性攻击,无源范式仍然有数据泄漏的风险。因此,提出了黑框设置,其中只能利用源模型的输出。在本文中,我们同时介绍了无源的适应和黑盒适应性,提出了一种新的方法,即来自频率混合和相互学习(FMML)的“更好的目标表示”。具体而言,我们引入了一种新的数据增强技术作为频率混音,该技术突出了插值中与任务相关的对象,从而增强了目标模型的类符合性和线性行为。此外,我们引入了一种称为相互学习的网络正则化方法,以介绍域的适应问题。它通过自我知识蒸馏传输目标模型内部的知识,从而通过学习多尺度目标表示来减轻对源域的过度拟合。广泛的实验表明,我们的方法在两种设置下都可以在几个基准数据集上实现最新性能。
translated by 谷歌翻译
人群计数是一项回归任务,它估计场景图像中的人数,在一系列安全至关重要的应用程序中起着至关重要的作用,例如视频监视,交通监控和流量控制。在本文中,我们研究了基于深度学习的人群计数模型对后门攻击的脆弱性,这是对深度学习的主要安全威胁。后门攻击者通过数据中毒将后门触发植入目标模型,以控制测试时间的预测。与已经开发和测试的大多数现有后门攻击的图像分类模型不同,人群计数模型是输出多维密度图的回归模型,因此需要不同的技术来操纵。在本文中,我们提出了两次新颖的密度操纵后门攻击(DMBA $^{ - } $和DMBA $^{+} $),以攻击模型以产生任意的大或小密度估计。实验结果证明了我们对五个经典人群计数模型和四种类型数据集的DMBA攻击的有效性。我们还深入分析了后门人群计数模型的独特挑战,并揭示了有效攻击的两个关键要素:1)完整而密集的触发器以及2)操纵地面真相计数或密度图。我们的工作可以帮助评估人群计数模型对潜在后门攻击的脆弱性。
translated by 谷歌翻译
图形离群值检测是一项具有许多应用程序的新兴但至关重要的机器学习任务。尽管近年来算法扩散,但缺乏标准和统一的绩效评估设置限制了它们在现实世界应用中的进步和使用。为了利用差距,我们(据我们所知)(据我们所知)第一个全面的无监督节点离群值检测基准为unod,并带有以下亮点:(1)评估骨架从经典矩阵分解到最新图形神经的骨架的14个方法网络; (2)在现实世界数据集上使用不同类型的注射异常值和自然异常值对方法性能进行基准测试; (3)通过在不同尺度的合成图上使用运行时和GPU存储器使用算法的效率和可扩展性。基于广泛的实验结果的分析,我们讨论了当前渠道方法的利弊,并指出了多个关键和有希望的未来研究方向。
translated by 谷歌翻译
我们考虑了一个联合表示的学习框架,在中央服务器的协助下,一组$ n $分布式客户通过其私人数据协作培训一组实体的表示(或嵌入)(例如,用户在一个中的用户社交网络)。在此框架下,对于以私人方式汇总在客户培训的本地嵌入的关键步骤,我们开发了一个名为SECEA的安全嵌入聚合协议,该协议为一组实体提供信息理论隐私保证,并在每个客户端提供相应的嵌入$同时$ $,对好奇的服务器和最多$ t <n/2 $勾结的客户。作为SECEA的第一步,联合学习系统执行了一个私人实体联盟,让每个客户在不知道哪个实体属于哪个客户的情况下学习系统中的所有实体。在每个聚合回合中,使用Lagrange插值在客户端中秘密共享本地嵌入,然后每个客户端构造编码的查询以检索预期实体的聚合嵌入。我们对各种表示的学习任务进行全面的实验,以评估SECEA的效用和效率,并从经验上证明,与没有(或具有较弱的)隐私保证的嵌入聚合协议相比,SECEA会造成可忽略的绩效损失(5%以内); SECEA的附加计算潜伏期减小,用于培训较大数据集的更深层次模型。
translated by 谷歌翻译
图形神经网络(GNN)已被广泛用于建模图形结构化数据,这是由于其在广泛的实用应用中令人印象深刻的性能。最近,GNNS的知识蒸馏(KD)在图形模型压缩和知识转移方面取得了显着进步。但是,大多数现有的KD方法都需要大量的真实数据,这些数据在实践中不容易获得,并且可能排除其在教师模型对稀有或难以获取数据集培训的情况下的适用性。为了解决这个问题,我们提出了第一个用于图形结构化数据(DFAD-GNN)的无数据对抗知识蒸馏的端到端框架。具体而言,我们的DFAD-GNN采用生成性对抗网络,主要由三个组成部分组成:预先训练的教师模型和学生模型被视为两个歧视者,并利用生成器来衍生训练图来从教师模型进入学生模型。在各种基准模型和六个代表性数据集上进行的广泛实验表明,我们的DFAD-GNN在图形分类任务中显着超过了最新的无数据基线。
translated by 谷歌翻译
最近联合学习(FL)范式的潜在假设是本地模型通常与全局模型共享与全局模型相同的网络架构,这对于具有不同的硬件和基础架构的移动和IOT设备变得不切实际。可扩展的联合学习框架应该解决配备不同计算和通信功能的异构客户端。为此,本文提出了一种新的联合模型压缩框架,它将异构低级模型分配给客户端,然后将它们聚合到全局全级模型中。我们的解决方案使得能够培训具有不同计算复杂性的异构本地模型,并汇总单个全局模型。此外,FEDHM不仅降低了设备的计算复杂性,而且还通过使用低秩模型来降低通信成本。广泛的实验结果表明,我们提出的\ System在测试顶-1精度(平均精度4.6%的精度增益)方面优于现行修剪的液体方法,在各种异构流域下较小的型号尺寸(平均较小为1.5倍) 。
translated by 谷歌翻译
本文介绍了一种基于图形的Concionome分析的基于图形的内核学习方法。具体地,我们演示了如何利用图表表示内的自然可用结构来编码内核中的先验知识。我们首先提出了一种矩阵分解,以直接从连接数据的自然对称图表表示中提取结构特征。然后,我们使用它们来导出一个结构悬停的图形内核将被馈送到支持向量机中。拟议的方法具有临床思考的优势。对挑战性HIV疾病分类的定量评估(DTI和FMRI衍生的连接数据)和情感识别(EEG导出的连接数据)任务证明了我们提出的方法对现有技术的卓越性能。结果表明,在情感监管任务期间,相关的EEG结合信息主要在Alpha带中编码。
translated by 谷歌翻译