最近,基于云的图形卷积网络(GCN)在许多对隐私敏感的应用程序(例如个人医疗保健和金融系统)中表现出了巨大的成功和潜力。尽管在云上具有很高的推理准确性和性能,但在GCN推理中保持数据隐私,这对于这些实际应用至关重要,但仍未得到探索。在本文中,我们对此进行了初步尝试,并开发了$ \ textit {cryptogcn} $ - 基于GCN推理框架的同型加密(HE)。我们方法成功的关键是减少HE操作的巨大计算开销,这可能比明文空间中的同行高的数量级。为此,我们开发了一种方法,可以有效利用GCN推断中基质操作的稀疏性,从而大大减少计算开销。具体而言,我们提出了一种新型的AMA数据格式方法和相关的空间卷积方法,该方法可以利用复杂的图结构并在HE计算中执行有效的矩阵矩阵乘法,从而大大减少HE操作。我们还开发了一个合作式框架,该框架可以通过明智的修剪和GCN中激活模块的多项式近似来探索准确性,安全级别和计算开销之间的交易折扣。基于NTU-Xview骨架关节数据集,即,据我们所知,最大的数据集对同型的评估,我们的实验结果表明,$ \ textit {cryptogcn} $均优胜于最先进的解决方案。同构操作的延迟和数量,即在延迟上达到3.10 $ \ times $加速,并将总代态操作数量减少77.4 \%,而准确度的较小精度损失为1-1.5 $ \%$。
translated by 谷歌翻译
同态加密(HE),允许对加密数据(Ciphertext)进行计算,而无需首先解密,因此可以实现对云中隐私性的应用程序的安全性缓慢的卷积神经网络(CNN)推断。为了减少推理潜伏期,一种方法是将多个消息打包到单个密文中,以减少密文的数量并支持同型多态多重蓄能(HMA)操作的大量并行性。尽管HECNN的推断速度更快,但主流包装方案密集的包装(密度)和卷积包装(Convpack)仍将昂贵的旋转开销引入了昂贵的旋转开销,这延长了HECNN的推断潜伏期,以实现更深和更广泛的CNN体​​系结构。在本文中,我们提出了一种名为FFCONV的低级分解方法,该方法专门用于有效的密文填料,用于减少旋转台面和HMA操作。 FFCONV近似于低级分解卷积的A D X D卷积层,其中D X D低率卷积具有较少的通道,然后是1 x 1卷积以恢复通道。 D X D低级别卷积带有密度,导致旋转操作显着降低,而1 x 1卷积的旋转开销接近零。据我们所知,FFCONV是能够同时减少densepack和Convpack产生的旋转头顶的第一项工作,而无需将其他特殊块引入HECNN推理管道。与先前的Art Lola和Falcon相比,我们的方法分别将推理潜伏期降低了88%和21%,其精度在MNIST和CIFAR-10上具有可比的精度。
translated by 谷歌翻译
保留保护解决方案使公司能够在履行政府法规的同时将机密数据卸载到第三方服务。为了实现这一点,它们利用了各种密码技术,例如同性恋加密(HE),其允许对加密数据执行计算。大多数他计划以SIMD方式工作,数据包装方法可以显着影响运行时间和内存成本。找到导致最佳性能实现的包装方法是一个艰难的任务。我们提出了一种简单而直观的框架,摘要为用户提供包装决定。我们解释其底层数据结构和优化器,并提出了一种用于执行2D卷积操作的新算法。我们使用此框架来实现他友好的AlexNet版本,在三分钟内运行,比其他最先进的解决方案更快的数量级,只能使用他。
translated by 谷歌翻译
保存隐私的神经网络(NN)推理解决方案最近在几种提供不同的延迟带宽权衡的解决方案方面获得了重大吸引力。其中,许多人依靠同态加密(HE),这是一种对加密数据进行计算的方法。但是,与他们的明文对应物相比,他的操作即使是最先进的计划仍然很慢。修剪NN模型的参数是改善推理潜伏期的众所周知的方法。但是,在明文上下文中有用的修剪方法可能对HE案的改善几乎可以忽略不计,这在最近的工作中也证明了这一点。在这项工作中,我们提出了一套新颖的修剪方法,以减少潜伏期和记忆要求,从而将明文修剪方法的有效性带到HE中。至关重要的是,我们的建议采用两种关键技术,即。堆积模型权重的置换和扩展,使修剪能够明显更多的密封性下文并分别恢复大部分精度损失。我们证明了我们的方法在完全连接的层上的优势,其中使用最近提出的称为瓷砖张量的包装技术填充了权重,该技术允许在非相互作用模式下执行Deep NN推断。我们在各种自动编码器架构上评估了我们的方法,并证明,对于MNIST上的小均值重建损失为1.5*10^{ - 5},我们将HE-SEAMABLE推断的内存要求和延迟减少了60%。
translated by 谷歌翻译
使用深度学习模型对敏感用户数据的处理是一个获得最近吸引力的领域。现有工作利用了同构加密(HE)方案来启用加密数据的计算。早期工作是加密货币,需要250秒才能进行一项MNIST推断。这种方法的主要局限性是在He-compypted Ciphertext上执行操作所需的类似FFT的操作。其他人建议使用模型修剪和有效的数据表示来减少所需的操作数量。我们专注于通过提出对CNN推断期间中间张量的表示的更改来改善现有工作。我们在MNIST和CIFAR-10数据集上构建和评估私有CNN,并在用于推广架构的推断的操作数量上减少了两倍。
translated by 谷歌翻译
神经网络的外包计算允许用户访问艺术模型的状态,而无需投资专门的硬件和专业知识。问题是用户对潜在的隐私敏感数据失去控制。通过同性恋加密(HE)可以在加密数据上执行计算,而不会显示其内容。在这种知识的系统化中,我们深入了解与隐私保留的神经网络相结合的方法。我们将更改分类为神经网络模型和架构,使其在他和这些变化的影响方面提供影响。我们发现众多挑战是基于隐私保留的深度学习,例如通过加密方案构成的计算开销,可用性和限制。
translated by 谷歌翻译
In skeleton-based action recognition, graph convolutional networks (GCNs), which model the human body skeletons as spatiotemporal graphs, have achieved remarkable performance. However, in existing GCN-based methods, the topology of the graph is set manually, and it is fixed over all layers and input samples. This may not be optimal for the hierarchical GCN and diverse samples in action recognition tasks. In addition, the second-order information (the lengths and directions of bones) of the skeleton data, which is naturally more informative and discriminative for action recognition, is rarely investigated in existing methods. In this work, we propose a novel two-stream adaptive graph convolutional network (2s-AGCN) for skeletonbased action recognition. The topology of the graph in our model can be either uniformly or individually learned by the BP algorithm in an end-to-end manner. This data-driven method increases the flexibility of the model for graph construction and brings more generality to adapt to various data samples. Moreover, a two-stream framework is proposed to model both the first-order and the second-order information simultaneously, which shows notable improvement for the recognition accuracy. Extensive experiments on the two large-scale datasets, NTU-RGBD and Kinetics-Skeleton, demonstrate that the performance of our model exceeds the state-of-the-art with a significant margin.
translated by 谷歌翻译
Machine learning is widely used in practice to produce predictive models for applications such as image processing, speech and text recognition. These models are more accurate when trained on large amount of data collected from different sources. However, the massive data collection raises privacy concerns.In this paper, we present new and efficient protocols for privacy preserving machine learning for linear regression, logistic regression and neural network training using the stochastic gradient descent method. Our protocols fall in the two-server model where data owners distribute their private data among two non-colluding servers who train various models on the joint data using secure two-party computation (2PC). We develop new techniques to support secure arithmetic operations on shared decimal numbers, and propose MPC-friendly alternatives to non-linear functions such as sigmoid and softmax that are superior to prior work. We implement our system in C++. Our experiments validate that our protocols are several orders of magnitude faster than the state of the art implementations for privacy preserving linear and logistic regressions, and scale to millions of data samples with thousands of features. We also implement the first privacy preserving system for training neural networks.
translated by 谷歌翻译
人类骨骼数据由于其背景鲁棒性和高效率而受到行动识别的越来越多。在基于骨架的动作识别中,图形卷积网络(GCN)已成为主流方法。本文分析了基于GCN的模型的基本因素 - 邻接矩阵。我们注意到,大多数基于GCN的方法基于人类天然骨架结构进行其邻接矩阵。根据我们以前的工作和分析,我们建议人类的自然骨骼结构邻接矩阵不适合基于骨架的动作识别。我们提出了一个新的邻接矩阵,该矩阵放弃了所有刚性邻居的连接,但使该模型可以适应地学习关节的关系。我们对两个基于骨架的动作识别数据集(NTURGBD60和FINEGYM)进行了验证模型进行广泛的实验和分析。全面的实验结果和分析表明,1)最广泛使用的人类天然骨骼结构邻接矩阵在基于骨架的动作识别中不适合; 2)所提出的邻接矩阵在模型性能,噪声稳健性和可传递性方面表现出色。
translated by 谷歌翻译
在基于骨架的动作识别中,图形卷积网络将人类骨骼关节模拟为顶点,并通过邻接矩阵将其连接起来,可以将其视为局部注意力掩码。但是,在大多数现有的图形卷积网络中,局部注意力面膜是根据人类骨架关节的自然连接来定义的,而忽略了例如头部,手和脚关节之间的动态关系。此外,注意机制已被证明在自然语言处理和图像描述中有效,在现有方法中很少研究。在这项工作中,我们提出了一个新的自适应空间注意层,该层将局部注意力图扩展到基于相对距离和相对角度信息的全局。此外,我们设计了一个连接头部,手脚的新初始图邻接矩阵,该矩阵在动作识别精度方面显示出可见的改进。在日常生活中人类活动领域的两个大规模且挑战性的数据集上,评估了该模型:NTU-RGB+D和动力学骨架。结果表明,我们的模型在两个数据集上都有很强的性能。
translated by 谷歌翻译
While machine learning is traditionally a resource intensive task, embedded systems, autonomous navigation, and the vision of the Internet of Things fuel the interest in resource-efficient approaches. These approaches aim for a carefully chosen trade-off between performance and resource consumption in terms of computation and energy. The development of such approaches is among the major challenges in current machine learning research and key to ensure a smooth transition of machine learning technology from a scientific environment with virtually unlimited computing resources into everyday's applications. In this article, we provide an overview of the current state of the art of machine learning techniques facilitating these real-world requirements. In particular, we focus on deep neural networks (DNNs), the predominant machine learning models of the past decade. We give a comprehensive overview of the vast literature that can be mainly split into three non-mutually exclusive categories: (i) quantized neural networks, (ii) network pruning, and (iii) structural efficiency. These techniques can be applied during training or as post-processing, and they are widely used to reduce the computational demands in terms of memory footprint, inference speed, and energy efficiency. We also briefly discuss different concepts of embedded hardware for DNNs and their compatibility with machine learning techniques as well as potential for energy and latency reduction. We substantiate our discussion with experiments on well-known benchmark datasets using compression techniques (quantization, pruning) for a set of resource-constrained embedded systems, such as CPUs, GPUs and FPGAs. The obtained results highlight the difficulty of finding good trade-offs between resource efficiency and predictive performance.
translated by 谷歌翻译
我们日常生活中的深度学习是普遍存在的,包括自驾车,虚拟助理,社交网络服务,医疗服务,面部识别等,但是深度神经网络在训练和推理期间需要大量计算资源。该机器学习界主要集中在模型级优化(如深度学习模型的架构压缩),而系统社区则专注于实施级别优化。在其间,在算术界中提出了各种算术级优化技术。本文在模型,算术和实施级技术方面提供了关于资源有效的深度学习技术的调查,并确定了三种不同级别技术的资源有效的深度学习技术的研究差距。我们的调查基于我们的资源效率度量定义,阐明了较低级别技术的影响,并探讨了资源有效的深度学习研究的未来趋势。
translated by 谷歌翻译
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems.This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry.The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.
translated by 谷歌翻译
在这项工作中,我们提出了一种新颖的矩阵编码方法,该方法对于神经网络特别方便,使用同构加密以隐私性的方式进行预测。基于这种编码方法,我们实施了一个卷积神经网络,以通过加密进行手写图像分类。对于两个矩阵$ a $和$ b $以执行同型乘法,其背后的主要想法是,在一个简单的版本中,分别将矩阵$ a $和矩阵$ b $的转置分别加密到两个密文中。通过其他操作,可以有效地通过加密的矩阵来计算同型矩阵乘法。对于卷积操作,我们提前跨越每个卷积内核到与输入图像相同大小的矩阵空间,以生成几个密文,后来将它们与密文加密输入图像一起使用,以计算一些最终的最终卷积结果。我们积累了所有这些中间结果,从而完成了卷积操作。在具有40 VCPU的公共云中,我们在MNIST测试数据集上的卷积神经网络实现需要$ \ sim $ 287秒,以计算十个可能的32个大小的加密图像$ 28 \ times 28 $同时。数据所有者只需要上传一个Ciphertext($ \ sim 19.8 $ MB),将这32张图像加密到公共云。
translated by 谷歌翻译
多年来,通过广泛研究了与量化的神经网络。遗憾的是,在GPU上的有限精度支持(例如,INT1和INT4)上通常限制具有多样化的精度(例如,1位重量和2位激活)的事先努力。为了打破这种限制,我们介绍了第一个任意精密神经网络框架(APNN-TC),以充分利用对AMPERE GPU张量核心的量化优势。具体地,APNN-TC首先结合了一种新的仿真算法来支持与INT1计算基元和XOR /和BOOLEAN操作的任意短比特宽度计算。其次,APNN-TC集成了任意精密层设计,以有效地将仿真算法映射到带有新型批处理策略和专业内存组织的张量核心。第三,APNN-TC体现了一种新型任意精密NN设计,可最大限度地减少层次的内存访问,并进一步提高性能。广泛的评估表明,APNN-TC可以通过Cutlass内核和各种NN模型实现显着加速,例如Reset和VGG。
translated by 谷歌翻译
图形卷积网络由于非欧几里得数据的出色建模能力而广泛用于基于骨架的动作识别。由于图形卷积是局部操作,因此它只能利用短距离关节依赖性和短期轨迹,但无法直接建模遥远的关节关系和远程时间信息,这些信息对于区分各种动作至关重要。为了解决此问题,我们提出了多尺度的空间图卷积(MS-GC)模块和一个多尺度的时间图卷积(MT-GC)模块,以在空间和时间尺寸中丰富模型的接受场。具体而言,MS-GC和MT-GC模块将相应的局部图卷积分解为一组子图形卷积,形成了层次的残差体系结构。在不引入其他参数的情况下,该功能将通过一系列子图卷积处理,每个节点都可以与其邻域一起完成多个空间和时间聚集。因此,最终的等效接收场被扩大,能够捕获空间和时间域中的短期和远程依赖性。通过将这两个模块耦合为基本块,我们进一步提出了一个多尺度的空间时间图卷积网络(MST-GCN),该网络(MST-GCN)堆叠了多个块以学习有效的运动表示行动识别的运动表示。拟议的MST-GCN在三个具有挑战性的基准数据集(NTU RGB+D,NTU-1220 RGB+D和动力学 - 骨骼)上实现了出色的性能,用于基于骨架的动作识别。
translated by 谷歌翻译
Spatial-temporal graphs have been widely used by skeleton-based action recognition algorithms to model human action dynamics. To capture robust movement patterns from these graphs, long-range and multi-scale context aggregation and spatial-temporal dependency modeling are critical aspects of a powerful feature extractor. However, existing methods have limitations in achieving (1) unbiased long-range joint relationship modeling under multiscale operators and (2) unobstructed cross-spacetime information flow for capturing complex spatial-temporal dependencies. In this work, we present (1) a simple method to disentangle multi-scale graph convolutions and (2) a unified spatial-temporal graph convolutional operator named G3D. The proposed multi-scale aggregation scheme disentangles the importance of nodes in different neighborhoods for effective long-range modeling. The proposed G3D module leverages dense cross-spacetime edges as skip connections for direct information propagation across the spatial-temporal graph. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model 1 outperforms previous state-of-the-art methods on three large-scale datasets: NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400.
translated by 谷歌翻译
建模各种时空依赖项是识别骨架序列中人类动作的关键。大多数现有方法过度依赖于遍历规则或图形拓扑的设计,以利用动态关节的依赖性,这是反映远处但重要的关节的关系不足。此外,由于本地采用的操作,因此在现有的工作中探索了重要的远程时间信息。为了解决这个问题,在这项工作中,我们提出了LSTA-Net:一种新型长期短期时空聚合网络,可以以时空的方式有效地捕获长/短距离依赖性。我们将我们的模型设计成纯粹的分解体系结构,可以交替执行空间特征聚合和时间特征聚合。为了改善特征聚合效果,还设计和采用了一种通道明智的注意机制。在三个公共基准数据集中进行了广泛的实验,结果表明,我们的方法可以在空间和时域中捕获长短短程依赖性,从而产生比其他最先进的方法更高的结果。代码可在https://github.com/tailin1009/lsta-net。
translated by 谷歌翻译
Dynamics of human body skeletons convey significant information for human action recognition. Conventional approaches for modeling skeletons usually rely on hand-crafted parts or traversal rules, thus resulting in limited expressive power and difficulties of generalization. In this work, we propose a novel model of dynamic skeletons called Spatial-Temporal Graph Convolutional Networks (ST-GCN), which moves beyond the limitations of previous methods by automatically learning both the spatial and temporal patterns from data. This formulation not only leads to greater expressive power but also stronger generalization capability. On two large datasets, Kinetics and NTU-RGBD, it achieves substantial improvements over mainstream methods.
translated by 谷歌翻译
胶囊网络(CAPSNET)是图像处理的新兴趋势。与卷积神经网络相反,CAPSNET不容易受到对象变形的影响,因为对象的相对空间信息在整个网络中保存。但是,它们的复杂性主要与胶囊结构和动态路由机制有关,这使得以其原始形式部署封闭式以由小型微控制器(MCU)供电的设备几乎是不合理的。在一个智力从云到边缘迅速转移的时代,这种高复杂性对在边缘的采用capsnets的采用构成了严重的挑战。为了解决此问题,我们提出了一个API,用于执行ARM Cortex-M和RISC-V MCUS中的量化capsnet。我们的软件内核扩展了ARM CMSIS-NN和RISC-V PULP-NN,以用8位整数作为操作数支持胶囊操作。随之而来的是,我们提出了一个框架,以执行CAPSNET的训练后量化。结果显示,记忆足迹的减少近75%,准确性损失范围从0.07%到0.18%。在吞吐量方面,我们的ARM Cortex-M API可以分别在仅119.94和90.60毫秒(MS)的中型胶囊和胶囊层执行(STM32H7555ZIT6U,Cortex-M7 @ 480 MHz)。对于GAP-8 SOC(RISC-V RV32IMCXPULP @ 170 MHz),延迟分别降至7.02和38.03 ms。
translated by 谷歌翻译