自动驾驶(AD)相关功能代表了下一代移动机器人和专注于越来越智能,自主和互连系统的自动驾驶汽车的重要元素。根据定义,必须提供涉及使用这些功能的应用程序,并且此属性是避免灾难性事故的关键。此外,所有决策过程都必须需要低功耗,以增加电池驱动系统的寿命和自主权。这些挑战可以通过有效实施神经形态芯片上的尖峰神经网络(SNN)以及使用基于事件的摄像机而不是传统基于框架的摄像机来解决这些挑战。在本文中,我们提出了一种新的基于SNN的方法,称为Lanesnn,用于使用基于事件的相机输入来检测街道上标记的车道。我们开发了四种以低复杂性和快速响应为特征的小说SNN模型,并使用离线监督的学习规则训练它们。之后,我们将学习的SNNS模型实施并映射到Intel Loihi神经形态研究芯片上。对于损耗函数,我们基于加权二进制交叉熵(WCE)和均方误差(MSE)度量的线性组成而开发了一种新颖的方法。我们的实验结果表明,与联合(IOU)度量的最大交叉点约为0.62,功耗非常低约1W。最好的IOU是通过SNN实现实现的,该实现仅占据Loihi处理器上的36个神经可孔,同时提供低潜伏期少于8 ms识别图像,从而实现实时性能。我们网络提供的IOU措施与最先进的措施相当,但功率消耗为1W。
translated by 谷歌翻译
现代车辆配备各种驾驶员辅助系统,包括自动车道保持,这防止了无意的车道偏离。传统车道检测方法采用了手工制作或基于深度的学习功能,然后使用基于帧的RGB摄像机进行通道提取的后处理技术。用于车道检测任务的帧的RGB摄像机的利用易于照明变化,太阳眩光和运动模糊,这限制了车道检测方法的性能。在自主驾驶中的感知堆栈中结合了一个事件摄像机,用于自动驾驶的感知堆栈是用于减轻基于帧的RGB摄像机遇到的挑战的最有希望的解决方案之一。这项工作的主要贡献是设计车道标记检测模型,它采用动态视觉传感器。本文探讨了使用事件摄像机通过设计卷积编码器后跟注意引导的解码器的新颖性应用了车道标记检测。编码特征的空间分辨率由致密的区域空间金字塔池(ASPP)块保持。解码器中的添加剂注意机制可提高促进车道本地化的高维输入编码特征的性能,并缓解后处理计算。使用DVS数据集进行通道提取(DET)的DVS数据集进行评估所提出的工作的功效。实验结果表明,多人和二进制车道标记检测任务中的5.54 \%$ 5.54 \%$ 5.54 \%$ 5.03 \%$ 5.03 \%$ 5.03。此外,在建议方法的联盟($ iou $)分数上的交叉点将超越最佳最先进的方法,分别以6.50 \%$ 6.50 \%$ 6.5.37 \%$ 9.37 \%$ 。
translated by 谷歌翻译
The term ``neuromorphic'' refers to systems that are closely resembling the architecture and/or the dynamics of biological neural networks. Typical examples are novel computer chips designed to mimic the architecture of a biological brain, or sensors that get inspiration from, e.g., the visual or olfactory systems in insects and mammals to acquire information about the environment. This approach is not without ambition as it promises to enable engineered devices able to reproduce the level of performance observed in biological organisms -- the main immediate advantage being the efficient use of scarce resources, which translates into low power requirements. The emphasis on low power and energy efficiency of neuromorphic devices is a perfect match for space applications. Spacecraft -- especially miniaturized ones -- have strict energy constraints as they need to operate in an environment which is scarce with resources and extremely hostile. In this work we present an overview of early attempts made to study a neuromorphic approach in a space context at the European Space Agency's (ESA) Advanced Concepts Team (ACT).
translated by 谷歌翻译
由于它们的时间加工能力及其低交换(尺寸,重量和功率)以及神经形态硬件中的节能实现,尖峰神经网络(SNNS)已成为传统人工神经网络(ANN)的有趣替代方案。然而,培训SNNS所涉及的挑战在准确性方面有限制了它们的表现,从而限制了他们的应用。因此,改善更准确的特征提取的学习算法和神经架构是SNN研究中的当前优先级之一。在本文中,我们展示了现代尖峰架构的关键组成部分的研究。我们在从最佳执行网络中凭经验比较了图像分类数据集中的不同技术。我们设计了成功的残余网络(Reset)架构的尖峰版本,并测试了不同的组件和培训策略。我们的结果提供了SNN设计的最新版本,它允许在尝试构建最佳视觉特征提取器时进行明智的选择。最后,我们的网络优于CIFAR-10(94.1%)和CIFAR-100(74.5%)数据集的先前SNN架构,并将现有技术与DVS-CIFAR10(71.3%)相匹配,参数较少而不是先前的状态艺术,无需安静转换。代码在https://github.com/vicenteax/spiking_resnet上获得。
translated by 谷歌翻译
尖峰神经网络已显示出具有人工神经网络的节能替代品。但是,对于常见的神经形态视觉基准(如分类),了解传感器噪声和输入编码对网络活动和性能的影响仍然很困难。因此,我们提出了一种使用替代梯度下降训练的单个对象定位的尖峰神经网络方法,用于基于框架和事件的传感器。我们将我们的方法与类似的人工神经网络进行比较,并表明我们的模型在准确性,对各种腐败的鲁棒性方面具有竞争力/更好的性能,并且能耗较低。此外,我们研究了神经编码方案对准确性,鲁棒性和能源效率的静态图像的影响。我们的观察结果与以前关于生物成分学习规则的研究重要差​​异,该规则有助于设计替代梯度训练的体系结构,并就噪声特征和数据编码方法方面的未来神经形态技术设计优先级。
translated by 谷歌翻译
穗状花序的神经形状硬件占据了深度神经网络(DNN)的更节能实现的承诺,而不是GPU的标准硬件。但这需要了解如何在基于事件的稀疏触发制度中仿真DNN,否则能量优势丢失。特别地,解决序列处理任务的DNN通常采用难以使用少量尖峰效仿的长短期存储器(LSTM)单元。我们展示了许多生物神经元的面部,在每个尖峰后缓慢的超积极性(AHP)电流,提供了有效的解决方案。 AHP电流可以轻松地在支持多舱神经元模型的神经形状硬件中实现,例如英特尔的Loihi芯片。滤波近似理论解释为什么AHP-Neurons可以模拟LSTM单元的功能。这产生了高度节能的时间序列分类方法。此外,它为实现了非常稀疏的大量大型DNN来实现基础,这些大型DNN在文本中提取单词和句子之间的关系,以便回答有关文本的问题。
translated by 谷歌翻译
在本文中,我们为两个静态的美国手语(ASL)手势分类任务(即ASL字母和ASL数字)开发了四个尖峰神经网络(SNN)模型。SNN模型部署在英特尔的神经形态平台上,然后与部署在边缘计算设备(Intel神经计算棒2(NCS2))上的等效深神经网络(DNN)模型进行了比较。在准确性,延迟,功耗和能源方面,我们进行了两种系统之间的全面比较。最佳DNN模型在ASL字母数据集上的精度为99.6%,而最佳性能SNN模型的精度为99.44%。对于ASL数字数据集,最好的SNN模型以99.52%的精度优于其所有DNN对应物。此外,我们获得的实验结果表明,与NCS2相比,Loihi神经形态硬件的实现分别可降低14.67倍和4.09倍。
translated by 谷歌翻译
Spiking neural networks (SNN) are a viable alternative to conventional artificial neural networks when energy efficiency and computational complexity are of importance. A major advantage of SNNs is their binary information transfer through spike trains. The training of SNN has, however, been a challenge, since neuron models are non-differentiable and traditional gradient-based backpropagation algorithms cannot be applied directly. Furthermore, spike-timing-dependent plasticity (STDP), albeit being a spike-based learning rule, updates weights locally and does not optimize for the output error of the network. We present desire backpropagation, a method to derive the desired spike activity of neurons from the output error. The loss function can then be evaluated locally for every neuron. Incorporating the desire values into the STDP weight update leads to global error minimization and increasing classification accuracy. At the same time, the neuron dynamics and computational efficiency of STDP are maintained, making it a spike-based supervised learning rule. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41% and 87.56%, respectively. Furthermore, we show that desire backpropagation is computationally less complex than backpropagation in traditional neural networks.
translated by 谷歌翻译
Event-based simulations of Spiking Neural Networks (SNNs) are fast and accurate. However, they are rarely used in the context of event-based gradient descent because their implementations on GPUs are difficult. Discretization with the forward Euler method is instead often used with gradient descent techniques but has the disadvantage of being computationally expensive. Moreover, the lack of precision of discretized simulations can create mismatches between the simulated models and analog neuromorphic hardware. In this work, we propose a new exact error-backpropagation through spikes method for SNNs, extending Fast \& Deep to multiple spikes per neuron. We show that our method can be efficiently implemented on GPUs in a fully event-based manner, making it fast to compute and precise enough for analog neuromorphic hardware. Compared to the original Fast \& Deep and the current state-of-the-art event-based gradient-descent algorithms, we demonstrate increased performance on several benchmark datasets with both feedforward and convolutional SNNs. In particular, we show that multi-spike SNNs can have advantages over single-spike networks in terms of convergence, sparsity, classification latency and sensitivity to the dead neuron problem.
translated by 谷歌翻译
我们最近提出了S4NN算法,基本上是对多层尖峰神经网络的反向化的适应,该网上网络使用简单的非泄漏整合和火神经元和一种形式称为第一峰值编码的时间编码。通过这种编码方案,每次刺激最多一次都是神经元火灾,但射击令携带信息。这里,我们引入BS4NN,S4NN的修改,其中突触权重被约束为二进制(+1或-1),以便减少存储器(理想情况下,每个突触的一个比特)和计算占地面积。这是使用两组权重完成:首先,通过梯度下降更新的实际重量,并在BackProjagation的后退通行证中使用,其次是在前向传递中使用的迹象。类似的策略已被用于培训(非尖峰)二值化神经网络。主要区别在于BS4NN在时域中操作:尖峰依次繁殖,并且不同的神经元可以在不同时间达到它们的阈值,这增加了计算能力。我们验证了两个流行的基准,Mnist和Fashion-Mnist上的BS4NN,并获得了这种网络的合理精度(分别为97.0%和87.3%),具有可忽略的准确率,具有可忽略的重量率(0.4%和0.7%,分别)。我们还展示了BS4NN优于具有相同架构的简单BNN,在这两个数据集上(分别为0.2%和0.9%),可能是因为它利用时间尺寸。建议的BS4NN的源代码在HTTPS://github.com/srkh/bs4nn上公开可用。
translated by 谷歌翻译
Spiking Neural Networks (SNNs) are bio-plausible models that hold great potential for realizing energy-efficient implementations of sequential tasks on resource-constrained edge devices. However, commercial edge platforms based on standard GPUs are not optimized to deploy SNNs, resulting in high energy and latency. While analog In-Memory Computing (IMC) platforms can serve as energy-efficient inference engines, they are accursed by the immense energy, latency, and area requirements of high-precision ADCs (HP-ADC), overshadowing the benefits of in-memory computations. We propose a hardware/software co-design methodology to deploy SNNs into an ADC-Less IMC architecture using sense-amplifiers as 1-bit ADCs replacing conventional HP-ADCs and alleviating the above issues. Our proposed framework incurs minimal accuracy degradation by performing hardware-aware training and is able to scale beyond simple image classification tasks to more complex sequential regression tasks. Experiments on complex tasks of optical flow estimation and gesture recognition show that progressively increasing the hardware awareness during SNN training allows the model to adapt and learn the errors due to the non-idealities associated with ADC-Less IMC. Also, the proposed ADC-Less IMC offers significant energy and latency improvements, $2-7\times$ and $8.9-24.6\times$, respectively, depending on the SNN model and the workload, compared to HP-ADC IMC.
translated by 谷歌翻译
深度估计是一个重要的计算机视觉任务,特别是用于自主车辆中的导航,或者在机器人中的对象操纵。在这里,我们使用端到端的神经形态方法解决了它,将两个事件的相机和尖峰神经网络(SNN)与略微修改的U-Net的编码器 - 解码器架构结合起来,我们命名为Sterepike。更具体地说,我们使用了多车辆立体声事件相机数据集(MVSEC)。它提供了深度地面真理,用于使用替代梯度下降以监督方式训练立体摩托车。我们提出了一种新颖的读数范式来获得密集的模拟预测 - 从解码器的尖峰中获得每个像素的深度。我们证明,这种体系结构概括得非常好,甚至比其非尖峰对应物更好,导致最先进的测试精度。据我们所知,这是第一次通过完全尖峰网络解决了这样一个大规模的回归问题。最后,我们表明,可以通过规范化获得低发射速率(<10%),精度最低的成本。这意味着可以在神经芯片上有效地实现Sterepositike,用于为低功率和实时嵌入式系统开门。
translated by 谷歌翻译
Emergence of deep neural networks (DNNs) has raised enormous attention towards artificial neural networks (ANNs) once again. They have become the state-of-the-art models and have won different machine learning challenges. Although these networks are inspired by the brain, they lack biological plausibility, and they have structural differences compared to the brain. Spiking neural networks (SNNs) have been around for a long time, and they have been investigated to understand the dynamics of the brain. However, their application in real-world and complicated machine learning tasks were limited. Recently, they have shown great potential in solving such tasks. Due to their energy efficiency and temporal dynamics there are many promises in their future development. In this work, we reviewed the structures and performances of SNNs on image classification tasks. The comparisons illustrate that these networks show great capabilities for more complicated problems. Furthermore, the simple learning rules developed for SNNs, such as STDP and R-STDP, can be a potential alternative to replace the backpropagation algorithm used in DNNs.
translated by 谷歌翻译
这项研究提出了依赖电压突触可塑性(VDSP),这是一种新型的脑启发的无监督的本地学习规则,用于在线实施HEBB对神经形态硬件的可塑性机制。拟议的VDSP学习规则仅更新了突触后神经元的尖峰的突触电导,这使得相对于标准峰值依赖性可塑性(STDP)的更新数量减少了两倍。此更新取决于突触前神经元的膜电位,该神经元很容易作为神经元实现的一部分,因此不需要额外的存储器来存储。此外,该更新还对突触重量进行了正规化,并防止重复刺激时的重量爆炸或消失。进行严格的数学分析以在VDSP和STDP之间达到等效性。为了验证VDSP的系统级性能,我们训练一个单层尖峰神经网络(SNN),以识别手写数字。我们报告85.01 $ \ pm $ 0.76%(平均$ \ pm $ s.d。)对于MNIST数据集中的100个输出神经元网络的精度。在缩放网络大小时,性能会提高(400个输出神经元的89.93 $ \ pm $ 0.41%,500个神经元为90.56 $ \ pm $ 0.27),这验证了大规模计算机视觉任务的拟议学习规则的适用性。有趣的是,学习规则比STDP更好地适应输入信号的频率,并且不需要对超参数进行手动调整。
translated by 谷歌翻译
由于稀疏,异步和二进制事件(或尖峰)驱动加工,尖峰神经网络(SNNS)最近成为深度学习的替代方案,可以在神经形状硬件上产生巨大的能效益。然而,从划痕训练高精度和低潜伏期的SNN,患有尖刺神经元的非微弱性质。要在SNNS中解决此培训问题,我们重新批准批量标准化,并通过时间(BNTT)技术提出时间批量标准化。大多数先前的SNN工程到现在忽略了批量标准化,认为它无效地训练时间SNN。与以前的作品不同,我们提出的BNTT沿着时轴沿着时间轴解耦的参数,以捕获尖峰的时间动态。在BNTT中的时间上不断发展的可学习参数允许神经元通过不同的时间步长来控制其尖峰率,从头开始实现低延迟和低能量训练。我们对CiFar-10,CiFar-100,微小想象特和事件驱动的DVS-CIFAR10数据集进行实验。 BNTT允许我们首次在三个复杂的数据集中培训深度SNN架构,只需25-30步即可。我们还使用BNTT中的参数分布提前退出算法,以降低推断的延迟,进一步提高了能量效率。
translated by 谷歌翻译
由于具有高生物学合理性和低能消耗在神经形态硬件上的特性,因此尖峰神经网络(SNN)非常重要。作为获得深SNN的有效方法,转化方法在各种大型数据集上表现出高性能。但是,它通常遭受严重的性能降解和高时间延迟。特别是,以前的大多数工作都集中在简单的分类任务上,同时忽略了与ANN输出的精确近似。在本文中,我们首先从理论上分析转换误差,并得出时间变化极端对突触电流的有害影响。我们提出尖峰校准(Spicalib),以消除离散尖峰对输出分布的损坏,并修改脂肪,以使任意最大化层无损地转换。此外,提出了针对最佳标准化参数的贝叶斯优化,以避免经验设置。实验结果证明了分类,对象检测和分割任务的最新性能。据我们所知,这是第一次获得与ANN同时在这些任务上相当的SNN。此外,我们只需要先前在检测任务上工作的1/50推理时间,并且可以在0.492 $ \ times $ $下在分段任务上实现相同的性能。
translated by 谷歌翻译
尖峰神经网络(SNNS)是一种实用方法,可以通过模拟神经元对时间信息的杠杆作用来进行更高的数据有效学习。在本文中,我们提出了时间通道联合注意(TCJA)架构单元,这是一种有效的SNN技术,依赖于注意机制,通过有效地沿空间和时间维度沿着尖峰序列的相关性来实现。我们的基本技术贡献在于:1)通过采用挤压操作,将尖峰流压缩为平均矩阵,然后使用具有高效1-D卷积的两种局部注意机制来建立时间和渠道关系,以在频道和渠道关系中进行特征提取灵活的时尚。 2)利用交叉卷积融合(CCF)层在时间范围和通道范围之间建模相互依赖性,从而破坏了两个维度的独立性,并实现了特征之间的相互作用。通过共同探索和重新启用数据流,我们的方法在所有测试的主流静态和神经形态数据集上,在包括时尚量的所有测试的主流静态数据集上,最高可先进的(SOTA)高达15.7% ,CIFAR10-DVS,N-Caltech 101和DVS128手势。
translated by 谷歌翻译
图形卷积网络(GCN)由于学习图信息的显着表示能力而实现了令人印象深刻的性能。但是,GCN在深网上实施时需要昂贵的计算功率,因此很难将其部署在电池供电的设备上。相比之下,执行生物保真推理过程的尖峰神经网络(SNN)提供了节能的神经结构。在这项工作中,我们提出了SpikingGCN,这是一个端到端框架,旨在将GCN的嵌入与SNN的生物层性特征相结合。原始图数据根据图形卷积的合并编码为尖峰列车。我们通过利用与神经元节点结合的完全连接的层来进一步对生物信息处理进行建模。在各种场景(例如引用网络,图像图分类和推荐系统)中,我们的实验结果表明,该方法可以针对最新方法获得竞争性能。此外,我们表明,在神经形态芯片上的SpikingGCN可以将能源效率的明显优势带入图形数据分析中,这表明了其构建环境友好的机器学习模型的巨大潜力。
translated by 谷歌翻译
由于它们的低能量消耗,对神经形态计算设备上的尖刺神经网络(SNNS)越来越兴趣。最近的进展使培训SNNS在精度方面开始与传统人工神经网络(ANNS)进行竞争,同时在神经胸壁上运行时的节能。然而,培训SNNS的过程仍然基于最初为ANNS开发的密集的张量操作,这不利用SNN的时空稀疏性质。我们在这里介绍第一稀疏SNN BackPropagation算法,该算法与最新的现有技术实现相同或更好的准确性,同时显着更快,更高的记忆力。我们展示了我们对不同复杂性(时尚 - MNIST,神经影像学 - MNIST和Spiking Heidelberg数字的真实数据集的有效性,在不失精度的情况下实现了高达150倍的后向通行证的加速,而不会减少精度。
translated by 谷歌翻译
Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems.This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry.The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.
translated by 谷歌翻译