To increase the quality of citizens' lives, we designed a personalized smart chair system to recognize sitting behaviors. The system can receive surface pressure data from the designed sensor and provide feedback for guiding the user towards proper sitting postures. We used a liquid state machine and a logistic regression classifier to construct a spiking neural network for classifying 15 sitting postures. To allow this system to read our pressure data into the spiking neurons, we designed an algorithm to encode map-like data into cosine-rank sparsity data. The experimental results consisting of 15 sitting postures from 19 participants show that the prediction precision of our SNN is 88.52%.
translated by 谷歌翻译
Emergence of deep neural networks (DNNs) has raised enormous attention towards artificial neural networks (ANNs) once again. They have become the state-of-the-art models and have won different machine learning challenges. Although these networks are inspired by the brain, they lack biological plausibility, and they have structural differences compared to the brain. Spiking neural networks (SNNs) have been around for a long time, and they have been investigated to understand the dynamics of the brain. However, their application in real-world and complicated machine learning tasks were limited. Recently, they have shown great potential in solving such tasks. Due to their energy efficiency and temporal dynamics there are many promises in their future development. In this work, we reviewed the structures and performances of SNNs on image classification tasks. The comparisons illustrate that these networks show great capabilities for more complicated problems. Furthermore, the simple learning rules developed for SNNs, such as STDP and R-STDP, can be a potential alternative to replace the backpropagation algorithm used in DNNs.
translated by 谷歌翻译
The term ``neuromorphic'' refers to systems that are closely resembling the architecture and/or the dynamics of biological neural networks. Typical examples are novel computer chips designed to mimic the architecture of a biological brain, or sensors that get inspiration from, e.g., the visual or olfactory systems in insects and mammals to acquire information about the environment. This approach is not without ambition as it promises to enable engineered devices able to reproduce the level of performance observed in biological organisms -- the main immediate advantage being the efficient use of scarce resources, which translates into low power requirements. The emphasis on low power and energy efficiency of neuromorphic devices is a perfect match for space applications. Spacecraft -- especially miniaturized ones -- have strict energy constraints as they need to operate in an environment which is scarce with resources and extremely hostile. In this work we present an overview of early attempts made to study a neuromorphic approach in a space context at the European Space Agency's (ESA) Advanced Concepts Team (ACT).
translated by 谷歌翻译
为了在专门的神经形态硬件中进行节能计算,我们提出了尖峰神经编码,这是基于预测性编码理论的人工神经模型家族的实例化。该模型是同类模型,它是通过在“猜测和检查”的永无止境过程中运行的,神经元可以预测彼此的活动值,然后调整自己的活动以做出更好的未来预测。我们系统的互动性,迭代性质非常适合感官流预测的连续时间表述,并且如我们所示,模型的结构产生了局部突触更新规则,可以用来补充或作为在线峰值定位的替代方案依赖的可塑性。在本文中,我们对模型的实例化进行了实例化,该模型包括泄漏的集成和火灾单元。但是,我们系统所在的框架自然可以结合更复杂的神经元,例如Hodgkin-Huxley模型。我们在模式识别方面的实验结果证明了当二进制尖峰列车是通信间通信的主要范式时,模型的潜力。值得注意的是,尖峰神经编码在分类绩效方面具有竞争力,并且在从任务序列中学习时会降低遗忘,从而提供了更经济的,具有生物学上的替代品,可用于流行的人工神经网络。
translated by 谷歌翻译
In the past years, artificial neural networks (ANNs) have become the de-facto standard to solve tasks in communications engineering that are difficult to solve with traditional methods. In parallel, the artificial intelligence community drives its research to biology-inspired, brain-like spiking neural networks (SNNs), which promise extremely energy-efficient computing. In this paper, we investigate the use of SNNs in the context of channel equalization for ultra-low complexity receivers. We propose an SNN-based equalizer with a feedback structure akin to the decision feedback equalizer (DFE). For conversion of real-world data into spike signals we introduce a novel ternary encoding and compare it with traditional log-scale encoding. We show that our approach clearly outperforms conventional linear equalizers for three different exemplary channels. We highlight that mainly the conversion of the channel output to spikes introduces a small performance penalty. The proposed SNN with a decision feedback structure enables the path to competitive energy-efficient transceivers.
translated by 谷歌翻译
尖峰神经网络(SNN)提供了一个新的计算范式,能够高度平行,实时处理。光子设备是设计与SNN计算范式相匹配的高带宽,平行体系结构的理想选择。 CMO和光子元件的协整允许将低损耗的光子设备与模拟电子设备结合使用,以更大的非线性计算元件的灵活性。因此,我们在整体硅光子学(SIPH)过程上设计和模拟了光电尖峰神经元电路,该过程复制了超出泄漏的集成和火(LIF)之外有用的尖峰行为。此外,我们探索了两种学习算法,具有使用Mach-Zehnder干涉法(MZI)网格作为突触互连的片上学习的潜力。实验证明了随机反向传播(RPB)的变体,并在简单分类任务上与标准线性回归的性能相匹配。同时,将对比性HEBBIAN学习(CHL)规则应用于由MZI网格组成的模拟神经网络,以进行随机输入输出映射任务。受CHL训练的MZI网络的性能比随机猜测更好,但不符合理想神经网络的性能(没有MZI网格施加的约束)。通过这些努力,我们证明了协调的CMO和SIPH技术非常适合可扩展的SNN计算体系结构的设计。
translated by 谷歌翻译
这项研究提出了依赖电压突触可塑性(VDSP),这是一种新型的脑启发的无监督的本地学习规则,用于在线实施HEBB对神经形态硬件的可塑性机制。拟议的VDSP学习规则仅更新了突触后神经元的尖峰的突触电导,这使得相对于标准峰值依赖性可塑性(STDP)的更新数量减少了两倍。此更新取决于突触前神经元的膜电位,该神经元很容易作为神经元实现的一部分,因此不需要额外的存储器来存储。此外,该更新还对突触重量进行了正规化,并防止重复刺激时的重量爆炸或消失。进行严格的数学分析以在VDSP和STDP之间达到等效性。为了验证VDSP的系统级性能,我们训练一个单层尖峰神经网络(SNN),以识别手写数字。我们报告85.01 $ \ pm $ 0.76%(平均$ \ pm $ s.d。)对于MNIST数据集中的100个输出神经元网络的精度。在缩放网络大小时,性能会提高(400个输出神经元的89.93 $ \ pm $ 0.41%,500个神经元为90.56 $ \ pm $ 0.27),这验证了大规模计算机视觉任务的拟议学习规则的适用性。有趣的是,学习规则比STDP更好地适应输入信号的频率,并且不需要对超参数进行手动调整。
translated by 谷歌翻译
Spiking neural networks (SNN) are a viable alternative to conventional artificial neural networks when energy efficiency and computational complexity are of importance. A major advantage of SNNs is their binary information transfer through spike trains. The training of SNN has, however, been a challenge, since neuron models are non-differentiable and traditional gradient-based backpropagation algorithms cannot be applied directly. Furthermore, spike-timing-dependent plasticity (STDP), albeit being a spike-based learning rule, updates weights locally and does not optimize for the output error of the network. We present desire backpropagation, a method to derive the desired spike activity of neurons from the output error. The loss function can then be evaluated locally for every neuron. Incorporating the desire values into the STDP weight update leads to global error minimization and increasing classification accuracy. At the same time, the neuron dynamics and computational efficiency of STDP are maintained, making it a spike-based supervised learning rule. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41% and 87.56%, respectively. Furthermore, we show that desire backpropagation is computationally less complex than backpropagation in traditional neural networks.
translated by 谷歌翻译
更具体地说,神经系统能够简单有效地解决复杂的问题,超过现代计算机。在这方面,神经形态工程是一个研究领域,重点是模仿控制大脑的基本原理,以开发实现此类计算能力的系统。在该领域中,生物启发的学习和记忆系统仍然是要解决的挑战,这就是海马涉及的地方。正是大脑的区域充当短期记忆,从而从大脑皮层的所有感觉核中学习,非结构化和快速存储信息及其随后的回忆。在这项工作中,我们提出了一个基于海马的新型生物启发的记忆模型,具有学习记忆的能力,从提示中回顾它们(与其他内容相关的记忆的一部分),甚至在尝试时忘记记忆通过相同的提示学习其他人。该模型已在使用尖峰神经网络上在大型摩托车硬件平台上实现,并进行了一组实验和测试以证明其正确且预期的操作。所提出的基于SPIKE的内存模型仅在接收输入,能提供节能的情况下才能生成SPIKES,并且需要7个时间步,用于学习步骤和6个时间段来召回以前存储的存储器。这项工作介绍了基于生物启发的峰值海马记忆模型的第一个硬件实现,为开发未来更复杂的神经形态系统的发展铺平了道路。
translated by 谷歌翻译
神经形态计算是一种新兴的计算范式,它从批处理的处理转向在线,事件驱动的流数据处理。当神经形态芯片与基于尖峰的传感器结合在一起时,只有在峰值时间内记录相关事件并证明对变化条件的低延迟响应时,才能通过消耗能量来固有地适应数据分布的“语义”。环境。本文为神经形态无线网络系统系统提出了端到端设计,该系统集成了基于尖峰的传感,处理和通信。在拟议的神经系统系统中,每个传感设备都配备了神经形态传感器,尖峰神经网络(SNN)和带有多个天线的脉冲无线电发射器。传输发生在配备了多Antenna脉冲无线电接收器和SNN的接收器上的共享褪色通道上进行。为了使接收器适应褪色的通道条件,我们引入了一项超网络,以使用飞行员控制解码SNN的权重。飞行员,编码SNN,解码SNN和超网络经过多个通道实现的共同训练。该系统被证明可以显着改善基于传统的基于框架的数字解决方案以及替代性非自适应训练方法,从时间到准确性和能源消耗指标方面。
translated by 谷歌翻译
尽管人工智能模型的进步,神经网络仍然无法实现人的表现,部分原因是由于信息是如何编码,并与人脑处理分歧。在一个人工神经网络(ANN)信息是使用统计方法来表示和处理为拟合函数,使在图像,文本和语音处理处理的结构模式。然而,实质性的变化的数据,例如统计特性,扭转的图像的背景,显着降低性能。在这里,我们提出了一个量子叠加扣球量子机制和现象在大脑中,它能够处理图像背景色的反转激发神经网络(QS-SNN)。的QS-SNN结合量子理论与脑启发从计算的角度来看尖峰神经网络模型,从而产生更鲁棒的性能与传统的人工神经网络模型进行比较,处理嘈杂输入时尤其如此。这里给出的结果将成为今后努力开发大脑启发的人工智能。
translated by 谷歌翻译
Event-based simulations of Spiking Neural Networks (SNNs) are fast and accurate. However, they are rarely used in the context of event-based gradient descent because their implementations on GPUs are difficult. Discretization with the forward Euler method is instead often used with gradient descent techniques but has the disadvantage of being computationally expensive. Moreover, the lack of precision of discretized simulations can create mismatches between the simulated models and analog neuromorphic hardware. In this work, we propose a new exact error-backpropagation through spikes method for SNNs, extending Fast \& Deep to multiple spikes per neuron. We show that our method can be efficiently implemented on GPUs in a fully event-based manner, making it fast to compute and precise enough for analog neuromorphic hardware. Compared to the original Fast \& Deep and the current state-of-the-art event-based gradient-descent algorithms, we demonstrate increased performance on several benchmark datasets with both feedforward and convolutional SNNs. In particular, we show that multi-spike SNNs can have advantages over single-spike networks in terms of convergence, sparsity, classification latency and sensitivity to the dead neuron problem.
translated by 谷歌翻译
尖峰神经网络(SNN)引起了脑启发的人工智能和计算神经科学的广泛关注。它们可用于在多个尺度上模拟大脑中的生物信息处理。更重要的是,SNN是适当的抽象水平,可以将大脑和认知的灵感带入人工智能。在本文中,我们介绍了脑启发的认知智力引擎(Braincog),用于创建脑启发的AI和脑模拟模型。 Braincog将不同类型的尖峰神经元模型,学习规则,大脑区域等作为平台提供的重要模块。基于这些易于使用的模块,BrainCog支持各种受脑启发的认知功能,包括感知和学习,决策,知识表示和推理,运动控制和社会认知。这些受脑启发的AI模型已在各种受监督,无监督和强化学习任务上有效验证,并且可以用来使AI模型具有多种受脑启发的认知功能。为了进行大脑模拟,Braincog实现了决策,工作记忆,神经回路的结构模拟以及小鼠大脑,猕猴大脑和人脑的整个大脑结构模拟的功能模拟。一个名为BORN的AI引擎是基于Braincog开发的,它演示了如何将Braincog的组件集成并用于构建AI模型和应用。为了使科学追求解码生物智能的性质并创建AI,Braincog旨在提供必要且易于使用的构件,并提供基础设施支持,以开发基于脑部的尖峰神经网络AI,并模拟认知大脑在多个尺度上。可以在https://github.com/braincog-x上找到Braincog的在线存储库。
translated by 谷歌翻译
我们提出了Memprop,即采用基于梯度的学习来培训完全的申请尖峰神经网络(MSNNS)。我们的方法利用固有的设备动力学来触发自然产生的电压尖峰。这些由回忆动力学发出的尖峰本质上是类似物,因此完全可区分,这消除了尖峰神经网络(SNN)文献中普遍存在的替代梯度方法的需求。回忆性神经网络通常将备忘录集成为映射离线培训网络的突触,或者以其他方式依靠关联学习机制来训练候选神经元的网络。相反,我们直接在循环神经元和突触的模拟香料模型上应用了通过时间(BPTT)训练算法的反向传播。我们的实现是完全的综合性,因为突触重量和尖峰神经元都集成在电阻RAM(RRAM)阵列上,而无需其他电路来实现尖峰动态,例如模数转换器(ADCS)或阈值比较器。结果,高阶电物理效应被充分利用,以在运行时使用磁性神经元的状态驱动动力学。通过朝着非同一梯度的学习迈进,我们在以前报道的几个基准上的轻巧密集的完全MSNN中获得了高度竞争的准确性。
translated by 谷歌翻译
神经形态计算是一个新兴的研究领域,旨在通过整合来自神经科学和深度学习等多学科的理论和技术来开发新的智能系统。当前,已经为相关字段开发了各种软件框架,但是缺乏专门用于基于Spike的计算模型和算法的有效框架。在这项工作中,我们提出了一个基于Python的尖峰神经网络(SNN)模拟和培训框架,又名Spaic,旨在支持脑启发的模型和算法研究,并与深度学习和神经科学的特征集成在一起。为了整合两个压倒性学科的不同方法,以及灵活性和效率之间的平衡,SpaiC设计采用神经科学风格的前端和深度学习后端结构设计。我们提供了广泛的示例,包括神经回路模拟,深入的SNN学习和神经形态应用,展示了简洁的编码样式和框架的广泛可用性。 Spaic是一个专用的基于SPIKE的人工智能计算平台,它将显着促进新模型,理论和应用的设计,原型和验证。具有用户友好,灵活和高性能,它将有助于加快神经形态计算研究的快速增长和广泛的适用性。
translated by 谷歌翻译
尖峰神经网络(SNN)被认为是执行各种学习任务的透视基础 - 无监督,监督和强化学习。通过突触可塑性实施SNN的学习 - 根据通常和突触后神经元的活性确定突触权重的规则。各种学习制度的多样性假设不同形式的突触可塑性可能是最有效的,例如,无监督和监督学习,因为它在生活神经元中观察到了从基本尖峰定时依赖性塑性(STDP)模型的许多偏差。在本文中,我们向无监督学习问题施加的可塑性规则制定具体要求,并构建新的可塑性模型概括STDP并满足这些要求。这种可塑性模型作为本工作中提出的新型监督学习算法的主要逻辑组成部分,称为Scobul(基于尖峰相关的学习)。我们还介绍了确认这些突触塑性规则和算法Scobul效率的计算机仿真实验结果。
translated by 谷歌翻译
Neuromorphic systems require user-friendly software to support the design and optimization of experiments. In this work, we address this need by presenting our development of a machine learning-based modeling framework for the BrainScaleS-2 neuromorphic system. This work represents an improvement over previous efforts, which either focused on the matrix-multiplication mode of BrainScaleS-2 or lacked full automation. Our framework, called hxtorch.snn, enables the hardware-in-the-loop training of spiking neural networks within PyTorch, including support for auto differentiation in a fully-automated hardware experiment workflow. In addition, hxtorch.snn facilitates seamless transitions between emulating on hardware and simulating in software. We demonstrate the capabilities of hxtorch.snn on a classification task using the Yin-Yang dataset employing a gradient-based approach with surrogate gradients and densely sampled membrane observations from the BrainScaleS-2 hardware system.
translated by 谷歌翻译
神经形态计算机通过模拟人脑进行计算,并使用极低的功率。预计将来对于节能计算是必不可少的。尽管它们主要用于尖峰基于神经网络的机器学习应用程序,但已知神经形态计算机是Turing-Complete,因此能够进行通用计算。但是,为了充分意识到它们的通用,节能计算的潜力,重要的是要设计有效的编码数字机制。当前的编码方法的适用性有限,可能不适合通用计算。在本文中,我们将虚拟神经元视为整数和理性数字的编码机制。我们评估虚拟神经元在物理和模拟神经形态硬件上的性能,并表明它可以使用基于混合信号的Memristor神经形态处理器平均使用23 nj的能量执行加法操作。我们还通过在某些MU回复功能中使用它来证明其实用性,这些功能是通用计算的构建块。
translated by 谷歌翻译
近年来,尖峰神经网络(SNN)由于其丰富的时空动力学,各种编码方法和事件驱动的特征而自然拟合神经形态硬件,因此在脑启发的智能上受到了广泛的关注。随着SNN的发展,受到脑科学成就启发和针对人工通用智能的新兴研究领域的脑力智能变得越来越热。本文回顾了最新进展,并讨论了来自五个主要研究主题的SNN的新领域,包括基本要素(即尖峰神经元模型,编码方法和拓扑结构),神经形态数据集,优化算法,软件,软件和硬件框架。我们希望我们的调查能够帮助研究人员更好地了解SNN,并激发新作品以推进这一领域。
translated by 谷歌翻译
穗状花序的神经形状硬件占据了深度神经网络(DNN)的更节能实现的承诺,而不是GPU的标准硬件。但这需要了解如何在基于事件的稀疏触发制度中仿真DNN,否则能量优势丢失。特别地,解决序列处理任务的DNN通常采用难以使用少量尖峰效仿的长短期存储器(LSTM)单元。我们展示了许多生物神经元的面部,在每个尖峰后缓慢的超积极性(AHP)电流,提供了有效的解决方案。 AHP电流可以轻松地在支持多舱神经元模型的神经形状硬件中实现,例如英特尔的Loihi芯片。滤波近似理论解释为什么AHP-Neurons可以模拟LSTM单元的功能。这产生了高度节能的时间序列分类方法。此外,它为实现了非常稀疏的大量大型DNN来实现基础,这些大型DNN在文本中提取单词和句子之间的关系,以便回答有关文本的问题。
translated by 谷歌翻译