|
近日，华中科技大学郭新团队以「Memristor-based spiking neural networks: cooperative development of neural network architecture/algorithms and memristors」¹为题在Chip上发表长篇综述论文，详细阐述了脉冲神经网络的基本原理、忆阻器技术及其应用挑战。第一作者为彭辉辉，通讯作者为甘霖和郭新。本文被遴选为本期Featured in Chip编辑特选文章之一。Chip是全球唯一聚焦芯片类研究的综合性国际期刊，入选了国家高起点新刊计划的「三类高质量论文」期刊之一。
脉冲神经网络（Spiking Neural Network, SNN）是一种新型人工神经网络，受到人脑结构和原理启发，以低功耗脉冲传输和大规模并行特性备受关注。在SNN中，神经元需要对脉冲信号进行突触权重相乘和累加，但其结果并不是直接传递给激活函数。而是神经元在时间上累积这些信号的作用效果，当膜电位达到阈值时发放脉冲向后传递²，如图1a所示。SNN的硬件实现是其能耗进一步降低的关键。尽管SNN在软件层面已经取得了一定的成功，但在硬件实现上仍然面临挑战。忆阻器技术不仅可以通过非易失性模拟突触而且可以通过易失性模拟神经元，而且与互补式金属氧化物半导体（Complementary Metal-Oxide Semiconductor，CMOS）工艺兼容，被视为实现SNN硬件的理想候选者。
图1 脉冲神经网络模型示意图。a，典型的SNN结构，由前神经元驱动的后神经元组成；b，SNN流程图；c，展示了LIF脉冲神经元的动力学；d，脉冲时间依赖可塑性(Spike-Timing-Dependent Plasticity，STDP)规则。
在脉冲神经网络中，非易失性忆阻器交叉阵列被用于模拟突触网络结构。这种交叉阵列不仅能够存储权重参数而且提供计算能力³（存算一体），而且可以基于欧姆定律和基尔霍夫电流定律实现网络大规模并行计算中主要工作负载——矢量矩阵乘法（Vector Matrix Multiplication, VMM），其1T1R结构的示意图如图2所示。
图2 1T1R结构的阵列示意图。
除了非易失性忆阻器作为突触，具有易失性的忆阻器也逐渐被开发用于神经网络中的脉冲神经元，相较于非易失性忆阻器，易失性忆阻器仍需与CMOS器件集成实现复杂度较高的神经元功能，但忆阻器的引入简化了神经元电路。例如使用忆阻器模拟 Leaky Integrate-and-Fire (LIF) 神经元的功能⁴（图3a），为进一步减弱LIF神经元对相同信息的响应，自适应环境的LIF神经元也使用易失性忆阻器成功搭建⁵（图3e）。
图3 基于易失性忆阻器的脉冲神经元。a，生物神经元细胞膜离子通道示意图；b，设备对多个电压脉冲的响应曲线；c，设备在不同Icc下的开关机制示意图；d，设备在100 μA (左)和500 μA (右) Icc下的典型I-V曲线；e，在重复刺激下神经元中的适应化示意图(左)和基于忆阻器的神经元电路图(右)；f，系统在特定频率脉冲下的适应性行为。
文章还分析了忆阻器在实现SNN硬件中面临的挑战，如电导调控的非线性、器件的随机性等。最后，论文提出了针对忆阻器非理想特性的算法优化策略，以实现高效节能的SNN硬件系统。
Memristor-based spiking neural networks: cooperative development of neural network architecture/algorithms and memristors¹
Spiking neural networks (SNNs) are a novel type of artificial neural network inspired by the structure and principles of the human brain, and have gained significant attention for their low-power pulse transmission and large-scale parallelism. In SNNs, neurons need to multiply and sum the synaptic weights of incoming spike signals; however, the result is not directly passed to an activation function. Instead, the neuron accumulates the effects of these signals over time and emits a spike backward when the membrane potential reaches a threshold, as shown in Figure 1a². The hardware implementation of SNNs is crucial for further reducing energy consumption. Although SNNs have achieved some success at the software level, they still face challenges in hardware implementation. Memristor technology can not only simulate non-volatile synapses but also volatile neurons, and is compatible with CMOS technology, making it an ideal candidate for implementing SNN hardware.
Fig. 1 | Schematic of SNNs model. a, Classical SNNs structure, comprising a post-neuron driven by input pre-neurons. b, SNNs flow chart. c, The dynamics of LIF spiking neurons is shown. d, The spike-timing-dependent plasticity (STDP) Rule.
In SNNs, non-volatile memristor crossbar arrays are used to simulate synaptic network structures. These crossbar arrays can not only store weight parameters but also provide computational capabilities³ (compute-in-memory), and can implement vector-matrix multiplication (VMM), the main workload in large-scale parallel computing of the network, based on Ohm’s Law and Kirchhoff’s Current Law. The schematic diagram of its 1T1R structure is shown in Figure 2.
Fig. 2 | Schematic diagram of the 1T1R array.
In addition to non-volatile memristors as synapses, volatile memristors are also being developed for spiking neurons in neural networks. Compared to non-volatile memristors, volatile memristors still need to be integrated with CMOS devices to implement more complex neuronal functions, but the introduction of memristors simplifies the neuronal circuits. For example, using memristors to simulate the function of LIF neurons⁴ (Figure 3a), and for further reducing the response of LIF neurons to the same information, adaptive environment LIF neurons have also been successfully built using volatile memristors⁵ (Figure 3e).
Fig. 3 | Volatile memristor-based spiking neurons. a, Illustration of an ion channel embedded in the cell membrane near the soma of a biological neuron. b, Experimental plot of the device’s response to multiple voltage pulses. c, Schematic of habitation in the SNS under the repetition of an identical stimulus (left) and the fully memristor-based artificial SNS circuit (right). d, The habitual behavior of the system under a certain frequency pulse, the pulse emission is from fast to slow. e, Schematic of the switching mechanism of the device under different Icc. f, The typical I–V curve of device under Icc of 100 μA (left) and 500 μA (right).
The article also analyzes the challenges faced by memristors in implementing SNN hardware, such as the non-linearity of conductance modulation and the stochasticity of devices. Finally, the paper proposes algorithm optimization strategies tailored to the non-ideal characteristics of memristors to achieve an efficient and energy-saving SNN hardware system.
参考文献： 1. Peng, H., Gan, L. & Guo, X. Memristor-based spiking neural networks: cooperative development of neural network architecture/algorithms and memristors. Chip 3, 100093 (2024). 2. Roy, K., Jaiswal, A. & Panda, P. Towards spike-based machine intelligence with neuromorphic computing. Nature 575, 607-617 (2019). 3. Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. & Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol. 15, 529-544 (2020). 4. Wang, Z. et al. Fully memristive neural networks for pattern classification with unsupervised learning. Nat. Electron. 1, 137-145 (2018). 5. Wu, Z. et al. A Habituation Sensory Nervous System with Memristors. Adv. Mater. 32 (2020).
论文链接： https://www.sciencedirect.com/science/article/pii/S270947232400011X
作者简介
彭辉辉，华中科技大学材料科学与工程学院硕士，主要研究方向为忆阻器及神经网络仿真。
Huihui Peng, Master of Material Science and Engineering from Huazhong University of Science and Technology, with a focus on the research of memristors and neural network simulation.
甘霖，华中科技大学材料科学与工程学院副教授，目前已在Advanced Materials、Angewandte、ACS Nano、NPG Asia Materials、Chemistry of Materials 和Small等国际主流期刊上发表80余篇研究论文，论文被引用次数总计5000余次（截至2021/10）。
Lin Gan, Associate Professor at the School of Material Science and Engineering, Huazhong University of Science and Technology, has published more than 80 research papers in leading international journals such as Advanced Materials, Angewandte, ACS Nano, NPG Asia Materials, Chemistry of Materials, and Small. As of October 2021, these papers have been cited over 5,000 times.
郭新，华中科技大学材料科学与工程学院教授、国家特聘专家、中国固态离子学会理事、国际期刊Solid State Ionics编委、国际固态离子学会（International Society for Solid State Ionics）学术奖评选委员会五名委员之一。在国内外主流学术会议（如MRS, E-MRS, ECS, MS&T, SSI等）及国内外著名高校和研究机构（如麻省理工学院、斯坦福大学、瑞士联邦工学院、德国马普研究所、清华大学、北京大学和中科院物理所等）作过80余场大会报告和邀请报告，在Science、Advanced Materials等学术期刊发表论文160余篇。 | 
