An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes

Yuling Luo; Qiang Fu; Junxiu Liu; Jim Harkin; Liam McDaid; Yi Cao

doi:10.1007/978-3-319-59153-7_49

An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes

Yuling Luo
, Qiang Fu
, Junxiu Liu^*
, Jim Harkin
, Liam McDaid
, Yi Cao

^*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

8 Citations (Scopus)

Abstract

This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij’s algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij’s algorithm - i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.

Original language	English
Title of host publication	Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings
Editors	Andreu Catala, Ignacio Rojas, Gonzalo Joya
Publisher	Springer Verlag
Pages	569-579
Number of pages	11
ISBN (Print)	9783319591520
DOIs	https://doi.org/10.1007/978-3-319-59153-7_49
Publication status	Published - 2017
Externally published	Yes
Event	14th International Work-Conference on Artificial Neural Networks, IWANN 2017 - Cadiz, Spain Duration: 14 Jun 2017 → 16 Jun 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10305 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th International Work-Conference on Artificial Neural Networks, IWANN 2017
Country/Territory	Spain
City	Cadiz
Period	14/06/17 → 16/06/17

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Learning rate
Momentum
Self-adaptation
Spiking neural networks

Access to Document

10.1007/978-3-319-59153-7_49

Cite this

Luo, Y., Fu, Q., Liu, J., Harkin, J., McDaid, L., & Cao, Y. (2017). An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes. In A. Catala, I. Rojas, & G. Joya (Eds.), Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings (pp. 569-579). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10305 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-59153-7_49

Luo, Yuling ; Fu, Qiang ; Liu, Junxiu et al. / An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes. Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings. editor / Andreu Catala ; Ignacio Rojas ; Gonzalo Joya. Springer Verlag, 2017. pp. 569-579 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ae8a6d0a61aa4b47a38d36a9e9189c89,

title = "An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes",

abstract = "This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij{\textquoteright}s algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij{\textquoteright}s algorithm - i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.",

keywords = "Learning rate, Momentum, Self-adaptation, Spiking neural networks",

author = "Yuling Luo and Qiang Fu and Junxiu Liu and Jim Harkin and Liam McDaid and Yi Cao",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 14th International Work-Conference on Artificial Neural Networks, IWANN 2017 ; Conference date: 14-06-2017 Through 16-06-2017",

year = "2017",

doi = "10.1007/978-3-319-59153-7\_49",

language = "English",

isbn = "9783319591520",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "569--579",

editor = "Andreu Catala and Ignacio Rojas and Gonzalo Joya",

booktitle = "Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings",

}

Luo, Y, Fu, Q, Liu, J, Harkin, J, McDaid, L & Cao, Y 2017, An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes. in A Catala, I Rojas & G Joya (eds), Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10305 LNCS, Springer Verlag, pp. 569-579, 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Cadiz, Spain, 14/06/17. https://doi.org/10.1007/978-3-319-59153-7_49

An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes. / Luo, Yuling; Fu, Qiang; Liu, Junxiu et al.
Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings. ed. / Andreu Catala; Ignacio Rojas; Gonzalo Joya. Springer Verlag, 2017. p. 569-579 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10305 LNCS).

Research output: Chapter in Book or Report/Conference proceeding › Conference Proceeding › peer-review

TY - GEN

T1 - An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes

AU - Luo, Yuling

AU - Fu, Qiang

AU - Liu, Junxiu

AU - Harkin, Jim

AU - McDaid, Liam

AU - Cao, Yi

N1 - Publisher Copyright: © Springer International Publishing AG 2017.

PY - 2017

Y1 - 2017

N2 - This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij’s algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij’s algorithm - i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.

AB - This paper presents two methods of using the dynamic momentum and learning rate adaption, to improve learning performance in spiking neural networks where neurons are modelled as spiking multiple times. The optimum value for the momentum factor is obtained from the mean square error with respect to the gradient of synaptic weights in the proposed algorithm. The delta-bar-delta rule is employed as the learning rate adaptation method. The XOR and Wisconsin breast cancer (WBC) classification tasks are used to validate the proposed algorithms. Results demonstrate no error and a minimal error of 0.08 are achieved for the XOR and WBC classification tasks respectively, which are better than the original Booij’s algorithm. The minimum number of epochs for XOR and Wisconsin breast cancer tasks are 35 and 26 respectively, which are also faster than the original Booij’s algorithm - i.e. 135 (for XOR) and 97 (for WBC). Compared with the original algorithm with static momentum and learning rate, the proposed dynamic algorithms can control the convergence rate and learning performance more effectively.

KW - Learning rate

KW - Momentum

KW - Self-adaptation

KW - Spiking neural networks

UR - https://www.scopus.com/pages/publications/85020500171

U2 - 10.1007/978-3-319-59153-7_49

DO - 10.1007/978-3-319-59153-7_49

M3 - Conference Proceeding

AN - SCOPUS:85020500171

SN - 9783319591520

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 569

EP - 579

BT - Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings

A2 - Catala, Andreu

A2 - Rojas, Ignacio

A2 - Joya, Gonzalo

PB - Springer Verlag

T2 - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017

Y2 - 14 June 2017 through 16 June 2017

ER -

Luo Y, Fu Q, Liu J, Harkin J, McDaid L, Cao Y. An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes. In Catala A, Rojas I, Joya G, editors, Advances in Computational Intelligence - 14th International Work-Conference on Artificial Neural Networks, IWANN 2017, Proceedings. Springer Verlag. 2017. p. 569-579. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-59153-7_49

An extended algorithm using adaptation of momentum and learning rate for spiking neurons emitting multiple spikes

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this