Balanced and Efficient Spiking Neural Networks

Abstract

Spiking Neural Networks promise an efficient and robust alternative to traditional Artificial Neural Networks. When compared to the human brain, however, they are far from achieving their full potential, the reason unclear so far. Recent neuroscience research offers a clue to solve this puzzle - tight balance. The excitatory and inhibitory input currents flowing into individual neurons have been observed to be balanced very tightly on a millisecond scale. This substantially exceeds the traditional E/I balance that is established on population level. Accompanying theoretical works have shown that spiking networks operating in this tight regime can implement more efficient and robust codes in their spike trains. In this work, we consolidate these theoretical insights with modern neuromorphic computing approaches to apply them to complex classification problems. We show up to 99.9% reductions in firing rates in balanced networks compared to unbalanced ones on the neuroscience-inspired cue-based navigation task, and outperform prior work by achieving 97.60% accuracy. The resulting approach can easily be mapped to neuromorphic hardware as its core relies on a balanced initialization scheme only. Moreover, even complex variants only involve adaptions of neuron and synapse models. Ultimately, our work shows the importance of integrating novel insights from neuroscience research into modern spiking networks to increase their efficiency and robustness using neuromorphic methods.