Hebbian-like learning has been repeatedly confirmed experimentally, yet computational models usually require non-local signals, such as backpropagating errors, to solve complex cognitive tasks. Recent cortical electron microscopy data suggests a model where synapses follow different plasticity rules depending on whether they are in a large or small state. Large synapses often include a spine apparatus, a calcium reservoir that influences synaptic dynamics and can alter rules of synaptic plasticity. Here, we test the computational outcomes of networks which compute with synapses switching their plasticity rules based on their strength. We designed a recurrent neural network (RNN) with synapses that switch between two learning rules: a Hebbian-like rule for weak synapses and a credit-assignment rule (backpropagation, BP) for strong synapses. We found that our plasticity-switching RNN (psRNN) learns cognitive tasks (e.g. working memory) in fewer trials than BP-only RNNs, despite fewer synapses using credit assignment. Three mechanisms underlie this advantage: BP samples multiple parameter configurations for better gradient estimation, Hebbian plasticity creates a dynamic task-relevant initialization, and the switching mechanism prevents Hebbian synapses from growing into unfavorable parameter regions. The interaction between rules also produces lower-rank, more feedforward recurrent structure, providing testable connectomic predictions and a framework for reconciling local learning rules observed in the brain with non-local rules used in computational models.
Turcu, D., Cornford, J., Dorkenwald, S., Mihalas, S.
Advertisement
Stats
- Recommendations n/a n/a positive of 0 vote(s)
- Views 9
- Comments 0