SHD

Abstract

We trained two-layer conductance-based networks to recognize spoken digits represented by cochlear spike events. Both COBA and PING learned under the calibrated initialization and optimization recipe, with PING achieving higher development-validation accuracy in the paired replicate. Matched inference rasters show how the trained layers respond to one held-out utterance. This single-seed comparison and illustrative response do not establish competitive test performance or a general benefit from inhibition.

TODO

1. Use the full utterance time ranges in the next training run (Methods, point 1).
2. Apply no input corruption in the next training run (Methods, point 2).

Results

COBA training

After 50 epochs, online training accuracy was 84% and validation accuracy 82.6% (Fig. 1B). Validation cross-entropy fell from 3.0055 to 0.5919 (Fig. 1A). All three learned matrices changed and final weights were finite. The condition took 58.4 minutes, including 52.7 minutes training and 5.7 minutes validation, excluding acquisition, setup and transfer. This is one development-holdout replicate, not an official-test evaluation.

PING training

With fixed reciprocal feedback in both 128 E/32 I layers, online training accuracy was 89% and validation accuracy 84.4% (Fig. 2B). Validation cross-entropy fell from 3.1025 to 0.4725 (Fig. 2A). All three learned matrices changed and final weights were finite. The condition took 100.8 minutes, including 91.6 minutes training and 9.2 minutes validation. It ran after COBA had completed. Higher validation accuracy in this pair does not establish a general inhibitory advantage or its mechanism.

COBA two-layer response

We replayed the first validation utterance through the final trained weights, without optimizer updates. The true class was 0, and COBA predicted class 0. Both layers responded during the utterance (Fig. 3). This fixed example illustrates activity propagation; it was not selected for correct classification or its raster appearance and cannot summarize performance across utterances.

PING two-layer response

PING predicted class 0 for the same true-class-0 utterance. The raster shows intermittent excitatory responses and synchronized inhibitory events in both layers (Fig. 4). Identical inhibitory parameters and uniform incoming feedback produce identical inhibitory spike trains within a layer. This response is illustrative; the raster alone does not establish a rhythm’s frequency or show why PING trained more accurately.

Methods

The architecture and simulation settings are listed in Table 1.

1. Prepare SHD. We split 8,156 official training utterances [1] (20 classes, 700 channels) into 7,340 training and 816 validation examples using a seed-42 per-class shuffle. Original spike-train lengths varied: last-spike times ranged from 0.233 to 1.369 s. We kept only events before 1 s to fit the simulation, flooring times into binary 1 ms bins. Speakers could overlap between splits; the official test set was unused.

2. Corrupt inputs. Per channel and 1 ms bin, we deleted existing spikes with probability 0.0008 and filled empty bins with probability 0.0016 (approximately 1.6 Hz). Both models shared example order and noise draws; validation noise stayed fixed across epochs for consistent comparison. Corruption exposed models to missing and extra spikes, but its benefit was not tested against uncorrupted training.

3. Initialize hidden weights. Both models shared seed-42 learned matrices. SNNLab’s LowerClampedNormal used parent mean 0.09 μS and SD 0.009 μS. Input→E1 and E1→E2 initial zero fractions were 0.95 and 0.90, followed by division by retained fan-in (35 and 12.8). Negative draws were clamped; initial zeros remained trainable and weights remained nonnegative.

4. Initialize readout and feedback. Signed E2→readout weights had Gaussian mean zero and SD 0.1 μS, centered across source neurons for each class, without sparsification or fan-in scaling. PING feedback was fixed at $0.05/128$ μS per E→I edge and $0.05/32$ μS per I→E edge: 0.05 μS summed incoming conductance in either direction. COBA had no inhibitory feedback.

5. Score utterances. We used a custom rate-normalized mean-voltage score:

   $$z_c=G\,\bar u_c\,\frac{r_{\mathrm{ref}}}{\max(\bar r_{E2},r_{\mathrm{floor}})}. \tag{1}$$

   Here $z_c$ is the score for class $c$, $\bar u_c$ its mean non-spiking readout voltage, and $\bar r_{E2}$ the per-neuron E2 rate, both over the full second. We used gain $G=20$, reference rate $r_{\mathrm{ref}}=20\,\mathrm{Hz}$ and rate floor $r_{\mathrm{floor}}=1\,\mathrm{Hz}$, predicting the largest score. The positive scaling preserves that choice for fixed weights but changes cross-entropy and training gradients.

6. Train both models. We used SNNLab’s train for 50 epochs with class-stratified batches of 256, merging tails smaller than 20. Custom Adam minimized mean cross-entropy per represented class; input/interlayer/readout learning rates were $5\times10^{-6}/2\times10^{-5}/0.01$, with betas 0.9/0.999 and no weight decay. The FastSigmoid surrogate used slope $100/15\,\mathrm{mV}^{-1}$ and gradient scale 0.01. States reset per utterance. Reported losses were example means; validation used epoch-end weights, including epoch zero.

7. Record matched responses. Using SNNLab’s infer, we replayed the first validation utterance with final-epoch weights, one shared fresh corruption draw (seed 46) and no updates. Selection did not depend on prediction or spike appearance.

8. Calculate autocorrelograms. We applied SNNLab’s spike_autocorrelogram separately to each population over the full utterance:

   $$A_{\mathrm{corr},P}[\ell] =\frac{\displaystyle\sum_{k=0}^{N_t-\ell-1}n_P[k]\,n_P[k+\ell]} {(N_t-\ell)\,\bar n_P^{\,2}}. \tag{2}$$

   Here $A_{\mathrm{corr},P}$ is dimensionless autocorrelation for population $P$ (E or I in one layer), $n_P[k]$ its count in bin $k$, and $\bar n_P$ its mean across $N_t=1000$ bins. We used 1 ms bins and integer lags $\ell=1,\ldots,200$ (1–200 ms). Independent stationary activity has baseline one; zero lag is omitted, and silent or too-short recordings give undefined curves. No smoothing, transient removal or trial averaging was applied; the stimulus-evoked activity envelope contributes to the correlation.

References

1. Cramer, B., Stradmann, Y., Schemmel, J., and Zenke, F. The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 33, 2744–2757 (2022). doi:10.1109/TNNLS.2020.3044364.