Decoder Accuracy Improves with Input Rate

Abstract

We asked which input rates preserve enough information in filtered MNIST pixel events for nonlinear decoding. We trained 3 decoders on finite-window, nonspiking membrane features and evaluated held-out digits. Mean accuracy increased from 28.6% at 0.1 Hz to 96.1% at 25 Hz. The tested range calibrates this representation and decoder; it does not establish PING-network accuracy or predict performance between tested rates.

Results

Mixed-rate validation trajectories

The retained calibration contains three independently initialized nonlinear decoders trained on fresh encoding draws sampled across the complete input-rate grid (Fig. 1).

If mixed-rate training learned digit structure that survived stochastic filtering, validation accuracy should improve across all three training replicates rather than only one trajectory.

Each replicate reached its maximum late in training. The selected epochs were 50, 25, 44, with validation accuracies 77%, 76.4%, 76.8%.

Notes.

1. The first occurrence of the maximum validation accuracy selected the checkpoint for each training replicate; later ties did not replace it.

Filtered digit features

A reused illustration shows one MNIST digit after the same finite-window synaptic and membrane filtering at three input rates (Fig. 2).

Because sparse Bernoulli input produces a small, random number of events, reducing the rate should remove responses from different pixels rather than attenuate the whole digit uniformly.

At 0.5 Hz only isolated fragments remained. The digit became progressively more complete at 5 and 25 Hz, consistent with stochastic event loss rather than uniform contrast scaling.

Held-out rate-accuracy curve

The three selected decoders received identical held-out feature vectors and encoding draws. The practical floor required every decoder to reach 50% accuracy at a tested rate (Fig. 3).

If higher input rates preserved more digit structure, held-out accuracy should rise before approaching a plateau.

Mean accuracy across training replicates increased monotonically from 28.6% at 0.1 Hz to 96.1% at 25 Hz.

The selected floor was 0.5 Hz: all three decoders first crossed the criterion there, and mean accuracy was 63.3%.

The resulting interval is a decoder-relative calibration without interpolation; it does not establish PING-network performance.

Methods

1. Retained computation. We reused a completed calibration containing three trained decoders, their validation histories and their held-out correctness records. We did not rerun feature simulation or decoder training for this article.

2. MNIST partitions. Of the 60000 official training images, the first 20000 trained the decoders and the next 5000 selected checkpoints; the remaining 35000 were unused. Evaluation used the first 5000 images from the separate official test partition, with no overlap.

3. Generate input events. Each normalized pixel intensity $x_{i} \in [ 0 , 1 ]$ generated an independent binary event $s_{i [ k ]}$ at integration timestep $\Delta t_{\text{sim}} =$ 0.1 ms:

   $$p_{\text{event} , i} = \frac{r_{\text{input,max}} x_{i} \Delta t_{\text{sim}}}{1000} , \quad s_{i [ k ]} \sim \text{Bernoulli} ( p_{\text{event} , i} ) . \tag{1}$$

   Here $i$ indexes pixels, $k$ is the simulation-step index, $p_{\text{event} , i}$ is event probability, and $r_{\text{input,max}}$ is maximum-pixel encoding rate in spikes/s; 1000 converts milliseconds to seconds.

4. Filter synaptic conductance. Excitatory conductance $g_{i [ k ]}$, in μS, decayed each step by $\exp(-\Delta t_{\text{sim}}/\tau_{\text{AMPA}})$ before an event added $w_{\text{event}}$. The AMPA time constant was $\tau_{\text{AMPA}} = 2$ ms and event strength was $w_{\text{event}} =$ 1.2 μS.

5. Integrate membrane voltage. During simulation step $k$, the updated conductance $g_{i [ k ]}$ was held fixed while each non-spiking membrane voltage $V_{m , i} ( t )$ obeyed

   $$C_{m} \frac{d V_{m , i}}{d t} = g_{L} ( E_{L} - V_{m , i} ) + g_{i [ k ]} ( E_{e} - V_{m , i} ) . \tag{2}$$

   Capacitance was $C_{m} = 1$ nF, leak conductance $g_{L} = 0.05$ μS, leak reversal $E_{L} = - 65$ mV and excitatory reversal $E_{e} = 0$ mV. Starting at zero conductance and $E_{L}$, voltage advanced by the exact exponential solution for that step. Simulation and decoder arithmetic used single precision.

6. Form pixel features. The feature $z_{\text{feature} , i}$, in mV, averaged post-update voltages above rest:

   $$z_{\text{feature} , i} = \frac{1}{N_{t}} \sum_{k = 1}^{N_{t}} ( V_{m , i} ( k \Delta t_{\text{sim}} ) - E_{L} ) \approx \frac{1}{T_{\text{present}}} \int_{0}^{T_{\text{present}}} ( V_{m , i} ( t ) - E_{L} ) \, \mathrm{d} t . \tag{3}$$

   Here $T_{\text{present}} =$ 200 ms, $N_{t} = \frac{T_{\text{present}}}{\Delta t_{\text{sim}}}$ is the timestep count, and physical time at step $k$ is $t_{k} = k \Delta t_{\text{sim}}$. Fresh encoding draws retained finite-window shot-noise effects without a stationary Gaussian approximation[1].

7. Train mixed-rate decoders. At every epoch, we sampled the input rate for each training and validation presentation uniformly from 0.1, 0.25, 0.5, 1, 2, 5, 10, 25 Hz and generated a fresh encoding draw. Each 784–1024–10 ReLU decoder was trained for 50 epochs using cross-entropy, Adam with learning rate 0.001, no weight decay and batch size 1.

8. Independent training replicates. Stochastic-stream identifiers 42, 43, 44 defined independent model initializations, rate assignments and encoding draws.

9. Select checkpoints. At each of the 50 eligible epochs, validation accuracy was the fraction correct across 5000 validation presentations. The earliest epoch attaining the maximum validation accuracy supplied the selected checkpoint for each training replicate.

10. Evaluate shared test features. Every held-out image was simulated once at each tested rate, and all selected decoders received the same feature vector for that image and rate; the feed-forward decoder had no state across presentations. The predicted class $\hat{y}$ was the class $c$ with the largest output logit $z_{c}$, and accuracy was measured per training replicate and rate.

11. Aggregate held-out accuracy. For each rate and training replicate, we averaged correctness across 5000 held-out images. We then recorded the mean, minimum and maximum accuracy across the 3 training replicates at each rate.

12. Select the tested interval. The practical floor was the lowest tested rate where every decoder reached 50% accuracy, and the ceiling was the highest tested rate. No interpolation was used; an empty qualifying set was reported as right-censored.

13. Present retained evidence. We redrew the validation trajectories and rate-accuracy summary from recorded measurements, using the aggregation defined above without interpolation. We reused the original finite-window feature illustration unchanged rather than implying a new simulation.