Neural Coding

The Spike Train Representation

Given spike times $\{t_1, t_2, \ldots, t_n\}$, the spike train is:

$$\rho(t) = \sum_{i=1}^{n} \delta(t - t_i)$$

The instantaneous firing rate is obtained by averaging over many trials (ensemble average) or by smoothing with a kernel $K(t)$:

$$r(t) = \langle \rho(t) \rangle = \lim_{\Delta t \to 0} \frac{\langle n(t, t+\Delta t) \rangle}{\Delta t}$$

For a Poisson process with constant rate $\lambda$, the interspike intervals (ISIs) follow an exponential distribution: $P(\tau) = \lambda e^{-\lambda \tau}$ with mean $\langle \tau \rangle = 1/\lambda$ and coefficient of variation$\text{CV} = 1$.

Derivation 1: Tuning Curves and the Cosine Model

Georgopoulos et al. (1982) found that motor cortex neurons have cosine tuning to movement direction. For a neuron with preferred direction $\theta_{\text{pref}}$:

$$r(\theta) = r_0 + r_{\max} \cos(\theta - \theta_{\text{pref}})$$

where $r_0$ is the baseline rate and $r_{\max}$ is the modulation depth. For a population of $N$ neurons with uniformly distributed preferred directions, the population vector estimate of movement direction is:

$$\hat{\theta}_{\text{pop}} = \arg\left(\sum_{i=1}^{N} r_i \, e^{i \theta_{\text{pref},i}}\right)$$

As $N \to \infty$, this estimator converges to the true direction with variance scaling as $\sigma^2 \propto 1/N$. The population vector averages out noise across neurons, achieving a population-level signal-to-noise ratio that grows as $\sqrt{N}$.

Derivation 2: Spike Timing Precision and the Victor-Purpura Metric

To quantify temporal coding, we need a distance metric between spike trains. The Victor-Purpura (1996) metric defines the distance as the minimum cost to transform one spike train into another using three operations:

• Insert a spike: cost = 1
• Delete a spike: cost = 1
• Move a spike by $\Delta t$: cost = $q |\Delta t|$

The parameter $q$ (units: 1/time) sets the temporal resolution. When$q \to 0$, the metric counts only spike number differences (rate code). When $q \to \infty$, it counts coincident spikes only (temporal code). The metric is computed via dynamic programming with complexity $O(n_1 n_2)$ where$n_1, n_2$ are spike counts:

$$D(i,j) = \min\begin{cases} D(i-1, j) + 1 \\ D(i, j-1) + 1 \\ D(i-1, j-1) + q|t_i^{(1)} - t_j^{(2)}| \end{cases}$$

Applying this metric to neural data and varying $q$ reveals the timescale at which temporal information becomes relevant. For cortical neurons, $q_{\text{opt}} \sim 10\text{--}100$ s$^{-1}$, corresponding to temporal precision of 10–100 ms.

Derivation 3: Information in a Poisson Spike Count

Consider a neuron with Poisson statistics whose rate $\lambda(s)$ depends on stimulus$s$. The spike count $n$ in a window $T$ follows $P(n|s) = \frac{(\lambda T)^n e^{-\lambda T}}{n!}$. The response entropy is:

$$H(R) = -\sum_n P(n) \log_2 P(n)$$

where $P(n) = \langle P(n|s) \rangle_s$. The noise entropy is:

$$H(R|S) = \left\langle -\sum_n P(n|s) \log_2 P(n|s) \right\rangle_s$$

For a Poisson distribution with mean $\mu = \lambda T$, the entropy is approximately:

$$H_{\text{Poisson}}(\mu) \approx \frac{1}{2} \log_2(2\pi e \mu) \quad \text{for } \mu \gg 1$$

The mutual information is then bounded by the difference between the total response variability and the intrinsic noise: $I(S;R) \leq \frac{1}{2}\log_2\left(1 + \frac{\text{Var}[\lambda]}{\langle \lambda \rangle}\, T\right)$. This shows that information grows logarithmically with time window $T$, and linearly with the signal-to-noise ratio $\text{Var}[\lambda]/\langle \lambda \rangle$.

Derivation 4: The Direct Method for Entropy Estimation

Strong et al. (1998) developed the "direct method" for estimating information in spike trains without assuming a coding model. Discretize time into bins of width $\Delta t$, encoding the spike train as a binary string. The total entropy of response words of length $L$ is:

$$H_{\text{total}}(L) = -\sum_{\text{words } w} P(w) \log_2 P(w)$$

The noise entropy uses stimulus-locked responses:

$$H_{\text{noise}}(L) = \left\langle -\sum_w P(w|s) \log_2 P(w|s) \right\rangle_s$$

The information rate is:

$$\dot{I} = \frac{H_{\text{total}}(L) - H_{\text{noise}}(L)}{L \cdot \Delta t} \quad \text{(bits/s)}$$

Extrapolation to $L \to \infty$ and correction for finite data bias (quadratic extrapolation in $1/N$) yields the true information rate. Fly H1 neurons transmit ~90 bits/s, approaching 50% of the channel capacity, indicating remarkably efficient coding.

Derivation 5: Fisher Information and the Cramér-Rao Bound

Fisher information quantifies the precision of neural population codes. For a population of $N$ neurons with tuning curves $f_i(s)$ and independent Poisson noise, the Fisher information is:

$$J(s) = \sum_{i=1}^{N} \frac{[f_i'(s)]^2}{f_i(s)}$$

The Cramér-Rao bound sets a lower limit on the variance of any unbiased estimator$\hat{s}$ of the stimulus:

$$\text{Var}(\hat{s}) \geq \frac{1}{J(s)}$$

For Gaussian tuning curves $f_i(s) = A \exp[-(s - s_i)^2 / (2\sigma^2)]$ with uniformly spaced preferred stimuli (spacing $\Delta s$), the total Fisher information is:

$$J_{\text{total}} = \frac{N A}{\sigma^2} \cdot C(\sigma / \Delta s)$$

where $C$ is a coverage factor that depends on the ratio of tuning width to spacing. Optimal coding requires $\sigma / \Delta s \approx 1$, balancing resolution against coverage. This result predicts that wider tuning curves are preferred when noise is high.