Efficiency metrics for auditory neuromorphic spike encoding techniques using information theory

A dynamic signal $x[n]$ is encoded into a spike train $w[n]$ by a given spike encoding technique ² . Every spike encoding technique has parameters that modulate the number of spikes produced in the spike train. These parameters can be configured to yield spike trains of varying density, ranging from very sparse (few spikes) to very dense (many spikes).

In this work, a spike train $w[n]$ is taken to be a discrete binary time signal (figure 1) with ones representing spikes. Different encoding parameter configurations can lead to spike trains of different firing rates. The maximum attainable firing rate is when there are spikes at every time step. In such a case, the firing rate is equal to the sampling rate of the input signal. Spike density ρ is defined as the normalized mean firing rate,

$$\begin{equation} \rho = \frac{1}{N} \sum_{n=0}^{N-1} |w[n]| \end{equation} \tag{ 1 }$$

where N is the total number of samples in the spike train $w[n]$, and the absolute value accounts for the case of spike encoding techniques which yield bipolar spike trains (where spikes can be positive or negative, such as for example SoD coding). Spike density is comprised between 0 (representing the extreme case of no spikes at all) and 1 (representing the extreme case of a spike at every sample).

Zoom In Zoom Out Reset image size

By varying the parameters of the spike encoding techniques that modulate the spike density and evaluating the information content of spike trains at different densities, a complete picture of the performance of a spike encoding technique can be established. Moreover, spike density offers a common scale to compare the performance of different spike encoding techniques which have different parameters.

2.2.1. Discrete random variables

2.2.2. Mutual information

The mutual information $I(X;W)$ between the signal and the spikes is estimated in bits:

$$\begin{equation} I(X;W) = \displaystyle\sum_{x\in X}\sum_{w\in W} P(x,w) \log_{2}\left( \frac{P(x,w)}{P(x) P(w)} \right) \end{equation} \tag{ 2 }$$

where $P(x,w)$ is the joint probability between X and W, and P(x) and P(w) are the marginal probabilities [4].

2.2.3. Encoding latency

2.2.4. Sampling bias

This subsection presents the three proposed efficiency metrics. The first metric, coding efficiency, estimates the fraction of information encoded in the spikes from the total information available in the signal to be encoded. The other two efficiency metrics take into account the cost of the spike encoding techniques, which characterizes the trade-off between encoding the maximum information and consuming the least energy. Computational efficiency attributes abstract costs to the algorithmic operations of the spike encoding techniques, whereas energy efficiency estimates the empirical energy expenditure specific to the data and implementation of the spike encoding techniques.

2.3.1. Coding efficiency

The maximum achievable mutual information is determined by the statistical structure of the signal's random variable X and is given by the entropy

$$\begin{equation} H(X) = - \displaystyle\sum_{x\in X} P(x) \log_{2} P(x) \end{equation} \tag{ 3 }$$

which is the total amplitude information present in the signal to be encoded. If W completely characterizes X, then the mutual information is equal to H(X), which means that the encoding is perfect. The coding efficiency ε is defined as the fraction of information that is encoded in the spikes:

$$\begin{equation} \epsilon = \frac{I(X;W)}{H(X)}. \end{equation} \tag{ 4 }$$

2.3.2. Computational efficiency

Every spike encoding technique has algorithmic operations that incur costs at every timestep. The cost can be decomposed into fixed and variable costs, respectively C_f and C_v . The fixed cost C_f is incurred whether or not a spike is produced at the timestep, and represents all the algorithmic operations that are needed to update the variables. The variable cost C_v , however, is only incurred when a spike is produced, and involves the operation of spike production, as well as any other algorithmic operations required after a spike is produced [14, 17, 35].

The computational cost of the algorithmic operations of spike encoding techniques is characterized using the abstract unit 'op' (corresponding to operations). This permits the abstract decomposition of the cost into fixed and variable computational costs in an implementation-independent way, based solely on the pseudo-code of the spike encoding technique.

For a given spike encoding technique, the average computational cost rate $C(\rho)$ (in op/timestep) only depends on the spike density ρ (which can be seen as the average number of spikes produced per timestep),

$$\begin{equation} C(\rho) = C_f + \rho \, C_v \end{equation} \tag{ 5 }$$

with the fixed cost C_f and the variable cost C_v estimated in op/timestep from the pseudo-code of the spike encoding techniques.

The computational efficiency η_C , measured in bit/op, is then

$$\begin{equation} \eta_C = \frac{I(X;W)}{C(\rho)} .\end{equation} \tag{ 6 }$$

2.3.3. Energy efficiency

Energy efficiency is based on the direct estimation of the energy cost rate $E(\rho)$ of the spike encoding on the specific data used and the specific implementation of the spike encoding techniques. This energy cost rate depends on the spike density.

In this work, the energy cost rate $E(\rho)$ is estimated using the pyJoules Python library [33], which monitors the energy consumed by CPUs using Intel's Running Average Power Limit technology. This approach gives an empirical measurement of the energy consumed, but it is dependent on the specific input data, the implementation of the spike encoding techniques, and the processor architecture used.

The energy efficiency η_E , measured in bit/nJ, is then

$$\begin{equation} \eta_E = \frac{I(X;W)}{E(\rho)}. \end{equation} \tag{ 7 }$$

The speech stimulus used is an audio sequence of 20 min made by concatenating spoken digits from the TIDIGITS adult dataset [11] chosen at random and sampled at 20 kHz.

The input speech stimulus is transformed into a cochleagram, which gives an approximation of auditory nerve fiber spiking probabilities. To compute the cochleagram (figure 2), the signal is first passed through an infinite impulse response Gammatone filterbank with eight channels. The filerbank has center frequencies distributed on the equivalent rectangular bandwidth scale spanning the range from 100 Hz to 8 kHz. It approximates the vibrations of the basilar membrane [23, 27]. Inner hair cells are then modeled with half-wave rectification followed by cubic-root compression and low-pass filtering at 10 Hz. This low-pass filtering allows to extract the envelope of the oscillations. Finally, the signal is decimated to 1 kHz and normalized to form the cochleagram. The brian2hears Python library [9] is used for the cochleagram extraction.

Zoom In Zoom Out Reset image size

Four spike encoding techniques are evaluated: ISC, SoD coding, BSA, and LIF coding. They take as input a dynamic signal (i.e. the output $x[n]$ of a given cochleagram channel). Thus, each of the cochleagram's channels is encoded separately and independently into a single spike train. The pseudo-code for each of these techniques is provided in the appendix.

3.2.1. Independent Spike Coding (ISC)

In ISC, the instantaneous firing probability in a cochleagram channel is used to emit spikes stochastically and independently, resulting in a non homogeneous Poisson process. Specifically, the probability of a spike at sample n is

$$\begin{equation} p[n] = x[n] \times a \end{equation} \tag{ 8 }$$

where a is a scaling factor that modulates the density of the resulting spike train.

ISC is used in [34] to encode cochleagrams for speech recognition. It is also the standard approach to generate spikes in computational models of the peripheral auditory system, such as those of Carney [3] and Meddis [22]. The Meddis model is used in [5] to produce the Heidelberg audio spike datasets for SNNs.

3.2.2. SoD coding

In SoD [24], a spike is emitted when the net change in the signal's amplitude is at least equal to a fixed value (Δ), with the spike being an ON event if the change is positive, and an OFF event if the change is negative. This makes it possible to have a reconstruction of the original signal from the spikes, since between any two consecutive spikes, the signal has changed by $\pm \Delta$. Specifically, the first and the kth spikes are emitted at samples:

$$\begin{align} n_0 &= \inf_{n \geqslant 0} \quad \{n, \quad |x[n] - x[0]| \geqslant \Delta\} \end{align} \tag{ 9 }$$

$$\begin{align} n_k &= \inf_{n \geqslant n_{k-1}} \{n, \quad |x[n] - x[n_{k-1}]| \geqslant \Delta\} . \end{align} \tag{ 10 }$$

Bipolar spikes (+1 and −1) are used, although it is equivalent to use two different unipolar spike trains. The threshold Δ modulates the density of the spike train.

SoD is used in [47] for speech recognition, and in the silicon cochlea model in [43]. SoD is discussed in other works under the name 'step-forward encoding' [28].

3.2.3. Ben's Spiker Algorithm (BSA)

BSA [32] is based on the reverse convolution of the signal with a FIR filter h of length M. It emits spikes based on a simple heuristic comparing the filter's impulse response with the signal, such that the convolution of the resulting spike train with the filter gives a reconstruction of the original signal. At each instant, two error metrics are computed:

$$\begin{align} e_1 &= \sum_{k=0}^{M-1} |x[n-k] - h[M-1-k]| \end{align} \tag{ 11 }$$

$$\begin{align} e_2 &= \sum_{k=0}^{M-1} |x[n-k]|. \end{align} \tag{ 12 }$$

If $e_1 \leqslant e_2 - \theta$, with θ a threshold, a spike is emitted and the filter is subtracted from the signal. BSA requires input signals to be comprised between 0 and 1, as the maximum signal value that can be reconstructed is equal to the sum of the filter's coefficients. BSA's threshold parameter θ modulates the density of the spike train.

BSA is used in works by Schrauwen and colleagues [31, 40–42], Maass and colleagues [15, 16], and Li and colleagues [13, 46].

3.2.4. LIF coding

In this method, the signal is fed directly as a membrane current to be integrated by a neuron with discretized equation

$$\begin{align} u[n+1] &= u[n] \mathrm{e}^{-\frac{1}{\tau}} + x[n] \end{align} \tag{ 13 }$$

$$\begin{align} u[0] &= 0 \end{align} \tag{ 14 }$$

where u is the neuron's membrane potential, τ its time constant, and the signal to be encoded $x[n]$ becomes the membrane current. The resistance in the model is assumed to be equal to 1. If the potential crosses a threshold θ, the neuron emits a spike, and its potential is reset to zero (resting potential). LIF coding's firing threshold θ modulates the density of the spike train.

LIF coding is used for example in speech recognition in [1] and [45]. LIF coding is also used to transduce the inner hair cell signal into spikes in the silicon cochleae proposed in [20, 21, 39].

The coding efficiency is estimated for each of the eight cochleagram channels, and the mean coding efficiency is shown in figure 3. At the boundaries of the spike density, where the spike trains are too sparse or too dense, not much information can be encoded, because there is not much variability in the spike trains. It is somewhere between the two extremes that the higher coding capacity offers room to maximally encode the signal. Therefore, mutual information curves have concave shapes relative to spike density. The results show that overall, ISC and SoD have low coding efficiency, while LIF and BSA have the highest coding efficiency (with BSA's coding efficiency being slightly higher than LIF's).

Zoom In Zoom Out Reset image size

The stochastic nature of ISC introduces noise that degrades the spike train's information content, which can explain its low coding efficiency. SoD encodes the changes in a signal. Intuitively, this means that the spike train can only inform on changes in levels (relative amplitude), not on the absolute values of the levels (exact value of the amplitude). To recover information on the latter, one would need to know the initial level, and keep track of all the changes up to the present. For this reason, SoD might show good performance if it is rather the derivative of the amplitude that is the information of interest, and not the amplitude itself as is the case here. BSA has the highest mean coding efficiency of 34.6% (standard deviation 3.2%) for a spike train of 30.6% density, but it should be noted that the threshold parameter is traditionally chosen for the optimal reconstruction of the signal from the spikes, and the threshold that is optimal for that signal's reconstruction might be different from the threshold that is optimal in terms of information. Interestingly, LIF's mean coding efficiency closely follows that of BSA, but peaks at a slightly lower efficiency of 32.6% (standard deviation 2.3%) for a spike train of 40% density, thus a higher density than for BSA's maximum coding efficiency.

It should be noted that overall the best mean coding efficiency is only 34.6%, which shows that most of the amplitude information is discarded when encoding a signal into a single spike train even in the best case scenario.

Next, the spike encoding techniques are compared in terms of their computational and energy efficiencies. Only the results for BSA and LIF are included in this article, as the other two encoding techniques (ISC and SoD) have significantly much less coding efficiency.

For the computational efficiency, rather than trying to capture the precise costs of the operations of each of the two spike encoding techniques, the cost is characterized abstractly in terms of the algorithmic operations involved. Costs of operations are taken as a rough approximation. Addition, subtraction, comparison, and absolute value are all assumed to cost 1 op. Multiplications are assumed to cost 2 op since they are usually significantly more costly compared to the other operations [8]. For the energy efficiency, the actual energy consumed in spike encoding is estimated, with this estimation being specific to the implementation, data, and architecture used.

For LIF (see algorithm 4 in the appendix), the fixed computational cost C_f involves updating the neuron's membrane potential at every time step, which results initially in a multiplication by the stored constant $\exp^{-\frac{1}{\tau}}$ (2 op) followed by addition of the input value (1 op). The membrane potential is then compared to the firing threshold (1 op). Thus, the fixed cost is $C_f = 4$ op. The variable cost C_v , incurred only when a spike is produced, involves producing the spike (1 op) and resetting the membrane potential to zero (1 op). Thus, the variable cost $C_v = 2$ op.

For BSA (see algorithm 3 in the appendix), both the fixed and variable computational costs depend on the filter length M. The fixed operations involve first calculating the two error terms. For the first error term e₁, this involves a subtraction and an absolute value, followed by an addition (1 op each), repeated M times (thus 3M op in total). For the second error term e₂, this involves an absolute value and an addition repeated M times (thus 2M op in total). The fixed cost also involves comparing the difference of the error terms to the threshold (2 op). Overall, the fixed cost is $C_f = 5M+2$ op. The variable cost, incurred only when a spike is produced, involves producing the spike (1 op) and updating the input by subtracting the filter's impulse response (M op). Thus, the variable cost is $C_v = M+1$ op.

Figure 4 shows the computational cost rate $C(\rho)$ (figure 4(a)) and the energy cost rate $E(\rho)$ (figure 4(a)), as well as the computational efficiency η_C (figure 4(c)) and the energy efficiency η_E (figure 4(d)) of both spike encoding techniques.

Zoom In Zoom Out Reset image size

As can be observed from figures 4(a) and (b), it is clear that BSA is more computationally costly than LIF. In fact, this characterization of computational costs (an implementation-independent approximation) arrives at very similar results when compared to the measurement of the actual energy cost rate using the pyJoules Python library (actual energy consumed in the specific implementation with an Intel® Xeon® Processor E5-2683 v4 architecture).

As can be observed from figures 4(c) and (d), both BSA and LIF have peak computational and energy efficiencies in the spike train density range between 25% and 35%. Unsurprisingly, LIF's computational efficiency is much higher than BSA, which makes it a more computationally and energetically efficient spike encoding technique than BSA.

The potential relationship between information-theoretic spike encoding evaluation and application performance is explored in this section on the example of isolated spoken digit recognition. The same TIDIGITS dataset is used. It has a total of 11 classes: 'zero' through 'nine', plus 'oh'. The training and test partitions have 2464 and 2486 audio examples, respectively. The cochleagram representation of the spoken digits is encoded into spikes using LIF coding, and the firing threshold is considered as the parameter to optimize.

First, the search complexity is discussed and compared between the decoding and the information-theoretic approaches. The former approach consists in choosing the encoding parameters that optimize classification accuracy as the performance metric, while the latter approach consists in choosing the parameters that optimize an information-theoretic efficiency metric instead. In this discussion, the optimization is performed with a grid search, i.e. an exhaustive search through a manually specified set of parameter values. The discussion on search complexity is followed by a comparison between the two approaches when performing a classification task using three different classifiers.

4.3.1. Search complexity

4.3.2. Classification results

The decoding and the information theoretic approaches are compared here on a classification task.

In the information-theoretic approach, energy efficiency (mutual information over energy rate) is taken as the metric to optimize. The LIF firing threshold in each channel is chosen as the value in the grid search that maximizes the energy efficiency in that channel. Note that in this case, the firing threshold can be different from channel to channel.

In the decoding approach, since it is not feasible to carry out the entire grid search in a tractable manner, a simplification is made: the LIF firing threshold is constrained to be the same for all channels. The results obtained with this simplification represent a lower bound on the best performance that could be obtained with a full grid search.

For a given spoken digit from the dataset, the spikes in each spike train are counted and averaged in 25 bins. The bins have 50% overlap. Since the audio files of the spoken digits are variable in duration, the bin width for each spoken digit is also variable. There is a total of 200 features per spoken digit (25 bins × 8 spike train channels). Three classifiers are used from the scikit-learn Python library: support vector machine with a radial basis kernel function, k-nearest neighbors with the Euclidian distance as a metric and k = 5 neighbors, and random forest with 100 estimators.

Classification results are shown in figure 5. In all graphs, classification accuracy for the decoding approach grid search (solid line) plateaus in the range approximately comprised between 5% and 40% spike density. In the information-theoretic approach, the grid search is carried out using the energy efficiency as performance metric, and the firing thresholds are chosen as the ones that maximize that metric in each channel. The classification task is then run with this single parameter configuration (red cross markers). The spike trains found have an average density of 32% and achieve a classification accuracy that is only slightly lower than the best accuracies found with the decoding approach. As can be observed in figure 5, the grid search of the decoding approach finds slightly better classification accuracies that can be achieved with lower spike densities.

Zoom In Zoom Out Reset image size

These results show that although the information-theoretic approach does not guarantee the best performance, it can be used to find an initial parameter configuration that gives a decent performance. This can be used in turn as a baseline in further parameter tuning with the aim of achieving better performance or lower spike density or both. Additional parameter tuning can be efficiently carried out for example with evolutionary optimization techniques.

It is also relevant to note that the shape of the curves in figure 5 is similar to those of the information-theoretic efficiency metrics proposed in this work (figures 3 and 4). Nevertheless, a direct comparison is not possible because the firing threshold has been constrained to be the same for all cochleagram channels in figure 5.

There are many techniques that can be used in the spike encoding of sound. These techniques have been evaluated using two approaches: decoding and information theory. In the decoding approach, performance is evaluated in applications that use models and further process the spike trains destructively, causing information loss. In the information-theoretic approach, the total information captured by the spikes is measured without introducing models or additional processing on the spikes.

In this work, the performance of four spike encoding techniques is quantitatively evaluated based on three proposed information-theoretic efficiency metrics

• (i)  
  Coding efficiency measures how much information is encoded in the spikes from the total information available in the signal.
• (ii)  
  Computational efficiency incorporates the cost of spike encoding by attributing abstract costs to the algorithmic operations of the spike encoding techniques (implementation-independent).
• (iii)  
  Energy efficiency incorporates the cost of spike encoding by empirically estimating the energy expended in the operation of spike encoding techniques (implementation-dependent).

These metrics allow to assess the overall performance of the spike encoding techniques:

• Independent Spike Coding, which is the standard method in computational auditory modeling, is shown to have poor coding efficiency, which is not surprising: the stochastic nature of ISC's spike generation introduces noise in the spike trains that degrades the information content.
• Send-on-Delta coding, which is an event-based sampling scheme that is popular in neuromorphic sensors, is shown to have poor coding efficiency as well, as it is unable to effectively encode the amplitude information: SoD only codes relative changes in amplitude, not absolute values.
• Ben's Spiker Algorithm, which is based on stimulus estimation, is found to have the highest coding efficiency out of the four techniques, but it is computationally complex and thus suffers in terms of computational and energy efficiencies.
• Leaky Integrate-and-Fire coding has slightly less coding efficiency than BSA, but has much higher computational and energy efficiencies because it is a less costly spike encoding technique. Overall, LIF coding shows the best performance out of the four spike encoding techniques considered in this work.

Furthermore, as shown on the example of isolated spoken digit recognition presented here, using the information-theoretic efficiency metrics to optimize spike encoding parameters is tractable and leads to an initial baseline solution that shows decent application performance. This baseline solution can then be used in more efficient parameter tuning to try to improve performance or lower spike density or both.