During my implementation of a digital filter, the verification work was carried out by comparing the output of my C++ program to the result from Octave scripts. The filter coefficients generated by Octave have the double type, while the third-party signal processing library I’m using can only produce float values. Except these differences, other parameters and implemented algorithms are completely the same.

For smoothing a same input signal, I found out that the relative 2-norm error between C++ and Octave can be as large as 0.6%, even though the error between the two sets of filter coefficients is only \(10^{-10}\). This contradicts my initial intuition. Then I applied filter coefficients with a same data type for both C++ and Octave, the result error was reduced to \(10^{-12}\).

Actually, the 0.6% discrepancy is caused by error accumulation during the integration or iteration of the filter’s state equation. And this experience teaches me these lessons:

  1. Don’t underestimate the difference between float and double types.
  2. Errors can be accumulated to an observable or significant amount during the integration or iteration of a state equation.
  3. When comparing algorithms of different implementations, the adopted input data and parameters should be exactly the same. This means the values of test parameters should not be manually copied from C++ debugger, Octave console or spreadsheet because some digits may be hidden, but should be exported with maximum accuracy using format controlled program, such as 15 digit long and 25 characters wide.
  4. Rigorous derivation of mathematical theories and careful coding processes are mandatory and crucial for numerical software development. However, they are still not enough. Besides, we need to establish both quantitatively accurate (including exact values, identities, equations) and qualitatively correct (including inequalities, magnitude) intuitions. Such intuitions cannot be acquired from a flash of ideas, but can only be accumulated from our plentiful practices and continuous focus.