Mathematical Principles for Scientific Computing and Visualization  
http://www.farinhansford.com/books/scv
by Gerald Farin and Dianne Hansford


Demonstration of Subtractive Cancellation Error

Cancellation error is a by-product of floating point calculations. It is the result of having very few significant digits, and the number of significant digits might vary. 

Let's look at the function f(x) = (1-cos(x))/x^2
Notice that evaluation around x=0 might be tricky because x is a divisor. Near x=0, 1 and cosx are nearly equal. Taking a difference of two nearly equal quantities results in fewer significant digits than we had with the original quantities. In fact, the number of significant digits will vary for different x-values close to the origin. This results in the oscillations in the plots below.

In the first plot below, we let x vary between -10 and 10.
In the second plot, we let x vary between -1E-05 to 1E05.
In the third plot, we let the range of x vary near x=0.

Try varying the ranges in the Manipulation plot.

Notice that sometimes (1-cosx) is evaluated to be a negative number!
cosx is approximated in a computer too -- so there is truncation error already present in its representation.

What you can't see won't hurt you?!

For more information, see Chapter 2 in Scientific Computing and Visualization by G. Farin and D. Hansford, AKPeters 2008.

f[x_] := N[(1- Cos[x])/x^2, 300];

Plot[f[x], {x, -10,10}, ImageSize->400, PlotStyle->Thick]
Plot[f[x], {x, -1.0*10^-5, 1.0*10^-5}, ImageSize->400]
Manipulate[Plot[f[x], {x, -range,range}, ImageSize->400],{{range,0.05},  1.0*10^-5,0.05, 1.0*10^-4 }]