Next: The Code Up: Roots of a Quadratic: Previous: Precision

## Round-off errors and loss of significance

When two values of nearly equal magnitude are subtracted, significant digits are lost. For example subtracting 0.12345678 and 0.12345676 results in with only one significant digit. If the two values subtracted had to be rounded to eight significant digits in order to fit into a 32 bit floating point number, the difference is clearly very inaccurate. Referring to the standard solution above, we see that we have trouble if is positive and is much less than . In that case the two terms in the numerator are nearly equal in magnitude, so one of the roots (the one involving a subtraction) is unreliable. The other root should be fine. Which one it is depends on the sign of . If is positive, then the solution with the positive square root involves a subtraction. If is negative, then the solution with the negative square root is risky.

There is a simple solution to this problem. We can divide the quadratic equation by so it reads

Notice that this is a quadratic in . We can use the traditional formula to solve for and then invert it to get

where the upper sign here correlates with the upper sign in the previous, standard formula. (Prove it yourself! It's just a simple exercise in algebra.) This formula also has one reliable root and one less reliable one. A little thought shows that the possibly bad root in one formula is the good root in the other.

So the best strategy is to use both formulas and to choose the more reliable root from each.

Next: The Code Up: Roots of a Quadratic: Previous: Precision
Carleton DeTar 2017-09-05