Floating-point math has an answer for everything, but sometimes that’s not what you want. Sometimes instead of getting an answer to the question sqrt(-1.0) (it’s NaN) it’s better to know that your …| Random ASCII - tech blog of Bruce Dawson
Riddle me this Batman: how much precision are these calculations evaluated at? If you answered ‘double’ and ‘float’ then you score one point for youthful idealism, but zero points for correctness. …| Random ASCII - tech blog of Bruce Dawson
How much precision does a float have? It depends on the float, and it depends on what you mean by precision. Typical reasonable answers range from 6-9 decimal digits, but it turns out that you can …| Random ASCII - tech blog of Bruce Dawson
This post is a more carefully thought out and peer reviewed version of a floating-point comparison article I wrote many years ago. This one gives solid advice and some surprising observations about…| Random ASCII - tech blog of Bruce Dawson
Last year I pointed out that float variables can be converted to text and then back to the same binary value using printf(“%1.8e”). You can also use %.9g for more compact results. I als…| Random ASCII - tech blog of Bruce Dawson