[fpc-pascal] Freepascal Floating Point Issue
James Richters
james at productionautomation.net
Tue Aug 22 00:02:01 CEST 2017
I am having an issue with a simple floating point application. I am setting a variable to a specific value and immediately after I set it, it is not exactly what I set it to. Here's an example
Draw_GX_Min:=999.999;
Writeln(Draw_GX_Min:3:30);
The writeln results in 999.999000000000020000000000000000
Why is it not 999.999000000000000000000000000000 where did 0.00000000000002 come from?
Draw_GX_Min is defined as a Double. I don't need that much precision for any reason. I'm just trying to understand why it's not what I set it to.
The reason I noticed this is because I have some conditional statements like
If Draw_GX_Min<>99.999 then
Something....
And sometimes they works as expected, and sometimes not.
I realize I could do:
If round(Draw_GX_Min*1000)<>999999 then
Something
and it would work, but why do I need to do this when I explicitly set it to 999.999?
Does anyone know what is going on here that would be kind enough to explain it to me?
I suspect this has something to do with floating point, but I don't really understand what's happening.
Is there a better way to define my variable, perhaps defined as fixed decimal point instead of allowing it to float.... so that say all numbers from -999.99999 to 999.99999 are guaranteed to be represented accurately to 5 decimal places so that if I set one to 999.99999 it would actually be 999.99999 exactly? I would rather have less precision but have variables retain exactly what they are set to, not something close
Thanks
James
More information about the fpc-pascal
mailing list