[fpc-pascal] Freepascal Floating Point Issue
Ralf Quint
freedos.la at gmail.com
Tue Aug 22 00:10:36 CEST 2017
On 8/21/2017 3:02 PM, James Richters wrote:
> I am having an issue with a simple floating point application. I am setting a variable to a specific value and immediately after I set it, it is not exactly what I set it to. Here's an example
>
> Draw_GX_Min:=999.999;
> Writeln(Draw_GX_Min:3:30);
>
> The writeln results in 999.999000000000020000000000000000
>
> Why is it not 999.999000000000000000000000000000 where did 0.00000000000002 come from?
>
Out of thin air... Well, kind of. Double floating point means 16 digits
of precision, so when you force a 30 digit precision output, anything
past 16 digits is random garbage, at best...
Ralf
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
More information about the fpc-pascal
mailing list