[fpc-pascal] Floating point question
James Richters
james.richters at productionautomation.net
Sun Feb 4 21:17:33 CET 2024
Here is a more concise example that illustrates the issue. For me, being a
human, I see 1440 and 1440.0 as exactly the same thing, but they are not
acting as the same thing, and the 1440.0 is causing all the grief here.
See how it makes a difference whether the .0 is there or not.
Then replace the 1440.1, and notice how it's no longer an issue, note it's
only a problem with .0 if it's a .1, or anything other than .0, it seems
fine.
program TESTDBL1 ;
Const
AA = 8427+33/1440.0;
BB = 8427+33/1440;
CC = 8427.0229166666666666666666666667; //Windows Calculator
Var
A_Ext : Extended;
B_Ext : Extended;
C_Ext : Extended;
A_Dbl : Double;
B_Dbl : Double;
C_Dbl : Double;
A_Sgl : Single;
B_Sgl : Single;
C_Sgl : Single;
begin
A_Ext := AA;
B_Ext := BB;
C_Ext := CC;
A_Dbl := AA;
B_Dbl := BB;
C_Dbl := CC;
A_Sgl := AA;
B_Sgl := BB;
C_Sgl := CC;
WRITELN ( 'A_Ext = ',A_Ext: 20 : 20 ) ;
WRITELN ( 'B_Ext = ',B_Ext: 20 : 20 ) ;
WRITELN ( 'C_Ext = ',C_Ext: 20 : 20 ) ;
WRITELN;
WRITELN ( 'A_Dbl = ',A_Dbl: 20 : 20 ) ;
WRITELN ( 'B_Dbl = ',B_Dbl: 20 : 20 ) ;
WRITELN ( 'C_Dbl = ',C_Dbl: 20 : 20 ) ;
WRITELN;
WRITELN ( 'A_Sgl = ',A_Sgl: 20 : 20 ) ;
WRITELN ( 'B_Sgl = ',B_Sgl: 20 : 20 ) ;
WRITELN ( 'C_Sgl = ',C_Sgl: 20 : 20 ) ;
end.
A_Ext = 8427.02246093750000000000
B_Ext = 8427.02291666666666625000
C_Ext = 8427.02291666666666625000
A_Dbl = 8427.02246093750000000000
B_Dbl = 8427.02291666666680000000
C_Dbl = 8427.02291666666680000000
A_Sgl = 8427.02246100000000000000
B_Sgl = 8427.02246100000000000000
C_Sgl = 8427.02246100000000000000
Notice for Double and Extended they are getting the value for Single for the
division, throwing off the result, but only with 1440.0, not with 1440
With the constants defined as:
Const
AA = 8427+33/1440.10;
BB = 8427+33/1440.1;
CC = 8427.0229150753419901395736407194; //Windows Calculator
Now notice:
A_Ext = 8427.02291507534198978000
B_Ext = 8427.02291507534198978000
C_Ext = 8427.02291507534198978000
A_Dbl = 8427.02291507534210000000
B_Dbl = 8427.02291507534210000000
C_Dbl = 8427.02291507534210000000
A_Sgl = 8427.02246100000000000000
B_Sgl = 8427.02246100000000000000
C_Sgl = 8427.02246100000000000000
All versions of Extended, Double, and Single are identical. As expected.
Everything I try works, except for .0
James
More information about the fpc-pascal
mailing list