[fpc-pascal] Floating point question

James Richters james.richters at productionautomation.net
Tue Feb 20 04:41:51 CET 2024


>I would not put too much trust in Windows calculator, since there you have
no control over the precision at all.
The actual CORRECT answer according to
https://www.wolframalpha.com/input?i=1%2F3.5 Is 
0.285714 Repeating forever

Which is what I get on windows only when using Variables.
   Var_Ans1 =  2.85714285714285714282E-0001
Which as you can see the .285714 repeats exactly 3 times with two more
correct digits, for a total of 20 correct digits before we run out of
precision. 

>It seems to be a windows-specific problem. Here is the result of your
program when executed on Linux:
>  Const_Ans1 =  2.85714298486709594727E-0001
>    Var_Ans1 =  2.85714298486709594727E-0001

>As you can see, the result is identical.
Great, Linux is consistently giving you the wrong answer, and I notice it's
the same wrong I answer I get when I use constants.

I don't know why it would be different in Windows than on Linux. 
I am curious what you get in Linux for:
A_const = (2/7);   // should be 0.285714 Repeating forever also
Or 
B_const = (2/7)-(1/3.5);  //should always be 0;

Just for fun I ran it on Turbo Pascal 7.0 for DOS and got:
 Const_Ans1 =  2.85714285714286E-0001
     Var_Ans1 =  2.85714285714286E-0001

I also noticed that re-casting constants is not allowed by Turbo Pascal 7.0,
so I removed the unnecessary re-casting so it would compile, but I did get
the expected answer even without re-casting it... as I expected I should. 
The fact that Turbo Pascal gives be the correct answer to 14 digits even
though in Turbo Pascal I also divided a byte by a single shows that I should
expect my pascal programs to provide THIS correct answer, 0.285714 repeating
until you run out of precision,  not something else. And it shows that a
byte divided by a single is in fact able to be an extended... it also proves
that in Pascal the result of a byte divided by a single is NOT limited to
single precision. 

2.85714298486709594727E-0001 makes no sense to me at all, It's like it did
the calculation in Single precision, then stored it in an extended, but if I
wanted to work in single precision, I would not have set my variable to
extended.

There is some flawed logic somewhere that says if you do math with a single,
the result is a single, but this is simply not correct, you can divide a
byte by a byte and get an extended, and you can divide a single by a single
and get an extended, Pascal has ALWAYS worked that way.
Forcing the result of an equation to be a single because one term is a
single is just not right. 

James



More information about the fpc-pascal mailing list