[fpc-devel] FormatFloat

[Jonas Maebe]

On 30 Dec 2009, at 18:25, darekm at emadar.com wrote:

>> 
>> On 30 Dec 2009, at 17:27, darekm at emadar.com wrote:
>> 
>>> this program:
>>> var
>>> d1,d2 : double;
>    d1,d2 : extended;
>>> begin
>>> d1:=105;
>>> d2:=1.05e2;
>>> writeln(d1-d2);
>>> end;
>> 
>> result: 6.9388939907E-18
> 
> sorry,
> I did not notice, that is different type (extended)
> 
> version 2.5.1 [2009/11/05] (linux and windows)

Ok, now I can reproduce it. It's caused by the fact that 1.05 cannot be represented exactly. 1.05e2 is first parsed as 1.05, and then multplied by 100. So you get an maximal error of extended_precision_delta*100. In that situation 6.9388939907E-18 is acceptable (extended_precision_delta is 1.0842021725e-19). The fact that the result is equal in case of double is simply because due to less available precision, the 1.05 value happens to be is rounded in a different way and 100*1.05 turns out the same as 105.0

Delphi apparently parses 1.05e2 in a different way (maybe it performs the "e2" multiplication on the string representation in order to minimise losses due to limited precision). Try the following program and you'll see that for the second subtraction, Delphi gives the same result as FPC:

var
 d1,d2,d3 : extended;
begin
 d1:=105;
 d2:=1.05e2;
 d3:=1.05;
 d3:=d3*100.0;
 writeln(d1-d2);
 writeln(d2-d3);
end.

Both results are correct in the context of IEEE floating point math.


Jonas