[fpc-pascal] Loss of precision when using math.Max()
Sven Barth
pascaldragon at googlemail.com
Fri Jun 29 22:55:01 CEST 2018
Am 29.06.2018 um 18:45 schrieb Alan Krause:
> I stumbled upon something the other day that was causing numerical
> differences between compiled Delphi and FPC code. Executing the
> following sample console application illustrates the issue clearly:
>
> program test;
>
> uses
> math, SysUtils;
>
> var
> arg1 : double;
> arg2 : double;
> res : double;
> begin
> arg1 := 100000.00;
> arg2 := 72500.51;
> writeln( 'arg1 = ' + FormatFloat( '0.00000000', arg1 ) );
> writeln( 'arg2 = ' + FormatFloat( '0.00000000', arg2 ) );
>
> res := arg1 - arg2;
> writeln( 'arg1 - arg2 = ' + FormatFloat( '0.00000000', res ) );
> writeln( 'Max( arg1 - arg2, 0 ) = ' + FormatFloat( '0.00000000',
> Max( res, 0 ) ) );
> writeln( 'Max( arg1 - arg2, 0.0 ) = ' + FormatFloat( '0.00000000',
> Max( res, 0.0 ) ) );
> end.
>
> --- begin output (Linux x86_64) ---
>
> arg1 = 100000.00000000
> arg2 = 72500.51000000
> arg1 - arg2 = 27499.49000000
> *Max( res, 0 ) = 27499.49023438*
> Max( res, 0.0 ) = 27499.49000000
>
> --- end output ---
>
> I am guessing that the integer value of zero is causing the wrong
> overloaded function to be called? I was able to solve the problem in
> my code by replacing the 0 with 0.0.
The compiler converts the 0 to the type with the lowest precision that
can hold the value (or the largest if none can hold it exactly). For 0
this is already satisfied by Single, so the compiler essentially has the
parameter types Double and Single. For some reason (I don't know whether
it's due to a bug or by design) it picks the Single overload instead of
the Double one.
Someone who knows more about the compiler's overload handling would need
to answer why it favors (Single, Single) over (Double, Double) for
(Double, Single) parameters (or (Single, Double), the order doesn't
matter here).
Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20180629/3957cb23/attachment.html>
More information about the fpc-pascal
mailing list