[fpc-pascal] Loss of precision when using math.Max()

Alan Krause alank at shermanloan.com
Fri Jun 29 18:45:31 CEST 2018


I stumbled upon something the other day that was causing numerical
differences between compiled Delphi and FPC code. Executing the following
sample console application illustrates the issue clearly:

program test;

uses
  math, SysUtils;

var
  arg1 : double;
  arg2 : double;
  res  : double;
begin
  arg1 := 100000.00;
  arg2 := 72500.51;
  writeln( 'arg1 = ' + FormatFloat( '0.00000000', arg1 ) );
  writeln( 'arg2 = ' + FormatFloat( '0.00000000', arg2 ) );

  res := arg1 - arg2;
  writeln( 'arg1 - arg2 = ' + FormatFloat( '0.00000000', res ) );
  writeln( 'Max( arg1 - arg2, 0 ) = ' + FormatFloat( '0.00000000', Max(
res, 0 ) ) );
  writeln( 'Max( arg1 - arg2, 0.0 ) = ' + FormatFloat( '0.00000000', Max(
res, 0.0 ) ) );
end.

--- begin output (Linux x86_64) ---

arg1 = 100000.00000000
arg2 = 72500.51000000
arg1 - arg2 = 27499.49000000
*Max( res, 0 ) = 27499.49023438*
Max( res, 0.0 ) = 27499.49000000

--- end output ---

I am guessing that the integer value of zero is causing the wrong
overloaded function to be called? I was able to solve the problem in my
code by replacing the 0 with 0.0.

Thanks,
  Alan
-- 
Alan Krause
*President @ Sherman & Associates, Inc.*
Office: (760) 634-1700    Web: https://www.shermanloan.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20180629/c145be1c/attachment.html>


More information about the fpc-pascal mailing list