[fpc-devel] Rounding inconsistent (critical)
Graeme Geldenhuys
graemeg.lists at gmail.com
Mon Oct 2 12:26:56 CEST 2006
On 02/10/06, Vincent Snijders <vsnijders at quicknet.nl> wrote:
> It is not a bug, it is using bankers rounding. See
> http://lazarus-ccr.sourceforge.net/docs/rtl/system/round.html
>
> Vincent
Exactly, but it is still wrong, except if I really don't know how
Banker's rounding works and what they mean by Even Number.
Take these to new tests.
AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445));
AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455));
Test 10:
Fails and said it expected 0.45 instead of my answer 0.44. Why?
This is how I came to my answer of 0.44. I ignore the "0." and only
look at 445 and I want that to 2 decimal digits. Lets now do what the
documentation says and go to the closest even number. 44 is even, 45
is not. So I would have thought it must round down to 0.44. It
doesn't.
Test 11:
Fails and said it expected 0.45 instead of my answer 0.46. Again,
the way I rounded it. Ignore the "0.' and only look at 455. I want it
to 2 decimal digits. What is the closest to a even number 45 or 46. In
this case, 46. Hence I expected it to round up and the final answer to
be 0.46. It doesn't.
I'm confused!! :-)
---- Under Delph 7 I got this -----------
Failing on 9: Expected <0.00> but got <0.01>
Failing on 10: Expected <0.44> but got <0.45>
------------------ end --------------------
So what rounding does Delphi 7 use?
Regards,
- Graeme -
More information about the fpc-devel
mailing list