[fpc-devel] Incorrect/Strange precision in FloatToStr and FloatToStrF

[Graeme Geldenhuys]

Any idea how Delphi handles this differently then without rounding
errors?  This behaviour was picked up in my unit tests I am porting
from Delphi to FPC.

Using Double or Real under Delphi worked.  Extended was the only type
I could use under FPC that gave me the correct results.

Regards,
  - Graeme -


On 1/17/06, Marco van de Voort <marcov at stack.nl> wrote:
> >
> > Is this a bug?  The actual value of 234.502 is not greater than either
> > data types (extended or double), but the conversion to string result
> > using the Double is not what I expected.
> >
> > Strangely enough 234.501 seem to work.  7.502 and 8.502 is another strange one!
>
>
> This is all normal. It has to do with the fact that floating point numbers
> variables are internally base 2 (binary) floating point notation.
>
> So if you assign a certain decimal value, say 0.1234, is converted to the
> closest binary notation and assigned to the float. Then when you print it is
> converted back to decimal. Both operations can cause rounding
>
>
> _______________________________________________
> fpc-devel maillist  -  fpc-devel at lists.freepascal.org
> http://lists.freepascal.org/mailman/listinfo/fpc-devel
>