[fpc-devel] Incorrect/Strange precision in FloatToStr and FloatToStrF
graemeg.lists at gmail.com
Tue Jan 17 16:02:24 CET 2006
Any idea how Delphi handles this differently then without rounding
errors? This behaviour was picked up in my unit tests I am porting
from Delphi to FPC.
Using Double or Real under Delphi worked. Extended was the only type
I could use under FPC that gave me the correct results.
- Graeme -
On 1/17/06, Marco van de Voort <marcov at stack.nl> wrote:
> > Is this a bug? The actual value of 234.502 is not greater than either
> > data types (extended or double), but the conversion to string result
> > using the Double is not what I expected.
> > Strangely enough 234.501 seem to work. 7.502 and 8.502 is another strange one!
> This is all normal. It has to do with the fact that floating point numbers
> variables are internally base 2 (binary) floating point notation.
> So if you assign a certain decimal value, say 0.1234, is converted to the
> closest binary notation and assigned to the float. Then when you print it is
> converted back to decimal. Both operations can cause rounding
> fpc-devel maillist - fpc-devel at lists.freepascal.org
More information about the fpc-devel