[fpc-pascal] Floating point question
James Richters
james.richters at productionautomation.net
Tue Feb 13 10:09:51 CET 2024
Sorry for the kind of duplicate post, I submitted it yesterday morning and I
thought it failed, so I re-did it and tried again.. then after that the
original one showed up.
A thought occurred to me. Since the complier math is expecting all the
constants would be in full precision, then the compiler math doesn't need to
change, it's just that the reduction in precision is just happening too
soon. It's evaluating and reducing each term of an expression, then the
math is happening, and the answer is not coming out right.
If instead everything was left full precision until after the compiler math
(because this is what the compiler math expects), and then the final answer
was reduced in precision where possible, then it would work flawlessly. So
the reduction in precision function only needs to run once on the final
answer, not on every term before the calculation.
Sorry again for the duplicate.
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20240213/8baf14cf/attachment-0001.htm>
More information about the fpc-pascal
mailing list