|
| [WD20] 0.10 in Db becomes 0.1000000014901 |
| Iniciado por guest, 17,dic. 2015 16:11 - 4 respuestas |
| |
| | | |
|
| |
| Publicado el 17,diciembre 2015 - 16:11 |
I have a tax rate stored as a 4 byte real. The tax rate is 0.10. When I query the Db and insert the tax rate into a table column the rate becomes 0.1000000014901. I've tried just about every Numeric input mask includingg 4 byte real. What's going on?
Also - I made sure that no other code is effecting this value. Right after TableAddLine() I did a trace to make sure the value was 0.1000000014901. |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,diciembre 2015 - 16:16 |
Hi,
I had something similat in a windev application when doingcalculations.
You can use this in your calculation
ROUND( calculation, 2 ) this is goning to round to two decimals.
regards
Allard |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,diciembre 2015 - 16:31 |
| I thought about that, but since I'm dealing with sales I need everything to be perfect. Also I want to allow tax rates up to 4 decimals. |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,diciembre 2015 - 16:36 |
Hi
this is unfortunately absolutely normal. Reals are a very OLD type of variable, created at the time where every byte counted. So the internal storage format does not allow to store every EXACT value.
This is why we have both the Currency and Numeric variable types, and you should use one of those two ALL the time, instead of reals, if you want to store/use EXACT value.
By the way, this is not a windev thing. Reals are a universal format and exhibit the same behavior in all languages.
Best regards |
| |
| |
| | | |
|
| | |
| |
| Publicado el 17,diciembre 2015 - 17:00 |
Thanks guys.
Thanks Fabrice. |
| |
| |
| | | |
|
| | | | |
| | |
|