I have a problem with a value I need to use for a math calculation. I’m using a lookup to bring in the value need for a simple tax calculation (amount*tax) that’s it, but! when I use that lookup value and multiply by the amount the result is no what it should be, it’s off.
First 2 columns are the values that I need to multiply, the last column is the result, but the result should be 231.88 not 231.66.
Although the numbers are correct the operation is incorrect. If I use a proper Number column it does the math correctly, but not if I use a lookup value. Have you encountered this problem or am I missing something here?
My thinking is that what you see on screen (the display) and the actual numbers under the hood are slightly different, and these slight differences could be further enhanced and clearly visible after the math which involves a multiplication? This is a hypothesis I would test.
Originally, I assume the numbers are in basic number columns. Even though you display them as 1.00, what happens when you visualize them with more numbers after the comma?
They are the same. If I choose to display 4 decimals I can “see” that all the numbers are correct. If that’s the case that numbers displayed and numbers stored are different then this is a BUG.
If I display more decimals it shows correctly, the problem is in the math operation, which is a ver BIG problem. If anyone else is using these values without proper validation then they’d be using incorrect results. I hope that’s not the case and I just need to use some “trick” to use have the expected result.
If in turn I use a Number column and I just use the number value in the operation instead of the lookup value, then the operation returns the correct result.
As far as I’ve tested, if you’re using a looked up value then the math column won’t return a correct result.
I’ve been testing more and it’s all the same. At some points I can SEE clearly the correct answer for this example (231.88) while the calculation is loading and when I click on the “Done” button, it displays the incorrect answer.
I’m working with a proposal header and details tables, I need the calculation to be performed where the items are (details). If I don’t find and answer, yes I would have to create an alternative method of getting the items values to do the necessary calculations. But it is not ideal.
Thank you for taking the time to take a look into this. The value comes from another math column (in the proposal header table) which is used to generate the value I need to send it to the details table.
Short answer is that you are not using 4.25. You are using 4.245967741935484. 4.25 is the “formatted/precious” version of a true underlying unrounded value. Your final math column is using the underlying value. Not the formatted value. Formatted values are used for display purposes only. Not for calculations.
Here’s what you need to do. You need to change this math column. The Precision setting is only for display purposes, but the true unrounded version of the number is calculated behind the scenes.
That’s exactly my mistake. I did now know that I had to use a round function. Just got off an email with support where they explained that the value it was used on the calculation was the complete value, not the displayed value.
