-
-
Notifications
You must be signed in to change notification settings - Fork 778
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix floating-point arithmetic issues when props are decimals. #619
base: master
Are you sure you want to change the base?
Fix floating-point arithmetic issues when props are decimals. #619
Conversation
export function getPrecision(step) { | ||
const stepString = step.toString(); | ||
let precision = 0; | ||
if (stepString.indexOf('.') >= 0) { | ||
precision = stepString.length - stepString.indexOf('.') - 1; | ||
} | ||
return precision; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is untouched, moved it up because I'm using it elsewhere.
parseFloat(closestPoint.toFixed(getPrecision(step))); | ||
withPrecision(closestPoint, getPrecision(step)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can back out this refactor if it's unwanted
Codecov Report
@@ Coverage Diff @@
## master #619 +/- ##
==========================================
+ Coverage 95.4% 95.83% +0.43%
==========================================
Files 2 2
Lines 87 96 +9
Branches 29 30 +1
==========================================
+ Hits 83 92 +9
Misses 4 4
Continue to review full report at Codecov.
|
Consider the following case:
I would expect the output of
getClosestPoint
to be5.3
. However, it returns5.25
.This is because of floating point arithmetic issues with the function.
I've added a test case and taken a stab at figuring it out, making sure I continue to respect the
Math.floor
logic onmaxSteps
.