I'm currently modeling a database using MongoDB in which users can transfer funds between accounts and buy products, the values of which are debited from their current balances. I'm working with a precision of two decimal places, for products values and for user's balance.
The problem is that when I add or subtract a value with decimal places using the $inc operator, in my user document I get some precision errors, like this:
{
"balance": 31513.210000000003,
}
I'm using node and mongoose to manipulate my DB, and I know about the floating point inaccuracies of the language, but I'd like to know if there's any way to overcome this issue in my mongodb database and force it to always work with two decimal places, so when I query for an user with a positive balance, values like 0.00000000003 won't be detected, as it should be 0.
Is there any way to control this in mongodb?
Instead of using a double which uses floating point math you've got several other methods you can choose from that offer different pros and cons.
Integer
Represent the balance not in the whole units, but in the fractional units. So you store 123.45 as 12345. If the math is addition and subtraction then this is easy, and all you need to make sure of is that any time you represent a value you insert the . at the appropriate place, but you can write a single function for this.
Object
Represent it as two integers: one for the whole part and one for the fractional part (123 and 45). This makes the math harder, but potentially makes the output easier to manage, and harder to accidentally output the wrong value.
String
With a string you'll have to write all your own math, but the output can be simple as the stored value would be '123.45'.
Related
In javascript, we know that some floating point arithmetic will result in values that contain a very small remainder. for example if I open my console and do this math I get this result as seen here
Typically one must know to round after doing these operations. When using parse, and using the increment function. If I have a column called Quantity in my DB and it currently has a value of 6.5 and I call myobject.increment("Quantity", -4.1) we end up with an unrounded value in the DB. the use of increment is required here as many entities may be adjusting this column at or near the same time, so race conditions are a concern.
since the arithmetic happens under the hood, how does parse expect one to handle cases like I mentioned above.
You may do one of two:
Change the logic inside increment function.
Do increment outside and use some setter.
Anyway in case you want to prevent non exact values, you should round it manually. Look here for number rounding answer
Why is this not a duplicate of these great SO articles?
While the two posts linked in the comments below are excellent I am specifically looking for information that helps me to address this issue in native JS. I know that JS shouldn't be the first choice for complex math, but given the limitation that this calculator is meant to run in the browser it is the tool that I have decided to work with.
Background
I'm trying to make a calculator with TypeScript without any libraries (like Big.js) and without using string concatenation in the inner logic of the calculator.
Examples
When a user wants to type the number 8.5:
The 8 key is pressed
The decimal key is pressed
The 5 key is pressed
Mathematically I create this number in the display with the following snippet:
8 + 5 * 0.1
This works but if I continue down the decimal places I encounter something unexpected:
8.5 + 5 * 0.01 // 8.55
8.55 + 5 * 0.001 // 8.555000000000001
Question
What is the best way to handle this without converting the number to a string? Is there an intelligent way to impose a limit on the precision of the calculator so that it only supports accuracy to so many decimal places?
Thanks for your help!
Use .toFixed():
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toFixed
or .toPrecision():
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toPrecision
depending on your needs.
Note that you don't need to convert numbers all the time. The only place to convert - is for the final output to the view. In this case we can even leave it in string format.
That is an answer to the question "how to get manage with" (like in description). About "why do we get such result" - first comment provides great answer.
The easiest way to get the Number value that is closest to what the user enters is to build a numeral in a string from the user’s keypress and then convert them with String.toNumber().
Numbers such as 8.55 or 8.555 are not exactly representable in the Number format. The closest values are 8.550000000000000710542735760100185871124267578125 and 8.55499999999999971578290569595992565155029296875. Converting the strings “8.55” and “8.555” with .toNumber() should produce these values.
Because these are the closest representable values, no calculation or algorithm can produce any closer values in the Number format.
For simple additions, subtractions, and limited multiplications, you can mimic decimal arithmetic by rounding to a desired number of decimal digits after each Number operation. However, this is generally not a feasible approach because other operations and various sequences of operations will exceed the ability to mimic decimal arithmetic reasonably.
It seems django, or the sqlite database, is storing datetimes with microsecond precision. However when passing a time to javascript the Date object only supports milliseconds:
var stringFromDjango = "2015-08-01 01:24:58.520124+10:00";
var time = new Date(stringFromDjango);
$('#time_field').val(time.toISOString()); //"2015-07-31T15:24:58.520Z"
Note the 58.520124 is now 58.520.
This becomes an issue when, for example, I want to create a queryset for all objects with a datetime less than or equal to the time of an existing object (i.e. MyModel.objects.filter(time__lte=javascript_datetime)). By truncating the microseconds the object no longer appears in the list as the time is not equal.
How can I work around this? Is there a datetime javascript object that supports microsecond accuracy? Can I truncate times in the database to milliseconds (I'm pretty much using auto_now_add everywhere) or ask that the query be performed with reduced accuracy?
How can I work around this?
TL;DR: Store less precision, either by:
Coaxing your DB platform to store only miliseconds and discard any additional precision (difficult on SQLite, I think)
Only ever inserting values with the precision you want (difficult to ensure you've covered all cases)
Is there a datetime javascript object that supports microsecond accuracy?
If you encode your dates as Strings or Numbers you can add however much accuracy you'd like. There are other options (some discussed in this thread). Unless you actually want this accuracy though, it's probably not the best approach.
Can I truncate times in the database to milliseconds..
Yes, but because you're on SQLite it's a bit weird. SQLite doesn't really have dates; you're actually storing the values in either a text, real or integer field. These underlying storage classes dictate the precision and range of the values you can store. There's a decent write up of the differences here.
You could, for example, change your underlying storage class to integer. This would truncate dates stored in that field to a precision of 1 second. When performing your queries from JS, you could likewise truncate your dates using the Date.prototype.setMilliseconds() function. Eg..
MyModel.objects.filter(time__lte = javascript_datetime.setMilliseconds(0))
A more feature complete DB platform would handle it better. For example in PostgreSQL you can specify the precision stored more exactly. This will add a timestamp column with precision down to miliseconds (matching that of Javascript)..
alter table "my_table" add "my_timestamp" timestamp (3) with time zone
MySQL will let you do the same thing.
.. or ask that the query be performed with reduced accuracy?
Yeah but this is usually the wrong approach.
If the criteria you're filtering by is to precise then you're ok; you can truncate the value then filter (like in the ..setMilliseconds() example above). But if the values in the DB you're checking against are too precise you're going to have a Bad Time.
You could write a query such that the stored values are formatted or truncated to reduce their precision before being compared to your criteria but that operation is going to need to be performed for every value stored. This could be millions of values. What's more, because you're generating the values dynamically, you've just circumvented any indexes created against the stored values.
I am implementing an invoice system, where everything is dynamically added on the dom through javascript and I am making some calculations on the browser itself with javascript.
for eg I am calculating each invoice line with quantity and price of unit and generating a total sum
price can be a floating point number
but I am not sure if this should be trusted or not, if someone has the same toughts about javascript please comment :)
I don't know but javascript doesn't seem to me to be trusted like other programming languages like PHP or so, this is my opinion, but if you can convince me please do
Thanks
Javascript uses the same data type that almost all languages use for floating point calculations. The double precision floating point data type is very common, because processors have built in support for it.
Floating point numbers have a limited precision, and most numbers with a fractional part can't be represented exactly. However, for what you are going to use it for, the precision is more than enough to show a correct result.
You should just be aware of the limited precision. When displaying the result, you should make sure that it's formatted (and rounded) to the precision that you want to show. Otherwise the limited precision might show up as for example a price of 14.9500000000000001 instead 14.95.
According to JavaScript's specifications, all numbers are 64bit precision (as in 64bit floating point precision).
From this post, you have 3 solutions:
use some implementation of Decimal for JavaScript, as BigDecimal.js
just choose a fixed number of digits to keep, like this (Math.floor(y/x) * x).toFixed(2)
switch to pure integers, treating prices as number of cents. This could lead you to big changes across the whole project
Financial calculations usually require specific fixed rules about (for example) when and how to round (in which direction), etc.
That means you'll often maintain an internal sub-total precision until you move to a next section of your calculation (like adding the tax, as per rules set).
IEEE-754 Floating point (as used in javascript) will give you a maximum accuracy of 2^53 (if you think about it like an integer).
Now your 'job' is to pretend javascript doesn't support floating point and substitute it yourself using the simplest possible way: decrease your maximum integer range to obtain the required floating point precision and see if that resulting range is suitable to your needs. If not, then you might need an external high precision math library (although most basic operations are pretty easy to implement).
First determine your desired internal precision (incl overflow digit for your expected rounding behavior): for example 3 digits:
FLOOR((2^53)/(10^3))=FLOOR(9.007.199.254.740.992/1000)=9.007.199.254.740,000
If this range is sufficient, then you need no other library, just multiply your input 10^float_digits and maintain that internal precision per calculation-section, while rounding each step according to the rules required for your calculation (you'd still need to do that when using a high-precision external math library).
For (visual) output, again, apply proper rounding and just divide your remaining value by 10^(floatDigits-roundingDigit(s)) and pass it through Number.prototype.toFixed() (which then just pads zero's when required).
As to your other question regarding trustworthiness of javascript vs other programming languages: one can even boot/run and use LINUX on javascript inside the browser: http://bellard.org/jslinux/
Let that sink in for a moment...
Now what if I told you this even works in IE6... Pretty humbling. Even servers can run on javascript (node.js)..
Hope this helps (it didn't fit in a comment).
Other answers have addressed issues that JavaScript has with using floating point numbers to represent money.
There's a separate issue with using JavaScript for calculations involving financial transactions that comes to mind.
Because the code is executed in a browser on the client machine, You can only trust the result to the extent that you can trust the client.
Therefore you should really only rely on JavaScript to calculate something that you could take for granted if the client told you.
For instance, if you were writing an e-commerce site, you could trust code that told you what the client wanted to buy, and what the clients shipping address was, but you would need to calculate the price of the goods yourself to prevent the client from telling you a lower price.
It's entirely possible that the invoicing system you're working on will only be used internally to your organisation.
If this is the case, you can disregard this entire answer.
But, if your applications is going to be used by customers to access and manipulate their invoices and orders, then this is something you'd have to consider.
How do you efficiently store very large and very small numbers from say 10^-100 to 10^100, so that you can use them to calculate values in a programming language like JavaScript.
JavaScript stores 10^100 as 1e+101, is there a way to do that in the database? The numbers would not often be that large, but I would like to do calculations with data such as 10^-34 * 2^16 or whatever, so the database should (I think) be storing these as numbers...
How does this work? How do you store numbers of this scale such that you can run computations with them?
By "the database", I'm thinking in general. I am messing around with MongoDB and Neo4j currently.
Databases themselves don't support numbers of arbitrary size in a native numeric format. Your general upper limit on numeric types is usually 8 bytes, which isn't anywhere near a googol.
You'll have to store the number either as a string (least efficient, easiest to work with, can be as precise as needed), as a byte array of arbitrary length (more efficient, harder to work with, still arbitrary precision), or in scientific notation (most efficient, harder to work with, and limited precision).
The first two, unfortunately, do eliminate the possibility of doing any server-side computation, since there wouldn't be a native numeric type that could support the range of valid values. All of the computation would have to be done client-side using a suitable numeric type.
If I were you, I'd separate the numerical value from the exponent. I personally don't have experience with MongoDB or Neo4j, but in MySQL (I'm sure they have similar terms) I'd create a table with an VARCHAR (text) column with whatever precision you'd like in your program (or how many unique numbers), and another VARCHAR column with length 3 (for max exponent 999). You can tinker with the values as you see fit, but that's all I can think of. If you want more flexible size values, I'd store the numbers on the server's file system using PHP rather than use databases.
You could use the double type.
The MySQL DOUBLE[(M,D)]
A normal-size (double-precision) floating-point number. Permissible
values are -1.7976931348623157E+308 to -2.2250738585072014E-308, 0,
and 2.2250738585072014E-308 to 1.7976931348623157E+308. These are the
theoretical limits, based on the IEEE standard. The actual range might
be slightly smaller depending on your hardware or operating system.