I am currently working on a project that requires me to iterate through a list of values and add a new value in between each value already in the list. This is going to be happening for every iteration so the list will grow exponentially. I decided that implementing the list as a Linked List would be a great idea. Now, JS has no default Linked List data structure, and I have no problem creating one.
But my question is, would it be worth it to create a simple Linked List from scratch, or would it be a better idea to just create an array and use splice() to insert each element? Would it, in fact, be less efficient due to the overhead?
Use a linked list, in fact most custom implementations done well in user javascript will beat built-in implementations due to spec complexity and decent JITting. For example see https://github.com/petkaantonov/deque
What george said is literally 100% false on every point, unless you take a time machine to 10 years ago.
As for implementation, do not create external linked list that contains values but make the values naturally linked list nodes. You will otherwise use way too much memory.
Inserting each element with splice() would be slower indeed (inserting n elements takes O(n²) time). But simply building a new array (appending new values and appending the values from the old one in lockstep) and throwing away the old one takes linear time, and most likely has better constant factors than manipulating a linked list. It might even take less memory (linked list nodes can have surprisingly large space overhead, especially if they aren't intrusive).
Javascript is an interpreted language. If you want to implement a linked list then you will be looping a lot! The interpreter will perform vely slowly. The built-in functions provided by the intrepreter are optimized and compiled with the interpreter so they will run faster. I would choose to slice the array and then concatenate everything again, it should be faster then implementing your own data structure.
As well javascript passes by value not by pointer/reference so how are you going to implement a linked list?
Related
JavaScript provides a variety of data structures to be used ranging from simple objects over arrays, sets, maps, the weak variants as well as ArrayBuffers.
Over the half past year I found myself in the spot to recreate some of the more common structures like Dequeues, count maps and mostly different variants of trees.
While looking at the Ecma specification I could not find a description on how arrays implemented on a memory level, supposedly this is up to the underlying engine?
Contrary to languages I am used to, arrays in JavaScript have a variable length, similar to list. Does that mean that elements are not necessarily aligned next to each other in memory? Does a splice push and pop actually result in new allocation if a certain threshold is reached, similar to for example ArrayLists in Java? I am wondering if arrays are the way to go for queues and stacks or if actual list implementations with references to the next element might be suited in JavaScript in some cases (e.g. regarding overhead opposed to the native implementation of arrays?).
If someone has some more in-depth literature, please feel encouraged to link them here.
While looking at the Ecma specification I could not find a description on how arrays implemented on a memory level, supposedly this is up to the underlying engine?
The ECMAScript specification does not specify or require a specific implementation. That is up to the engine that implements the array to decide how best to store the data.
Arrays in the V8 engine have multiple forms based on how the array is being used. A sequential array with no holes that contains only one data type is highly optimized into something similar to an array in C++. But, if it contains mixed types or if it contains holes (blocks of the array with no value - often called a sparse array), it would have an entirely different implementation structure. And, as you can imagine it may be dynamically changed from one implementation type to another if the data in the array changes to make it incompatible with its current optimized form.
Since arrays have indexed, random access, they are not implemented as linked lists internally which don't have an efficient way to do random, indexed access.
Growing an array may require reallocating a larger block of memory and copying the existing array into it. Calling something like .splice() to remove items will have to copy portions of the array down to the lower position.
Whether or not it makes more sense to use your own linked list implementation for a queue instead of an array depends upon a bunch of things. If the queue gets very large, then it may be faster to deal with the individual allocations of a list so avoid having to copy large portions of the queue around in order to manipulate it. If the queue never gets very large, then the overhead of a moving data in an array is small and the extra complication of a linked list and the extra allocations involved in it may not be worth it.
As an extreme example, if you had a very large FIFO queue, it would not be particularly optimal as an array because you'd be adding items at one end and removing items from the other end which would require copying the entire array down to insert or remove an item from the bottom end and if the length changed regularly, the engine would probably regularly have to reallocate too. Whether or not that copying overhead was relevant in your app or not would need to be tested with an actual performance test to see if it was worth doing something about.
But, if your queue was always entirely the same data type and never had any holes in it, then V8 can optimize it to a C++ style block of memory and when calling .splice() on that to remove an item can be highly optimized (using CPU block move instructions) which can be very, very fast. So, you'd really have to test to decide if it was worth trying to further optimize beyond an array.
Here's a very good talk on how V8 stores and optimizes arrays:
Elements Kinds in V8
Here are some other reference articles on the topic:
How do JavaScript arrays work under the hood
V8 array source code
Performance tips in V8
How does V8 optimize large arrays
I'm currently building a small application is Vanilla JS (without any dependencies like lodash/jquery) and I needed to compare two objects to check for equality in keys and values. I was just wondering about how to optimize this problem.
The keys of both objects are in the same order as they are derived from the same method. According to this answer, the fastest and most efficient way to do this is using JSON.stringify(object1) === JSON.stringify(object2).
However, in my app, if the two objects are not equal, then I loop through the two of them and perform some operations. The problem is that these operations are pretty performance heavy and run occasionally. I need to optimize my solution.
Therefore, I was wondering if JSON.stringify runs some sort of for loop internally as well. In my application, it is more likely for the two objects to be unequal. Therefore, if JSON.stringify also runs some sort of for loop, I could just remove the check and run the operations I need right away (which will only cause a difference in the program if the two objects are unequal) saving time and making it more optimized. If I don't do this, then I will technically be running two for loops for the exact same purpose when the two objects are unequal and running one for loop either way when the two objects are equal. If JSON.stringify is some sort of for loop internally, then I can just one for loop no matter if the objects are equal. Am I making sense here? Please let me know if you don't understand something. Is this check useless and should I remove it to optimize my code?
Your question touches 4 different areas:
The implementation (and thus performance) of JSON.stringify
The implementation (and thus performance) of object iteration
The quality and performance of the JIT compiler
The speed of memory allocation (JSON.stringify is a memory hog for big objects)
So it is quite clear, that there is now "Universal" answer for all JS engines and OSes.
I recommend you do checks your in code ... why?
While right now the order of attributes might be constant, future maintenance to your codebase might change that and introduce a hard to track down bug.
It is good practice to create an isEqual method for all object types you use
It is better readable.
Ofcourse there are also disadvantages:
Your code will become bigger (this might be linked to better readable)
ANything else I might have forgotten.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have read this article:
https://gamealchemist.wordpress.com/2013/05/01/lets-get-those-javascript-arrays-to-work-fast/
At the end of point 6 the author says:
Rq about shift/unshift : beware, those are always O(n) operations
(meaning : each operation will take a time proportionnal to the number
of the array length). Unless you really need to, you shouldn’t use
them. Rather build your own rotating array if you need such feature.
And in the 7th point:
Rq for shift/unshift users : apply the same principle, with two
indexes, to avoid copy/reallocation. One index on the left, one on the
right, both starting at the middle of the array. Then you’ll be again
in O(1) time. Better. Don’t forget to re-center the indexes when they
are ==.
I was wondering what does the author mean when he says build your own rotating array and two indexes,...One index on the left, one on the right, both starting at the middle of the array. How should be this considerations translated into code (the author doesn't make an example for this use cases)?
Could the principles applied to shift and unshift be applied to Array.prototype.splice too?
EDIT: I have an ordered array of x coordinates going from indexes 0 (lower values for x) to n (higher x values). I would need to use myArray.splice(index, 0, item); several times and insert some new x coordinates between the already existent ones if this coordinate is < of an higher one and > of a lower one (I can easily find that out through a binary search) and I don't want it to reorder the indexes every time I call splice cause I have thousands of elements in the array myArray.
Can it be improved using the principles mentioned by the author of the linked article?
Thanks for the attention.
All performance questions must be answered by coding a very specific solution and then measuring the performance of that solution compared to your alternative with representative data in the browsers you care about. There are very few performance questions that can be answered accurately with an abstract question that does not include precise code to be measured.
There are some common sense items like if you're going to put 1000 items in an array, then yes it is probably faster to preallocate the array to the final length and then just fill in the array values rather than call .push() 1000 times. But, if you want to know how much difference there is and whether it's actually relevant in your particular situation, then you will need to code up two comparisons and measure them in multiple browsers in a tool like http://jsperf.com.
The recommendation in that article to create your own .splice() function seems suspect to me without measuring. It seems very possible that a good native code implementation of .splice() could be faster than one written entirely in Javascript. Again, if you really wanted to know, you would have to measure a specific test case.
If you have lots of array manipulations to do and you want to end up with a sorted array, it might be faster to just remove items, add new items onto the end of the array and the call .sort() with a custom comparison function when you're doing rather than inserting every new item in sorted order. But, again which way is faster will depend entirely upon exactly what you are doing, how often you're doing it and what browsers you care about the most. Measure, measure, measure if you really want to know.
As to whether your specific situation in your edit can be improved with a custom .splice(), you'd have to code it up both ways with a representative data set and then test in a tool like perf in multiple browsers to answer the question. Since you haven't provide code or data to test, none of us can really answer that one for you. There is no generic answer that works for all possible uses of .splice() on all possible data sets in all possible browsers. The devil is in the details and the details are in all the specifics of your situation.
My guess is that if you're really performance tweaking, you will often find bigger fish to fry in making your overall algorithm smarter (so it has less work to do in the first place) than by trying to rewrite array methods. The goal is to test/measure enough to understand where your performance bottlenecks really are so you can concentrate on the one area that makes the most difference and not spend a lot of time guessing about what might make things faster. I'm always going to code a smart use of the existing array methods and only consider a custom coded solution when I've proven to myself that I have a bottleneck in one particular operation that actually matters to the experience of my app. Per-optimization will just make the code more complicated and less maintainable and will generally be time poorly spent.
I was thinking about the same problem and have ended up using B+tree data structure. It takes time and not easy to implement but really good result. It can be considered the combination of both good aspects of array and linked-list:
In term of search performance, it is similar to binary search on array and even better version ? (not so sure but at least it's tight).
Modifying (insert, delete) the set without affect all other element index (affect range is very small constant - length of a block)
I would like to hear your thought, you can check this link for the visualization of b+tree in action.
When using the new Array(size) ctor, if size is not a constant, JS seems to create a sparse array in some browsers (at least in Chrome), causing access to be much slower than when using the default ctor, as shown here.
That is exactly the opposite of what I want: I pre-allocate an array of given size to avoid dynamic re-allocation and thereby improving performance. Is there any way to achieve that goal?
Please note that this question is not about the ambiguity of the new Array(size) ctor. I posted a recommendation on that here.
100000 is 1 past the pre-allocation threshold, 99999 still pre-allocates and as you can see it's much faster
http://jsperf.com/big-array-initialize/5
Pre-allocated vs. dynamically grown is only part of the story. For
preallocated arrays, 100,000 just so happens to be the threshold where
V8 decides to give you a slow (a.k.a. "dictionary mode") array.
Also, growing arrays on demand doesn't allocate a new array every time
an element is added. Instead, the backing store is grown in chunks
(currently it's grown by roughly 50% each time it needs to be grown,
but that's a heuristic that might change over time).
You can find more information ..here.Thanks ..:)
I have a json nested object, similar to this.
In my case, I have a unique id field of type int(say instead name above). This is not a binary tree, but more depict parent-child relationship. I wanted a way to easy lookup the child tree (children) rooted at say id = 121. In a brute force way, I may compare all nodes till I find one, and return the children. But I was think of keeping a map of {id, node}. For example {"121" : root[1][10]..[1]}. This may be super wastefulness of memory (unless use a pointer to the array).Note sure any better way.
I have control over what to send from server, so may augment above data structure. but need a quick way to get child tree based on node id in the client side.
EDIT:
I am considering keeping another data structure, map of {id, []ids}, where ids is the ordered path from root. Any better way?
Objects in javascript are true pointer-based objects, meaning that you can keep multiple references to them without using much more memory. Why not do a single traversal to assign the sub-objects to a new id-based parent object? Unless your hierarchical object is simply enormous, this should be very fast.
In light of best practice and what would happen if the application you're building were to scale to millions of users, you might rethink whether you really want the server to do more work. The client's computer is sitting there, ready to provide you with remote computing power for FREE. Why move the work load to the server causing it to process fewer client requests per second? That may not be a direction you want to go.
Here is a fiddle demonstrating this index-building technique. You run through it once, and use the index over and over as you please. It only takes 4 or 5 ms to build said index. There is no performance problem!
One more note: if you are concerned with bandwith, one simple way to help with that is trim down your JSON. Don't put quotes around object key names, use one-letter key names, and don't use whitespace and line breaks. That will get you a very large improvement. Performing this change to your example JSON, it goes from 11,792 characters to 5,770, only 49% of the original size!
One minor note is that object keys in javascript are always Strings. The numeric ids I added to your example JSON are coerced to strings when used as a key name. This should be no impediment to usage, but it is a subtle difference that you may want to be aware of.
I don't assume that the ids are somehow ordered, but still it might help to prune at least parts of the tree if you add to each node the information about the minimum and maximum id value of its children (and sub... children).
This can be quite easily be achieved at server side and when searching the tree you can check if the id you're looking for is within the id-range of a node, before stepping inside and searching all children.