Does node.js provide a real array implementation? - javascript

I am using node.js as my server platform and I need to process a non sparse array of 65,000 items.
Javascript arrays are not true arrays, but actually hashes. Index access is accompagnied with conversion of the index to string and then doing a hash lookup. (see the Arrays section in http://www.crockford.com/javascript/survey.html).
So, my question is this. Does node.js implement a real array? The one that does cost us to resize or delete items, but with the true random access without any index-to-string-then-hash-lookup ?
Thanks.
EDIT
I may be asking for too much, but my array stores Javascript objects. Not numbers. And I cannot break it into many typed arrays, each holding number primitives or strings, because the objects have nested subobjects. Trying to use typed arrays will result in an unmaintainable code.
EDIT2
I must be missing something. Why does it have to be all or nothing? Either true Javascript with no true arrays or a C style extension with no Javascript benefits. Does having a true array of Javascript (untyped) objects contradicts the nature of Javascript in anyway? Java and C# have List<Object> which is essentially what I am looking for. C# even closer with List<DynamicObject>.

Node.js has the Javascript typed arrays: Int8Array, Uint8Array, Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array.
I think they are what you are asking for.

Node.js does offer a Buffer class that is probably what you're looking for:
A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.

Not intrinsically, no.
However depending on your level of expertise, you could write a "true" array extension using Node's C/C++ extension facility. See http://nodejs.org/api/addons.html

You want to use Low Level JavaScript (LLJS) to manipulate everything directly in C-style.
http://mbebenita.github.com/LLJS/
Notice that according to the link above, an LLJS array is more like the array you are looking for (true C-like array), rather than a Javascript array.
There is an implementation for LLJS in Node.js available , so maybe you do not have to write your own node.js C extension. Perhaps this implementation will do the trick: https://github.com/mbebenita/LLJS

Related

Is an array in JavaScript a pointers array? [duplicate]

This question already has answers here:
Are JavaScript Arrays actually implemented as arrays?
(2 answers)
How are JavaScript arrays implemented?
(8 answers)
Closed 2 years ago.
I am new to JavaScript and lately, I found out that arrays in JavaScript are like lists in Java and that they can contain different types of variables.
My question is if in JavaScript an array are made of pointers? How is it possible to have different types in the same array, because we must define the array size before we assign the variables?
I have tried to find some information on Google, but all I have found are examples on arrays ):
You do not have to define the array size before you assign the variables. You can go like:
let array = [];
array.push(12);
array.push("asd");
array.push({data:5});
array.forEach(element => {
console.log(element);
});
Also I think you should not think about pointers with such a high level language. The better way is to look at variables like 'primitives' and 'objects'. Here is a good read about it:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures
High level languages, and in particular scripting languages, tend to reference most things with pointers, and they make pointer access transparent. Javascript does this also. Most everything, even primitives like numbers and strings, are objects. Objects in javascript have properties that store things. Those properties are essentially pointers, in that they are references to other objects. Arrays are implemented in the same way, and are in fact objects with numeric properties (and a few utility methods a standard object doesn't have, such as .length, .push(), .map(), etc.). Arrays don't hav a fixed size anymore than objects do. So everything in javascript is stored in these object "buckets" that can store anything in their properties (although you can seal objects, like numbers and strings, so that they don't accidentally change).
Languages with fixed data types (C like languages for instance) implement things with fixed data structures, and the exact size is easily calculable and known. When you declare a variable, the compiler uses the type of that variable to reserve some space in memory. Javascript handles all that for you and doesn't assume anything is a fixed size, because it can't. The size of javascript objects can change at any time.
In C-Like languages, when you ask for an array, you are asking for a block of a specific size. The compiler needs to know how big that is so that it can determine where in memory to put everything, and it can use the type of objects in the array to easily calculate that. Interpreted languages use pointers behind the scenes to keep track of where everything is stored, because they can't assume it will always be in the same place, like a compiled program can. (This is somewhat of a simplification and there are caveats to this of course).
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
JavaScript is a loosely typed language, Therefore there is noting stoping you from having different types in javascript array. but I would strongly avoid structuring your data that way without static type-checking (Typescript)
const test = ['test', {test:'test'}, 1, true]

Does v8 still have C-style arrays that are contiguous blocks of memory, and how can I make sure I am using them?

There seems to be a conventional wisdom that arrays are represented as hashmaps from indices to values in v8. The only source I found that states otherwise is this:
https://www.youtube.com/watch?feature=player_detailpage&v=XAqIpGU8ZZk#t=994s
Seems authoritative, however, it dates back to 2012. A lot could have changed since.
Is it still true that
var a1 = Array(1000) is a contiguous array under the hood (unless you exceed array's boundaries) and var a2 = [] is not?
V8 will use true arrays if it can. For instance, if you fill the array in a contiguous way, don't use delete on it, etc. Basically, if you use it as though it were a true array (but one that magically grows for you), V8 is likely to be able to keep using a true array under the covers.
If your data is a fit for one of the typed arrays (Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array, or Float64Array), you can use them to ensure you're dealing with a true array.
Re the comment you added under the question: I don't have a specific reference I can cite for the above. The V8 source code is, of course, available on the V8 site, but digging through it for all places where arrays might fall back to dictionary behavior would probably be more work than you (or I) are going to want to do. :-)

What kind of list are arrays in JavaScript?

Probably a silly question, but I was wondering what implementation is behind arrays in JavaScript? Are they SLL (Singly Linked List) or DLL (Doubly Linked List) or something different?
You can't really tell, it can vary between implementations (V8, Rhino, etc.), it depends on what the current compiler's developer preferred.
Depends on the browser. Why does this matter?
Could be just a hash table.
And yes - it was a silly question
Taken from V8 object.h:
// The JSArray describes JavaScript Arrays
// Such an array can be in one of two modes:
// - fast, backing storage is a FixedArray and length <= elements.length();
// Please note: push and pop can be used to grow and shrink the array.
// - slow, backing storage is a HashTable with numbers as keys.
class JSArray: public JSObject {
So in V8, it can be in the best case an C++ array allocated in the heap or a HashTable implementation, it depends on the interpreter and its optimization processes.
Javascript array is just simply Object.
Arrays are list-like objects whose prototype has methods to perform traversal and mutation operations.
From MDN - array
Edit
As the comment of #Alnitak, the above is nothing about implementation. The implementation of Object in JS depends on the Javascript engine.

What is the best way to compile JavaScript-like structures to static, fast C++?

On the development of a compiler from a language very similar to JavaScript to C++, I need a way to represent data structures. JavaScript's main data structures are Arrays and Hash-Tables. Arrays are more straighforward: I can use a vector of untyped pointers. It needs to be a vector because JS arrays are dynamic, and of pointers because JS arrays can hold any kind of object, for example:
var array = [1,2,[3,4],"test"];
I can't see a way to represent this other than that (is there?). For the hashes, I could use something similar, except including the string hashing step on access.
The problem is: JavaScript hashes are JIT-compiled into actual C++ objects which probably are much faster than hashes. This way, I'm afraid my attempt to generate C++ like that will actually result in slower code than the JavaScript version!
Does that make sense?
What would be the best approach to my compiler?
If this is an AOT compiler you can only process the hash keys that you see at compile-time, obviously. In this case you can change hash accesses to known keys to array accesses, giving each known key a small integer as index.

Quickest way of checking JSON equivalence

I have two web services generating JSON output. Both services are using different technologies but are supposed to generate the exact same output.
I want to check if this output is exactly the same in the browser environment. I would really prefer to simple compare them as string, but the JSON output is not sorted.
I can convert JSON string to objects and then iterate over their keys to check the equivalence but that is basically a n^2 algorithm.
I was wondering if there is any quicker or better way. Something that browser environment already provides.
At the moment there is no such method provided by the browsers to do a deep comparison. WE have to do it ourselves or using some non native library.

Categories

Resources