In the Firebase documentation, it says:
Firebase stores all data as Objects, even Arrays are stored as objects with numerical keys.
As a convenience, the Firebase Web API automatically converts Array-like Objects into Arrays
for use JavaScript.
This "convenience" is a royal pain, in my opinion. Is there any way to disable the automatic conversion, while still using array-like objects with numerical keys? I am using the arrayjs library, and would like to maintain those array-like objects throughout the application.
It also says in the docs: "It's not currently possible to change or prevent this behavior." So no, you can't disable it.
You can work around it as follows:
add any non-numeric key to the path (e.g. "ignoreme": true), which will prevent any keys from being treated as numeric
prefix the item keys with a string (e.g. "rec1", "rec2"...)
make the numbers non-consecutive
But in reality, sequential numeric ids in distributed real-time data are ill advised and should be avoided for most cases. They cause nothing but heartache.
Related
If I have a JSON file that let's say represents a dictionary of words, where the word is the key and the value is the definition of that word. Would sorting and organizing the keys in alphabetical order in the JSON file make a difference to the performance when searching a word to find its definition (plus maybe other details about that word) in JS - that is if there were thousands of words or even more?
Or does JSON and Javascript already have an algorithm built in to find results optimally without the need to sort the data for better performance?
Also, I wouldn't mind having an alternative data structure or format or library that could give faster results for this kind of search problem suggested to me! (but of course, this suggestion isn't part of the main question)
There is no need to sort the keys in and json structure. The reading of a key/value will not be faster after sorting. See the discussion here
If you have very big structures, you can implement a TRIE by yourself. See: Wiki TRIE. Or see this: trie.js
Sorting the keys does not impact the search performance. Javascript objects may not remember the order of insertion
If possible, use Map object - The Map object holds key-value pairs and remembers the original insertion order of the keys. Any value (both objects and primitive values) may be used as either a key or a value
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
I’m new to data structure and I’m learning it in Javascript.
My Question is:
Why do we need hash tables when we 've objects in javascript?
Can anybody give me a situation where hash tables will be more useful than objects?
"Hashtable" is called different things in different languages. Java has Hashtable and HashMap, Ruby has Hash, Python has dict... in JavaScript, it's called Map.
Objects' keys are limited to strings; Map keys can be anything.
Objects support inheritance; a Map only contains what is specifically put into it.
Think you means Map instead of HashTable. IMHO Map may be more useful and perform better if you need one of that:
keep order of insertions of key/value pairs;
frequent additional and removal;
key which not String/Symbol.
I think you can obtain more information at MDN
The MDN docs on this are quite helpful: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map#Objects_and_maps_compared
Most notably, using a map gives you the advantage of using anything as a key, maps retain order, and may perform better when constantly adding and removing values.
As per the answer to the question, the order of properties on an object is not guaranteed. Then how does the sort function work flawlessly in MongoDB when we pass multiple sort fields as an object?
db.users.find({}).sort({firstName: 1, age:-1}).exec(callback);
How does mongodb know that it first has to sort by firstName and then by age? Is it working on a wrong assumption?
This is an interesting question.
It works as most ECMAScript implementations actually do preserve the key order (e.g. V8 which is used by Node). However, the official Node MongoDB driver does not encourage its use, instead it offers two different notations:
Sorting can be acieved with option parameter sort which takes an array
of sort preferences
{ "sort": [['field1','asc'], ['field2','desc']] }
With single ascending field the array can be replaced with the name of the field.
{ "sort": "name" }
The Mongoose documentation does not mention anything regarding key order (at least not that I know of). But it also offers two different approaches to sorting:
// sort by "field" ascending and "test" descending
query.sort({ field: 'asc', test: -1 });
// equivalent
query.sort('field -test');
Indeed it seems that the Mongoose sort functionality was based on wrong (but currently valid) assumption. It is probably best if you use the string notation when sorting with multiple keys.
Sorting with the object notation works properly when using MongoDB directly (e.g. in the MongoDB shell) as it does not use pure JSON but extended JSON.
Also see this very related question: How can you specify the order of properties in a javascript object for a MongoDB index in node.js?
There is also a Mongoose issue about this:
Yep, that's an unfortunate dilemma for both mongoose and the mongodb
driver. ECMAScript says that keys are not ordered, but they are
actually ordered in V8 (except for numeric keys, which is a nasty edge
case), and likely will continue to be ordered in V8 for the
foreseeable future. You can use the [['field1', 'asc']] syntax or the
mongoose-specific 'field1 -field2' syntax or the ES2015 Map class
(which guarantees insertion order for keys) if you're concerned with
key order.
So when using Mongoose, either use the string notation or the newly added support for ES2015 Maps which are guaranteed to be ordered by insertion order.
In your example you use indexes for sort operations in MongoDB and if you sort with multiple fields you create a compound index to support sorting on multiple fields. As you did when you give indexes, the query planner obtain the sort order from your indexes (your example's order: firstname, lastname) and it will sort by your order. In the other example it is a javascript object. Javascript doesn't warrant sorting.
I know in python I can use lists in order to make fast sortings and dictionaries in order to search things faster (because immutable objects can be hashed). Is that the same for javascript too? I haven't seen anything about the performance of datatypes in javascript after much search.
Yes. "Object vs Arrays in Javascript is as Python's Dictionaries vs Lists".
Performance pros and cons are also the same. With lists being more efficient if numeric indexes are appropriate to the task and dictionaries being more efficient for long lists that must be accessed by a string.
var dict = {};
dict['apple'] = "a sweet edible fruit";
dict['boy'] = "a young male human";
var list = [];
list.push("apples");
list.push("oranges");
list.push("pears");
I have been looking for some bibliography and other sources that could answer this question. I Know that this isn't the best answer, but let me try an answer that involve some concepts that lend us to discuss this topic.
Javascript and inheritance
Although the it could suggest that arrays and objects in javascript are like lists and dictionaries, they are different because each language are written in different ways, with different underlying philosophies, concepts and purposes.
In the case of Javascript, it seems that both Arryas and Objects are more like hash tables. Contrary to the intuition, Arrays are just an other type of built in object of javascript. In fact, as they say in the ECMAScript Specification 6.1.7
The Object Type
An Object is logically a collection of properties. Each property is
either a data property, or an accessor property:
A data property associates a key value with an ECMAScript language
value and a set of Boolean attributes. An accessor property associates
a key value with one or two accessor functions, and a set of Boolean
attributes. The accessor functions are used to store or retrieve an
ECMAScript language value that is associated with the property.
Properties are identified using key values. A property key value is
either an ECMAScript String value or a Symbol value. All String and
Symbol values, including the empty string, are valid as property keys.
A property name is a property key that is a String value.
An integer index is a String-valued property key that is a canonical
numeric String (see 7.1.16) and whose numeric value is either +0 or a
positive integer ≤ 253 - 1. An array index is an integer index whose
numeric value i is in the range +0 ≤ i < 232 - 1.
Property keys are used to access properties and their values. There
are two kinds of access for properties: get and set, corresponding to
value retrieval and assignment, respectively. The properties
accessible via get and set access includes both own properties that
are a direct part of an object and inherited properties which are
provided by another associated object via a property inheritance
relationship. Inherited properties may be either own or inherited
properties of the associated object. Each own property of an object
must each have a key value that is distinct from the key values of the
other own properties of that object.
And,about the arrays, it specifies:
22.1Array Objects
Array objects are exotic objects that give special treatment to a certain class of property names.
Following the logic above, and as it says in the specification, the language was thinked in such way that all types in javascript extends a global object, and then new methods and properties are added to have differents behaivors.
Memory Management
There are a gap between the language specifications and how they must be implemented in an actual runtime enviroment. Altought each implementation has its own logics, it seems that most of them has similarities.
As This Article Explains:
Most JavaScript interpreters use dictionary-like structures (hash
function based) to store the location of object property values in the
memory. This structure makes retrieving the value of a property in
JavaScript more computationally expensive than it would be in a
non-dynamic programming language like Java or C#. In Java, all of the
object properties are determined by a fixed object layout before
compilation and cannot be dynamically added or removed at runtime
(well, C# has the dynamic type which is another topic). As a result,
the values of properties (or pointers to those properties) can be
stored as a continuous buffer in the memory with a fixed-offset
between each. The length of an offset can easily be determined based
on the property type, whereas this is not possible in JavaScript where
a property type can change during runtime.
As this make javascript kind of ineffitient, the engineers had to came with some clever workarounds in order to solve this problem. Following this other article:
If you access a property, e.g. object.y, the JavaScript engine looks
in the JSObject for the key 'y', then loads the corresponding property
attributes, and finally returns the [[Value]].
But where are these property attributes stored in memory? Should we
store them as part of the JSObject? If we assume that we’ll be seeing
more objects with this shape later, then it’s wasteful to store the
full dictionary containing the property names and attributes on the
JSObject itself, as the property names are repeated for all objects
with the same shape. That’s a lot of duplication and unnecessarily
memory usage. As an optimization, engines store the Shape of the
object separately.
This Shape contains all the property names and the attributes, except
for their [[Value]]s. Instead the Shape contains the offset of the
values inside of the JSObject, so that the JavaScript engine knows
where to find the values. Every JSObject with this same shape points
to exactly this Shape instance. Now every JSObject only has to store
the values that are unique to this object.
The benefit becomes clear when we have multiple objects. No matter how
many objects there are, as long as they have the same shape, we only
have to store the shape and property information once!
All JavaScript engines use shapes as an optimization, but they don’t
all call them shapes:
Academic papers call them Hidden Classes (confusing w.r.t. JavaScript classes)
V8 calls them Maps (confusing w.r.t. JavaScript Maps)
Chakra calls them Types (confusing w.r.t. JavaScript’s dynamic types and typeof)
JavaScriptCore calls them Structures
*SpiderMonkey calls them Shapes
Python
Arrays
Python uses a different aproach for the implementation of lists, it seems that lists are more like some dynamics arrays than an actual array that you could find in C, But they are sill are focussed on saving spaces of time and complexity in a runtime. As this FAQ cited form the PyDocs says:
Python’s list objects are really variable-length arrays, not
Lisp-style linked lists. The implementation uses a contiguous array of
references to other objects, and keeps a pointer to this array and the
array’s length in a list head structure.
This makes indexing a list (L[i]) an operation whose cost is
independent of the size of the list or the value of the index.
When items are appended or inserted, the array of references is
resized. Some cleverness is applied to improve the performance of
appending items repeatedly; when the array must be grown, some extra
space is allocated so the next few times don’t require an actual
resize.
Like javascript, Python's lists are not required to be homogeneous, so they are not an actual implementation of other "strong typed" data structures that does have to contain only the same entities such as integers, strings, etc.
Same as javascript, the specifications of the language the actual implementation are two separate things. Depending on if you are using Cpython, Jython, IronPython, etc, the memory management and the actual functions that runs behind the scenes will be making diferent things in the process of interpreting python to machine code.
I know that this isnt the best source, but as I found discussed in Quora:
Contrary to what their name implies, Python lists are actually arrays(...).
Specifically, they are dynamic arrays with exponential
over-allocation, which allows code like the following to have linear
complexity:
lst = []
for i in xrange(0, 100000):
lst.append(i)
Alternative implementations like Jython and IronPython seem to use
whatever native dynamic array class their underlying language
(respectively Java and C#) provides, so they have the same performance
characteristics (the precise underlying classes seem to be ArrayList
for Jython and C# List for IronPython).
(...)arrays technically store pointers rather than the objects
themselves, which allows the array to contain only elements of a
specific size. Having pointers all over the place in the underlying
implementation is a common feature of dynamically typed languages, and
in fact of any language that tries to pretend it doesn't have
pointers.
Dictionaries
As the official docs puts in their "History and Design FAQ"
CPython’s dictionaries are implemented as resizable hash tables.
Compared to B-trees, this gives better performance for lookup (the
most common operation by far) under most circumstances, and the
implementation is simpler.
Dictionaries work by computing a hash code for each key stored in the
dictionary using the hash() built-in function. The hash code varies
widely depending on the key; for example, “Python” hashes to
-539294296 while “python”, a string that differs by a single bit, hashes to 1142331976. The hash code is then used to calculate a
location in an internal array where the value will be stored. Assuming
that you’re storing keys that all have different hash values, this
means that dictionaries take constant time – O(1), in computer science
notation – to retrieve a key. It also means that no sorted order of
the keys is maintained, and traversing the array as the .keys() and
.items() do will output the dictionary’s content in some arbitrary
jumbled order.
In Conclution
There are two separate things about a language: one involves how it should work, with it syntax, semantics, logic and philosophy. On the other hand you have the actual implementation of that language in a specific runtime, interpreter or compilation.
This way, although (in theory) you have one Python or one Javascript, you could have CPython, IronPython Jython, etc; and in the other hand, you have SpiderMonkey, V8, etc.
But referring to how each runtime implements the language features of Arrays/Lists and Objects/Dictionaries and how analogous they are, it seems that Javascript has chosen a inheritance model based on prototypes that makes everithing a kind of object; so both Objects and Dictionaries are more like a hash table than an actual array.
On the other hand, Python has a more flavores in respect of data structures, both in their libraries and in how the interpreters deal with them, making use of arrays or dynamic arrays to bring to life the Pyton's Lists, and using hash tables for the dictionaries, making them more similar to the objects in javascript.
I am using node.js as my server platform and I need to process a non sparse array of 65,000 items.
Javascript arrays are not true arrays, but actually hashes. Index access is accompagnied with conversion of the index to string and then doing a hash lookup. (see the Arrays section in http://www.crockford.com/javascript/survey.html).
So, my question is this. Does node.js implement a real array? The one that does cost us to resize or delete items, but with the true random access without any index-to-string-then-hash-lookup ?
Thanks.
EDIT
I may be asking for too much, but my array stores Javascript objects. Not numbers. And I cannot break it into many typed arrays, each holding number primitives or strings, because the objects have nested subobjects. Trying to use typed arrays will result in an unmaintainable code.
EDIT2
I must be missing something. Why does it have to be all or nothing? Either true Javascript with no true arrays or a C style extension with no Javascript benefits. Does having a true array of Javascript (untyped) objects contradicts the nature of Javascript in anyway? Java and C# have List<Object> which is essentially what I am looking for. C# even closer with List<DynamicObject>.
Node.js has the Javascript typed arrays: Int8Array, Uint8Array, Int16Array, Uint16Array, Int32Array, Uint32Array, Float32Array.
I think they are what you are asking for.
Node.js does offer a Buffer class that is probably what you're looking for:
A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.
Not intrinsically, no.
However depending on your level of expertise, you could write a "true" array extension using Node's C/C++ extension facility. See http://nodejs.org/api/addons.html
You want to use Low Level JavaScript (LLJS) to manipulate everything directly in C-style.
http://mbebenita.github.com/LLJS/
Notice that according to the link above, an LLJS array is more like the array you are looking for (true C-like array), rather than a Javascript array.
There is an implementation for LLJS in Node.js available , so maybe you do not have to write your own node.js C extension. Perhaps this implementation will do the trick: https://github.com/mbebenita/LLJS