Develop Reference

Allocation-free abstractions in Javascript

I have a general question which is about whether it is possible to make zero-allocation iterators in Javascript. Note that by "iterator" I am not married to the current definition of iterator in ECMAScript, but just a general pattern for iterating over user-defined ranges.
To make the problem concrete, say I have a list like [5, 5, 5, 2, 2, 1, 1, 1, 1] and I want to group adjacent repetitions together, and process it into a form which is more like [5, 3], [2, 2], [1, 4]. I then want to access each of these pairs inside a loop, something like "for each pair in grouped(array), do something with pair". Furthermore, I want to reuse this grouping algorithm in many places, and crucially, in some really hot inner loops (think millions of loops per second).
Question: Is there an iteration pattern to accomplish this which has zero overhead, as if I hand-wrote the loop myself?
Here are the things I've tried so far. Let's suppose for concreteness that I am trying to compute the sum of all pairs. (To be clear I am not looking for alternative ways of writing this code, I am looking for an abstraction pattern: the code is just here to provide a concrete example.)
Inlining the grouping code by hand. This method performs the best, but obscures the intent of the computation. Furthermore, inlining by hand is error-prone and annoying.
function sumPairs(array) {
let sum = 0
for (let i = 0; i != array.length; ) {
let elem = array[i++], count = 1
while (i < array.length && array[i] == elem) { i++; count++; }
// Here we can actually use the pair (elem, count)
sum += elem + count
}
return sum
}
Using a visitor pattern. We can write a reduceGroups function which will call a given visitor(acc, elem, count) for each pair (elem, count), similar to the usual Array.reduce method. With that our computation becomes somewhat clearer to read.
function sumPairsVisitor(array) {
return reduceGroups(array, (sofar, elem, count) => sofar + elem + count, 0)
}
Unfortunately, Firefox in particular still allocates when running this function, unless the closure definition is manually moved outside the function. Furthermore, we lose the ability to use control structures like break unless we complicate the interface a lot.
Writing a custom iterator. We can make a custom "iterator" (not an ES6 iterator) which exposes elem and count properties, an empty property indicating that there are no more pairs remaining, and a next() method which updates elem and count to the next pair. The consuming code looks like this:
function sumPairsIterator(array) {
let sum = 0
for (let iter = new GroupIter(array); !iter.empty; iter.next())
sum += iter.elem + iter.count
return sum
}
I find this code the easiest to read, and it seems to me that it should be the fastest method of abstraction. (In the best possible case, scalar replacement could completely collapse the iterator definition into the function. In the second best case, it should be clear that the iterator does not escape the for loop, so it can be stack-allocated). Unfortunately, both Chrome and Firefox seem to allocate here.
Of the approaches above, the custom-defined iterator performs quite well in most cases, except when you really need to put the pedal to the metal in a hot inner loop, at which point the GC pressure becomes apparent.
I would also be ok with a Javascript post-processor (the Google Closure Compiler perhaps?) which is able to accomplish this.

Check this out. I've not tested its performance but it should be good.
(+) (mostly) compatible to ES6 iterators.
(-) sacrificed ...GroupingIterator.from(arr) in order to not create a (imo. garbage) value-object. That's the mostly in the point above.
afaik, the primary use case for this is a for..of loop anyways.
(+) no objects created (GC)
(+) object pooling for the iterators; (again GC)
(+) compatible with controll-structures like break
class GroupingIterator {
/* object pooling */
static from(array) {
const instance = GroupingIterator._pool || new GroupingIterator();
GroupingIterator._pool = instance._pool;
instance._pool = null;
instance.array = array;
instance.done = false;
return instance;
}
static _pool = null;
_pool = null;
/* state and value / payload */
array = null;
element = null;
index = 0;
count = 0;
/* IteratorResult interface */
value = this;
done = true;
/* Iterator interface */
next() {
const array = this.array;
let index = this.index += this.count;
if (!array || index >= array.length) {
return this.return();
}
const element = this.element = array[index];
while (++index < array.length) {
if (array[index] !== element) break;
}
this.count = index - this.index;
return this;
}
return() {
this.done = true;
// cleanup
this.element = this.array = null;
this.count = this.index = 0;
// return iterator to pool
this._pool = GroupingIterator._pool;
return GroupingIterator._pool = this;
}
/* Iterable interface */
[Symbol.iterator]() {
return this;
}
}
var arr = [5, 5, 5, 2, 2, 1, 1, 1, 1];
for (const item of GroupingIterator.from(arr)) {
console.log("element", item.element, "index", item.index, "count", item.count);
}

Is an array.forEach with a splice the best way to remove an entry from an array of unique Ids?

I have an array of objects. Each object has a unique userTestrId. Here is the code that I am using when I want to delete one of the objects. Is this the most efficient way I can perform the delete? What I am concerned with is once a row has been deleted the code will still go all the way through the array even though there is no chance of another entry:
var id = 99;
self.tests.forEach(function (elem, index) {
if (elem['userTestId'] === id)
self.tests.splice(index, 1);
});
}

var id = 99;
self.tests.some(function (elem, index) {
if (elem['userTestId'] === id)
self.tests.splice(index, 1);
return true;
});
return false;
}
Could utilise Array.some? Stops looping once you return TRUE from a callback.

This is an alternative to #benhowdle89's answer.
Use Array.prototype.every
The .every method is used to iterate over an array and check whether each and every element passes a test or not. If the callback returns false for any single element, the loop breaks.
Take the following example:
var odds = [3, 5, 7, 9, 11, 12, 17, 19];
//an array with all odd numbers except one
var checkEven = function (n, i, arr) {
console.log ("Checking number ", n);
if (n%2===0) {
arr.splice(i, 1);
return false;
}
return true;
}
console.log(odds.every(checkEven), odds);
If you run the above and look at the console, the loop executed till number 12 only, where it spliced, and stopped.
You can employ similar logic in your code very easily :)

var id = 99;
self.tests.some(function (elem, index) {
if (elem['userTestId'] === id)
self.tests.splice(index, 1);
return true;
});
return false;
}
Polyfill :
some was added to the ECMA-262 standard in the 5th edition; as such it may not be present in all implementations of the standard. You can work around this by inserting the following code at the beginning of your scripts, allowing use of some in implementations which do not natively support it.
// Production steps of ECMA-262, Edition 5, 15.4.4.17
// Reference: http://es5.github.io/#x15.4.4.17
if (!Array.prototype.some) {
Array.prototype.some = function(fun /*, thisArg*/) {
'use strict';
if (this == null) {
throw new TypeError('Array.prototype.some called on null or undefined');
}
if (typeof fun !== 'function') {
throw new TypeError();
}
var t = Object(this);
var len = t.length >>> 0;
var thisArg = arguments.length >= 2 ? arguments[1] : void 0;
for (var i = 0; i < len; i++) {
if (i in t && fun.call(thisArg, t[i], i, t)) {
return true;
}
}
return false;
};
}
see in detail

While your concern is technically correct, it's unlikely to be an actual problem(Javascript is fast, this is a microoptimization).
What you should do is focus on using the appropriate interface, so your code could be easy to read and understand. .forEach() does not tell you what you want to do, unless you really do want to do something with each element of the array.
Lodash has the .remove() function, which removes all elements matching a predicate. Unfortunately, I couldn't find the exact specific function you wanted in JS's standard library or in lodash, so you would have to write your own wrapper:
var id = 99
removeFirst(tests, function (elem) { return elem.userTestId === id })
function removeFirst(array, callback) {
var index = array.findIndex(callback)
array.splice(index, 1)
}
Having noted that, you should avoid using an array at all - splicing is way more expensive than looping the whole array to begin with! Instead, since you have a unique identifier, you could use a map:
var map = {}
tests.forEach(function mapper(elem) {
map[elem.userTestId] = elem
})
Now, your removal function is simply delete map[id].

What are the Alternatives to eval in JavaScript?

I have a little bit of code that looks just like this:
function StrippedExample(i1, i2, i3, i4, i5, i6, i7, i8) {
this.i = [];
for (var i=1,j=0 ;i<9;i++) {
var k = eval("i"+i);
if (k > 0) {
this.i[j++] = k;
}
}
}
FireBug profiler claims that second longest function is eval(), taking up to nearly 6% of the run time.
Everyone says eval is EVIL (as in bad) and slow (as I have found), but I can't really do anything else - the server simply pulls the data out the database and pushes to the browser.
What alternatives do I have? I could do the same as I am doing here on the server but that just shifts the burden higher up the chain. I can't change the database layout since everything hooks into those 8 variables and is a massive undertaking.

function StrippedExample(i1, i2, i3, i4, i5, i6, i7, i8) {
var args = [i1, i2, i3, i4, i5, i6, i7, i8]; // put values in an array
this.i = [];
for (var i=0,j=0 ;i<8;i++) { // now i goes from 0-7 also
var k = args[i]; // get values out
if (k > 0) {
this.i[j++] = k;
}
}
}
The above code can be simplified further, I just made the minimal change to get rid of eval. You can get rid of j, for example:
function StrippedExample(i1, i2, i3, i4, i5, i6, i7, i8) {
var args = [i1, i2, i3, i4, i5, i6, i7, i8];
this.i = [];
for (var i = 0; i < args.length; i++) {
var k = args[i];
if (k > 0) { this.i.push(k); }
}
}
is equivalent. Or, to use the built-in arguments object (to avoid having your parameter list in two places):
function StrippedExample(i1, i2, i3, i4, i5, i6, i7, i8) {
this.i = [];
for (var i = 1; i < arguments.length; i++) {
var k = arguments[i];
if (k > 0) { this.i.push(k); }
}
}
Even if you weren't filtering the list, you don't want to do something like this.i = arguments because arguments is not a real Array; it has a callee property that you don't need and is missing some array methods that you might need in i. As others have pointed out, if you want to quickly convert the arguments object into an array, you can do so with this expression:
Array.prototype.slice.call(arguments)
You could use that instead of the var args = [i1, i2 ... lines above.

Eval alternative:
exp = '1 + 1'
x = Function('return ' + exp)()
console.log(x)

You are simply making an array from your function 8 arguments, removing the ones that are less than or equal to zero.
The following code is equivalent, and it will work for any arbitrary number of arguments:
function StrippedExample() {
var args = [];
for (var i = 0; i < arguments.length; i++) {
if (arguments[i] > 0) {
args.push(arguments[i]);
}
}
//...
}

Call the function with one argument — an Array
Use the arguments object

One alternative to to pass an array to your function, instead of individual arguments:
StrippedExample([3, 1, 4, 1, 5, 9, 2, 6])
Then your code would be:
function StrippedExample(inArray) {
this.i = [];
for (var i=0,j=0 ;i<inArray.length;i++) {
var k = inArray[i];
if (k > 0) {
this.i[j++] = k;
}
}
}
If you really need to pass in separate arguments, you can access them using your arguments array, which is an object that acts like an array (though it's not really; not all Array methods work on it) that exposes all arguments that have been passed in to your function; they do not even need to be declared in this case, but it's good form to include a comment indicating what sorts of arguments you are expecting for users of your code:
function StrippedExample(/*i1, i2, i3, i4, i5, i6, i7, i8*/) {
this.i = [];
for (var i=0,j=0 ;i<arguments.length;i++) {
var k = arguments[i];
if (k > 0) {
this.i[j++] = k;
}
}
}
If you're guaranteed to only have 8 elements, then you could use 8 in place of inArray.length or arguments.length; I decided to use the more general version in my examples in case that was helpful to you.

This code should be made to use the arguments array that every Javascript function has access to.
It's not that eval is evil (it's in Lisp, so it must be good) it's simply a sign of a hack - you need something to work and you forced it. It screams out to me "The author gave up on good programming design and just found something that worked".

function StrippedExample() {
this.i = [];
for (var i=1,j=0 ;i<arguments.length;i++) {
var k = arguments[i];
if (k > 0) {
this.i[j++] = k;
}
}
}

Short answer:
StrippedExample=(...a)=>a.filter(i=>i>0);
no need use eval for work with arguments at all.
Initial code and most proposed solutions doesn't return result by traditional way.

Given that there is a fixed amount of variables, you can build an array of them manually and loop through it. But if you have a variable amount of arguments, one way to get the variables passed to the function as an array is:
var args = Array.prototype.slice.call(arguments.callee.caller.arguments);
And your function would look like this:
function StrippedExample() {
var args = Array.prototype.slice.call(arguments.callee.caller.arguments);
for(var i in args) {
if (args[i] > 0) {
this.i[j++] = args[i];
}
}
}

You can also run string expressions with setTimeout. It works the same as the Function object.
let a=10;
window.a=100;
window.b=1;
setTimeout("let c=1000;console.log(a,b,c)");//10,1,1000

definitely try this as a drop in replacement.
I was looking for this answer and came to this post.
I read the developer doc and this worked as a direct replacement in my application
var func1 = "stringtxt" + object + "stringtxt";
instead of ---> eval(func1); --> use --> Function(func1)();
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#Never_use_eval!

How do I add syntactic sugar in my Javascript library?

Right now the library can translate this operation
Select * from List where name = k% order by desc
to
List.filter(function(x) { return x.first_char() == 'k' }).sort().reverse());
Whats the best hack to remove the () so that the developer can write statements like:
List.filter(fn(x) { return x.first_char == 'k' }).sort.reverse;
Naive approach:
maxfn = function() {this[0]..}; Array.prototype.max = maxfn();
But with this approach I can't access 'this'.
I wanted to add a syntactic sugar for
new Array("1","2","3")
to something like :)(suggestions needed)
_("1","2" ,"3")
like we have in scheme where list -> '
I tried to clone the arguments but failed.
Thanks.

For lists you can use JSON notation:
["1", "2", "3"]

You can use JSON notation as suggested by RoBorg, if you control the list... However, there's no cross-browser way to treat a property as a method. Note: spidermonkey (firefox) does support using a getter (get method for a property).

Whats the best hack to remove the ()
Property getters/setters in JavaScript. Unfortunately it's a relatively new JavaScript feature that won't work on IE6/7 (as well as various other older browsers), so it's not really ready for prime-time yet (despite the intro of the linked article).
You could do this particular example by making a JavaScript object that wrapped a String and shadowed all String's methods, then add a static ‘first_char’ property set to the String's first character on initialisation. But it's really not worth it.
new Array("1","2","3")
to something like :)(suggestions needed)
_("1","2" ,"3")
Well that's simple enough:
function _(/* items */) {
var a= new Array();
for (var i= 0; i<arguments.length; i++)
a[i]= arguments[i];
return a;
}
There's no point in doing it nowadays, though, since the array literal syntax:
['1', '2', '3']
has been available since JavaScript 1.1-1.2 era and is available in every browser today. (It predates JSON by many, many years.)

I'll try to answer one by one:
1) Why would you want to remove parenthesis from a functon call?
2) If the "naive" approach is failing it's probably because you are calling the maxFn and assigning the results to Array.prototype.max. It should be like this:
maxfn = function() {this[0]..}; Array.prototype.max = maxfn;
3) RoBorg is correct, just use literal notation to construct arrays on the fly.
Edit:
Here's one way of implementing a max function on an array object. The optional evaluator argument is a function that takes two parameters, the current max value and current value in array. It should return the object that is "greater". Useful for non-primitives.
Array.prototype.max = function(evaluator) {
var max, i = 1; len = this.length;
if (len > 0) max = this[0];
for (; i < len; i++) {
if (evaluator) {
max = evaluator(max, this[i]);
}
else if(max < this[i]) {
max = this[i];
}
}
return max;
};
var a = [1, 3, 4, 5, 6];
alert(a.max());
var b = ["Arnold", "Billy", "Caesar"];
alert(b.max());
var c = ["Arnold", new Date(), 99, true];
alert(c.max());
var d = [1, 3, 4, 5, 6];
alert(d.max(function (max, val) { return max < val ? val : max }));

Is there any good JavaScript hash(code/table) implementation out there?

Yes, I know you could use regular objects as associative arrays in JavaScript, but I'd like to use something closer to java's Map's implementation (HashMap, LinkedHashMap etc). Something that could have any kind of data used as key. Are there any good hash(code/table) in JavaScript implementation out there?

In javascript, objects are literally a hash implementation. A Java HashMap will be a little bit of a fake-out, so I'd challenge you to re-think your needs.
The straight answer is no, I don't believe that there is a great implementation of Java's HashMap in javascript. If there is, it's bound to be part of a library that you may or may not want to use, and you certainly don't need to include a library just to have a little hash table.
So let's go ahead and write one, just to examine the problem. You can use it if you like. We'll just start by writing a constructor, and we'll piggyback off of Array, which is Object, but has some useful methods that will keep this example from getting too tedious:
function HashMap () {
var obj = [];
return obj;
}
var myHashMap = HashMap();
We'll add some methods straight from the world of Java, but translate into javascript as we go...
function HashMap() {
var obj = [];
obj.size = function () {
return this.length;
};
obj.isEmpty = function () {
return this.length === 0;
};
obj.containsKey = function (key) {
for (var i = 0; i < this.length; i++) {
if (this[i].key === key) {
return i;
}
}
return -1;
};
obj.get = function (key) {
var index = this.containsKey(key);
if (index > -1) {
return this[index].value;
}
};
obj.put = function (key, value) {
if (this.containsKey(key) !== -1) {
return this.get(key);
}
this.push({'key': key, 'value': value});
};
obj.clear = function () {
this = null; // Just kidding...
};
return obj;
}
We could continue to build it out, but I think it's the wrong approach. At the end of the day, we end up using what javascript provides behind the scenes, because we just simply don't have the HashMap type. In the process of pretending, it lends itself to all kinds of extra work.
It's a bit ironic that one of the things that makes javascript such an interesting and diverse language is the ease with which it handles this kind of wrestling. We can literally do anything we'd like, and the quick example here does nothing if it doesn't illustrate the deceptive power of the language. Yet given that power, it seems best not to use it.
I just think javascript wants to be lighter. My personal recommendation is that you re-examine the problem before you try implement a proper Java HashMap. Javascript neither wants nor affords for one.
Remember the native alternative:
var map = [{}, 'string', 4, {}];
..so fast and easy by comparison.
On the other hand, I don't believe that there are any hard-and-fast answers here. This implementation really may be a perfectly acceptable solution. If you feel you can use it, I'd say give it a whirl. But I'd never use it if I felt that we have reasonably simpler and more natural means at our disposal.. which I'm almost certain that we do.
Sidenote:
Is efficiency related to style? Notice the performance hit.. there's a big O staring us in the face at HashMap.put()... The less-than-optimal performance probably isn't a show-stopper here, and you'd probably need to be doing something very ambitious or have a large set of data before you'd even notice a performance hickup a modern browser. It's just interesting to note that operations tend to become less efficient when you're working against the grain, almost as if there is a natural entropy at work. Javascript is a high level language, and should offer efficient solutions when we keep in line with its conventions, just as a HashMap in Java will be a much more natural and high performing choice.

I have released a standalone JavaScript hash table implementation that goes further than those listed here.
http://www.timdown.co.uk/jshashtable/

Note that java collections using "any kind of object" as a key isn't quite right. Yes, you can use any object, but unless that object has good hashcode() and equals() implementations then it won't work well. The base Object class has a default implementation for these, but for custom classes to work (effectively) as hashtable keys you need to override them. Javascript has no equivalent (that I know of).
To create a hashtable in javascript that can (effectively) use arbitrary objects as the key you'd need to enforce something similar on the objects you use, at least if you want to keep the performance gains of a hashtable. If you can enforce a 'hashcode()' method that returns a String, then you can just use an Object under the hood as the actual hashtable.
Otherwise, you'd need to so something like the other solutions posted, which as of right now do not perform like hashtables. They both do O(n) searches over the list to try and find the key, which pretty much defeats the purpose of a hashtable (hashtables are generally constant time for get/put).

Here's a naive implementation I just put together - as keparo mentioned in a comment, one of the big issues is equality checking:
var ObjectMap = function()
{
this._keys = [];
this._values = [];
};
ObjectMap.prototype.clear = function()
{
this._keys = [];
this._values = [];
};
ObjectMap.prototype.get = function(key)
{
var index = this._indexOf(key, this._keys);
if (index != -1)
{
return this._values[index];
}
return undefined;
};
ObjectMap.prototype.hasKey = function(key)
{
return (this._indexOf(key, this._keys) != -1);
};
ObjectMap.prototype.hasValue = function(value)
{
return (this._indexOf(value, this._values) != -1);
};
ObjectMap.prototype.put = function(key, value)
{
var index = this._indexOf(key, this._keys);
if (index == -1)
{
index = this._keys.length;
}
this._keys[index] = key;
this._values[index] = value;
};
ObjectMap.prototype.remove = function(key)
{
var index = this._indexOf(key, this._keys);
if (index != -1)
{
this._keys.splice(index, 1);
this._values.splice(index, 1);
}
};
ObjectMap.prototype.size = function()
{
return this._keys.length;
};
ObjectMap.prototype._indexOf = function(item, list)
{
for (var i = 0, l = list.length; i < l; i++)
{
if (this._equals(list[i], item))
{
return i;
}
}
return -1;
};
ObjectMap.prototype._equals = function(a, b)
{
if (a === b)
{
return true;
}
// Custom objects can implement an equals method
if (typeof a.equals == "function" &&
typeof b.equals == "function")
{
return a.equals(b);
}
// Arrays are equal if they're the same length and their contents are equal
if (a instanceof Array && b instanceof Array)
{
if (a.length != b.length)
{
return false;
}
for (var i = 0, l = a.length; i < l; i++)
{
if (!this._equals(a[i], b[i]))
{
return false;
}
}
return true;
}
// Checking object properties - objects are equal if they have all the same
// properties and they're all equal.
var seenProperties = {};
for (var prop in a)
{
if (a.hasOwnProperty(prop))
{
if (!b.hasOwnProperty(prop))
{
return false;
}
if (!this._equals(a[prop], b[prop]))
{
return false;
}
seenProperties[prop] = true;
}
}
for (var prop in b)
{
if (!(prop in seenProperties) && b.hasOwnProperty(prop))
{
if (!a.hasOwnProperty(prop))
{
return false;
}
if (!this._equals(b[prop], a[prop]))
{
return false;
}
}
}
return true;
};
Example usage:
>>> var map = new ObjectMap();
>>> var o = {a: 1, b: [1,2], c: true};
>>> map.put(o, "buns");
>>> map.get(o)
"buns"
>>> map.get({a: 1, b: [1,2], c: true});
"buns"
>>> map.get({a: 1, b: [1,2], c: true, d:"hi"});
>>> var a = [1,2,3];
>>> map.put(a, "cheese");
>>> map.get(a);
"cheese"
>>> map.get([1,2,3]);
"cheese"
>>> map.get([1,2,3,4]);
>>> var d = new Date();
>>> map.put(d, "toast");
>>> map.get(d);
"toast"
>>> map.get(new Date(d.valueOf()));
"toast"
This is in no way a complete implementation, just a pointer for a way to implement such an object. For example, looking at what I've given, you would also need to add constructor property checks before the object property check, as this currently works:
>>> function TestObject(a) { this.a = a; };
>>> var t = new TestObject("sandwich");
>>> map.put(t, "butter");
>>> map.get({a: "sandwich"})
"butter"