My docs contain lists of errors.
I have a view which counts the number of docs with a specific error, and an average of how many errors a doc with a given error has.
Map
function (doc) {
var i;
for (i = 0; i < doc.errors.length; i = i + 1){
emit([doc.errors[i], doc.flummery], [1, doc.errors.length])
}
}
Reduce
function (key, values, rereduce) {
var avg = [];
var cnt = [];
var i;
for (i = 0; i < values.length; i = i + 1) {
avg.push(values[i][1]);
cnt.push(values[i][0]);
}
return [sum(cnt), eval(avg.join("+")) / avg.length]
}
I have read many times that the use of eval() is the path to madness, but I can't think of any way that it could really trip me up here. Is eval() safe in couchdb views?
Note that Oleg came up with the obvious answer, which is "don't do that, couchdb gives you a sum() function". It is obvious in this situation that eval() is not needed. However, should I ever require it, is it safe to use given that I'll have good control over my inputs?
Can't you use sum(avg) / avg.length instead?
eval have obvious drawback of calling full power of compiler: it is very resource heavy and introduces a big security hole unless you carefully sanitize your input.
Related
This question already has answers here:
Nested for-loop only executing once
(2 answers)
Closed 2 years ago.
I am unable to figure out on why my loop is running only once.
I am sitting here for like 2 hours to figure it out, but no luck.
please help
var tempSheetValues1 = currentSheet.getRange("B2:B" + lastRow).getValues();
// considering length to be 5
for (var i = 0; i < 5; i++) {
var id = tempSheetValues1.slice(i, i + 1).join('');
var url = 'https://api.demo.hitmebro/' + id + '?language=en';
Logger.log(url);
// Check if the test id exists
var response = hitApi(url, "get", null, headers, "inputId");
Logger.log(response);
var json = response.getContentText();
var data = JSON.parse(json);
var Collection = data.data.inside;
var rows = [];
data;
for (i = 0; i < Collection.length; i++) {
data = Collection[i];
var more = rows.push([data.xid]);
}
dataRange = currentSheet.getRange(2, 1, rows.length, 1);
dataRange.setValues(rows);
}
Preamble
Since comments to the question deal with the issue, I added several optimization points to consider, paying attention to them will benefit you in the long run.
Suggestions
Move to the new V8 runtime to make use of the latest ES6 features and syntax:
Make use of built-in Array methods, like map, reduce, filter, etc. - it improves readability, reduces probability of bugs creeping in (because you forgot to change i to j - exactly your case).
Use destructuring assignment - again, it improves readability and, especially if combined with TypeScript (or JSDoc) is a joy to use (+improved type safety).
Use const as much as you can (unless you really need to reassign - and use let then, not var), most certainly never use undeclared variables (I assume dataRange is defined somewhere prior to the code sample, but just in case...).
[edit] it might sound counter-intuitive, but variables aren't meant to be rewritten, the name refers to the fact that they can hold any value. Thus, please, avoid resetting (like let data = 'some_val'; data; data = 'new_val') - it leads to messy and / or buggy code. Instead, decalre a new one or try to avoid it at all (see below).
Optimized code
This is not the most optimal version there is, but is definitely a start (e.g., you can take the getRange() out of the loop to improve execution speed on large collections drastically, issue concurrent requests to the API with Promise.all() - thank V8 - etc.)
const tempSheetValues1 = currentSheet.getRange("B2:B" + lastRow).getValues();
for (let i = 0; i < 5; i++) {
const id = tempSheetValues1.slice(i, i + 1).join('');
const url = `https://api.demo.hitmebro/${id}?language=en`;
Logger.log(url);
// Check if the test id exists
const response = hitApi(url, "get", null, headers, "inputId");
Logger.log(response);
const json = response.getContentText();
const parsed = JSON.parse(json);
const { data } = parsed;
const { inside } = data;
const rows = inside.map(elem => [elem.xid]);
const dataRange = currentSheet.getRange(2, 1, rows.length, 1);
dataRange.setValues(rows);
}
Notes
Hopefully I didn't miss anything when rewriting the inner loop logic, feel free to correct me if something's wrong.
References
Variable declaration guide
Array built-in methods reference on MDN
Migrating to V8 runtime guide
Destructuring assignment guide on MDN
I'm trying to find out if I should temporally copy an array item while I work with it inside a for-loop.
Is there any performance difference ? Other than making the code more readable.
(JavaScript)
var max = someArray.length;
for (var i = 0; i < max; i++) {
// Here I will use someArray[i] quite often. Like this:
if (someArray[i].position.y > blabla) {
//...
}
}
OR:
var max = someArray.length;
for (var i = 0; i < max; i++) {
var tmp = someArray[i]; // <---- CHANGE
// Here I will use tmp quite often. Like this:
if (tmp.position.y > blabla) {
//...
}
}
Caveat: Worry about performance when you have a specific performance problem to worry about. Until then, write whatever seems clearest and least error prone to you and your team. 99.999999% of the time, the specific performance of a given loop just won't matter in real world terms.
With that said:
With modern JavaScript, I'd probably use a for-of loop (ES2015+) instead:
for (const entry of someArray) {
if (entry.position.y > blabla) {
// ...
}
}
In theory, that uses an iterator behind the scenes, which involves function calls; in practice, it's likely that if you're dealing with an actual array and that array uses the standard iterator, the JavaScript engine will be able to optimize the loop if the loop is a hot spot (and if it isn't, it's not worth worrying about).
Re your two alternatives, if i and someArray don't change in the loop and someArray is just a normal array, the JavaScript engine is likely to be able to optimize it into something like your second loop. As a matter of style, before for-of I always used a local within the loop rather than retyping someArray[i] each time, but that's just because it's easier to type tmp (or some more meaningful name) than someArray[i].
If there's a reason to believe that a specific loop is being slow (so for some reason, it's not getting optimized), then I might go with something like your second example:
for (let i = 0, max = someArray.length; i < max; i++) {
const tmp = someArray[i];
if (tmp.position.y > blabla) {
//...
}
}
But again, this is mostly a matter of style until/unless you have a specific performance problem you're diagnosing and fixing.
According to the fundamentals of CS
the search functionality of an unsorted list has to occur in O(n) time where as direct access into an array will occur in O(1) time for HashMaps.
So is it more performant to map an array into a dictionary and then access the element directly or should I just use includes? This question is specifically for JavaScript because I believe this would come down to core implementation details of how includes() and {} is implemented.
let y = [1,2,3,4,5]
y.includes(3)
or...
let y = {
1: true,
2: true
3: true
4: true
5: true
}
5 in y
It's true that object lookup occurs in constant time - O(1) - so using object properties instead of an array is one option, but if you're just trying to check whether a value is included in a collection, it would be more appropriate to use a Set, which is a (generally unordered) collection of values, which can also be looked up in linear time. (Using a plain object instead would require you to have values in addition to your keys, which you don't care about - so, use a Set instead.)
const set = new Set(['foo', 'bar']);
console.log(set.has('foo'));
console.log(set.has('baz'));
This will be useful when you have to look up multiple values for the same Set. But, adding items to the Set (just like adding properties to an object) is O(N), so if you're just going to look up a single value, once, there's no benefit to this nor the object technique, and you may as well just use an array includes test.
Updated 04/29/2020
As the commenter rightly pointed out it would seem V8 was optimizing out the array includes calls. An updated version that assigns to a var and uses it produces more expected results. In that case Object address is fastest, followed by Set has and in a distant third is Array includes (on my system / browser).
All the same, I do stand by my original point, that if making micro-optimizations it is worth testing assumptions. Just make sure your tests are valid ;)
Original
Well. Despite the obvious expectation that Object address and Set has should outperform Array includes, benchmarks against Chrome indicate that implementation trumps expectation.
In the benches I ran against Chrome Array includes was far and away the best performer.
I also tested locally with Node and got more expected results. In that Object address wins, followed closely by Set has, then Array includes was marginally slower than both.
Bottom line is, if you're making micro-optimizations (not recommending that) it's worth benchmarking rather than assuming which might be best for your particular case. Ultimately it comes down to the implementation, as your question implies. So optimizing for the target platform is key.
Here's the results I got:
Node (12.6.0):
ops for Object address 7804199
ops for Array includes 5200197
ops for Set has 7178483
Chrome (75.0):
https://jsbench.me/myjyq4ixs1/1
This isn't necessarily a direct answer to the question but here is a related performance test I ran real quick in my chrome dev tools
function getRandomInt(max) {
return Math.floor(Math.random() * max);
}
var arr = [1,2,3];
var t = performance.now();
for (var i = 0; i < 100000; i++) {
var x = arr.includes(getRandomInt(3));
}
console.log(performance.now() - t);
var t = performance.now();
for (var i = 0; i < 100000; i++) {
var n = getRandomInt(3);
var x = n == 1 || n == 2 || n == 3;
}
console.log(performance.now() - t);
VM44:9 9.100000001490116
VM44:16 5.699999995529652
I find the array includes syntax nice to look at, so I wanted to know if the performance was likely to be an issue the way I use it, for checking if a variable is one of a set of enums for instance. It doesn't seem to be much of an impact for situations like this with a short list. Then I ran this.
function getRandomInt(max) {
return Math.floor(Math.random() * max);
}
var t = performance.now();
for (var i = 0; i < 100000; i++) {
var x = [1,2,3].includes(getRandomInt(3));
}
console.log(performance.now() - t);
var t = performance.now();
for (var i = 0; i < 100000; i++) {
var n = getRandomInt(3);
var x = n == 1 || n == 2 || n == 3;
}
console.log(performance.now() - t);
VM83:8 12.600000001490116
VM83:15 4.399999998509884
and so the way I actually use it and like lookin at it is quite worse with performance, despite still not being very significant unless run a few million times, so using it inside of an Array.filter that may run a lot as a react redux selector may not be a great idea like I was about to do when I decided to test this.
It seems that functional iterators are replacing the use of for loops in JS.
What is the advantage of passing a function such as map or reduce compared to a for/while loop?
var numbers = [1, 4, 9];
var doubles = numbers.map(function(num) {
return num * 2;
});
var doubles = [];
for (i = 0; i < numbers.length; i++) {
doubles[i] = numbers[i] * 2;
}
I have no idea why you would call the use of map a "closure". Closures are something else entirely. map is a higher-order function--defined as a function which operates on (takes or returns) a function. This style of programming can loosely be called "functional".
There are advantages and disadvantages to using functions like map. As one commenter pointed out, it's more compact:
function sum(array) {
var sum = 0;
for (var i = 0; i < array.length; i++) sum += array[i];
return sum;
}
vs.
function sum(array) {
return array.reduce(add);
}
Where add is function add(a, b) { return a + b; }.
More compact means more readable and less surface area for bugs. The use of the function named add also enhances readability; we can easily intuit that the operation is to add the elements of the array.
Basically, all the array functions have for-loop equivalents which require setting up more variables and writing more logic. For instance, map is
function map(array, fn) {
var result = [];
for (var i = 0; i < array.length; i++) result.push(fn(array[i]));
return result;
}
This can be written (much) more compactly as array.map(fn).
In many cases, we might already have functions defined which do the element mapping, or element filtering we want to do. In that case, we can simply use the functions as is with map, reduce etc.
map and its friends also have the advantage that they are friendly to sparse arrays. For instance:
var a = [];
a[1000000] = 1;
Now we double each element:
function double(array) {
var result = [];
for (var i = 0; i < array.length; i++) result.push(array[i] * 2);
return result;
}
This loops one million times and returns an array filled with NaNs. In contrast
array.map(elt => elt*2)
operates only on the single element which is present at position 1000000 and returns a sparse array as one would like.
The functional style also opens up additional possibilities for flexibility. Let's say we want to generalize the idea of multiplying things. I can write a higher-order function to create a function which multiplies some value by a particular factor:
function multiply(n) {
return function(x) {
return n * x;
};
}
Now I can write
array.map(multiply(2))
This level of conciseness and expressiveness is going to be hard to achieve in a for-loop solution.
forEach and map etc. are likely to be slower than a for loop. This could be an issue if your code is running a million times in a tight loop. In the real world, it rarely is an issue. It is better to prioritize code readability and compactness.
However, no one is forcing you to use map or filter. In ES7 or whatever it will be called, you will be able to use array comprehensions to accomplish the same thing in even more readable fashion:
[ for (i of array) if (i % 2) i + 1 ]
which combines a filter and a map.
A bit further afield, if you are planning to write a generator which iterates over an array, and yield some calculation from each element, you are going to need to use a for loop, since there is no way to yield from within a forEach callback:
function *double(array) {
for (var i = 0; i < array.length; i++) yield array[i]*2;
}
function *double(array) {
array.forEach(elt => yield elt*2); // DOESN'T WORK!!
}
It's a paradigm shift. The latter is a form of imperative programming, where the user is the one creating data that the computer consumes. The former is essentially a more functional based approach stemming from mathematics, utilizing data (code) that already exists.
There is no theoretical (processing) advantage of one over the other, though applied to the current state of computers, functional programming becomes more useful as processing power becomes greater.
Functional programming allows a form of reasoning based in mathematics emphasizing inputs and outputs. JavaScript in particular is good at handling this style due to functions being first-class data types
good answers here already. will just add one thing i came to appreciate with time: when you do things the "imperative"/old way it tends to encourage a style of programming with a lot of intermediate variables, mutable things all over and also the "while i'm iterating i might as well do this other thing on the same data" which is the biggest trap in code design - sacrificing separation of concerns for dubious performance gains. Consider this example
const numbers = [1,4,9,16];
let sum = 0;
let onlyEvenNumbers = [];
for(i = 0; i < numbers.length; i++) {
sum += numbers[i];
if(numbers[i] % 2 == 0) {
onlyEvenNumbers.push(numbers[i]);
}
}
this is bad imo cause you're gaining very little (if any) performance and the for loop doesn't have a clear single purpose. of course in this simple case it's probably ok, but it can get real ugly real fast. also it's not clear at first glance what gets stored in the onlyEvenNumbers var until you read to the bottom of the for loop - again, probably ok here, but if the for loop gets big it may get confusing.
functional approach for the win (also notice things can be not only const, but are even not mutated once constructed):
const numbers = [1,4,9,16];
const sum = numbers.reduce((acc, val) => acc + val);
const onlyEvenNumbers = numbers.filter(num => num % 2 == 0);
Is there any particular reason why I should choose either of these techniques for generating a random string in nodejs?
First:
//var TOKEN_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
var TOKEN_CHARS = 'abcdef0123456789';
var len = 24;
var chars = [];
for (var i = 0; i < len; i++) {
var index = Math.floor(Math.random() * TOKEN_CHARS.length);
chars.push(TOKEN_CHARS[index]);
}
console.log(chars.join(''));
Second:
var token = require('crypto').randomBytes(len/2).toString('hex');
console.log(token);
At first glance the output of these look similar. I don't understand fully, but as far as I can tell from researching Math.random() may not be the best technique based on the fact the "seed" has to do with the system time and is not truly random. However the highly used connect library uses the first technique so I assume it must be pretty good.
If I were to use the first technique, would the token be "more" secure using the commented out TOKEN_CHARS (simply due to more possibilities for each character)?
Math.random() is created as a general purpose PRNG, crypto.pseudoRandomBytes is a part of OpenSSL library and is created as a CSPRNG. So it's a good reason to use second one.
If I were to use the first technique, would the token be "more" secure using the commented out TOKEN_CHARS
No. However, if you want more entropy in your token, you can use .toString('base64') in the second case, this way it'll use 64 characters to represent your token.