Develop Reference

Dynamic, multidimensional array in javascript - counting up not working

I need a 3 dimensional array to count up - it needs to grow dynamically. It consists of index, string1 and string2.
This outputs exactly, what I want (for a single loop, since the array is just hardcoded to index 0)
var otr_entries=[[0,"",""]];
var otr_entries_count=0;
some_working_for_loop()
{
if(is_important_value_to_save())
{
//otr_entries_count=otr_entries_count+1;
otr_entries[otr_entries_count][1]=xx[i].previousElementSibling.innerHTML;
otr_entries[otr_entries_count][2]=xx[i].innerHTML;
window.alert(otr_entries[otr_entries_count][1]); // Expected output
window.alert(otr_entries[otr_entries_count][2]); // Expected output
}
}
but when I replace otr_entries[0][2] with otr_entries[otr_entries_count][2] the script suddenly fails, if the count is not 0. That means, that the array is not just growing. So how can this be archived?
var otr_entries=[[0,"",""]];
var otr_entries_count=0;
just_some_perfectly_working_for_loop(;;)
{
if(is_important_value_to_save())
{
otr_entries_count=otr_entries_count+1; // Counting up breaks the code
otr_entries[otr_entries_count][1]=xx[i].previousElementSibling.innerHTML;
otr_entries[otr_entries_count][2]=xx[i].innerHTML;
window.alert(otr_entries[otr_entries_count][1]); // No output, script totally stops
window.alert(otr_entries[otr_entries_count][2]); // No output, script totally stops
}
}
EDIT:
This is my solution, thanks to peters help. Works perfectly fine.
var otr_entries=[];
var otr_entries_count=-1;
some_working_for_loop()
{
if(is_important_value_to_save())
{
otr_entries_count=otr_entries_count+1;
otr_entries.push(otr_entries_count,xx[i].previousElementSibling.innerHTML,xx[i].innerHTML)
window.alert(otr_entries[otr_entries_count][1]); // Expected output
window.alert(otr_entries[otr_entries_count][2]); // Expected output
}
}

If in the loop you’re trying to add to the array you need to do otr_entries.push([count,"something","something2"]).
Also, if you’re adding to the array you shouldn’t be using that same array as your loop control.

You're just incrementing the counter before you add new item in the array. The solution is to move the counter incrementing line at the end:
This worked for me
const otr_entries=[[0,"",""]];
const otr_entries_count=0;
for(const entry of otr_entries) {
// condition
if(true) {
otr_entries[otr_entries_count][1]='something';
otr_entries[otr_entries_count][2]='something else'
window.alert(otr_entries[otr_entries_count][1]);
window.alert(otr_entries[otr_entries_count][2]);
otr_entries_count+=1;
}
}

Nested loop (FOR) with IF statement inside 2nd for print just one result

Basically I have 2 arrays, one with some code and another with codes and relative description, what I need to do is match the codes and print the description but my code (apparently) stops at the first loop of the inner FOR (I've attaches a screenshot to understand better).
If I remove the IF statement from the code it prints the counters of the 2 for as it should be.
for (x=0; x<causeoferrorlength; x++)
{
document.getElementById("mdataresult").innerHTML += "x "+causeoferrorsplit[x]+"</br>";
for(k=0; k<78; k++)
{
if ( causeoferrorsplit[x] === gbrucausesoferror[k][0] )
{
document.getElementById("mdataresult").innerHTML += "k "+gbrucausesoferror[k][0]+"</br>";
}
}
}
I have no errors from the console but it isn't printing as expected.

This is probably better handled in a declarative way versus imperative. It will be shorter and easier to reason about.
Given you're using two arrays, and that the codes in the first array will always be found somewhere in the second array:
let causes = ["001", "003", "005"];
let codes = [
["001","Earthquake"],
["002","Sunspots"],
["003","User Error"],
["004","Snakes"],
["005","Black Magic"]
];
let results = causes.map( cause => codes[ codes.findIndex( code => code[0] === cause ) ][1] );
console.log(results); // ["Earthquake", "User Error", "Black Magic"]
What's happening here? We're mapping the array of potential causes of error (the first array) to a list of descriptions taken from the second array.
Array.map takes a function that is invoked once with each array member. We'll call that member 'cause'.
Array.findIndex takes a function that is invoked once for each array member. We'll call that member 'code'.
For each 'cause' in causes we find the index in codes where the first array value is equal to the cause, then return the second array value, the description.
If you have the ability to change the second array to an object, then this gets way simpler:
let causes = ["001", "003", "005"];
let codes = {
"001":"Earthquake",
"002":"Sunspots",
"003":"User Error",
"004":"Snakes",
"005":"Black Magic"
};
let results = causes.map( cause => codes[cause] );
console.log(results); // ["Earthquake", "User Error", "Black Magic"]

dc.js Using two reducers without a simple dimension and second grouping stage

Quick question following up my response from this post:
dc.js Box plot reducer using two groups
Just trying to fully get my head around reducers and how to filter and collect data so I'll step through my understanding first.
Data Format:
{
"SSID": "eduroam",
"identifier": "Client",
"latitude": 52.4505,
"longitude": -1.9361,
"mac": "dc:d9:16:##:##:##",
"packet": "PR-REQ",
"timestamp": "2018-07-10 12:25:26",
"vendor": "Huawei Technologies Co.Ltd"
}
(1) Using the following should result in an output array of key value pairs (Key MAC Address & Value Count of networks connected to):
var MacCountsGroup = mac.group().reduce(
function (p, v) {
p[v.mac] = (p[v.mac] || 0) + v.counter;
return p;
},
function (p, v) {
p[v.mac] -= v.counter;
return p;
},
function () {
return {}; // KV Pair of MAC -> Count
}
);
(2) Then in order to use the object this must be passed flattened so it can be passed to a chart as follows:
function flatten_object_group(group) {
return {
all: function () {
return group.all().map(function (kv) {
return {
key: kv.key,
value: Object.values(kv.value).filter(function (v) {
return v > 0;
})
};
});
}
};
}
var connectionsGroup = flatten_object_group(MacCountsGroup);
(3) Then I pass mac as a piechart dimension & connectionsGroup as the group. This gives a chart back a chart with roughly 50,000 slices based on my dataset.
var packetPie = dc.pieChart("#packetPie");
packetPie
.height(495)
.width(350)
.radius(180)
.renderLabel(true)
.transitionDuration(1000)
.dimension(mac)
.ordinalColors(['#07453E', '#145C54', '#36847B'])
.group(connectionsGroup);
This works A'OK and I follow up to this point.
(4) Now I want to group by the values given out by the first reducer, i.e I want to combine all of the mac addresses with 1 network connection, 2 network connections and so on as slices.
How would this be done as a dimension of "Network connections"? How can I produce this summarized data which doesn't exist in my source data and is generated from mac?
Or would this require an intermediate function between the first reducer and flattening to combine all of the values from the first reducer?

You don't need to do all of that to get a pie chart of mac addresses.
There are a few faulty understandings in points 1-3, which I guess I'll address first. It looks like you copy and pasted code from the previous question, so I'm not really sure if this helps.
(1) If you have a dimension of mac addresses, reducing it like this won't have any further effect. The original idea was to dimension/group by vendor and then reduce counts for each mac address. This reduction will group by mac address and then further count instances of each mac address within each bin, so it's just an object with one key. It will produce a map of key value pairs like
{key: 'MAC-123', value: {'MAC-123': 12}}
(2) This will flatten the object within the values, dropping the keys and producing just an array of counts
{key: 'MAC-123', value: [12]}
(3) Since the pie chart is expecting simple key/value pairs with the value being a number, it is probably unhappy with getting values like the array [12]. The values are probably coerced to NaN.
(4) Okay, here's the real question, and it's actually not as easy as your previous question. We got off easy with the box plot because the "dimension" (in crossfilter terms, the keys you filter and group on) existed in your data.
Let's forget the false lead in points 1-3 above, and start from first principles.
There is no way to look at an individual row of your data and determine, without looking at anything else, if it belongs to the category "has 1 connection", "has 2 connections", etc. Assuming you want to be able to click on slices in the pie chart and filter all the data, we'll have to find another way to implement that.
But first let's look at how to produce a pie chart of "number of network connections". That's a little bit easier, but as far as I know, it does require a true "double reduce".
If we use the default reduction on the mac dimension, we'll get an array of key/value pairs, where the key is a mac address, and the value is the number of connections for that address:
[
{
"key": "1c:b7:2c:48",
"value": 8
},
{
"key": "1c:b7:be:ef",
"value": 3
},
{
"key": "6c:17:79:03",
"value": 2
},
...
How do we now produce a key/value array where the key is number of connections, and the value is the array of mac addresses for that number of connections?
Sounds like a job for the lesser-known Array.reduce. This function is the likely inspiration for crossfilter's group.reduce(), but it's a bit simpler: it just walks through an array, combining each value with the result of the last. It's great for producing an object from an array:
var value_keys = macPacketGroup.all().reduce(function(p, kv) {
if(!p[kv.value])
p[kv.value] = [];
p[kv.value].push(kv.key);
return p;
}, {});
Great:
{
"1": [
"b8:1d:ab:d1",
"dc:d9:16:3a",
"dc:d9:16:3b"
],
"2": [
"6c:17:79:03",
"6c:27:79:04",
"b8:1d:aa:d1",
"b8:1d:aa:d2",
"dc:da:16:3d"
],
But we wanted an array of key/value pairs, not an object!
var key_count_value_macs = Object.keys(value_keys)
.map(k => ({key: k, value: value_keys[k]}));
Great, that looks just like what a "real group" would produce:
[
{
"key": "1",
"value": [
"b8:1d:ab:d1",
"dc:d9:16:3a",
"dc:d9:16:3b"
]
},
{
"key": "2",
"value": [
"6c:17:79:03",
"6c:27:79:04",
"b8:1d:aa:d1",
"b8:1d:aa:d2",
"dc:da:16:3d"
]
},
...
Wrapping all that in a "fake group", which when asked to produce .all(), queries the original group and does the above transformations:
function value_keys_group(group) {
return {
all: function() {
var value_keys = group.all().reduce(function(p, kv) {
if(!p[kv.value])
p[kv.value] = [];
p[kv.value].push(kv.key);
return p;
}, {});
return Object.keys(value_keys)
.map(k => ({key: k, value: value_keys[k]}));
}
}
}
Now we can plot the pie chart! The only fancy thing here is that the value accessor should look at the length of the array for each value (instead of assuming the value is just a number):
packetPie
// ...
.group(value_keys_group(macPacketGroup))
.valueAccessor(kv => kv.value.length);
Demo fiddle.
However, clicking on slices won't work. I'll return to that in a minute - just want to hit "save" first!
Part 2: Filtering based on counts
As I remarked at the start, it's not possible to create a crossfilter dimension which will filter based on the count of connections. This is because crossfilter always needs to look at each row and determine, based only on the information in that row, whether it belongs in a group or filter.
If you add another chart at this point and try clicking on a slice, everything in the other charts will disappear. This is because the keys are now counts, and counts are invalid mac addresses, so we're telling it to filter to a key which doesn't exist.
However, we can obviously filter by mac address, and we also know the mac addresses for each count! So this isn't so bad. It just requires a filterHandler.
Although, hmmm, in producing the fake group, we seem to have forgotten value_keys. It's hidden away inside the function, and then let go.
It's a little ugly, but we can fix that:
function value_keys_group(group) {
var saved_value_keys;
return {
all: function() {
var value_keys = group.all().reduce(function(p, kv) {
if(!p[kv.value])
p[kv.value] = [];
p[kv.value].push(kv.key);
return p;
}, {});
saved_value_keys = value_keys;
return Object.keys(value_keys)
.map(k => ({key: k, value: value_keys[k]}));
},
value_keys: function() {
return saved_value_keys;
}
}
}
Now, every time .all() is called (every time the pie chart is drawn), the fake group will stash away the value_keys object. Not a great practice (.value_keys() would return undefined if you called it before .all()), but safe based on the way dc.js works.
With that out of the way, the filterHandler for the pie chart is relatively simple:
packetPie.filterHandler(function(dimension, filters) {
if(filters.length === 0)
dimension.filter(null);
else {
var value_keys = packetPie.group().value_keys();
var all_macs = filters.reduce(
(p, v) => p.concat(value_keys[v]), []);
dimension.filterFunction(k => all_macs.indexOf(k) !== -1);
}
return filters;
});
The interesting line here is another call to Array.reduce. This function is also useful for producing an array from another array, and here we use it just to concatenate all of the values (mac addresses) from all of the selected slices (connection counts).
Now we have a working filter. It doesn't make too much sense to combine it with the box plot from the last question, but the new fiddle demonstrates that filtering based on number of connections does work.
Part 3: what about zeroes?
As commonly comes up, crossfilter considers a bin with value zero to still exist, so we need to "remove the empty bins". However, in this case, we've added a non-standard method to the first fake group, in order to allow filtering. (We could have just used a global there, but globals are messy.)
So, we need to "pass through" the value_keys method:
function remove_empty_bins_pt(source_group) {
return {
all:function () {
return source_group.all().filter(function(d) {
return d.key !== '0';
});
},
value_keys: function() {
return source_group.value_keys();
}
};
}
packetPie
.group(remove_empty_bins_pt(value_keys_group(macPacketGroup)))
Another oddity here is we are filtering out the key zero, and that's a string here!
Demo fiddle!
Alternately, here's a better solution! Do the bin filtering before passing to value_keys_group, and then we can use the ordinary remove_empty_bins!
function remove_empty_bins(source_group) {
return {
all:function () {
return source_group.all().filter(function(d) {
//return Math.abs(d.value) > 0.00001; // if using floating-point numbers
return d.value !== 0; // if integers only
});
}
};
}
packetPie
.group(value_keys_group(remove_empty_bins(macPacketGroup)))
Yet another demo fiddle!!

Javascript toFixed behaviour with substraction

I'm manipulating a lot of numbers in my application. For this particular case, here is what I do : I retrieve two lists of numbers, I do an average for each of these list, then I substract the two average. To avoid average like 3.333333333333 I use .toFixed(3) on my results.
Here is what it looks like :
// I found this function somewhere on Stackoverflow
Array.prototype.average = function() {
if(this.length == 0){
return 0;
}
else{
return this.reduce(function (p, c) {
return p + c;
}) / this.length;
}
};
sumHigh = [ 10.965, 10.889, 10.659, 10.69, 10.599 ]
sumLow = [ 4.807, 3.065, 2.668, 2.906, 3.606, 4.074, 4.153 ]
// Ok normal
console.log(sumHigh.average()) // 10.760399999999999
console.log(sumLow.average()) // 3.6112857142857138
// Ok normal
console.log(sumHigh.average().toFixed(3)) // "10.760" Does the ".." has anything to do with my problem ?
console.log(sumLow.average().toFixed(3)) // "3.611"
// So here I have my two average values with no more than 3 numbers after the comma but it is not taken into account when substracting these two numbers...
// Not Ok, why 10.760 - 3.611 = 7.148999999999999 ?
console.log(sumHigh.average().toFixed(3) - sumLow.average().toFixed(3)) // 7.148999999999999
console.log(parseFloat(sumHigh.average().toFixed(3)) - parseFloat(sumLow.average().toFixed(3))) // 7.148999999999999
// Just as an example, this is working
console.log(parseFloat(sumHigh.average().toFixed(3)) + parseFloat(sumLow.average().toFixed(3))) // 14.371
console.log(parseFloat(sumHigh.average()) + parseFloat(sumLow.average())) // 14.371685714285713
Can someone explain this behaviour?
Why substraction is not working while addition is?
Ok I know I can solve my problem with :
console.log((sumHigh.average() - sumLow.average()).toFixed(3)) // "7.149"
But that doesn't explain this behaviour.
Thanks

mongo/mongoid MapReduce on batch inserted documents

Im creating my batch and inserting it to collection using command i specified below
batch = []
time = 1.day.ago
(1..2000).each{ |i| a = {:name => 'invbatch2k'+i.to_s, :user_id => BSON::ObjectId.from_string('533956cd4d616323cf000000'), :out_id => 'out', :created_at => time, :updated_at => time, :random => '0.5' }; batch.push a; }
Invitation.collection.insert batch
As stated above, every single invitation record has user_id fields value set to '533956cd4d616323cf000000'
after inserting my batch with created_at: 1.day.ago i get:
2.1.1 :102 > Invitation.lte(created_at: 1.week.ago).count
=> 48
2.1.1 :103 > Invitation.lte(created_at: Date.today).count
=> 2048
also:
2.1.1 :104 > Invitation.lte(created_at: 1.week.ago).where(user_id: '533956cd4d616323cf000000').count
=> 14
2.1.1 :105 > Invitation.where(user_id: '533956cd4d616323cf000000').count
=> 2014
Also, I've got a map reduce which counts invitations sent by each unique User (both total and sent to unique out_id)
class Invitation
[...]
def self.get_user_invites_count
map = %q{
function() {
var user_id = this.user_id;
emit(user_id, {user_id : this.user_id, out_id: this.out_id, count: 1, countUnique: 1})
}
}
reduce = %q{
function(key, values) {
var result = {
user_id: key,
count: 0,
countUnique : 0
};
var values_arr = [];
values.forEach(function(value) {
values_arr.push(value.out_id);
result.count += 1
});
var unique = values_arr.filter(function(item, i, ar){ return ar.indexOf(item) === i; });
result.countUnique = unique.length;
return result;
}
}
map_reduce(map,reduce).out(inline: true).to_a.map{|d| d['value']} rescue []
end
end
The issue is:
Invitation.lte(created_at: Date.today.end_of_day).get_user_invites_count
returns
[{"user_id"=>BSON::ObjectId('533956cd4d616323cf000000'), "count"=>49.0, "countUnique"=>2.0} ...]
instead of "count" => 2014, "countUnique" => 6.0 while:
Invitation.lte(created_at: 1.week.ago).get_user_invites_count returns:
[{"user_id"=>BSON::ObjectId('533956cd4d616323cf000000'), "count"=>14.0, "countUnique"=>6.0} ...]
Data provided by query, is accurate before inserting the batch.
I cant wrap my head around whats going on here. Am i missing something?

The part that you seemed to have missed in the documentation seem to be the problem here:
MongoDB can invoke the reduce function more than once for the same key. In this case, the previous output from the reduce function for that key will become one of the input values to the next reduce function invocation for that key.
And also later:
the type of the return object must be identical to the type of the value emitted by the map function to ensure that the following operations is true:
So what you see is your reduce function is returning a signature different to the input it receives from the mapper. This is important since the reducer may not get all of the values for a given key in a single pass. Instead it gets some of them, "reduces" the result and that reduced output may be combined with other values for the key ( possibly also reduced ) in a further pass through the reduce function.
As a result of your fields not matching, subsequent reduce passes do not see those values and do not count towards your totals. So you need to align the signatures of the values:
def self.get_user_invites_count
map = %q{
function() {
var user_id = this.user_id;
emit(user_id, {out_id: this.out_id, count: 1, countUnique: 0})
}
}
reduce = %q{
function(key, values) {
var result = {
out_id: null,
count: 0,
countUnique : 0
};
var values_arr = [];
values.forEach(function(value) {
if (value.out_id != null)
values_arr.push(value.out_id);
result.count += value.count;
result.countUnique += value.countUnique;
});
var unique = values_arr.filter(function(item, i, ar){ return ar.indexOf(item) === i; });
result.countUnique += unique.length;
return result;
}
}
map_reduce(map,reduce).out(inline: true).to_a.map{|d| d['value']} rescue []
end
You also do not need user_id in the values emitted or kept as it is already the "key" value for the mapReduce. The remaining alterations consider that both "count" and "countUnique" can contain an exiting value that needs to be considered, where you were simply resetting the value to 0 on each pass.
Then of course if the "input" has already been through a "reduce" pass, then you do not need the "out_id" values to be filtered for "uniqueness" as you already have the count and that is now included. So any null values are not added to the array of things to count, which is also "added" to the total rather than replacing it.
So the reducer does get called several times. For 20 key values the input will likely not be split, which is why your sample with less input works. For pretty much anything more than that, then the "groups" of the same key values will be split up, which is how mapReduce optimizes for large data processing. As the "reduced" output will be sent back to the reducer again, you need to be mindful that you are considering the values you already sent to output in the previous pass.