Develop Reference

Is there a more elegant way to push and return an array?

I am still figuring out Promises, but while working with them, I've realized it would be nice to reduce an array of fetch objects and put some throttles next to them. While creating my slow querying function, I realized I couldn't think of an elegant way to push onto an array and return that array better than this.
SO. My question is; Is there a more elegant way of pushing to an array and returning an array in one step in Javascript than this?
const mQry = q => fetch(q).then(r=>r.json()); // Fetches and returns json
const throttle = t => new Promise(r=>setTimeout(r,t)); // adds a promised timeout
const slowQrys = (q,t) => // pass in an array of links, and a number of milliseconds
Promise.all(q.reduce((r,o)=> // reduce the queries
// Here's the big issue. Is there any more elegant way
// to push two elements onto an array and return an array?
[...r, mQry(...o), throttle(t)]
,[]);
And before anyone says, I am super aware that splitting out an array could be not efficient, but I'm probably never using more than 10 items, so it's not a super big deal.

A cleaner and more efficient equivalent of the general operation
q.reduce((r, o) =>
[...r, f(...o), g(t)])
uses flatMap:
q.flatMap(o =>
[f(...o), g(t)])
However, in the context of your question, creating a throttle(t) next to each fetch operation in a Promise.all is completely and unambiguously wrong. All of the setTimeout timers will be running in parallel and resolve at the same time, so there’s no point in creating more than one. They don’t interact with the fetch operations, either, just delay the overall fulfilment of the promise slowQrys returns and muddle the array it resolves to.

I would guess that your intent is to chain your fetch(s), such that two consecutive fetch are mandatorily spaced by at least t ms
The chaining is thus
Promise.all([fetch, wait]), Promise.all([fetch, wait]), ...
The way to write that thus be
const slowQrys = (links, t)=>links.reduce((p, link)=>{
return p.then(_=>{
return Promise.all([
fetch(link),
wait(t)
])
})
}, Promise.resolve())

Javascript/NodeJS passing byref

I'm still learning JS. In some other languages, you can pass variables byref and then modify them elsewhere in code.
In an attempt to avoid having lots of duplicate code, I have structured a series of callbacks and parsing like so:
class MarketData {
constructor() {
//Arrays
this.OneMinuteData = [];
this.ThreeMinuteData = [];
this.initializeCandleData();
}
initializeData() {
var client = new Client();
this._initializeData(60, client, this.OneMinuteData);
this._initializeData(180, client, this.ThreeMinuteData);
}
_initializeData(granularity, client, dataStore) {
client.GetRates({ granularity: granularity }, function(err, msg, data) {
var items = data.map(item => ({
///data mapped here
}));
dataStore = dataStore.concat(items);
}
}
So essentially I have this 'private' _initializeData function with the hopes of passing in an array and having it add to the array, but since JS passes byval, I cannot achieve the desired effect (e.g. this.OneMinuteData array is not modified).
Because of this, the only way I currently know how to work around this problem is to essentially have the same function copy-pasted for each individual array, which I find incredibly sloppy. Is there a better way of doing this?

but since JS passes byval, I cannot achieve the desired effect (e.g. this.OneMinuteData array is not modified).
While JavaScript does pass by value, that value when dealing with an object (including any array) is a reference.
See the documentation for concat:
The concat() method is used to merge two or more arrays. This method does not change the existing arrays, but instead returns a new array.
So when you say dataStore = dataStore.concat(items);, you assign a new array to the local dataStore variable and discard the old one.
Outside the function, the original array is unchanged.
The reason the array assigned to OneMinuteData is not modified is because you never modify any array.
Push the values of items into dataStore instead.
dataStore.push.apply(dataStore, items);
NB: GetRates has the signature of an asynchronous function, so make sure you don't try to inspect the modifications to OneMinuteData before they are made.

Promises and upserting to database in bulk

I am currently parsing a list of js objects that are upserted to the db one by one, roughly like this with Node.js:
return promise.map(list,
return parseItem(item)
.then(upsertSingleItemToDB)
).then(all finished!)
The problem is that when the list sizes grew very big (~3000 items), parsing all the items in parallel is too memory heavy. It was really easy to add a concurrency limit with the promise library and not run out of memory that way(when/guard).
But I'd like to optimize the db upserts as well, since mongodb offers a bulkWrite function. Since parsing and bulk writing all the items at once is not possible, I would need to split the original object list in smaller sets that are parsed with promises in parallel and then the result array of that set would be passed to the promisified bulkWrite. And this would be repeated for the remaining sets if list items.
I'm having a hard time wrapping my head around how I can structure the smaller sets of promises so that I only do one set of parseSomeItems-BulkUpsertThem at time (something like Promise.all([set1Bulk][set2Bulk]), where set1Bulk is another array of parallel parser Promises?), any pseudo code help would be appreciated (but I'm using when if that makes a difference).

It can look something like this, if using mongoose and the underlying nodejs-mongodb-driver:
const saveParsedItems = items => ItemCollection.collection.bulkWrite( // accessing underlying driver
items.map(item => ({
updateOne: {
filter: {id: item.id}, // or any compound key that makes your items unique for upsertion
upsert: true,
update: {$set: item} // should be a key:value formatted object
}
}))
);
const parseAndSaveItems = (items, offset = 0, limit = 3000) => { // the algorithm for retrieving items in batches be anything you want, basically
const itemSet = items.slice(offset, limit);
return Promise.all(
itemSet.map(parseItem) // parsing all your items first
)
.then(saveParsedItems)
.then(() => {
const newOffset = offset + limit;
if (items.length >= newOffset) {
return parseAndSaveItemsSet(items, newOffset, limit);
}
return true;
});
};
return parseAndSaveItems(yourItems);

The first answer looks complete. However here are some other thoughts that came to mind.
As a hack-around, you could call a timeout function in the callback of your write operation before the next write operation performs. This can give your CPU and Memory a break inbetween calls. Even if you add one millisecond between calls, that is only adding 3 seconds if you have a total of 3000 write objects.
Or you can segment your array of insertObjects, and send them to their own bulk writer.

Creating a filterable list with RxJS

I'm trying to get into reactive programming. I use array-functions like map, filter and reduce all the time and love that I can do array manipulation without creating state.
As an exercise, I'm trying to create a filterable list with RxJS without introducing state variables. In the end it should work similar to this:
I would know how to accomplish this with naive JavaScript or AngularJS/ReactJS but I'm trying to do this with nothing but RxJS and without creating state variables:
var list = [
'John',
'Marie',
'Max',
'Eduard',
'Collin'
];
Rx.Observable.fromEvent(document.querySelector('#filter'), 'keyup')
.map(function(e) { return e.target.value; });
// i need to get the search value in here somehow:
Rx.Observable.from(list).filter(function() {});
Now how do I get the search value into my filter function on the observable that I created from my list?
Thanks a lot for your help!

You'll need to wrap the from(list) as it will need to restart the list observable again every time the filter is changed. Since that could happen a lot, you'll also probably want to prevent filtering when the filter is too short, or if there is another key stroke within a small time frame.
//This is a cold observable we'll go ahead and make this here
var reactiveList = Rx.Observable.from(list);
//This will actually perform our filtering
function filterList(filterValue) {
return reactiveList.filter(function(e) {
return /*do filtering with filterValue*/;
}).toArray();
}
var source = Rx.Observable.fromEvent(document.querySelector('#filter'), 'keyup')
.map(function(e) { return e.target.value;})
//The next two operators are primarily to stop us from filtering before
//the user is done typing or if the input is too small
.filter(function(value) { return value.length > 2; })
.debounce(750 /*ms*/)
//Cancel inflight operations if a new item comes in.
//Then flatten everything into one sequence
.flatMapLatest(filterList);
//Nothing will happen until you've subscribed
source.subscribe(function() {/*Do something with that list*/});
This is all adapted from one of the standard examples for RxJS here

You can create a new stream, that takes the list of people and the keyups stream, merge them and scans to filter the latter.
const keyup$ = Rx.Observable.fromEvent(_input, 'keyup')
.map(ev => ev.target.value)
.debounce(500);
const people$ = Rx.Observable.of(people)
.merge(keyup$)
.scan((list, value) => people.filter(item => item.includes(value)));
This way you will have:
-L------------------ people list
------k-----k--k---- keyups stream
-L----k-----k--k---- merged stream
Then you can scan it. As docs says:
Rx.Observable.prototype.scan(accumulator, [seed])
Applies an accumulator function over an observable sequence and returns each
intermediate result.
That means you will be able to filter the list, storing the new list on the accumulator.
Once you subscribe, the data will be the new list.
people$.subscribe(data => console.log(data) ); //this will print your filtered list on console
Hope it helps/was clear enough

You can look how I did it here:
https://github.com/erykpiast/autocompleted-select/
It's end to end solution, with grabbing user interactions and rendering filtered list to DOM.

You could take a look at WebRx's List-Projections as well.
Live-Demo
Disclosure: I am the author of the Framework.

How do I sort a JSON object by a nested value?

I have an ajax call that returns a JSON object that is pretty complex and I'm having a hard time sorting it.
My call:
$.post('/reports-ajax',arguments, function(data) {}
The response:
{
"10001":{
"unitname":"Fort Worth",
"discounts":{"12-02-2012":"34.810000","12-03-2012":"20.810000","12-04-2012":"27.040000"},
"gross":{"12-02-2012":"56.730000","12-03-2012":"19.350000","12-04-2012":"66.390000"},
"net":{"12-02-2012":"61.920000","12-03-2012":"98.540000","12-04-2012":"39.350000"},
"discounts_total":82.66,
"gross_total":82.47,
"net_total":99.81,
"number":10001
},
"10002":{
"unitname":"Dallast",
"discounts":{"12-02-2012":"12.600000","12-03-2012":"25.780000","12-04-2012":"47.780000","12-05-2012":"45.210000"},
"gross":{"12-02-2012":"29.370000","12-03-2012":"91.110000","12-04-2012":"60.890000","12-05-2012":"51.870000"},
"net":{"12-02-2012":"16.770000","12-03-2012":"65.330000","12-04-2012":"13.110000","12-05-2012":"06.660000"},
"discounts_total":131.37,
"gross_total":33.24,
"net_total":101.87,
"number":10002
},
"32402":{
"unitname":"Austin",
"discounts":{"12-05-2012":"52.890000","12-02-2012":"22.430000","12-03-2012":"58.420000","12-04-2012":"53.130000"},
"gross":{"12-05-2012":"25.020000","12-02-2012":"2836.010000","12-03-2012":"54.740000","12-04-2012":"45.330000"},
"net":{"12-04-2012":"92.200000","12-05-2012":"72.130000","12-02-2012":"13.580000","12-03-2012":"96.320000"},
"discounts_total":186.87,
"gross_total":161.1,
"net_total":174.23,
"number":32402
}
}
I go over the function with a standard each call and do some awesome stuff with highcharts but now I'm trying to sort the responses by the net_total call and I can't figure it out.
I tried .sort() and it errors out that its not a function. I've been reading for a while but guess I'm not finding the right results. This looked promising: Sorting an array of JavaScript objects but it failed with the .sort is not a function. It seems most .sort are on [] arrays not full objects..
Any help would be greatly appreciated.

Sorting objects doesn't make sense since object keys have no positional value. For example, this:
{ a:1, b:2 }
and this:
{ b:2, a:1 }
are exactly the same object. They're not just similar, they're the same.
Nothing in javascript per se gives object keys any positional value. Some people perhaps are mistaken in the belief that:
for (var key in obj) {
iterates through the object keys in a specific sequence. But this is wrong. You should always assume that the for .. in loop processes object keys in random order, always, all the time.
Obviously, if you're going to write a web browser, you're not going to implement a random number generator to parse a for .. in loop. Therefore most web browsers have an accidental stability to how the for .. in loop processes object keys.
Developers who learn javascript by playing around with the browser may figure out that their browser iterates through objects in alphabetical order for example, or the order the keys were added to the object. But this is totally accidental and cannot be relied upon. The browser vendor may change this behavior in the future without violating any backwards compatability (except with buggy scripts written by people who believe objects have a sort order). Not to mention that different browsers have different implementations of javascript and therefore not necessarily have the same internal key ordering of objects.
All the above is besides the point. "Key sort order" does not make any sense in javascript and any behavior observed is merely implementation detail. In short, javascript object does not have key order, just assume it's random.
Solution
Now, what you're really trying to do is not sort the object (you can't, it doesn't make sense). What you're really trying to do is process the object attributes in a specific order. The solution is to simply create an array (which has sorting order) of object keys and then process the object using that array:
// First create the array of keys/net_total so that we can sort it:
var sort_array = [];
for (var key in Response) {
sort_array.push({key:key,net_total:Response[key].net_total});
}
// Now sort it:
sort_array.sort(function(x,y){return x.net_total - y.net_total});
// Now process that object with it:
for (var i=0;i<sort_array.length;i++) {
var item = Response[sort_array[i].key];
// now do stuff with each item
}

What you have there isn't an array and has no order, so you'll have to transform it into an array so you can give it order.
Vaguely:
var array = [];
$.each(data, function(key, value) {
array.push(value);
});
array.sort(function(a, b) {
return a.net_total - b.net_total;
});
Live Example | Source
As GolezTroi points out in the comments, normally the above would lose the key that each entry is stored under in data and so you'd add it back in the first $.each loop above, but in this case the entries already have the key on them (as number), so there's no need.
Or you can replace the first $.each with $.map:
var array = $.map(data, function(entry) {
return entry;
});
// ...and then sort etc.
...whichever you prefer.