Access the data from two observables without multiple subscriptions - javascript

My first observable emits data concerning my user (from Firebase). On this data being emitted, I'd like to trigger a second call to Firebase, retrieving various extra user information from a distinct collection.
I can manage the two operations individually just fine, but obviously the latter call should only really take place after the first call has succeeded. I imagine this is something to do with mergeMap but I can't figure out quite how to do it. Should point out to the close/downvoter that I've, despite trying, not found similar questions that are both answered and present the solution in a way I can match to exactly what I'm failing to understand.
this._auth.userState$
.mergeMap(user => this._userMetaData.getData(user.uid))
.subscribe(data => {
console.log(data);
});
This is my initial effort. I was hoping that the console.log would return (in some fashion) the data from both calls. It just logs an empty array, instead.
Can anyone point me in the right direction here?

I see 2 different points worth to be highlighted.
1) Within the submit, you want to access the data returned by the 2 calls:
This can be accomplished in the following way
.mergeMap(user =>
this._userMetaData.getData(user.uid)
.map(data => ({user, data})
)
Here you basically create, via the last map, a return which is an object referencing both the user returned by the first call and the data returned by the second call
2) In the log you see an empty Array:
Here you need to look at your logic since, according to your current code, in the log you should see the result of the second call

Related

MongoDB weird behavior needs explanation

So yesterday I started messing around with MongoDB in Node and when it comes to retrieving data I encountered a weird practice.
You retrieve data from a collection that is nested within a database by calling.
data = client.db([dbname]).collection([collectionname]).find([searchcriteria])
and this returns what seems to be an object at least in the eyes of typeof
the sample code then uses the following lines to log it to the console:
function iterate(x){
console.log(x)
}
data.forEach(iterate)
The output is as expected in this case two objects with 2 pairs everything is fine so far.
I thought it is a bit unnecessary to have the iterate function so I changed that to just
console.log(data)
expecting the 2 objects in a array or nested in another object but what i get is this huge object with all kinds of different things in it EXCEPT the two objects that we saw before.
So now to my Question and what I need deeper explanation on:
why can i actually use .forEach() on this object I cannot recreate this on other objects.
and the second thing is why is console.log(data) giving me all this output that is hidden if I call it through .forEach()?
and is there any other way to quickly within one or 2 lines of code retrieve data from Mongo ?
this seems to be a very not useful way of doing things.
and how does this .forEach() thing on objects work I found a article here on stack however this was not very detailed and not very easy to understand.
The find function returns a cursor - this is the huge object that you are seeing. Checkout documentation for more details here: https://docs.mongodb.com/manual/reference/method/db.collection.find/#db.collection.find
The reason why you can call forEach on the returned object (=cursor), is because it is one of its methods. See https://docs.mongodb.com/manual/reference/method/cursor.forEach/#cursor.forEach
Overview of all cursor methods is here: https://docs.mongodb.com/manual/reference/method/js-cursor/
To get the array of data that you are looking for you need to use the toArray method, like so:
const data = client.db([dbname]).collection([collectionname]).find([searchcriteria]).toArray()

Split array into chunks/bulks and made operations with them one by one

I'm a bit sorry about tags, probably I understood my problem not right and used them wrong but..
The problem I'm faced with my project is new for me and I never before experienced it. So in my case I have a huge dataset response from DB (Mongo, 100'000+ docs) and I needed to http-request every specific field from doc.
Example array from dataset will be like:
{
_id: 1,
http: http.request.me
},
{
//each doc of 99k docs more
}
So guess you already understood that I cannot use default for loop because
if it async I'll be made a huge amount request to API and will
be banned/restricted/whatever
if I made it one-by-one it will take me about 12-23H of
waiting before my loop completes itself. (actually, this way is in
use)
This is what I'm trying to do right now
there is also another way and that's why I'm here. I could split my huge array in to chunks for example each 5/10/100..N and request them one-by-one
│→await[request_map 0,1,2,3,4]→filled
│→await[request_map 5..10]→filled
│→await[request_map n..n+5]→filled
↓
According to the Split array into chunks I could easily do it. But then I should use 2 for cycles, first one will split default array and second async-request this new array (length 5/10/100...N)
But I have recently heard about reactive paradigm and RxJS that (probably) could solve this. Is this right? What operator should I use? What keyword should I use to find relative problems? (if I google reactive programming I'll receive a lot of useless result with react.js but not what I want)
So should I care about all this and just write an unoptimized code or there is an npm-module for that or another-better-pattern/solution?
Probably I found and answer here
RxJS 1 array item into sequence of single items - operator I'm checking it now, but I also appreciate any relevant contribution
to this question
RxJS has truly been helpful in this case and worth looking. It's an
elegant solution for this kind of problems
Make use of bufferCount and concatMap
range(0,100).pipe(
// save each http call into array as observable but not executing them
map(res=>http(...)),
//5 at a time
bufferCount(5),
//execute calls concurrently and in a queue of 5 calls each time
concatMap(res=>forkJoin(res))
).subscribe(console.log)
There's actually an even easier way to do what you want with mergeMap operator and it's second optional argument which sets the number of concurrent inner Observables:
from([obj1, obj2, obj3, ...]).pipe(
mergeMap(obj => /* make a request out of `obj` */, 5), // keep only 5 concurrent requests
).subscribe(result => ...)

Riak MapReduce in single node using javascript and python

I want to perform MapReduce job on data in Riak DB using javascript. But stuck in very begining, i couldnot understand how it is returning value.
client = riak.RiakClient()
query = client.add('user')
query.map("""
function(v){
var i=0;
i++;
return [i];
}
""")
for result in query.run():
print "%s" % (result);
For simplicity i have checked the above example.
Here query is bucket and user contain five sets of data in RiakDB.
i think map() returns single value but it returns array with 5 value, i think equivalent to five set of data in RiakDB.
1
1
1
1
1
And here, why I can return only array? it treats each dataset independently, and returns for each. so i think i have five 1's. Due to this reason when i process fetched data inside map(), returns gives unexpected result for me.
so please give me some suggestion. i think it is basic thing but i couldnot get it. i highly appreciate your help.
When you run a MapReduce job, the map phase code is sent out to the vnodes where the data is stored and executed for each value in the data. The resulting arrays are collected and passed to a single reduce phase, which also returns an array. If there are sufficiently many results, the reduce phase may be run multiple times, with the previous reduce result and a batch of map results as input.
The fact that you are getting 5 results implies that 5 keys were seen in your bucket. There is no global state shared between instances of the map phase function, so each will have an independent i, which is why each result is 1.
You might try returning [v.key] so that you have something unique for each one, or if the values are expected to be small, you could return [JSON.stringify(v)] so you can see the entire structure that is passed to the map.
You should note that according to the docs site javascript Map Reduce has been officially deprecated, so you may want to use Erlang functions for new development.

How to find all entries of more than one type of record in Ember.js?

I have two types of records in the Ember DS.Store: users and location. (In fact I have more, but for the sake of simplicity).
Now to get all entries of, say, 'user', I would simply do
this.store.find('user')
Now say I have a variable allResults and I assign it to this command.
allResults = this.store.find('user')
This gives me what Ember calls a promiseArray, and to do anything after this promise array loads I simply
allResults.then(successfunction(), failurefunction())
Now this works great when I need only one type of record - say I need only users, I can easily call my successfunction() as the first argument.
However, my need goes beyond that: I am basically building a searchbar that searches through these records, so if someone types in "mary", it needs to both show the user "Mary" and the location "mary Ave" for example.)
So I need the combined results of
this.store.find('user')
&
this.store.find('location')
Therefore, here are my questions: [I feel that either of them would work.]
Is there a way I can fetch all data pertaining to both 'user' and 'location' and have it returned as one glorious promiseArray? This seems likely and also the best way of approaching this issue.
Can you concatenate two promiseArrays to make one larger one, then user the .then function for the larger one? If so, how?
You can combine promises.
http://emberjs.com/api/classes/Ember.RSVP.Promise.html
Something like this should work for you:
var bothPromise = Promise.all([
store.find('user'),
store.find('location')
]).then(function(values){
//merge the values arrays
var all = Em.A();
all.addObjects(values[0]); //users
all.addObjects(values[1]); //locations
return all;
});
bothPromise.then(function(allObjects){...
See the "Combine" section at this promise library called Q for another explanation of combining promises.

JavaScript Pub/Sub property access issue

I am having a strange issue here I hope you all can help with.
Project Details
I am working on a simple pub/sub implementation for a larger application that includes a pubsub.subscribe_once() method. This method enables the creation of one-off subscriptions, meaning that a generic subscription is created, and then once the correct "publish" event fires and the subscription callback is run, the subscription deletes itself.
subscribe_once: function(topic, func) {
var sub = pubsub.subscribe(topic, func),
old_func = sub.func;
// rewrite our subscription's method to remove itself after invocation
sub.func = function() {
// call the original function
old_func.apply(this);
// remove subscription from topic
pubsub.unsubscribe(sub);
};
return sub;
}
Problem
I seem to be having some kind of issue with the memory flow of this process. (In order to best understand the following explanation I suggest you walk through the jsfiddle demo below as I go.) I create a subscribe_once('someevent') subscription, and then fire publish('someevent'). What you would expect to see when the publish method is invoked is that the topics hashtable contains a "someevent" key, which references an array of Subscription objects. And in fact, if you reference topics["someevent"], you see an array with a single Subscription. If, however, you reference topics you see the "someevent" key, but the array is empty!
By commenting out pubsub.unsubscribe(sub); the problem is eliminated, even though this function does not appear to be fired until after we run console.log(topics).
Further, this does not seem to be an issue with the way a given browser "threads" console.log; try console.log(topics, topics[topic], topics, topics[topic]) and you get the same result.
Demo: http://jsfiddle.net/4Ab6c/
Any help would be greatly appreciated! Thanks.
I'm still looking for some documentation to back me up, but I suspect the object display in the console is performing lazy evaluation on your topics object. I added console.log(topics) to the subscribe method after sub is pushed onto the array and I get the same result as your 'but not here' log. When I wrap the final line of your fiddle pubsub.publish('someevent') in setTimeout and I get the Object tree open before the publish callback runs, then it shows the subscription in the array and it stays that way even after the callback runs. If I don't open the object tree in the console before the callback runs then I see the empty array.
I will keep searching for a blog post or something that confirms lazy evaluation is occurring.
Just in case I haven't made it obvious enough, by lazy I mean, the console isn't gathering the details of the object until the tree view is clicked open in the console.
I am working in Chrome.
UPDATE
I have found similar behavior on Firefox as well. Firefox recognizes that there is one object in the array but if you don't drill down into the array before the publish even fires then the drill-down on the array will be empty.
I updated the fiddle from your comment:
http://jsfiddle.net/4Ab6c/2/
Please try this:
Run the fiddle and expand the object tree for the first console.log before the publish event fires, I set it to a five second timeout but you could make it longer or shorter depending on how quickly you can get down to the console and click the inspector open.
You should see the subscribe object in the array as expected.
Clear the console and run the fiddle again. This time do not open the object inspector until after the publish event has fired and all the code is done running.
Now when you open the object inspector of the first console.log you should not see the subscription event in the array.
UPDATE 2
Here is a much simpler fiddle that exhibits the same behavior:
http://jsfiddle.net/4Ab6c/3/
If you expand first before second shows up then you will get foo: bar. If you expand first after second shows up you will get foo: baz.
UPDATE 3
And, voila, another SO question seconds the motion on lazy evaluation.

Categories

Resources