I'm trying to query an API to get some data then I want to upsert all of it into my table.
But for some reason I'm not having any luck.
What's the best way to go about this?
I don't think my method of doing a query in a loop is best.
var coin = new Parse.Object.extend("Coins");
axios.get('https://api.coinmarketcap.com/v1/ticker/')
.then(response => {
let data = response.data;
// Put into database
data.map(entry => {
let q = new Parse.Query(Model.Coins.className());
q.equalTo('symbol', entry.symbol);
q.first()
.then(record => {
record.set('symbol', entry.symbol);
record.set('price_usd', entry.price_usd);
return record.save(null, {useMasterKey: true});
});
});
res.success(response);
});
you should avoid fetching and updating objects in a loop. In order to make it better you need to use 2 things:
In your query, instead of using equalTo and first you need to use containedIn for query all the records in one call. Then you need to iterate on the query results, for each record in the loop you need to do the following:
record.set('symbol', entry.symbol);
record.set('price_usd', entry.price_usd);
Finally you need to use saveAll to save all the objects in one call (please notice that saveAll is static function of Parse.Object and you should pass an array into it. Please review the docs before doing it)
Check your data. You may find you have unexpectedly updated records.
Assuming that's the entire body of a cloud function, your function initiates an asynchronous server call, then immediately tells the requestor that the operation was successful, but response isn't populated yet so you pass back undefined. However, parse-server will still run that asynchronous code.
The solution is to put the res.success call inside another .then chain, so the function won't return a response until after the server call + save finishes.
You're also going to get an uncaught error if the symbol doesn't exist on your table, though. You don't check to make sure the query returned a response to the first() call.
Related
I'm new to the javascript world and have been tinkering with Actions on Google. I had an action that was previously running but after attempting to simplify my code, I've been running into a new error I cannot seem to figure out. The code is supposed to make a call to a database and return information to the user based on the date they have selected.
Problem:
The code is supposed to make a call to a database and return information to the user based on the date they have selected. The intent seems to break down when I call the URL using axios. When I test my function I receive the following error from the Google Cloud Platform: "Error: No response has been set. Is this being used in an async call that was not returned as a promise to the intent handler?"
app.intent('Moon', (conv, {date}) => {
// Sets date to today if none given
if (!date) {
date = new Date().toISOString();
}
// Slices date string to match date format in database
const dateSlice = date.slice(5, 9);
// Call to row in dabase for the given date
function getData() {
return axios.get(`example.com/search?Date=${dateSlice}`);
}
return getData().then(res => {
res.data.map(con => {
conv.ask(`On X date there will be a ${con.Object1}`);
});
});
});
I don't know much about Promise and await but that seems to be the issue. I'm not sure how I was able to get my code to run before without these objects. I've tried to insert a Promise object before my return but it makes the rest of the function unreachable. I also checked to see if Axios had any updates but it has not, I am on the latest version. Does the error have to do with one of the returns perhaps?
It could be related to Promises, but you seem to be handling them correctly.
The call to axios.get() is returning a Promise...
... which you are returning in getData()...
... which is returned in the 'Moon' Intent handler as part of the getData().then() block.
I suspect more that this is a logic problem. If res.data is an empty array, then there will be no calls to conv.ask(), so you end up not asking anything.
There is also a problem if res.data has more than two items. In this case, you'll generate an error because you've replied with more than two "simple" responses.
Either way - you may wish to log res and/or res.data to make sure you're getting back what you think.
I am using the listCollection method in mongodb and looping through each returned collection with a query using find.
The problem is that I use the loop to construct an object that I wish to return with response.json after the loop is done but since find is async I am stuck at figuring how to "wait" until every find callback has finished before returning response.json()
var data = {};
database.listCollections({name: {$ne: 'system.indexes'}}).toArray(function(err, collections) {
if (err) return res.json({});
for(i=0; i<collections.length; i++){
var collection = collections[i].name;
database.collection(collection).find(query, limit)
.sort({"Date": -1}).toArray(function(err, docs){
if (err) return res.json({
});
/* Do stuff with docs, push stuff to data */
});
/** Console shows blank **/
console.log("Data " + i+ ": " + JSON.stringify(data));
}
/** Response is blank.. **/
res.json(data);
});
The problem is that the for loop returns way before the find()s are done. How do I deal with this in a JS/Node way? I can hack together a solution but I might encounter a similar problem later..
Edit: I am certain that data is actually returned and processed properly, as a console.log() inside each find on data shows that it actually has content.
You can use Promise if your NodeJS version supports it or use any Promise library otherwise. In this case you can wrap your code inside for loop into some function that returns Promise, then push all those promises objects into array. After loop call
Promise.all(promisesArray).then(/* Callback function that sends response */)
Another option is to use recursion instead of for loop and send request when you reach stop condition. But I don't think it's good option because in case if collections array is big you can exceed maximum stack size.
So I'm trying to go through one Firebase database to find entries in the database matching a criteria. Therefore I'm using the deferred object of jQuery to handle the database calls.
Once I get a return value from this first database I want to get the user info from a second database for each of those values in the first db. Then the results are added to a JSON array
so its:
<search for value, find one>
<<<search other db for oher info>>>
<continue search for outer value>
But this only returns one value - although everything else is running fine (and the console logs all the info correct).
Here's the code:
function find(searchLocation, profileID) {
var requestUserData = {
data: []
};
var def = $.Deferred();
//This will be executed as long as there are elements in the database that match the criteria and that haven't been loaded yet (so it's a simple loop)
Ref.orderByChild("location").equalTo(searchLocation).on("child_added", function(snapshot) {
def.ressolve(snapshot.val().ID);
});
return def.promise();
};
I hope you guys have any ideas on what to do or how I could solve this. Thanks in advance!
Edit: upon further testing I discovered that this problem already exists in the outer loop - so only the first value is being returned. I think this is related to the posission of the resolve() method but I didn't find a posibility on how to change this behaviour.
Firebase is a real-time database. The events stream as changes occur at the server. You're attempting to take this real-time model and force it into CRUD strategy and do a GET operation on the data. A better solution would be to simply update the values in real-time as they are modified.
See AngularFire, ReactFire, or BackboneFire for an example of how you can do this with your favorite bindings framework.
To directly answer the question, if you want to retrieve a static snapshot of the data, you want to use once() callback with a value event, not a real-time stream from child_added:
Ref.orderByChild("location").equalTo(searchLocation).once("value", function(snapshot) {
def.resolve(snapshot.val());
});
I have such a problem (structured english)
GET HTTP Resource
FOR every data item received do
GET another HTTP Resource
Alter the original data from the outer loop with data from the inner GET
RETURN altered data
How would you pass the outer data to the inner request, e.g. does not work (in pseudo code):
GET HTTP (callback function (recDataOUTER){
GET NEW HTTP ((recDataOUTER, recDataINNER){
Alter both data accordingly
})
return altered data
})
Is there a more elegant way? The background is that I am using MongoDB and am struggling with Joins (there are no joins I know).
To verify, you are making a GET request. This request is returning a list of IDs. You then want to loop through those IDs and make a GET request for each of those ids. You then want to pass the data to the parent scope. Is this correct?
$http(...).success(function(data) {
data.forEach(data, function(value, key) {
$http(...).success(function(childData) {
//Save childData to parent data object
data[key] = childData.whatever;
});
})
});
In this situation you should be able to simply alter the data variable directly from within the child HTTP callback function above. You shouldn't have any scope issues within the child callback function.
As an aside, it is usually best practice to request all of the data in a single call, rather than making a series of GET calls. I don't know the specifics of your project here, but you may reconsider your approach.
I have a series of methods I run using async.waterfall which then return a result. I store that result into a request.session variable because I use Ajax and work with the result later on. However, I find that I can set the value of the session variable, but later on I cannot retrieve it.
Here's the code from the waterfall.
async.waterfall([
//Some functions
], function(err, result) {
request.session.cases = result;
response.render('template.jade', {
items : result
});
I then perform a jquery load to another url which then filters cases in request.session.cases.
But, I find that request.session.cases is undefined!
app.get('/case/:bugid', function(request, response) {
console.log(request.session.cases); //undefined
}
The weird part is that I can set request.session.cases to anything other than result (I can even set it to result[0]) and it will be fine. I've also printed out the value of result before I do the Ajax call and it is indeed populated with an array of objects.
cookieSession has a very small size of data that you can store into it. The data I was trying to store was too big for a cookieSession. Therefore I switched to using a session for that type of data, and using a normal cookie for user login.