So i've been doing some reading and I think I have a general grasp on this subject but could use some insight from someone more experienced. I've been trying to write a simple RSS reader in Meteor and have been facing some issues with calling the Meteor method asynchronously. I currently define the method on the server(synchronously) and call it on the client(asynchronously). What I don't understand is that when I try to make the HTTP.call on the server, I return an undefined value passed to my client if I pass a callback into the request. But when I make the API request synchronously everything seems to work fine. Is this the normal behavior I should expect/the way I should be making the API call?
Meteor.methods({
getSubReddit(subreddit) {
this.unblock();
const url = 'http://www.reddit.com/r/' + subreddit + '/.rss';
const response = HTTP.get(url, {}, (err, res) => {
if(!err) {
//console.log(res.content);
return res;
} else {
return err;
}
});
}
});
Here's the method defined on the server side. Note that logging res.content shows that I'm actually getting the right content back from the call. I've tried reading some other answers on the topic and seen some things about using Future/wrapAsync, but I'm not sure I get it. Any help would be greatly appreciated!
The HTTP.get is doing async work, so callback passed to it will be called out of this meteor method call context.
To get desired result you should do it like this:
Meteor.methods({
getSubReddit(subreddit) {
// IMPORTANT: unblock methods call queue
this.unblock();
const url = 'http://www.reddit.com/r/' + subreddit + '/.rss';
const httpGetSync = Meteor.wrapAsync(HTTP.get);
try {
const response = httpGetSync(url, {});
//console.log(response.content);
return response.content;
} catch (err) {
// pass error to client
throw new Meteor.Error(...);
}
}
});
Related
I am experimenting with t library, and am trying to install the functionality into my own search bar to practice (still a student).
Can anyone provide advice to the format this would be in??
#PascalLamers answer is the cleanest way to go about it, using the "await" pattern.
To see what this looks like with promise chaining:
app.get('/:url', async function (req, res) {
return wappalyzer
.init()
.then(function(){
return wappalyzer.open(decodeURIComponent(req.params.url))
})
.then(function(site){
return site.analyze()
})
.then(function(data){
return res.status(200).json(data);
})
.catch(function(err){
return res.status(500).json({ message : err.message })
})
}
Each then executes a single async operation, and on success passes its result to the next then() operation.
Each then() is only called upon the success of the previous then(). If an error occurs at any point, the catch() is executed and the function returns (in this case, returns an error 500).
Completely ignoring what wappalyzer actually is or does, I would suggest the following, since you are already providing an async function as callback to your route controller :
app.get('/:url', async function (req, res) {
try {
await wappalyzer.init();
const site = await wappalyzer.open(decodeURIComponent(req.query.url));
const data = await site.analyze();
// if you end up here, everything was successfull
return res.status(200).json(data);
} catch (ex) {
// if you end up here, something went wrong
return res.status(500).json({ message : ex.message });
}
});
The above doesn't really make much sense, since you are telling your backend to look for an url param but using a query parameter instead. I woudln't recommend to send an url as either, param or query. Use the request body instead :
// receive request in backend, with an endpoint that makes a bit more sense
// also using POST method, otherwise you won't be able to send a body
app.post('/search', async function (req, res) {
const url = req.body.url;
// do stuff
});
// sending request to backend, using axios for example
const respond = await axios.post('/search', { url : 'www.google.com' });
// or using fetch
const respond = await fetch('/search', {
method: 'post',
body: JSON.stringify({ url : 'www.google.com' });
Please be aware these are just pointers, nothing to copy & paste :) .
If you are using Express, I suggest reading the documentation again. I think you are misunderstanding how the router works : https://expressjs.com/de/guide/routing.html#route-parameters
Im' trying to understand using promises with Google Cloud Functions a bit better. I just learned about the 'finally' method on promises, which is called after all promises in the chain are fully resolved or rejected. In a http function is it good practice to put response.send() inside of the finally method?
The below code uses request-promise-native for the http request. In the first .then() I call parseSchedule, which uses the cheerio web scraping api to loop through some data and on a website, and add it to the scheduledGames array (synchronously, I think).
I return from that and the then log that data to the console in writeDB, but one thing I noticed is that I see response.send() log 'execution finished' before I see the data from scheduleGames in the log. Is that correct?
Should I be using the 'finally' block like this?
Thanks,
const options = {
uri: 'https://www.cbssports.com/nba/schedule/' + urlDate,
Connection: 'keep-alive',
transform: function (body) {
return cheerio.load(body);
}
};
return request(options)
.then(parseSchedule)
.then(writeSchedule)
.catch((err) => console.log("there was an error: " + err))
.finally(res.send("execution finished"));
function parseSchedule($){
const scheduledGames = [];
$('tbody').children('tr').each((i, element) => {
const gameTime = $(element).children('td').eq(2).find('a').text()
const scheduledGame = { gameTime: gameTime};
scheduledGames.push(scheduledGame);
});
return scheduledGames;
}
function writeDB(scheduledGames){
console.log(scheduledGames);
}
}
It typically makes more sense to send a success response at the time in the promise chain when everything is successful, or send an error response in a catch handler. If you do these two things, it doesn't make sense to use finally at all, since success and error are the only two cases you really need to handle. Unless you have some special case, stick to just success and error.
I am running in to an issue with Firebase cloud functions. I have an onWrite cloud function that triggers a sequence of events. I have a path for requests that the onWrite cloud function is tied to. When that cloud function executes, it deletes the new request for the requests path and pushes the request in to a render path/que that will be used client side for rendering UI elements/data. Once the data has been written to the render path, I call a vanilla javascript function that is not tied to any cloud events. The vanilla javascript function is supposed to reach out to an external API and fetch some data to later be updated on the render object that was pushed in to the render path.
The problem is that the vanilla javascript function never executes. I have been looking all over the web to figure out why this happening but can't seem to figure out why. I am on the Flame plan so outbound api requests should be allowed to my knowledge. Here an example of my code:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const request = require('request');
admin.initializeApp();
exports.requestModule = functions.database.ref('/requests').onWrite((change, context) => {
// Create reference to database
let db = admin.database();
if (context && context.auth && context.auth.uid) {
const afterData = change.after.val();
let uid = context.auth.uid;
let cleanData = afterData[uid];
cleanData.status = "loading";
// Remove the requested module from the requests path
let cleansePath = db.ref('/requests/' + uid);
cleansePath.remove().then((snapshot) => {
return true;
}).catch((error) => {
console.log(error);
return false;
});
// Add requested module to the render path
let renderPath = db.ref('/render/' + uid);
renderPath.push(cleanData).then((snapshot) => {
let val = snapshot.val();
let key = snapshot.key;
// Trigger the get weather api call
getWeather(uid, key, val);
return true;
}).catch((error) => {
console.log(error);
return false;
});
}
});
// Fetches data from external api
function getWeather (uid, key, obj) {
console.log('Fetching weather!');
let db = admin.database();
request('https://api.someweathersite.net/forecast/', (error, response, body) => {
if (!error && Number(response.statusCode) === 200) {
console.log('error:', error);
console.log('statusCode:', response && response.statusCode);
console.log('body:', body);
obj.data = body;
obj.status = 'loaded';
// Set data from api response in render object to be shown client side
let render = db.ref('/render/' + uid + '/' + key );
render.set(obj).then(() => {
return true;
}).catch((error) => {
console.log(error)
return false;
});
}
});
}
The console.log message at the top of the "getWeather" function never executes. I don't think that the "getWeather" function is ever executing.
If I put the api call directly in the onWrite "requestModule" function, the api call will work. However, when it calls an external function it never gets called/works. I basically want to have the "requestModule" function handle all requests and plan to have a module dispatcher that handles which module function/api data should be fetched from. That's why I don't want to keep the api call in the "requestModule" function. Any idea of why this happening or how I can get this working?
getWeather is performing asynchronous work to fetch some data, but it's not returning a promise to indicate when that work is complete. In fact, none of the async work you're performing here is correctly using the promises returned by the various API calls. It's not sufficient to simply use then() on each promise.
You need to keep track of all of the async work, and return a single promise that resolves only after all the work is complete. Otherwise, Cloud Functions may terminate and clean up your function before the work is complete. (Note that it's not deterministic which work may or may not actually complete before forced termination, but the only way to ensure that all work completes is through that single promise you return.)
You may want to watch my tutorials on using promises in Cloud Functions to get a better handle on what you're required to do make your functions work correctly: https://firebase.google.com/docs/functions/video-series/
I'm trying to validate certain URLs based on their status code. I tried to do something like this:
const http = require('http');
let link = 'http://source.unsplash.com/random/100x100';
const getStatusCode = (link, callback) => {
http.get(link, res => {
callback(res.statusCode);
})
}
const codeCallback = code => {
console.log('Status code inside callback: ', code); //works, displays 301
return code
}
let statusCode = getStatusCode(link, codeCallback)
console.log(statusCode) //undefined
if (statusCode) {...} //do something
Is there a way to store status code or should I implement validation logic inside callback?
Welcome to the callback hell ;)
The way you're using it - yes, you need to do your validation logic inside the callback.
If you decide to explore this area further, I'd suggest looking promises and async/await in JavaScript.
Good luck!
Your code is a good example of a typical problem that arises when asynchronous calls happen in javascript. Specifically, the problem is that you are trying to print a variable before its value has been made available.
To explain more, all the http requests are asynchronous in nature due to the fact that they should not be blocking your application while you are waiting for a response from a server.
So when you are calling console.log(statusCode), this line gets executed before the http.get callback has been executed, so at the moment of execution the statusCode is undefined.
Indeed, the solution would be to handle the statusCode inside the callback. Something like this.
const http = require('http');
let link = 'http://source.unsplash.com/random/100x100';
const getStatusCode = (link, callback) => {
http.get(link, res => {
if (res.statusCode){
callback(res.statusCode);
}
})
}
const codeCallback = code => {
console.log('Status code inside callback: ', code); //works, displays 301
return code
}
getStatusCode(link, codeCallback);
If you are interested in understanding how the whole asynchronous logic execution works here is a good intro https://www.youtube.com/watch?v=8aGhZQkoFbQ
Question: Would you consider dangling callbacks as bad node.js style or even dangerous? If so under which premise?
Case: as described below, imagine you need to make calls to a DB in an express server that updates some data. Yet the client doesn't need to be informed about the result. In this case you could return a response immediately, not waiting for the asynchronous call to complete. This would be described as dangling callback for lack of a better name.
Why is this interesting?: Because tutorials and documentation in most cases show the case of waiting, in worst cases teaching callback hell. Recall your first experiences with say express, mongodb and passport.
Example:
'use strict'
const assert = require('assert')
const express = require('express')
const app = express()
function longOperation (value, cb) {
// might fail and: return cb(err) ...here
setTimeout(() => {
// after some time invokes the callback
return cb(null, value)
}, 4000)
}
app.get('/ping', function (req, res) {
// do some declartions here
//
// do some request processesing here
// call a long op, such as a DB call here.
// however the client does not need to be
// informed about the result of the operation
longOperation(1, (err, val) => {
assert(!err)
assert(val === 1)
console.log('...fired callback here though')
return
})
console.log('sending response here...')
return res.send('Hello!')
})
let server = app.listen(3000, function () {
console.log('Starting test:')
})
Yeah, this is basically what called a "fire and forget" service in other contexts, and could also be the first step in a good design implementing command-query response separation.
I don't consider it a "dangling callback", the response in this case acknowledges that the request was received. Your best bet here would be to make sure your response includes some kind of hypermedia that lets clients get the status of their request later, and if it's an error they can fix have the content at the new resource URL tell them how.
Think of it in the case of a user registration workflow where the user has to be approved by an admin, or has to confirm their email before getting access.