I'm trying to validate certain URLs based on their status code. I tried to do something like this:
const http = require('http');
let link = 'http://source.unsplash.com/random/100x100';
const getStatusCode = (link, callback) => {
http.get(link, res => {
callback(res.statusCode);
})
}
const codeCallback = code => {
console.log('Status code inside callback: ', code); //works, displays 301
return code
}
let statusCode = getStatusCode(link, codeCallback)
console.log(statusCode) //undefined
if (statusCode) {...} //do something
Is there a way to store status code or should I implement validation logic inside callback?
Welcome to the callback hell ;)
The way you're using it - yes, you need to do your validation logic inside the callback.
If you decide to explore this area further, I'd suggest looking promises and async/await in JavaScript.
Good luck!
Your code is a good example of a typical problem that arises when asynchronous calls happen in javascript. Specifically, the problem is that you are trying to print a variable before its value has been made available.
To explain more, all the http requests are asynchronous in nature due to the fact that they should not be blocking your application while you are waiting for a response from a server.
So when you are calling console.log(statusCode), this line gets executed before the http.get callback has been executed, so at the moment of execution the statusCode is undefined.
Indeed, the solution would be to handle the statusCode inside the callback. Something like this.
const http = require('http');
let link = 'http://source.unsplash.com/random/100x100';
const getStatusCode = (link, callback) => {
http.get(link, res => {
if (res.statusCode){
callback(res.statusCode);
}
})
}
const codeCallback = code => {
console.log('Status code inside callback: ', code); //works, displays 301
return code
}
getStatusCode(link, codeCallback);
If you are interested in understanding how the whole asynchronous logic execution works here is a good intro https://www.youtube.com/watch?v=8aGhZQkoFbQ
Related
Brief Project Overview
I am working on a full page caching system using Node and Express. I have this working, however I am trying to add cache "hole-punching", where portions of a document can have different caching policies.
Attempted Approach
I am attempting to achieve this by injecting the rendered output of one Express route into the rendered output of another Express route. Unfortunately, the most popular methods of firing an http request within Node involve asynchronous promises. I have not been able to get promises working in a synchronous request because... well, they don't work that way.
From what I can tell, I am left with 2 options:
Make the entire route/request/response asynchronous (is this possible?)
Find a synchronous way of rendering an Express route
I don't know how to achieve either of these. I know many folks will say not to do #2 because performance - I'm open to other suggestions.
Code
In the code below, I override the res.render() method to analyze the output of the current response and determine if there is a secondary request that should be made, rendered, and injected.
async function getRemoteInclude(req, content) {
// remote include format: {{ includeRemote('path') }}
const handlebarRegex = '(?<={{)(.*?)(?=}})';
const includeRegex = '(?<=includeRemote[(])(.*?)(?=[)])';
const replaceHandlebarRegex = /\{\{[^\}]+\}\}/g;
let parsedContent = content;
let foundHandlebars = content.match(handlebarRegex); // does string with {{ * }} exist?
if (foundHandlebars) {
let foundInclude = foundHandlebars[0].match(includeRegex); // does string with includeRemote('*') exist?
if (foundInclude) {
const axios = require('axios');
let path = 'http://expresscache.com:3000/test'; // sample hardcoded route
let response = await axios.get(path); // fire async request to express route
return parsedContent.replace(replaceHandlebarRegex, response.body); // replace remote include string with route response
}
}
return content;
}
function interceptRender() {
return function (req, res, next) {
res.render = function (view, options, callback) {
var self = this;
options = options || res.locals;
self.app.render(view, options, function (err, content) {
if (!callback) {
content = getRemoteInclude(req, content); // check if content should be modified with rendered remote include
self.locals.body = content;
self.send(content);
} else {
callback();
}
});
}
next();
}
}
module.exports = {
interceptRender: interceptRender
};
Obviously this code performs an async request through use of the axios node library. I am expecting that this will not be part of the final solution, unless option 1 from above is possible.
Ask
Does anyone have an idea how to either get this async request working in a synchronous context or to modify this request so that it is synchronous itself? Or maybe a completely different approach?
I am creating an HTTP-callable function that makes contact with an API. However, I am getting an error with sending back a 200. I think it has something to do with a mistake I made in using asynchronous functions. Here is my code.
exports.organisationDataToTemp = functions.region('europe-west3').https.onRequest((req, res) => {
res.set('Access-Control-Allow-Origin', '*');
const GETparam = req.query.kvk;
const KvK = GETparam.toString();
//Test if KvK number is already in temp collection
const snapshot = db.collection('temp').where('information.kvkNumber', '==', KvK).get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
//This is where it needs to send the header and it fails. I do get here only when I need this code to run, but it still fails
console.log('kvk number already in temp collection');
res.status(200).send(doc.id);
return;
});
});
//Irrelevant code
//Make API call using the provided KvK number
const keyName = 'VitozFMIS';
const API_key = 'sdfghjqwertyuiopdfghytrdcvbjftyujnbvc';
//Call the first JSON
const _EXTERNAL_URL = 'https://api.kvk.nl/api/v2/testprofile/companies?kvkNumber=' + KvK + '&' + keyName + '=' + API_key + '&startPage=1';
fetch(_EXTERNAL_URL)
.then(response => response.json())
.then(data => {
const total = data.data.totalItems;
for(n=1;n<=total;n++){
const API = 'https://api.kvk.nl/api/v2/testprofile/companies?kvkNumber=' + KvK + '&' + keyName + '=' + API_key + '&startPage=' + n;
//irrelevant code
//Make the API call
fetch(API)
.then(resp => resp.json())
.then(data => {
//irrelevant code
});
}
});
//Return 200 if no errors occured
res.status(200).send(cleanupID);
return;
});
Normally, the code runs exactly as needed, but when the kvk number is already in the collection, it needs to send the document ID back in a 200. I am not sure, but I think it is because it sends another code before this one, but I do not understand why it fails. Does someone know what fails?
This line of code is always going to immediately return a 200 response, before anything else happens:
res.status(200).send(cleanupID);
That's because the Firestore and fetch APIs you're using are asynchronous and return immediately. The callbacks you provide execute some time later, after the results are available. Calling then does not block your code from continuing - it just establishes a callback to be run when the promise resolves.
Cloud Functions requires that sending the response must be the very last thing your function does, so you should wait until all of the work fully succeeds or fails before sending the response.
You will need to structure your code to use the callbacks from the promises to decide what to do. All of the logic much be inside the callback, or deferred to another callback by returning another promise. I strongly suggest finding a tutorial to learn how this works. It's essential for writing effective JavaScript. It's also highly valuable to learn async/await syntax to make promises easier to work with.
Im' trying to understand using promises with Google Cloud Functions a bit better. I just learned about the 'finally' method on promises, which is called after all promises in the chain are fully resolved or rejected. In a http function is it good practice to put response.send() inside of the finally method?
The below code uses request-promise-native for the http request. In the first .then() I call parseSchedule, which uses the cheerio web scraping api to loop through some data and on a website, and add it to the scheduledGames array (synchronously, I think).
I return from that and the then log that data to the console in writeDB, but one thing I noticed is that I see response.send() log 'execution finished' before I see the data from scheduleGames in the log. Is that correct?
Should I be using the 'finally' block like this?
Thanks,
const options = {
uri: 'https://www.cbssports.com/nba/schedule/' + urlDate,
Connection: 'keep-alive',
transform: function (body) {
return cheerio.load(body);
}
};
return request(options)
.then(parseSchedule)
.then(writeSchedule)
.catch((err) => console.log("there was an error: " + err))
.finally(res.send("execution finished"));
function parseSchedule($){
const scheduledGames = [];
$('tbody').children('tr').each((i, element) => {
const gameTime = $(element).children('td').eq(2).find('a').text()
const scheduledGame = { gameTime: gameTime};
scheduledGames.push(scheduledGame);
});
return scheduledGames;
}
function writeDB(scheduledGames){
console.log(scheduledGames);
}
}
It typically makes more sense to send a success response at the time in the promise chain when everything is successful, or send an error response in a catch handler. If you do these two things, it doesn't make sense to use finally at all, since success and error are the only two cases you really need to handle. Unless you have some special case, stick to just success and error.
So i've been doing some reading and I think I have a general grasp on this subject but could use some insight from someone more experienced. I've been trying to write a simple RSS reader in Meteor and have been facing some issues with calling the Meteor method asynchronously. I currently define the method on the server(synchronously) and call it on the client(asynchronously). What I don't understand is that when I try to make the HTTP.call on the server, I return an undefined value passed to my client if I pass a callback into the request. But when I make the API request synchronously everything seems to work fine. Is this the normal behavior I should expect/the way I should be making the API call?
Meteor.methods({
getSubReddit(subreddit) {
this.unblock();
const url = 'http://www.reddit.com/r/' + subreddit + '/.rss';
const response = HTTP.get(url, {}, (err, res) => {
if(!err) {
//console.log(res.content);
return res;
} else {
return err;
}
});
}
});
Here's the method defined on the server side. Note that logging res.content shows that I'm actually getting the right content back from the call. I've tried reading some other answers on the topic and seen some things about using Future/wrapAsync, but I'm not sure I get it. Any help would be greatly appreciated!
The HTTP.get is doing async work, so callback passed to it will be called out of this meteor method call context.
To get desired result you should do it like this:
Meteor.methods({
getSubReddit(subreddit) {
// IMPORTANT: unblock methods call queue
this.unblock();
const url = 'http://www.reddit.com/r/' + subreddit + '/.rss';
const httpGetSync = Meteor.wrapAsync(HTTP.get);
try {
const response = httpGetSync(url, {});
//console.log(response.content);
return response.content;
} catch (err) {
// pass error to client
throw new Meteor.Error(...);
}
}
});
Question: Would you consider dangling callbacks as bad node.js style or even dangerous? If so under which premise?
Case: as described below, imagine you need to make calls to a DB in an express server that updates some data. Yet the client doesn't need to be informed about the result. In this case you could return a response immediately, not waiting for the asynchronous call to complete. This would be described as dangling callback for lack of a better name.
Why is this interesting?: Because tutorials and documentation in most cases show the case of waiting, in worst cases teaching callback hell. Recall your first experiences with say express, mongodb and passport.
Example:
'use strict'
const assert = require('assert')
const express = require('express')
const app = express()
function longOperation (value, cb) {
// might fail and: return cb(err) ...here
setTimeout(() => {
// after some time invokes the callback
return cb(null, value)
}, 4000)
}
app.get('/ping', function (req, res) {
// do some declartions here
//
// do some request processesing here
// call a long op, such as a DB call here.
// however the client does not need to be
// informed about the result of the operation
longOperation(1, (err, val) => {
assert(!err)
assert(val === 1)
console.log('...fired callback here though')
return
})
console.log('sending response here...')
return res.send('Hello!')
})
let server = app.listen(3000, function () {
console.log('Starting test:')
})
Yeah, this is basically what called a "fire and forget" service in other contexts, and could also be the first step in a good design implementing command-query response separation.
I don't consider it a "dangling callback", the response in this case acknowledges that the request was received. Your best bet here would be to make sure your response includes some kind of hypermedia that lets clients get the status of their request later, and if it's an error they can fix have the content at the new resource URL tell them how.
Think of it in the case of a user registration workflow where the user has to be approved by an admin, or has to confirm their email before getting access.