Develop Reference

Wait for response from request before returning

I am trying to create a function with a GET request that returns a portion of the data from the GET request. However, it keeps returning before the data is retrieved, so I keep getting "undefined". How can I set this up so it actually waits for the data to be set before returning?
let getInfo = async () => {
const request = net.request({
url: URL
})
return new Promise((resolve, reject) => { // Promise being here DOES work
request.on('response', (response) => {
response.on('data', (chunk) => {
//return new Promise((resolve, reject) => { //Promise being here does NOT work
let body = JSON.parse(chunk)
let info = body.data
if (info){
resolve(info);
}
reject();
//})
});
});
request.write('')
request.end()
}).then(data => {
console.log("From then: "+data)
return data
})
}
getInfo().then(data => {
console.log("From outside: "+data)
})
Edit: This is the updated version that still does not work. I am trying to use the native electron method and I don't see why this doesn't work. The "From then:" part displays the info correctly. But when run "From outside:" it prints undefined. Does the issue have anything to do with the response.on being nested inside the request.on?
Solution: As #NidhinDavid showed in his answer, the issue was that the promise was inside the 'response' listener. Moving the 'GET' request from start to finish inside the Promise fixed it to giving the correct output. I have updated my code to reflect that for future individuals.

let getInfo = () => {
let info;
const request = net.request({
url: URL
})
return new Promise((resolve, reject) => {
request.on('response', (response) => {
response.on('data', (chunk) => {
request.write('')
request.end()
let body = JSON.parse(chunk)
info = body.data
if (info) {
resolve(info)
} else {
reject('Something went wrong');
}
});
});
})
}
getInfo()
.then(data => {
// this will be your info object
console.log(data)
})
.catch(err => {
// this will log 'Something went wrong' in case of any error
console.log(err)
})
You need to return inside your, on type event handler. Read more about asynchronous code and synchronous code here

I couldn't find the net module and the one which is included with Nodejs do not have request method. So to get the similar concept of event emiters and promise I am using http module and doing a http request to fetch json and parse it
'use strict'
var https = require('https');
const getInfo = async () => {
// create a new promise chain
// remember it is a chain, if one return is omitted
// then the chain is broken
return new Promise((resolve, reject) => {
var options = {
host: 'support.oneskyapp.com',
path: '/hc/en-us/article_attachments/202761727/example_2.json'
};
// start the request
https.request(options, function (response) {
var str = '';
// data arrives in chunks
// chunks needs to be stitched together before parsing
response.on('data', function (chunk) {
str += chunk;
});
// response body obtained
// resolve (aka return) the result
// or parse it, or do whatever you want with it
response.on('end', function () {
resolve(str)
});
// errors are another event
// listen for errors and reject when they are encountered
response.on('error', function (err) {
reject(err)
})
}).end()
})
}
//*********************************************
// using async await
//*********************************************
// if this is the entry point into app
// then top-level async approach required
(async ()=>{
try{
let data = await getInfo()
console.log("From ASYNC AWAIT ")
console.log(JSON.stringify(JSON.parse(data)))
}
catch (err) {
console.log("operation failed, error: ", err)
}
})();
//************************************************
// using promise chains
//************************************************
getInfo()
.then((data)=>{
console.log("FROM PROMISE CHAIN ")
console.log(JSON.stringify(JSON.parse(data)))
})
.catch((err)=>{
console.log("operation failed, error: ", err)
})

Tyr this, it might works for you,
let info;
const getInfo = async (_url)=>{
const response = await fetch(_url);
const data = await response.json();
info = data;
} ;
const url = "some url";
getInfo(url);
console.log(info);
Async function always returns a promise, so either consume that promise or internally await the data and assign it to some variable.
Check for the valid data required in info by logging it to the console.

Javascript/Node/Express: res.json needs to wait for a function to finish running before returning... but res.json is impatient?

Hello friends! I hope you are well.
So in this snippet of code, I have a list of ten servers I want to query. These ten servers are represented by the ten ports I have defined up top in the const "ports".
The idea is to use forEach() on "ports", and run the query on each server. An object is returned by each run of the query and added into the initially empty array "data".
THEN! After "data" has been loaded with the ten objects containing the status of my servers, I want it dished back to the client side!
However..... this is not what happens. Port.forEach() runs the query just fine, and adds into the array "data"... but not before res.json jumps the gun and sends off an empty data back to my soon to be disappointed client.
So far... I've tried callbacks and async/await... but I haven't figured out the syntax for it.. Any tips would be awesome! Thank you for your time friendos!
module.exports = app => {
app.get("/api/seMapServerStatus", (req, res) => {
const ports = [ "27111", "27112", "27117", "27118", "27119", "27110", "27115", "27116", "27113", "27114" ]
const data = []
function hitServers(port){
Gamedig.query({
type: "aGameType",
host: "theServer'sIP",
port: port
}).then((state) => {
data.push(state)
console.log("this is the server", state)
}).catch((error) => {
console.log("Server is offline");
});
};
ports.forEach(port => {
hitServers(port)
})
});
console.log("and here is the final server list", data)
res.json(data);
}

The above code executes synchronously, therefore you return in the same frame before any promise has the chance to resolve.
We can clean the above code up as follows:
module.exports = app => {
app.get("/api/seMapServerStatus", (req, res) => {
const ports = ["27111", "27112", "27117", "27118", "27119", "27110", "27115", "27116", "27113", "27114"]
function hitServers(port) {
return Gamedig.query({
type: "aGameType",
host: "theServer'sIP",
port: port
})
}
// With error handling
function hitServersSafe(port) {
return hitServers(port)
.then(result => {
return {
success: true,
result: result
}
})
.catch(error => {
return {
success: false,
// you probably need to serialize error
error: error
}
})
}
const promises = ports.map(port => hitServers(port))
// With error handling
// const promises = ports.map(port => hitServersSafe(port))
Promise
.all(promises)
.then(data => res.json(data))
.catch(error => {
// do something with error
})
})
}
We map each port to a promise. After we have a list of X promises we wait for all of them to finish.
The calling Promise.all() returns an array of resolved values, or rejects when any promise rejects.
Only after all of the promises have resolved we can move on and send the result to the client.

Multiple paginated GET API calls in parallel/async in Node

I am making call to the bitbucket API to get all the files that are in a repo. I have reached to a point where I can get the list of all the folders in the repo and make the first API call to all the root folders in the repo in parallel and get the the list of first 1000 files for all folders.
But the problem is bitbucket api can give me only 1000 files per folder at a time.
I need to append a query param &start =nextPageStart and make the call again, until it is null and isLastPage is true per API. How can I achieve that with below code??
I get the nextPageStart from first call to the api. See the API response below.
Below is the code that I have so far.
Any help or guidance is appreciated.
Response from individual API thats called per folder.
{
"values": [
"/src/js/abc.js",
"/src/js/efg.js",
"/src/js/ffg.js",
...
],
"size": 1000,
"isLastPage": false,
"start": 0,
"limit": 1000,
"nextPageStart": 1000
}
function where i made asynchronous calls to get the list of files
export function getFilesList() {
const foldersURL: any[] = [];
getFoldersFromRepo().then((response) => {
const values = response.values;
values.forEach((value: any) => {
//creating API URL for each folder in the repo
const URL = 'https://bitbucket.abc.com/stash/rest/api/latest/projects/'
+ value.project.key + '/repos/' + value.slug + '/files?limit=1000';
foldersURL.push(URL);
});
return foldersURL;
}).then((res) => {
// console.log('Calling all the URLS in parallel');
async.map(res, (link, callback) => {
const options = {
url: link,
auth: {
password: 'password',
username: 'username',
},
};
request(options, (error, response, body) => {
// TODO: How do I make the get call again so that i can paginate and append the response to the body till the last page.
callback(error, body);
});
}, (err, results) => {
console.log('In err, results function');
if (err) {
return console.log(err);
}
//Consolidated results after all API calls.
console.log('results', results);
});
})
.catch((error) => error);
}

I was able to get it working be creating a function with callback.
export function getFilesList() {
const foldersURL: any[] = [];
getFoldersFromRepo().then((response) => {
const values = response.values;
values.forEach((value: any) => {
//creating API URL for each folder in the repo
const URL = 'https://bitbucket.abc.com/stash/rest/api/latest/projects/'
+ value.project.key + '/repos/' + value.slug + '/files?limit=1000';
foldersURL.push(URL);
});
return foldersURL;
}).then((res) => {
// console.log('Calling all the URLS in parallel');
async.map(res, (link, callback) => {
const options = {
url: link,
auth: {
password: 'password',
username: 'username',
},
};
const myarray = [];
// This function will consolidate response till the last Page per API.
consolidatePaginatedResponse(options, link, myarray, callback);
}, (err, results) => {
console.log('In err, results function');
if (err) {
return console.log(err);
}
//Consolidated results after all API calls.
console.log('results', results);
});
})
.catch((error) => error);
}
function consolidatePaginatedResponse(options, link, myarray, callback) {
request(options, (error, response, body) => {
const content = JSON.parse(body);
content.link = options.url;
myarray.push(content);
if (content.isLastPage === false) {
options.url = link + '&start=' + content.nextPageStart;
consolidatePaginatedResponse(options, link, myarray, callback);
} else {
// Final response after consolidation per API
callback(error, JSON.stringify(myarray));
}
});
}

I think the best way is to wrap it in a old school for loop (forEach doesn't work with async, since it's synchronous and it will cause all the requests to be spawn at the same time).
What I understood is that you do some sort of booting query where you get the values array and then you should iterate among the pages. Here some code, I didn't fully grasp the APIs so I'll give a simplified (and hopefully readable) answer, you should be able to adapt it:
export async function getFilesList() {
logger.info(`Fetching all the available values ...`);
await getFoldersFromRepo().then( async values => {
logger.info("... Folders values fetched.");
for (let i = 0; ; i++ ) {
logger.info( `Working on page ${i}`);
try {
// if you are using TypeScript, the result is not the promise but the succeeded value already
const pageResult: PageResult = await yourPagePromise(i);
if (pageResult.isLastPage) {
break;
}
} catch(err) {
console.err(`Error on page ${i}`, err);
break;
}
}
logger.info("Done.");
});
logger.info(`All finished!`);
}
The logic behind is that first getFoldersFromRepo() returns a promise which returns the values, and then I sequentially iterate on all available pages through the yourPagePromise function (which returns a promise). The async/await construct allows to write more readable code, rather then having a waterfall of then().
I'm not sure it respects your APIs specs, but it's the logic you can use as foundation! ^^

How to define a function which returns Promise to a Express Route function?

I have a business leve database module called "db_location" which uses the node-fetch module to get some data from a remote server via REST API.
**db_location.js** DB LOGIC
const p_conf = require('../parse_config');
const db_location = {
getLocations: function() {
fetch(`${p_conf.SERVER_URL}/parse` + '/classes/GCUR_LOCATION', { method: 'GET', headers: {
'X-Parse-Application-Id': 'APPLICATION_ID',
'X-Parse-REST-API-Key': 'restAPIKey'
}})
.then( res1 => {
//console.log("res1.json(): " + res1.json());
return res1;
})
.catch((error) => {
console.log(error);
return Promise.reject(new Error(error));
})
}
};
module.exports = db_location
I would need to call this function within a Route function so as to separate database processing from controller.
**locations.js** ROUTE
var path = require('path');
var express = require('express');
var fetch = require('node-fetch');
var router = express.Router();
const db_location = require('../db/db_location');
/* GET route root page. */
router.get('/', function(req, res, next) {
db_location.getLocations()
.then(res1 => res1.json())
.then(json => res.send(json["results"]))
.catch((err) => {
console.log(err);
return next(err);
})
});
When I ran http://localhost:3000/locations, I received the following error.
Cannot read property 'then' of undefined
TypeError: Cannot read property 'then' of undefined
It seems the Promise was empty or something wrong down the Promise chain going from one response object to another? What is a best practise for solving this kind of scenario?
EDIT 1
If I changed the getLocations to return res1.json() (which I think is a non-empty Promise according to the node-fetch documentation):
fetch(`${p_conf.SERVER_URL}/parse` + '/classes/GCUR_LOCATION', { method: 'GET', headers: {
'X-Parse-Application-Id': 'APPLICATION_ID',
'X-Parse-REST-API-Key': 'restAPIKey'
}})
.then( res1 => {
return res1.json(); // Not empty as it can be logged to `Promise Object`
})
.catch((error) => {
console.log(error);
return Promise.reject(new Error(error));
})
And the route code was changed to :
db_location.getLocations()
.then(json => res.send(json["results"]))
.catch((err) => {
console.log(err);
return next(err);
})
The exactly same error was thrown.

You need getLocations to return a Promise. At the moment, it's running a fetch, but that fetch isn't connected to anything else, and getLocations is returning undefined (and of course you can't call .then on uundefined)
Instead, change to:
const db_location = {
getLocations: function() {
return fetch( ...
Also, since you're not doing anything special in the getLocations catch block, you might consider omitting it entirely and let the caller handle it.

Your function doesn't return anything.
If you want to use a promise, you need return it.

jest everything after expect isn't called

my jest is not working as I expect it. See:
const res = {
send: (content) => {
expect(content).toEqual({
app_status: 501,
errors: {
jwt: {
location: 'body',
param: 'jwt',
value: undefined,
msg: 'The jwt is required'
}
}
});
console.log("after expect");
done();
},
};
Basically EVERYTHING after the expect(content).toEqual ... in res.send is not called. I find that very confusing. I am getting no error except for that my test's are taking too long (because done) is not called and the test is not "closed". So my question is, am I missing something obviously?

The following should work fine. I added asynchronous since send may be called asynchronously:
const createResponse = () => {
var resolve;
const p = new Promise(
(r,reject)=>resolve=r
);
return [
{
send: (value) => {
resolve(value)
}
},
p
];
};
test('(async) fail', done => {
//Router
const router = express.Router();
//Endpoint to fetch version
router.get('/api/version', (req, res) => {
setTimeout(x=>res.send('v1'),10)
});
const request = {
method: 'GET'
};
let [response,p] = createResponse()
router.stack[0].handle(request, response, () => {});
p.then(
x=>expect(x).toBe('v2'),
reject=>expect("should not reject").toBe(reject)
).then(
x=>done()
//,x=>done() //this will cause all tests to pass
);
});
To answer your question in the comment; consider the following code:
const express = require('express');
const router = express.Router();
//Endpoint to fetch version
router.get('/api/version', (req, res) => {
res.send("Hello World");
});
const request = {
method: 'GET'
};
const response = {
send: (value) => {
debugger;//pause here and see the stack
throw new Error("Hello Error.");
}
};
router.stack[0].handle(
request,
response,
//this is the function done in route.js:
// https://github.com/expressjs/express/blob/master/lib/router/route.js#L127
err => {
console.error(err.message);//console errors Hello Error.
}
);
Send throws an error but send is called by your mock which is called by express here. So express catches the exception and then ends up here (done is not from just but your callback).
So it'll call your callback with an error, skipping done from jist and not throwing anything (maybe showing something in log). Since your callback doesn't do anything it times out.
You could try to call done from jist in the callback (at console.error(err.message);).
[UPDATE]
Careful trying to catch the error thrown by expect the following will tell me 1 test passed:
test('(async) fail', done => {
try{
expect(true).toBe(false);
}catch(e){
}
done();
});