I am currently working an API/router which is to be integrated in the user's own web server. My code will do its required work but also allow the user to implement their own logic afterwards such as in this snippet: myApi.on('action', (err, result, req, res) => {/*user's logic here*/}).
By design, my code will not end the response if the user decides to override the default behavior and implement their own logic. This is to allow the request and response to fall through to the next middleware function if desired by the user.
The control flow I desire is:
Run my initial route logic
Run the user's custom logic
Pass control to the next middleware function if response hasn't ended
My issue arises when the user wants to implement any async logic. I have no way of allowing the user's custom logic to conclude before passing control onto the next middleware function. I believe it is because of the interface I have set up. I desire as little busywork as possible to be performed by the user. This means I would prefer not passing the user some equivalent of a callback() or next() function which they would need to call at every endpoint of their logic (as my router involves a number of different routes).
I've looked into promise wrappers such as promisify-node, but they seem only capable of wrapping functions which take a callback as their final argument. I've also considered requiring the user return a promise from their custom implementation, but that goes against my desire of not requiring busywork/boilerplate code.
Is there any way using promises -- or any other construct -- I can deal with this async issue without having to pain the user?
No, you cannot promisify a function that does not take a callback. Such a function either isn't asynchronous or doesn't provide you any way to know when it's finished - you're out of luck.
My issue arises when the user wants to implement any async logic.
Just require the user to pass a function that returns a promise if he wants to implement something asynchronous. It will probably do him a favour having to work with promises anyway :-)
const myApi = {
on(name, fn) {
Promise.resolve(result) // or whenever you need to run the user logic
.then(result =>
fn(req, result) // or whatever you want to pass
)
.then(customresult => {
res.write(customresult); // or whatever you want to do with its result
});
}
};
myApi.on('action', (req, result) => {
// user's logic here
// throw exceptions
// return values
// return promises for values
// or whatever the user needs to do
})
If I understand the question you would inspect the value the users' custom function returns. If it is a promise-like object then you can wait for it to resolve before moving on. This is equivalent to passing a next function to the users' custom middle ware.
In your example the user(s) would use your API like so:
myApi.on('action', (err, result, req, res) => {
return myPromiseAwareAsyncFunction();
})
And in your code:
let userMiddleWare = lookupNextActionHandler();
let result = userMiddleWare(err, result, req, res);
// Wrap any value (promise or not) in a promise
Promise.resolve(result)
.then(result => {
// Move on to next middle ware
})
.catch(error => {
console.error(error);
process.exit(1); // Or what ever you use
// to end the middle ware chain.
});
If you wish to offer both promises and a next callback you could do something like this:
let userMiddleWare = lookupNextActionHandler();
new Promise((resolve, reject) => {
function next(err) {
if (err) {
reject(err);
} else {
resolve();
}
}
let result = userMiddleWare(err, result, req, res, next);
Promise.resolve(result).then(resolve).catch(reject);
})
.then(result => {
// Move on to next middle ware
})
.catch(error => {
console.error(error);
process.exit(1); // Or what ever you use
// to end the middle ware chain.
});
Since promises can only handle one resolution and ignores further incantations after set this will work and which ever the user uses (next or return a Promise) will work.
Related
I've created a code snippet here that fetches data about a certain country using REST Countries API. The function works fine, and returns a pending promise. That promise's fulfilled value will equal an object containing key value pairs for the country's capital, name, and code.
But if I wanted to use them for something, why wouldn't I just set that very same object equal to a variable and continue to program new actions inside my async function? Why bother trying to use values gained asynchronously on the global scope?
function getJSON(url, errorMSG = 'Something went wrong') {
// feed it the fetch URL, then your customized error message
return fetch(url).then(response => {
if (!response.ok) throw new Error(errorMSG);
return response.json();
});
}
async function countryData(nation) {
try {
const info = await getJSON(
`https://restcountries.eu/rest/v2/name/${nation}?fullText=true`,
'Invalid country selected'
);
return {
// Fullfilled value of promise
capital: info[0].capital,
name: info[0].name,
code: info[0].cioc,
};
} catch (err) {
console.error(err); // display your custom error message
}
}
console.log(countryData('Canada'));
fetch is an async function. Why do they resolve the promise to a response object, instead of continuing to run extra actions once they have it? Because they don't know what you want to do with it. It would be impossible for fetch to handle every possible thing that should happen next, so it just has a single job: get the data, then return it (in a promise). You can then combine this with whatever other code you like.
On a smaller scale, this may happen with countryData too. You might have 10 different parts of your app that want to do things with the result from countryData. It may not be "impossible" for countryData to implement all 10 things, but it's definitely impractical and not a good idea. Instead, countryData can be written to have one job: get the country data and return it. Then each of the 10 pieces of code can do their own things with the result.
This isn't about it being async, the same principles apply to synchronous code to. If you can keep code focused on a single task, without entangling it with the needs of other pieces of code, then your code becomes easier to maintain.
I know one must not block the loop, I know use call backs, and I know ES6 await. The more I research this the more it reaffirms it.
But sometimes your hands are tied. Is there a way to tell JavaScript, please go check on your event queue, and service those, then come back here before continuing execution.
Something like inspired by the MDN docs:
if (queue.hasNextMessage()) {
queue.processNextMessage()
}
There are similar threads about use datetime to wait a duration, but I don't know how the long other event thread will take, been looking at polling the promise status, but it appears to be a dead end.
The context is I have to override a validation callback. The caller of the callback does not wait for a promise to resolve (That I cant change).
Here is the test setup showing the concept. I have made a few attempts, but none of them work because they are always stuck in the main loop.
// The validate function depends on a fetch call which takes time.
// Free to change this.
function validate() {
return fetch(url).then(response => response.json())
.then(data => {console.log(data); return true;})
.catch(msg => {console.log(msg); return false;})
}
// Cannot change this function, I am not in control of it
function CallValidate() {
console.log("Validation Result: ", Boolean(validate()));
}
// This is the setup for when test passes
let url = 'http://api.open-notify.org/astros.json';
CallValidate();
// This is the setup for when test fails
// This currently fails because the promise objects is being evaluated
// to true, instead of waiting for its response.
url = 'http://DUMMY.NOT.WORKING.URL';
CallValidate();
Is there a way to tell JavaScript, please go check on your event queue, and service those, then come back here before continuing execution.
No, but you can do something very similar: divide your computation into several tasks. In Node.js, you can use setImmediate() to queue a task. In the browser, you can use messages, as outlined by this answer.
Example:
function setImmediate(callback) {
const channel = new MessageChannel();
channel.port1.onmessage = () => {
callback();
};
channel.port2.postMessage("");
}
console.log("Task 1");
setImmediate( () => {
console.log("Task 2");
});
Boolean(validate()) will always evaluate to true, because validate returns a Promise which is a truthy value no matter what it resolves to.
If you can't change CallValidate, you might still be able to lift up the fetch call. But I'm not sure if that's an option for you.
async function validateUrl(url) {
const valid = await fetch(url).then(response => response.json())
.then(data => {console.log(data); return true;})
.catch(msg => {console.log(msg); return false;});
function validate() {
return valid;
}
function CallValidate() {
console.log("Validation Result: ", Boolean(validate()));
}
CallValidate();
}
// assuming you are in async function context
await validateUrl('http://api.open-notify.org/astros.json');
await validateUrl('http://DUMMY.NOT.WORKING.URL');
If moving the function is also no option I would ditch the CallValidate code and either write something yourself, or pick another library/helper/etc that does handle asynchronous functions (with promise return values).
I have a router that runs several middlewares. So it looks like this.
router.route("/").get(something, something_else, one_more_thing)
the functions something, something_else, one_more_thing are all async. and "one_more_thing" returns a response.
does this mean the entire router.route() is async. Or do i need to switch it to something like,
router.get('/', async (req, res) => {
try {
await something
await something_else
results = await one_more_thing
res.json(results)
} catch (err) {
res.send(err);
}
});
UPDATE:: Adding actual code
Here is my router.route call
router.route('/clock_in').post(timeClockValidator, isManager, startingOnTime, isClockedIn, clockIn);
Here is timeClockValidator
export const timeClockValidator = async (req, res, next) => {
logger.info('validating time clock json.');
let schema = '';
let results = { errors: [] };
req.originalUrl === '/api/timeclock/clock_in'
? (schema = time_clock_schema)
: (schema = schema);
try {
results = await v.validate(req.body, schema);
} catch (error) {
logger.error(error);
return res.status(400).json('Error");
}
return results.errors.length !== 0
? (logger.error(results.errors),
res.status(400).json("Error")
: next();
};
The rest of the functions are set up the same way.
nodejs router.route contains all async middle ware, is the router itself async?
Yes, if any part of any middleware or handler involved in processing an incoming request is asynchronous, then the result will be delivered asynchronously (on some future tick of the event loop). "Asynchronous" is often referred to as "infectious" in that once anything in a chain of code is asynchronous, then the whole result becomes asynchronous because once any part of the chain is delayed to a future tick of the event loop by any part of the chain, then the final result will always be delayed to some future tick of the event loop.
It's kind of like taking a trip that involves getting on four different trains. If one train breaks down and you're stranded for 24 hours, then you're going to be at least 24 hours late on all subsequent trains in your trip too.
There's a difference between a middleware function that uses asynchronous operations and a plain async function that isn't specifically designed to be a middleware function.
To do something like:
router.route("/").get(something, something_else, one_more_thing)
Both something and something_else have to accept arguments such as (req, res, next) and they have to call next() when they are done in order to continue on to the next handler in the list for this route. They can be either synchronous or asynchronous as long as they either call next() when they are done to advance to the next middleware or send a response and stop further routing by not calling next().
That requires a completely different function than running things like this:
router.get('/', async (req, res) => {
try {
await something
await something_else
results = await one_more_thing
res.json(results)
} catch (err) {
res.send(err);
}
});
If something() and something_else() return promises as this implies, then you cannot use these functions in the first form - you would only be able to use them like this.
So, this is why I strongly dislike these theoretical questions without pseudo-code and no real code (they are less productive for everyone). If you show us some ACTUAL code for something() and something_else(), then we could see exactly how they are written and could advise you either how to fix them to work in one of these forms or could advise exactly which form they work with.
I'm trying to use Promise.all() with an array of Promises that is being populated inside of a foreach looop right before but it seems like the Promise.all() is not waiting the promises to be all completed before executing its callback.
What's wrong with the following code? (I tried to simplify it before posting, so parts of it might not make complete sense, but that promised and loops are all there).
class test {
constructor(sql) {
Promise.all([this.sync(sql, 0), this.sync(sql, 1)]).then((data) => {
console.log(data);
});
}
sync(sql, id = 0) {
return new Promise((resolve, reject) => {
request.get('http://localhost/test/' + id, {
json: true
}, (req, res) => {
var promises = [];
res.body['items'].forEach(item => {
promises.push(new Promise((resolve, reject) => {
this.existingRecord(sql, item['id']).then(() => {
resolve(false);
}).catch(() => {
this.add(sql, item).then(resolve(id));
})
}))
});
Promise.all(promises).then((data) => resolve(data));
});
});
}
add(sql, data) {
return new Promise((resolve, reject) => {
console.log('Inserting ' + data['id']);
var request = new sql.Request();
var query = `INSERT INTO test (col1, col2) VALUES (${utils.prepareInsertdata(data)})`;
request.query(query, (err, result) => {
if (err) {
console.log('ERROR INSERTING: ' + data['id']);
console.log(err);
}
resolve();
});
});
}
}
First off, you're making it much harder to write good, clean, error-free code when you have a mix of promises and regular callbacks in your control flow. I find that the best way to write asynchronous code using promises is to first take any asynchronous operations that are not promise based and create promise-based wrappers for them and then write my logic and control flow using only promises. This makes a consistent path for flow of control and for error handling and it removes the mess of promisifying things from the actual main logic.
Then, I see several significant issues in your code.
Asynchronous operations in the constructor
It's almost never a good idea to put asynchronous operations in the constructor. This is because the constructor HAS to return the object itself so that leaves no simple way to communicate back to the code that created your object when the asynchronous operations are actually done and if they succeeded of failed. It is not entirely clear to me what you're trying to accomplish with those async operations, but this is likely a bad design pattern. I favor a factory function that returns a promise that resolves to the new object for combining the creation of an object with asynchronous operations. This gives you everything you need, a fully formed object, knowledge of when the async operations are done and an ability to have error handling for the async operations. You can see more about this factory function option and some other design options here:
Asynchronous operations in constructor
Improve .then() handler construction
When you do this:
this.add(sql, item).then(resolve(id));
You are calling resolve(id) immediately and passing that to .then() rather than waiting for the .then() handler to be called before calling resolve(id). All of this is complicated because you're mixing regular callbacks and promises.
Creating new wrapped promises rather than just returning existing promises
This is related to your mix of regular callbacks and regular promises, but you'd much rather just return an existing promise than wrap it in a new promise that you have to manually resolve and reject. More than half the time, you will miss proper error handling when manually wrapping things in a new promise and it just results in more code than is needed.
Race Conditions
In any sort of multi-user database environment, you can't write database code such as:
if (record exists) {
do one thing
} else {
create new record
}
This is a race condition. If some other database request comes in during the processing of this, it could change the database in the middle of this and you'd be trying to create a record that just got created by another piece of code.
The usual solution varies by database (and you don't say exactly which database library you're using). Usually, you want to let the database manage the creation of unique records making it so that a duplicate record (by whatever key you're managing uniqueness in this table by) isn't allowed in the database and the concurrency of that is managed by the database itself. Some databases have an atomic operation such as findOrCreate() that will find an existing record or create a new one in an atomic fashion. Other databases have other approaches. But, it's important to make sure that adding unique records to the database is an atomic operation that can't ever create unwanted duplicates.
I'd suggest this implementation:
// use promise version of request library (already promisified for us)
const rp = require('request-promise');
class test {
constructor() {
}
init(sql) {
return Promise.all([this.sync(sql, 0), this.sync(sql, 1)]).then((data) => {
console.log(data);
// do something with the data here - probably store it in instance data
});
}
sync(sql, id = 0) {
return rp.get('http://localhost/test/' + id, {json: true}).then(res => {
// process all items
return Promise.all(res.body.items.map(item => {
return this.existingRecord(sql, item.id).then(() => {
return false;
}).catch(() => {
// it's probably bad form here to do this for all possible database errors
// probably this should be looking for a specific error of id not found
// or something like that.
// This is also likely a race condition. You would typically avoid the race
// condition by making the item key unique in the database and just doing an add and letting
// the database tell you the add failed because the item already exists
// This will allow the database to control the concurrency and avoid race conditions
return this.add(sql, item);
});
}));
});
}
}
// factory function that returns promise that resolves to a new object
// don't use new test() elsewhere
function createTestObj(sql) {
let t = new test();
return t.init(sql).then(() => {
// resolve to our new object
return t;
});
}
For your add() method, I'd switch to using the promise interface in your sql database. There should either be one built-in or a 3rd party package that will add one on top of your database interface. This will prevent the manual creation of promises and the incomplete error handling in your add() method.
What is the best way to determine if the subscriber has finished executing or better yet return something and catch it up-stream? For example:
this._subscriptions.push(this._client
.getCommandStream(this._command) // Returns an IObservable from a Subject stream
.subscribe(msg => {
// Do some processing maybe some promise stuff
http.request(url).then(
// some more stuff
);
});
What's the best know to determine that subscription has finished. I've implemented it as follows:
this._subscriptions.push(this._client
.getCommandStream(this._command)
.subscribe(msg => {
// Do some processing maybe some promise stuff
http.request(url).then(re => {
// some more stuff
msg.done()
}).catch(err => msg.done(err));
});
i.e. added a done method to the object being passed in to determine if this is finished. The issue with that is I'll have to call done in every promise or catch block and find that a little too exhaustive. Is there a cleaner and more automated way of doing this?
I think the examples I've given are not good enough. This implementation is using RX to build an internal messaging bus. The get command stream is actually returning a read-only channel (as an Observable) to get commands and process them. Now the processing could be a http request followed by many other things or just an if statement.
this._client
.getCommandStream(this._command) // Returns an IObservable from a Subject stream
.subscribe(msg => {
// Do some processing maybe some promise stuff
http.request(url).then({
// some more stuff
}).then({
// Here I wanna do some file io
if(x) {
file.read('path', (content) => {
msg.reply(content);
msg.done();
});
} else {
// Or maybe not do a file io or maybe even do some image processing
msg.reply("pong");
msg.done()
}
});
});
I feel like this is a fine usage of the Observable pattern as this is exactly a sequence of commands coming in and this logic would like to act on them. The question is notice msg.done() being called all over the place. I want to know what is the best way to limit that call and know when the entire thing is done. Another option is to wrap it all in a Promise but then again what's the difference between resolve or msg.done()?
Actually, making another asynchronous request inside subscribe() isn't recommended because it just makes things more complicated and using Rx in this way doesn't help you make your code more understandable.
Since you need to make a request to a remote service that returns a PRomise you can merge it into the chain:
this._subscriptions.push(this._client
.getCommandStream(this._command)
.concatMap(msg => http.request(url))
.subscribe(...)
Also the 3rd parameter to subscribe is a callback that is called when the source Observable completes.
You can also add your own teardown logic when the chain is being disposed. This is called after the complete callback in subscribe(...) is called:
const subscription = this._subscriptions.push(this._client
...
.subscribe(...)
subscription.add(() => doWhatever())
Btw, this is equivalent to using the finally() operator.
As per RxJs subscribe method documentation, the last Argument is completed function
var source = Rx.Observable.range(0, 3)
var subscription = source.subscribe(
function (x) {
console.log('Next: %s', x);
},
function (err) {
console.log('Error: %s', err);
},
function () {
console.log('Completed');
});
please refer this documentation
https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/subscribe.md