I use readable and transform streams which I later consume using for await.
I cannot find a way to process callee's stream errors so they can be caught in the caller function.
For example if transform throws, it results in uncaught error.
If I add on error listener to transform, naturally it doesn't propagate the error to main's catch.
function handler(err) {
// Log the error here
// If throw from here it won't be cauch in main
// How do I propagate it to main?
}
function getStream() {
// transform1 and transform2 are custom transform streams
readable.pipe(csvParser).pipe(transform1).pipe(transform2);
readable.on('error', handler);
csvParser.on('error', handler);
transform1.on('error', handler);
transform2.on('error', handler);
return transform2;
}
async function main() {
try {
const stream = getStream();
for await(const chunk of stream) {
// process chunk
}
} catch (ex) {
// how to catch transform errors here?
}
}
Is there any way to do it?
I ended up solving this with Promise.race:
function readStream(stream) {
return Promise.race([
new Promise((_, reject) => stream.once('error', reject)),
iterate(stream)
]);
}
async function iterate(stream) {
for await (const object of stream) {
// ...
}
}
This will reject with any error from either the stream or the iterate function, or resolve with the result of iterate, whichever comes first.
As I understand it async/await is largely syntactic sugar on top of promises. Specifically I think that since ReadableStream.pipe follows an event-based (on("error", errorHandler)) rather than promise-based pattern, the try { ... await ... } catch (ex) { ... } construction isn't going to handle asynchronous errors "thrown" within ReadableStream.pipe as seamlessly as you might hope.
Assuming my understanding is correct (and to be honest I don't use the aysnc/await syntax often, so it's possible I'm missing something too) one straightforward work-around is to fall back to adding a callback-based event handler like this:
function handleError(err) { ... }
readable.once("error", handleError);
// and if you want, re-use that handler within your catch block, like:
// `} catch (ex) { handleError(ex); }`
// but I'm not sure how often that will come up if you follow this pattern
but you may already be aware of that.
Failing that, if you really want to use the aysnc/await/try/catch-style construction in this case you could use something like util.promisify to convert the on("error", handler)-event-based API that ReadableStream is using to promises. This StackOverflow post - How to use Async await using util promisify? - covers that topic in more depth but (IMO) that seems like a lot of hoops to jump through just to avoid adding readable.once("error" /* ...whatever you'd otherwise have in your catch block ... */)
In short, I think because ReadableStream.pipe isn't designed around promises, the async/await syntax isn't enough (in and of itself) to ensure that the asynchronous errors that might be emitted as on-error events are trapped by the try/catch block. You need to handle those errors in the old-school way, either directly (by registering a handler for the error events emitted by your readable, in which case the await and try/catch stuff isn't directly applicable) or "indirectly" (by creating an adapter that makes those emitted-events bubble up like the catch case on a resolved promise, in which case you can make it look like a synchronous-style try/catch using async/await).
I think a transform stream function should not throw an exception. Instead it should emit an error event or pass the error to the call back function.
Here is an example.
Edit
Wrap everything in the transform method with a try catch block. And it propagates to the main function.
const { Transform } = require("stream");
//Added on 1st edit
function handler(ex) {
console.error("Logged By Handler: ", ex);
}
function getStream() {
// A simple transform stream to test
const transform = new Transform({
transform(chunk, encoding, callback) {
try {
chunk = chunk.toString();
// if chunk == "err" then we want to throw an error
// to simulate a real life error
if (chunk === "err\n")
return callback("Oops! Sth failed in the transform stream.", null);
// or this.emit("error", "Oops! Sth failed in the transform stream.");
this.push(chunk.toUpperCase());
// Simulating exception
throw new Error(`Fatal error.`);
callback();
} catch (ex) {
handler(ex);
callback(ex, null);
}
},
});
process.stdin.pipe(transform);
// readable -> transform
return transform;
}
async function main(transform) {
try {
for await (const chunk of transform) process.stdout.write(chunk);
} catch (ex) {
console.error("Handled by main:", ex);
}
}
main(getStream());
Related
I'm facing a promise rejection that escapes a try catch block. The rejection causes an Uncaught exception which is something I can not comprehend.
If I add a reject handler in the Promise.then or a Promise.catch, the rejection gets captured. But I was hopping try catch will work in this situation.
What is happening here?
class HttpResponse {
json() {
return Promise.reject('parse error')
}
}
function fetch() {
return new HttpResponse();
}
const res = fetch();
try {
res.json().then(json => {
alert(`json: ${json}`);
}
//,reason => {
// alert(`promise.reject ${reason}`);
//}
)
//.catch(reason => {
// alert(`promise.catch ${reason}`);
//})
} catch (e) {
alert(`catch{} from try catch: ${e}`);
}
Promises have their own mechanism of handling errors with the catch() method. A try / catch block can't control what's happening when chaining methods.
function fetch() {
return Promise.reject('parse error');
}
const res = fetch();
res.then(json => {
console.log(`json: ${json}`);
}).catch((e) => {
console.log(`catch{} from chained catch: ${e}`);
});
However, using async / await changes that. There you don't use the methods of a promise but handle errors with a try / catch block.
function fetch() {
return Promise.reject('parse error');
}
(async function() {
try {
const res = await fetch();
} catch (e) {
console.log(`catch{} from try catch: ${e}`);
}
})();
The technical reason behind this behavior is because a JavaScript promise invokes one of two callback functions for success or failure.
A promise does not emit an Error that is required for the try to work. it is not an Error (or more technically accurate) an instance of Error. It emits and event. You are encouraged to emit a new Error() if you need it to emit one. As pointed out here: https:developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…
It emits an event that you can set up a handler for: https://developer.mozilla.org/en-US/docs/Web/API/PromiseRejectionEvent –
Finally, await throws and Error as described in the spec: https://tc39.es/ecma262/#await-rejected
Technical reasons behind:
HostPromiseRejectionTracker is an implementation-defined abstract
operation that allows host environments to track promise rejections.
An implementation of HostPromiseRejectionTracker must complete
normally in all cases. The default implementation of
HostPromiseRejectionTracker is to unconditionally return an empty
normal completion.
https://www.ecma-international.org/ecma-262/10.0/index.html#sec-host-promise-rejection-tracker
Basically javascript engines can freely implement this spec. In the case of browsers you don't get the Error captured inside the try/catch because the error is not emitted where you think it should be. But instead it's tracked with this special event that throws the error in the console.
Also on nodejs, this event causes the process to exit if you have node set to exit on unhandled exceptions.
On the other side, if you instead use async/await, the rejection is treated like an 'error' in practical terms. Meaning that the newer async/await feature behaves in a different fashion showing that it is not only syntactic sugar for Promises.
https://tc39.es/ecma262/#sec-throwcompletion
In sum, if you use Promise.then you are forced to provide a reject argument or chain it with .catch to have the rejection captured or else it will reach the console and in case of nodejs to exit the process if configured to do so (I believe new nodejs does this by default).
But if you use the newer async/await syntax you not only have a concise code (which is secondary) but a better rejection handling because it can be properly nested in a try/catch and it will behave like an standard Error.
I've been seeing a couple of different patterns for handling errors.
One is like:
let result;
try {
result = await forSomeResult(param);
} catch {
throw new Error();
}
And the other is like:
const result = await forSomeResult(param).catch(() => throw new Error());
I prefer the latter since it looks like a cleaner solution. But I've also heard talks that the first solution is better since there could be some race condition where the .catch doesn't execute before the next command is run.
I was wondering if someone had a technical answer about why one method is better than the other.
First of all, there's no reason to catch and throw or .catch() and throw unless you're going to do something else inside the catch handler or throw a different error. If you just want the same error thrown and aren't doing anything else, then you can just skip the catch or .catch() and let the original promise rejection go back to the caller.
Then, it is generally not recommended that you mix await with .catch() as the code is not as easy to follow. If you want to catch an exception from await, use try/catch around it. .catch() works, it's just not a preferred style if you're already awaiting the promise.
The one technical difference between these two styles:
async function someFunc()
let x = await fn().catch(err => {
console.log(err);
throw err;
});
// do something else with x
return something;
}
And, this:
async function someFunc()
let x;
try {
x = await fn();
} catch(err) {
console.log(err);
throw err;
}
// do something else with x
return something;
}
is if the function fn() you are calling throws synchronously (which it shouldn't by design, but could by accident), then the try/catch option will catch that synchronous exception too, but the .catch() will not. Because it's in an async function, the async function will catch the synchronous exception and turn it into a rejected promise for you automatically for the caller to see as a rejected promise, but it wouldn't get logged or handled in your .catch() handler.
One of the more beneficial cases for try/catch with await is when you have multiple await statements and you don't need to handle errors on any of them individually. Then, you can surround them with one try/catch and catch all the errors in one place.
async function someFunc(someFile) {
let fileHandle;
try {
// three await statements in a row, all using same try/catch
fileHandle = await fsp.open(someFile);
let data = await fsp.read(...);
// modify data
await fsp.write(...)
} catch(err) {
// just log and re-throw
console.log(err);
throw err;
} finally {
// close file handle
if (fileHandle) {
await fileHandle.close();
}
}
}
It depends.
Are you going to be calling multiple asynchronous functions that could error? You can wrap them all in a try/catch and perform common error handling without having to repeat yourself:
try {
result = await forSomeResult(param);
await forSomeOtherResult();
return await finalResult(result);
} catch { //catches all three at once!
throw new Error();
}
Do you only need to handle errors for this one, specific call? The .catch() pattern is fine. There is no race condition to worry about, await waits for the promise to resolve or reject, and this includes all success and failure callbacks attached to the promise. However, in your example, you're catching an error only to throw an empty one - in that case, it may be preferable to simply write this:
const result = await forSomeResult(param);
...and let the error propagate to the caller naturally.
I've seen a mixture of both styles used common enough that I think it's fine either way - they each have a particular strength.
From node doc:
A handful of typically asynchronous methods in the Node.js API may
still use the throw mechanism to raise exceptions that must be handled
using try / catch. There is no comprehensive list of such methods;
please refer to the documentation of each method to determine the
appropriate error handling mechanism required.
Can someone bring example of such function which is async and still throws? How and when do you catch the exception then?
More particularly. Do they refer to such function:
try
{
obj.someAsync("param", function(data){
console.log(data);
});
}catch(e)
{
}
Now normally I know above doesn't make sense -because when the callback executes, try block could have been already exited.
But what kind of example does the excerpt from documentation refer to? If async method throws as they say it, where, when and how should I handle it? (or maybe if you show such function can you show where in its doc it says how to handle it as mentioned on the quote?)
The async methods like the one from your example usually throw for programmer errors like bad parameters and they call the callback with error for operational errors.
But there are also async functions in ES2017 (declared with async function) and those signal errors by rejecting the promise that they return - which turn into a thrown exception when you use them with await keyword.
Examples:
function x(arg, cb) {
if (!arg) throw new Error('Programmer error: bad arguments');
setTimeout(() => {
cb(new Error('Operational error: something bad happened'));
}, 2000);
}
Now when you use it you usually don't want to handle programmer errors - you want to fix them. So you don't do this:
try {
x();
} catch (err) {
// Why should I handle the case of bad invocation
// instead of fixing it?
}
And the operational errors you handle like this:
x(function (err) {
if (err) {
// handle error
} else {
// success
}
});
Now, if you have a function that doesn't take a callback but returns a promise:
function y() {
return new Promise((res, rej) => setTimeout(() => rej('error'), 2000));
}
Then you handle the error like this:
y().catch(error => {
// handle error
});
or, while using await:
try {
await y();
} catch (err) {
// handle error
}
For more info on the difference between operational errors and programmer errors see:
Best Practices for Error Handling in Node.js by Dave Pacheco
Error Handling in Node.js
For more info on the promises and async/await see the links in this answer.
afaik there are three ways a async function could "throw"; and how to catch each of these:
as any other function (aka. someone messed up): I'd not catch these cases because they should not be in my code, and catching such errors makes it harder to find and fix em.
function foo(){
//someone messed up, better fixing than catching this
return new Prooooooooooomise((resolve) => 42);
}
try {
foo();
}catch(err){
console.error(err);
}
Promises:
function foo(){ return Promise.resolve('bar') }
foo().then(value => value =========> 'error')
.catch(err => {
console.error(err);
return "fixedValue";
});
And Nodes callback syntax/pattern:
function foo(value, callback){
setTimeout(function(){
if(Math.random() < .5){
callback("I don't like to", undefined);
}else{
callback(null, value * 2);
}
}, 500);
}
foo(21, function(err, data){
if(err){
//no try..catch at all
console.error(err);
}else{
//do whatever with data
}
})
These are the most common async errors you'll come along; well, the first one is just a plain bug in an async mothod.
I'm writing a JavaScript function that makes an HTTP request and returns a promise for the result (but this question applies equally for a callback-based implementation).
If I know immediately that the arguments supplied for the function are invalid, should the function throw synchronously, or should it return a rejected promise (or, if you prefer, invoke callback with an Error instance)?
How important is it that an async function should always behave in an async manner, particularly for error conditions? Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
e.g:
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
throw new Error('userId is not valid')
}
// make async call
}
// OR...
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
return cb(new Error('userId is not valid'))
}
// make async call
}
Ultimately the decision to synchronously throw or not is up to you, and you will likely find people who argue either side. The important thing is to document the behavior and maintain consistency in the behavior.
My opinion on the matter is that your second option - passing the error into the callback - seems more elegant. Otherwise you end up with code that looks like this:
try {
getUserById(7, function (response) {
if (response.isSuccess) {
//Success case
} else {
//Failure case
}
});
} catch (error) {
//Other failure case
}
The control flow here is slightly confusing.
It seems like it would be better to have a single if / else if / else structure in the callback and forgo the surrounding try / catch.
This is largely a matter of opinion. Whatever you do, do it consistently, and document it clearly.
One objective piece of information I can give you is that this was the subject of much discussion in the design of JavaScript's async functions, which as you may know implicitly return promises for their work. You may also know that the part of an async function prior to the first await or return is synchronous; it only becomes asynchronous at the point it awaits or returns.
TC39 decided in the end that even errors thrown in the synchronous part of an async function should reject its promise rather than raising a synchronous error. For example:
async function someAsyncStuff() {
return 21;
}
async function example() {
console.log("synchronous part of function");
throw new Error("failed");
const x = await someAsyncStuff();
return x * 2;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
There you can see that even though throw new Error("failed") is in the synchronous part of the function, it rejects the promise rather than raising a synchronous error.
That's true even for things that happen before the first statement in the function body, such as determining the default value for a missing function parameter:
async function someAsyncStuff() {
return 21;
}
async function example(p = blah()) {
console.log("synchronous part of function");
throw new Error("failed");
const x = await Promise.resolve(42);
return x;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
That fails because it tries to call blah, which doesn't exist, when it runs the code to get the default value for the p parameter I didn't supply in the call. As you can see, even that rejects the promise rather than throwing a synchronous error.
TC39 could have gone the other way, and had the synchronous part raise a synchronous error, like this non-async function does:
async function someAsyncStuff() {
return 21;
}
function example() {
console.log("synchronous part of function");
throw new Error("failed");
return someAsyncStuff().then(x => x * 2);
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
But they decided, after discussion, on consistent promise rejection instead.
So that's one concrete piece of information to consider in your decision about how you should handle this in your own non-async functions that do asynchronous work.
How important is it that an async function should always behave in an async manner, particularly for error conditions?
Very important.
Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
Yes, I personally think it is OK when that is a very different error from any asynchronously produced ones, and needs to be handled separately anyway.
If some userids are known to be invalid because they're not numeric, and some are will be rejected on the server (eg because they're already taken) you should consistently make an (async!) callback for both cases. If the async errors would only arise from network problems etc, you might signal them differently.
You always may throw when an "unexpected" error arises. If you demand valid userids, you might throw on invalid ones. If you want to anticipate invalid ones and expect the caller to handle them, you should use a "unified" error route which would be the callback/rejected promise for an async function.
And to repeat #Timothy: You should always document the behavior and maintain consistency in the behavior.
Callback APIs ideally shouldn't throw but they do throw because it's very hard to avoid since you have to have try catch literally everywhere. Remember that throwing error explicitly by throw is not required for a function to throw. Another thing that adds to this is that the user callback can easily throw too, for example calling JSON.parse without try catch.
So this is what the code would look like that behaves according to these ideals:
readFile("file.json", function(err, val) {
if (err) {
console.error("unable to read file");
}
else {
try {
val = JSON.parse(val);
console.log(val.success);
}
catch(e) {
console.error("invalid json in file");
}
}
});
Having to use 2 different error handling mechanisms is really inconvenient, so if you don't want your program to be a fragile house of cards (by not writing any try catch ever) you should use promises which unify all exception handling under a single mechanism:
readFile("file.json").then(JSON.parse).then(function(val) {
console.log(val.success);
})
.catch(SyntaxError, function(e) {
console.error("invalid json in file");
})
.catch(function(e){
console.error("unable to read file")
})
Ideally you would have a multi-layer architecture like controllers, services, etc. If you do validations in services, throw immediately and have a catch block in your controller to catch the error format it and send an appropriate http error code. This way you can centralize all bad request handling logic. If you handle each case youll end up writing more code. But thats just how I would do it. Depends on your use case
I'm writing a JavaScript function that makes an HTTP request and returns a promise for the result (but this question applies equally for a callback-based implementation).
If I know immediately that the arguments supplied for the function are invalid, should the function throw synchronously, or should it return a rejected promise (or, if you prefer, invoke callback with an Error instance)?
How important is it that an async function should always behave in an async manner, particularly for error conditions? Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
e.g:
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
throw new Error('userId is not valid')
}
// make async call
}
// OR...
function getUserById(userId, cb) {
if (userId !== parseInt(userId)) {
return cb(new Error('userId is not valid'))
}
// make async call
}
Ultimately the decision to synchronously throw or not is up to you, and you will likely find people who argue either side. The important thing is to document the behavior and maintain consistency in the behavior.
My opinion on the matter is that your second option - passing the error into the callback - seems more elegant. Otherwise you end up with code that looks like this:
try {
getUserById(7, function (response) {
if (response.isSuccess) {
//Success case
} else {
//Failure case
}
});
} catch (error) {
//Other failure case
}
The control flow here is slightly confusing.
It seems like it would be better to have a single if / else if / else structure in the callback and forgo the surrounding try / catch.
This is largely a matter of opinion. Whatever you do, do it consistently, and document it clearly.
One objective piece of information I can give you is that this was the subject of much discussion in the design of JavaScript's async functions, which as you may know implicitly return promises for their work. You may also know that the part of an async function prior to the first await or return is synchronous; it only becomes asynchronous at the point it awaits or returns.
TC39 decided in the end that even errors thrown in the synchronous part of an async function should reject its promise rather than raising a synchronous error. For example:
async function someAsyncStuff() {
return 21;
}
async function example() {
console.log("synchronous part of function");
throw new Error("failed");
const x = await someAsyncStuff();
return x * 2;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
There you can see that even though throw new Error("failed") is in the synchronous part of the function, it rejects the promise rather than raising a synchronous error.
That's true even for things that happen before the first statement in the function body, such as determining the default value for a missing function parameter:
async function someAsyncStuff() {
return 21;
}
async function example(p = blah()) {
console.log("synchronous part of function");
throw new Error("failed");
const x = await Promise.resolve(42);
return x;
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
That fails because it tries to call blah, which doesn't exist, when it runs the code to get the default value for the p parameter I didn't supply in the call. As you can see, even that rejects the promise rather than throwing a synchronous error.
TC39 could have gone the other way, and had the synchronous part raise a synchronous error, like this non-async function does:
async function someAsyncStuff() {
return 21;
}
function example() {
console.log("synchronous part of function");
throw new Error("failed");
return someAsyncStuff().then(x => x * 2);
}
try {
console.log("before call");
example().catch(e => { console.log("asynchronous:", e.message); });
console.log("after call");
} catch (e) {
console.log("synchronous:", e.message);
}
But they decided, after discussion, on consistent promise rejection instead.
So that's one concrete piece of information to consider in your decision about how you should handle this in your own non-async functions that do asynchronous work.
How important is it that an async function should always behave in an async manner, particularly for error conditions?
Very important.
Is it OK to throw if you know that the program is not in a suitable state for the async operation to proceed?
Yes, I personally think it is OK when that is a very different error from any asynchronously produced ones, and needs to be handled separately anyway.
If some userids are known to be invalid because they're not numeric, and some are will be rejected on the server (eg because they're already taken) you should consistently make an (async!) callback for both cases. If the async errors would only arise from network problems etc, you might signal them differently.
You always may throw when an "unexpected" error arises. If you demand valid userids, you might throw on invalid ones. If you want to anticipate invalid ones and expect the caller to handle them, you should use a "unified" error route which would be the callback/rejected promise for an async function.
And to repeat #Timothy: You should always document the behavior and maintain consistency in the behavior.
Callback APIs ideally shouldn't throw but they do throw because it's very hard to avoid since you have to have try catch literally everywhere. Remember that throwing error explicitly by throw is not required for a function to throw. Another thing that adds to this is that the user callback can easily throw too, for example calling JSON.parse without try catch.
So this is what the code would look like that behaves according to these ideals:
readFile("file.json", function(err, val) {
if (err) {
console.error("unable to read file");
}
else {
try {
val = JSON.parse(val);
console.log(val.success);
}
catch(e) {
console.error("invalid json in file");
}
}
});
Having to use 2 different error handling mechanisms is really inconvenient, so if you don't want your program to be a fragile house of cards (by not writing any try catch ever) you should use promises which unify all exception handling under a single mechanism:
readFile("file.json").then(JSON.parse).then(function(val) {
console.log(val.success);
})
.catch(SyntaxError, function(e) {
console.error("invalid json in file");
})
.catch(function(e){
console.error("unable to read file")
})
Ideally you would have a multi-layer architecture like controllers, services, etc. If you do validations in services, throw immediately and have a catch block in your controller to catch the error format it and send an appropriate http error code. This way you can centralize all bad request handling logic. If you handle each case youll end up writing more code. But thats just how I would do it. Depends on your use case