In my script I need to make a XMLrequest, get the response from the server (if resolved), parse it and place the parsed result in a variable.
Only later, with this variable I can do lot of stuff.
So basically, I am trying to separate the creation of such parser object from the rest of my code, in order to:
place the above process into a function in an external script which I can call whenever I need.
not having all my code within a .then() method
organize my code into logical files each devoted to a specific task
So I came up with this piece of code (just a sample) to explain my needs.
As I am fairly new to asyncronous programming, would it be fine to do like so, given that the response should be very fast to resovle (or to reject) in my case?
If it's ok, I would then put this inside a separate file to be able to import it and call testing() (or whatever the name will be) from anywhere I need.
function delay(input) {
return new Promise(function(resolve, reject) {
// some async operation here
setTimeout(function() {
// resolve the promise with some value
resolve(input + 10);
}, 500);
});
}
function testing() {delay(5).then(result => {return document.write(result)})};
testing();
EDIT
Ok, so I think I hit the problem thanks to the answer of #DrewReese and the links in the comments.
I am trying to solve this situation: the code above was misleading to understand my point, but I guess tere's no really simple solution.
Look at this code (it's basically the same as the one above, except fot the last three lines):
function delay(input) {
return new Promise(function(resolve, reject) {
// some async operation here
setTimeout(function() {
// resolve the promise with some value
resolve(input + 10);
}, 500);
});
}
function testing() {delay(5).then(result => {return result})};
var test = testing();
document.write(test);
So in this case, I know when I am defining test the output is 'undefined' because the Promise in testing() has not yet resolved.
The problem I'm trying to solve is: is there (if any) a way to define test only when the Promise is resolved without wrapping it in a then() and maybe outputting something different when it's not resolved (like "LOADING...").
I don't know basically if it is possible to check if a variable has a Promise pending and outputting two different values: when it's waiting and when it's resolved/rejected.
Hopefully I was clear enough, otherwise I'll test more and come back with another question if needed.
The whole point of promises (promise chains) is to escape from "nesting hell" by flattening your "chain", so yes, when your promise chain is resolving blocks of code, it is returning a promise which is then-able. I hope this helps illustrate:
someAsyncHttpRequest(parameter) // <- Returns Promise
.then(result => {
// do something with result data, i.e. extract response data,
// mutate it, save it, do something else based on value
...
// You can even return a Promise
return Promise.resolve(mutatedData);
})
.then(newData => { // <- mutadedData passed as parameter
// do new stuff with new data, etc... even reject
let rejectData = processNewData(newData);
return Promise.reject(rejectData);
})
.catch(err => {
console.log('Any caught promise rejection no matter where it came from:', err);
})
.finally(() => {// You can even run this code no matter what});
If you need to use any variable values you set within the chain, then you need to make the outer function async and await on the resolution of the promise chain:
asyncFunction = async (parameter) => {
let asyncValue;
await someAsyncHttpRequest(parameter)
.then(result => {
...
asyncValue = someValue;
...
});
// safe to use asyncValue now
};
So for you:
function delay(input) {
return new Promise(function(resolve, reject) {
// some async operation here
setTimeout(function() {
// resolve the promise with some value
resolve(input + 10);
}, 500);
});
}
**async** function testing() { // Declare this an asynchronous function
let value = **await** delay(5); // Now you can await the resolution of the Promise
console.log(value); // Outputs resolved value 15!
return value; // Just returns resolved Promise
}
var test = testing();
console.log(test);
/** Outputs the Promise!
Promise {<pending>}
__proto__:Promise
[[PromiseStatus]]:"resolved"
[[PromiseValue]]:15
*/
test.then(console.log)); // Outputs returned resolved value, 15
Related
I'm doing some unit testing. The test framework loads a page into an iFrame and then runs assertions against that page. Before each test begins, I create a Promise which sets the iFrame's onload event to call resolve(), sets the iFrame's src, and returns the promise.
So, I can just call loadUrl(url).then(myFunc), and it will wait for the page to load before executing whatever myFunc is.
I use this sort of pattern all over the place in my tests (not just for loading URLs), primarily in order to allow changes to the DOM to happen (e.g. mimick clicking a button, and wait for divs to hide and show).
The downside to this design is that I'm constantly writing anonymous functions with a few lines of code in them. Further, while I have a work-around (QUnit's assert.async()), the test function that defines the promises completes before the promise is run.
I'm wondering if there is any way to get a value from a Promise or wait (block/sleep) until it has resolved, similar to .NET's IAsyncResult.WaitHandle.WaitOne(). I know JavaScript is single-threaded, but I'm hoping that doesn't mean that a function can't yield.
In essence, is there a way to get the following to spit out results in the correct order?
function kickOff() {
return new Promise(function(resolve, reject) {
$("#output").append("start");
setTimeout(function() {
resolve();
}, 1000);
}).then(function() {
$("#output").append(" middle");
return " end";
});
};
function getResultFrom(promise) {
// todo
return " end";
}
var promise = kickOff();
var result = getResultFrom(promise);
$("#output").append(result);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div id="output"></div>
I'm wondering if there is any way to get a value from a Promise or
wait (block/sleep) until it has resolved, similar to .NET's
IAsyncResult.WaitHandle.WaitOne(). I know JavaScript is
single-threaded, but I'm hoping that doesn't mean that a function
can't yield.
The current generation of Javascript in browsers does not have a wait() or sleep() that allows other things to run. So, you simply can't do what you're asking. Instead, it has async operations that will do their thing and then call you when they're done (as you've been using promises for).
Part of this is because of Javascript's single threadedness. If the single thread is spinning, then no other Javascript can execute until that spinning thread is done. ES6 introduces yield and generators which will allow some cooperative tricks like that, but we're quite a ways from being able to use those in a wide swatch of installed browsers (they can be used in some server-side development where you control the JS engine that is being used).
Careful management of promise-based code can control the order of execution for many async operations.
I'm not sure I understand exactly what order you're trying to achieve in your code, but you could do something like this using your existing kickOff() function, and then attaching a .then() handler to it after calling it:
function kickOff() {
return new Promise(function(resolve, reject) {
$("#output").append("start");
setTimeout(function() {
resolve();
}, 1000);
}).then(function() {
$("#output").append(" middle");
return " end";
});
}
kickOff().then(function(result) {
// use the result here
$("#output").append(result);
});
This will return output in a guaranteed order - like this:
start
middle
end
Update in 2018 (three years after this answer was written):
If you either transpile your code or run your code in an environment that supports ES7 features such as async and await, you can now use await to make your code "appear" to wait for the result of a promise. It is still programming with promises. It does still not block all of Javascript, but it does allow you to write sequential operations in a friendlier syntax.
Instead of the ES6 way of doing things:
someFunc().then(someFunc2).then(result => {
// process result here
}).catch(err => {
// process error here
});
You can do this:
// returns a promise
async function wrapperFunc() {
try {
let r1 = await someFunc();
let r2 = await someFunc2(r1);
// now process r2
return someValue; // this will be the resolved value of the returned promise
} catch(e) {
console.log(e);
throw e; // let caller know the promise was rejected with this reason
}
}
wrapperFunc().then(result => {
// got final result
}).catch(err => {
// got error
});
async functions return a promise as soon as the first await is hit inside their function body so to the caller an async function is still non-blocking and the caller must still deal with a returned promise and get the result from that promise. But, inside the async function, you can write more sequential-like code using await on promises. Keep in mind that await only does something useful if you await a promise so in order to use async/await, your asynchronous operations must all be promise-based.
If using ES2016 you can use async and await and do something like:
(async () => {
const data = await fetch(url)
myFunc(data)
}())
If using ES2015 you can use Generators. If you don't like the syntax you can abstract it away using an async utility function as explained here.
If using ES5 you'll probably want a library like Bluebird to give you more control.
Finally, if your runtime supports ES2015 already execution order may be preserved with parallelism using Fetch Injection.
Another option is to use Promise.all to wait for an array of promises to resolve and then act on those.
Code below shows how to wait for all the promises to resolve and then deal with the results once they are all ready (as that seemed to be the objective of the question); Also for illustrative purposes, it shows output during execution (end finishes before middle).
function append_output(suffix, value) {
$("#output_"+suffix).append(value)
}
function kickOff() {
let start = new Promise((resolve, reject) => {
append_output("now", "start")
resolve("start")
})
let middle = new Promise((resolve, reject) => {
setTimeout(() => {
append_output("now", " middle")
resolve(" middle")
}, 1000)
})
let end = new Promise((resolve, reject) => {
append_output("now", " end")
resolve(" end")
})
Promise.all([start, middle, end]).then(results => {
results.forEach(
result => append_output("later", result))
})
}
kickOff()
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
Updated during execution: <div id="output_now"></div>
Updated after all have completed: <div id="output_later"></div>
I have this function:
export async function promisePlay(example: someInterface): Promise<string> {
const data = await fetch(example.url)
return new Promise((resolve, reject) => {
resolve("Lets test this")
console.log("This point shouldnt be reached");
})
}
When I run this I'm seeing the text This point shouldnt be reached in the console output. What I want (and thought) was that the resolve() should return and stop from running further.
Could someone please tell me what I'm fundamentally misunderstanding and how to correct?
Thanks.
resolve is a function in a variable.
You are calling the resolve function and passing the return value of the promise as the parameter.
However, that does not stop execution or break out.
If you want to exit the function, you must use return;.
Calling resolve does stop the pending await command of wherever you called the original function.
Whatever you value you pass to return will not be passed to the variable that resolves the promise and it will not stop the pending promise in the await command.
function test(yourWord)
{
return new Promise((resolve, reject) =>
{
// This will allow the command
// let res = await test("hello");
// to continue executing. It will stop executing on that line
// unless you call resolve or reject
resolve("you said: " + yourWord);
// this will continue to execute
console.log("I'm still alive");
// this string return value will be ignored
return "blah blah";
// this will not be executed since the function has returned
console.log("I'm dead");
});
}
(async () =>
{
// this will pause execution until resolve or reject is called and
// whatever value is passed into resolve() will be assigned to res
let res = await test("hello");
// this will echo out "you said hello"
console.log("res: " + res);
})();
You can return the resolve
export async function promisePlay(example: someInterface): Promise<string> {
const data = await fetch(example.url)
return new Promise((resolve, reject) => {
return resolve("Lets test this")
console.log("This point shouldnt be reached");
})
}
Promise body is synchronous, when you resolve the promise, it won't return(stop executing further statements).
I think you are confused between promises resolve and returning from them.
Consider below example,
new Promise((resolve) => {
console.log("inside promise");
resolve("promise resolved");
console.log("after resolving");
}).then(val => console.log(val));
In the above snippet, the console will print inside promise then after removing. It means it will execute all the statement in the promise first then it will wait for the promise to resolve. If you don't want to execute any statement further resolve, you can add return resolve in the function, cause its a normal function anyhow.
A JS promise has 3 states initial, fulfilled and rejected.
When awaiting a promise, the code waits for the state to change from initial to either rejected or resolved before execution continues.
When you await your promisePlay() function, execution will continue after the await as soon as reject() or resolve() is called.
Your case is special because you have code after you have resolved() within your promise.
It appears that during some experimentation of my own, it appears the JS engine waits until all the code is executed after the resolve(), I suppose its a bad idea to rely on this behaviour, as other answers suggest perhaps you should be return the result you want within in the resolve
Hi I am new to javascript and have recently been learning about promises. I am trying to wait on the result of a network call before moving forward with my logic. To do this I think I need a promise that waits on the network call's result. I have been practicing with promises and noticed some behavior I don't understand.
var mybool = false;
// Promise
var mypromise = new Promise(
function (resolve, reject) {
if (mybool) {
var pvalue = 'PASS';
resolve(pvalue);
} else {
var fvalue = new Error('FAIL');
reject(fvalue);
}
}
);
// call promise
var callpromise = function () {
mypromise
.then(function (fulfilled) {
// yay, you got a new phone
console.log(fulfilled);
})
.catch(function (error) {
// ops, mom don't buy it
console.log(error.message);
});
}
callpromise();
console.log('This will appear first since prev function call is still thinking');
mybool = true;
console.log(mybool);
callpromise();
So as you can see the var mybool determines the output of the promise function. when I run this program this is the output:
"This will appear first since prev function call is still thinking"
true
"FAIL"
"FAIL"
Can someone explain why even after flipping the mybool var the 2ndcallpromise() is still outputting a value I would only expect from mybool = false?
Promises don't get "called", they get created. You create mypromise once, before callpromise gets actually called. By the time, mybool is false. If you want to create a Promise at a specific point in your code, or if you want multiple promises, create the promise inside of a function.
The simplest change to what you're doing is probably to pass mybool into your promise-creator, which as others have pointed out should be a function.
I'm also adding a counter, so that you can see the results which resolve/reject in a different order than their creation.
var mypromise = function(mybool, counter) {
return new Promise(
function (resolve, reject) {
if (mybool) {
var pvalue = 'PASS-' + counter;
resolve(pvalue);
} else {
var fvalue = new Error('FAIL-' + counter);
reject(fvalue);
}
}
)
}
// call promise
var callpromise = function (mybool, counter) {
mypromise(mybool, counter)
.then(function (fulfilled) {
// yay, you got a new phone
console.log(fulfilled);
})
.catch(function (error) {
// ops, mom don't buy it
console.log(error.message);
});
}
callpromise(false, 1);
console.log('This will appear first since prev function call is still thinking');
callpromise(true, 2);
The function ("callback" or "handler" is these are often called) you specify as the first argument to the then function, will be called when the promise "resolves", and the argument to that function (the "callback", not then) is the value you yourself supplied as argument to the resolve call.
Meaning that in your example, fulfilled is "PASS" (the value of pvalue that you specify as argument to resolve). The name fulfilled doesn't seem like a good candidate here, perhaps due to how you understood (incorrectly) promises work. A fitting name for it would be value, or pvalue (the same thing you resolve the promise with).
But the crucial piece of puzzle you're missing is that every time you call then with some callback function, the callback function will execute if the promise is in the "resolved" state (meaning that resolve has already been called earlier).
So you can call then 1000 times, and you will get your callback invoked 1000 times, with that one and the same resolved ("promised" in loose terms) value.
Let's break down your example:
By the time the var mypromise = ...; statement is executed, the mypromise promise is created and resolved -- the Promise constructor immediately invokes the provided function (called "executor" in promise terms), and this executor function in your case immediately calls resolve which puts the promise in the "resolved" state.
callpromise is just a function, which, when executed, essentially causes the user agent to call the function provided as argument to then as soon as and if the mypromise promise is in "resolved" state, which it is already (see 1.)
You call callpromise twice, but the promise was already resolved even before the first call, and so the value of mybool is of absolutely no effect as far as the promised value is concerned -- the promise was already resolved once, and that's it.
Try to add a console.log call in the promise executor function (the one evaluating mybool) and observe how your promise is resolved once, before you even create the callback function.
This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 5 years ago.
I have an es6 class, with an init() method responsible for fetching data, transforming it, then update the class's property this.data with newly transformed data.
So far so good.
The class itself has another getPostById() method, to just do what it sounds like. Here is the code for the class:
class Posts {
constructor(url) {
this.ready = false
this.data = {}
this.url = url
}
async init() {
try {
let res = await fetch( this.url )
if (res.ok) {
let data = await res.json()
// Do bunch of transformation stuff here
this.data = data
this.ready = true
return data
}
}
catch (e) {
console.log(e)
}
}
getPostById(id){
return this.data.find( p => p.id === id )
}
}
Straightforward, except I have an async/await mechanism in the init() method.
Now, this code will work correctly:
let allPosts = new Posts('https://jsonplaceholder.typicode.com/posts')
allPosts.init()
.then( d => console.log(allPosts.getPostById(4)) )
// resulting Object correctly logged in console
but it only gets printed into the console:
How could I use allPosts.getPostById(4) as a return of a function ?
Like:
let myFunc = async () => {
const postId = 4
await allPosts.init() // I need to wait for this to finish before returning
// This is logging correct value
console.log( 'logging: ' + JSON.stringify(allPosts.getPostById( postId ), null, 4) )
// How can I return the RESULT of allPosts.getPostById( postId ) ???
return allPosts.getPostById( postId )
}
myFunc() returns a Promise but not the final value. I have read several related posts on the subject but they all give example of logging, never returning.
Here is a fiddle that includes two ways of handling init(): using Promise and using async/await. No matter what I try, I can't manage to USE the FINAL VALUE of getPostById(id).
The question of this post is: how can I create a function that will RETURN the VALUE of getPostById(id) ?
EDIT:
A lot of good answers trying to explain what Promises are in regards to the main execution loop.
After a lot of videos and other good reads, here is what I understand now:
my function init() correctly returns. However, within the main event loop: it returns a Promise, then it is my job to catch the result of this Promise from within a kinda parallel loop (not a new real thread). In order to catch the result from the parallel loop there are two ways:
use .then( value => doSomethingWithMy(value) )
use let value = await myAsyncFn(). Now here is the foolish hiccup:
await can only be used within an async function :p
thus itself returning a Promise, usable with await which should be embed in an async function, which will be usable with await etc...
This means we cannot really WAIT for a Promise: instead we should catch parallel loop indefinitely: using .then() or async/await.
Thanks for the help !
As for your comment; I'll add it as answer.
The code you write in JavaScript is run on one thread, that means that if your code could actually wait for something it will block any of your other code from getting executed. The event loop of JavaScript is explained very well in this video and if you like to read in this page.
A good example of blocking code in the browser is alert("cannot do anything until you click ok");. Alert blocks everything, the user can't even scroll or click on anything in the page and your code also blocks from executing.
Promise.resolve(22)
.then(x=>alert("blocking")||"Hello World")
.then(
x=>console.log(
"does not resolve untill you click ok on the alert:",
x
)
);
Run that in a console and you see what I mean by blocking.
This creates a problem when you want to do something that takes time. In other frameworks you'd use a thread or processes but there is no such thing in JavaScript (technically there is with web worker and fork in node but that's another story and usually far more complicated than using async api's).
So when you want to make a http request you can use fetch but fetch takes some time to finish and your function should not block (has to return something as fast as possible). This is why fetch returns a promise.
Note that fetch is implemented by browser/node and does run in another thread, only code you write runs in one thread so starting a lot of promises that only run code you write will not speed up anything but calling native async api's in parallel will.
Before promises async code used callbacks or would return an observable object (like XmlHttpRequest) but let's cover promises since you can convert the more traditional code to a promise anyway.
A promise is an object that has a then function (and a bunch of stuff that is sugar for then but does the same), this function takes 2 parameters.
Resolve handler: A function that will be called by the promise when the promise resolves (has no errors and is finished). The function will be passed one argument with the resolve value (for http requests this usually is the response).
Reject handler: A function that will be called by the promise when the promise rejects (has an error). This function will be passed one argument, this is usually the error or reason for rejection (can be a string, number or anything).
Converting callback to promise.
The traditional api's (especially nodejs api's) use callbacks:
traditionalApi(
arg
,function callback(err,value){
err ? handleFail(err) : processValue(value);
}
);
This makes it difficult for the programmer to catch errors or handle the return value in a linear way (from top to bottom). It gets even more impossible to try and do things parallel or throttled parallel with error handling (impossible to read).
You can convert traditional api's to promises with new Promise
const apiAsPromise = arg =>
new Promise(
(resolve,reject)=>
traditionalApi(
arg,
(err,val) => (err) ? reject(err) : resolve(val)
)
)
async await
This is what's called syntax sugar for promises. It makes promise consuming functions look more traditional and easier to read. That is if you like to write traditional code, I would argue that composing small functions is much easier to read. For example, can you guess what this does?:
const handleSearch = search =>
compose([
showLoading,
makeSearchRequest,
processRespose,
hideLoading
])(search)
.then(
undefined,//don't care about the resolve
compose([
showError,
hideLoading
])
);
Anayway; enough ranting. The important part is to understand that async await doesn't actually start another thread, async functions always return a promise and await doesn't actually block or wait. It's syntax sugar for someFn().then(result=>...,error=>...) and looks like:
async someMethod = () =>
//syntax sugar for:
//return someFn().then(result=>...,error=>...)
try{
const result = await someFn();
...
}catch(error){
...
}
}
The examples allways show try catch but you don't need to do that, for example:
var alwaysReject = async () => { throw "Always returns rejected promise"; };
alwaysReject()
.then(
x=>console.log("never happens, doesn't resolve")
,err=>console.warn("got rejected:",err)
);
Any error thrown or await returning a rejected promise will cause the async function to return a rejected promise (unless you try and catch it). Many times it is desirable to just let it fail and have the caller handle errors.
Catching errors could be needed when you want the promise to succeed with a special value for rejected promises so you can handle it later but the promise does not technically reject so will always resolve.
An example is Promise.all, this takes an array of promises and returns a new promise that resolves to an array of resolved values or reject when any one of them rejects. You may just want to get the results of all promises back and filter out the rejected ones:
const Fail = function(details){this.details=details;},
isFail = item => (item && item.constructor)===Fail;
Promise.all(
urls.map(//map array of urls to array of promises that don't reject
url =>
fetch(url)
.then(
undefined,//do not handle resolve yet
//when you handle the reject this ".then" will return
// a promise that RESOLVES to the value returned below (new Fail([url,err]))
err=>new Fail([url,err])
)
)
)
.then(
responses => {
console.log("failed requests:");
console.log(
responses.filter(//only Fail type
isFail
)
);
console.log("resolved requests:");
console.log(
responses.filter(//anything not Fail type
response=>!isFail(response)
)
);
}
);
Your question and the comments suggest you could use a little intuition nudge about the way the event loop works. It really is confusing at first, but after a while it becomes second nature.
Rather than thinking about the FINAL VALUE, think about the fact that you have a single thread and you can't stop it — so you want the FUTURE VALUE -- the value on the next or some future event loop. Everything you write that is not asynchronous is going to happen almost immediately — functions return with some value or undefined immediately. There's nothing you can do about. When you need something asynchronously, you need to setup a system that is ready to deal with the async values when they return sometime in the future. This is what events, callbacks, promises (and async/await) all try to help with. If some data is asynchronous, you simply can not use it in the same event loop.
So what do you do?
If you want a pattern where you create an instance, call init() and then some function that further process it, you simply need to setup a system that does the processing when the data arrives. There are a lot of ways to do this. Here's one way that's a variation on your class:
function someAsync() {
console.log("someAsync called")
return new Promise(resolve => {
setTimeout(() => resolve(Math.random()), 1000)
})
}
class Posts {
constructor(url) {
this.ready = false
this.data = "uninitilized"
this.url = url
}
init() {
this.data = someAsync()
}
time100() {
// it's important to return the promise here
return this.data.then(d => d * 100)
}
}
let p = new Posts()
p.init()
processData(p)
// called twice to illustrate point
processData(p)
async function processData(posts) {
let p = await posts.time100()
console.log("randomin * 100:", p)
}
init() saves the promise returned from someAsync(). someAsync() could be anything that returns a promise. It saves the promise in an instance property. Now you can call then() or use async/await to get the value. It will either immediately return the value if the promise has already resolved or it will deal with it when it has resolved. I called processData(p) twice just to illustrate that it doesn't calle the someAsync() twice.
That's just one pattern. There are a lot more — using events, observables, just using then() directly, or even callbacks which are unfashionable, but still can be useful.
NOTE: Wherever you use await it has to be inside an async function.
Check out the UPDATED FIDDLE
You need to use await myFunc() to get the value you expect from getPostById because an async function always returns a promise.
This sometimes is very frustrating as the whole chain needs to be converted into async functions but that's the price you pay for converting it to a synchronous code, I guess. I am not sure if that can be avoided but am interested in hearing from people who have more experience on this.
Try out the below code in your console by copying over the functions and then accessing final and await final.
NOTE:
An async function CAN contain an await expression.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
There is no rule that is must have await in order to even declare an async function.
The example below uses an async function without await just to show that an async function always returns a promise.
const sample = async () => {
return 100;
}
// sample() WILL RETURN A PROMISE AND NOT 100
// await sample() WILL RETURN 100
const init = async (num) => {
return new Promise((resolve, reject) => {
resolve(num);
});
}
const myFunc = async (num) => {
const k = await init(num);
return k;
}
// const final = myFunc();
// final; This returns a promise
// await final; This returns the number you provided to myFunc
I have a function that reads a directory and copies and creates a new file within that directory.
function createFiles (countryCode) {
fs.readdir('./app/data', (err, directories) => {
if (err) {
console.log(err)
} else {
directories.forEach((directory) => {
fs.readdir(`./app/data/${directory}`, (err, files) => {
if (err) console.log(err)
console.log(`Creating ${countryCode}.yml for ${directory}`)
fs.createReadStream(`./app/data/${directory}/en.yml`).pipe(fs.createWriteStream(`./app/data/${directory}/${countryCode}.yml`))
})
})
}
})
}
How do I do this using promises or Promise All to resolve when it's complete?
First, you need to wrap each file stream in a promise that resolves when the stream emits the finish event:
new Promise((resolve, reject) => {
fs.createReadStream(`./app/data/${directory}/en.yml`).pipe(
fs.createWriteStream(`./app/data/${directory}/${countryCode}.yml`)
).on('finish', resolve);
});
The you need to collect these promises in an array. This is done by using map() instead of forEach() and returning the promise:
var promises = directories.map((directory) => {
...
return new Promise((resolve, reject) => {
fs.createReadStream( ...
...
});
});
Now you have a collection of promises that you can wrap with Promise.all() and use with a handler when all the wrapped promises have resolved:
Promise.all(promises).then(completeFunction);
In recent versions of Node (8.0.0 and later), there's a new util.promisify function you can use to get a promise. Here's how we might use it:
// Of course we'll need to require important modules before doing anything
// else.
const util = require('util')
const fs = require('fs')
// We use the "promisify" function to make calling promisifiedReaddir
// return a promise.
const promisifiedReaddir = util.promisify(fs.readdir)
// (You don't need to name the variable util.promisify promisifiedXYZ -
// you could just do `const readdir = util.promisify(fs.readdir)` - but
// I call it promisifiedReaddir here for clarity.
function createFiles(countryCode) {
// Since we're using our promisified readdir function, we'll be storing
// a Promise inside of the readdirPromise variable..
const readdirPromise = promisifiedReaddir('./app/data')
// ..then we can make something happen when the promise finishes (i.e.
// when we get the list of directories) by using .then():
return readdirPromise.then(directories => {
// (Note that we only get the parameter `directories` here, with no `err`.
// That's because promises have their own way of dealing with errors;
// try looking up on "promise rejection" and "promise error catching".)
// We can't use a forEach loop here, because forEach doesn't know how to
// deal with promises. Instead we'll use a Promise.all with an array of
// promises.
// Using the .map() method is a great way to turn our list of directories
// into a list of promises; read up on "array map" if you aren't sure how
// it works.
const promises = directory.map(directory => {
// Since we want an array of promises, we'll need to `return` a promise
// here. We'll use our promisifiedReaddir function for that; it already
// returns a promise, conveniently.
return promisifiedReaddir(`./app/data/${directory}`).then(files => {
// (For now, let's pretend we have a "copy file" function that returns
// a promise. We'll actually make that function later!)
return copyFile(`./app/data/${directory}/en.yml`, `./app/data/${directory}/${countryCode}.yml`)
})
})
// Now that we've got our array of promises, we actually need to turn them
// into ONE promise, that completes when all of its "children" promises
// are completed. Luckily there's a function in JavaScript that's made to
// do just that - Promise.all:
const allPromise = Promies.all(promises)
// Now if we do a .then() on allPromise, the function we passed to .then()
// would only be called when ALL promises are finished. (The function
// would get an array of all the values in `promises` in order, but since
// we're just copying files, those values are irrelevant. And again, don't
// worry about errors!)
// Since we've made our allPromise which does what we want, we can return
// it, and we're done:
return allPromise
})
}
Okay, but, there's probably still a few things that might be puzzling you..
What about errors? I kept saying that you don't need to worry about them, but it is good to know a little about them. Basically, in promise-terms, when an error happens inside of a util.promisify'd function, we say that that promise rejects. Rejected promises behave mostly the same way you'd expect errors to; they throw an error message and stop whatever promise they're in. So if one of our promisifiedReaddir calls rejects, it'll stop the whole createFiles function.
What about that copyFile function? Well, we have two options:
Use somebody else's function. No need to re-invent the wheel! quickly-copy-file looks to be a good module (plus, it returns a promise, which is useful for us).
Program it ourselves.
Programming it ourselves isn't too hard, actually, but it takes a little bit more than simply using util.promisify:
function copyFile(from, to) {
// Hmm.. we want to copy a file. We already know how to do that in normal
// JavaScript - we'd just use a createReadStream and pipe that into a
// createWriteStream. But we need to return a promise for our code to work
// like we want it to.
// This means we'll have to make our own hand-made promise. Thankfully,
// that's not actually too difficult..
return new Promise((resolve, reject) => {
// Yikes! What's THIS code mean?
// Well, it literally says we're returning a new Promise object, with a
// function given to it as an argument. This function takes two arguments
// of its own: "resolve" and "reject". We'll look at them separately
// (but maybe you can guess what they mean already!).
// We do still need to create our read and write streams like we always do
// when copying files:
const readStream = fs.createReadStream(from)
const writeStream = fs.createWriteStream(to)
// And we need to pipe the read stream into the write stream (again, as
// usual):
readStream.pipe(writeStream)
// ..But now we need to figure out how to tell the promise when we're done
// copying the files.
// Well, we'll start by doing *something* when the pipe operation is
// finished. That's simple enough; we'll just set up an event listener:
writeStream.on('close', () => {
// Remember the "resolve" and "reject" functions we got earlier? Well, we
// can use them to tell the promise when we're done. So we'll do that here:
resolve()
})
// Okay, but what about errors? What if, for some reason, the pipe fails?
// That's simple enough to deal with too, if you know how. Remember how we
// learned a little on rejected promises, earlier? Since we're making
// our own Promise object, we'll need to create that rejection ourself
// (if anything goes wrong).
writeStream.on('error', err => {
// We'll use the "reject" argument we were given to show that something
// inside the promise failed. We can specify what that something is by
// passing the error object (which we get passed to our event listener,
// as usual).
reject(err)
})
// ..And we'll do the same in case our read stream fails, just in case:
readStream.on('error', err => {
reject(err)
})
// And now we're done! We've created our own hand-made promise-returning
// function, which we can use in our `createFiles` function that we wrote
// earlier.
})
}
..And here's all the finished code, so that you can review it yourself:
const util = require('util')
const fs = require('fs')
const promisifiedReaddir = util.promisify(fs.readdir)
function createFiles(countryCode) {
const readdirPromise = promisifiedReaddir('./app/data')
return readdirPromise.then(directories => {
const promises = directory.map(directory => {
return promisifiedReaddir(`./app/data/${directory}`).then(files => {
return copyFile(`./app/data/${directory}/en.yml`, `./app/data/${directory}/${countryCode}.yml`)
})
})
const allPromise = Promies.all(promises)
return allPromise
})
}
function copyFile(from, to) {
return new Promise((resolve, reject) => {
const readStream = fs.createReadStream(from)
const writeStream = fs.createWriteStream(to)
readStream.pipe(writeStream)
writeStream.on('close', () => {
resolve()
})
writeStream.on('error', err => {
reject(err)
})
readStream.on('error', err => {
reject(err)
})
})
}
Of course, this implementation isn't perfect. You could improve it by looking at other implementations - for example this one destroys the read and write streams when an error occurs, which is a bit cleaner than our method (which doesn't do that). The most reliable way would probably to go with the module I linked earlier!
I highly recommend you watch funfunfunction's video on promises. It explains how promises work in general, how to use Promise.all, and more; and he's almost certainly better at explaining this whole concept than I am!
First, create a function that returns a promise:
function processDirectory(directory) {
return new Promise((resolve, reject) => {
fs.readdir(`./app/data/${directory}`, (err, files) => {
if (err) reject(err);
console.log(`Creating ${countryCode}.yml for ${directory}`);
fs.createReadStream(`./app/data/${directory}/en.yml`)
.pipe(fs.createWriteStream(`./app/data/${directory}/${countryCode}.yml`))
.on('finish', resolve);
});
});
}
Then use Promise.all:
Promise.all(directories.map(processDirectory))
.then(...)
.catch(...);