I am trying to read the contents of several files in Node.js using promises. Since the standard fs module does not provide a sufficient promise interface, I decided to use fs-extra instead which provides pretty much the same functions as the default fs module with an additional promise interface.
Reading the contents of a single file as shown below works as desired and logs the file's contents to the console:
const fse = require('fs-extra')
const filePath = './foo.txt'
fse.readFile(filePath, 'utf8')
.then(filecontents => {
return filecontents
})
.then(filecontents => {
console.log(filecontents)
})
However, I need to handle several files inside a given directory. To do this I need to implement the following steps:
get an array of all files inside the directory using fse.readdir() - done
join filenames and directory name to get a kind of a base file path using path.join with .map() to avoid iterating over the array - done
read file contents using fse.readFile() inside another .map()
These three steps are implemented as follows:
const fse = require('fs-extra');
const path = require('path');
const mailDirectory = './mails'
fse.readdir(mailDirectory)
.then(filenames => {
return filenames.map(filename => path.join(mailDirectory, filename))
})
.then(filepaths => {
// console.log(filepaths)
return filepaths
.map(filepath => fse.readFile(filepath).then(filecontents => {
return filecontents
}))
})
.then(mailcontents => {
console.log(mailcontents)
})
As stated above, steps 1 and 2 are working quite nice. However, I am unable to read the file contents using fse.readFile() inside the last .map() which results in an
[ Promise { <pending> },
Promise { <pending> },
Promise { <pending> },
Promise { <pending> },
Promise { <pending> } ]
output indicating that the promise is not resolved, yet. I assume that this unresolved promise is the promise returned by the fse.readFile() function. However I am unable to resolve it properly since a comparable approach in my very first snippet works like a charm.
How could I solve this issue? Where does it exactly come from since I am a newbie in the field of JS and especially in the field of Node.js?
You have an Array of Promises. You should wait on them using Promise.all():
const fse = require('fs-extra');
const path = require('path');
const mailDirectory = './mails'
fse.readdir(mailDirectory)
.then(filenames => {
return filenames.map(filename => path.join(mailDirectory, filename))
})
.then(filepaths => {
// console.log(filepaths)
return filepaths
.map(filepath => fse.readFile(filepath).then(filecontents => {
return filecontents
}))
})
// Promise.all consumes an array of promises, and returns a
// new promise that will resolve to an array of concrete "answers"
.then(mailcontents => Promise.all(mailcontents))
.then(realcontents => {
console.log(realcontents)
});
Also, if you don't want to have to have an additional dependency on fs-extra you can use node 8's new util.promisify() to make fs follow a Promise oriented API.
Related
I have an array of objects that I get from database.
const users = await User.findAll();
To each of them, I want to call an async function user.getDependency() to get another related table, and devolve it nested in the object to a final format like this:
[
User {
attr1: value,
attr2: value,
...
Dependency: {
attr1: value,
...
}
},
User {
...
and so on
Now, I get my problem. The only possibilities I am being able to think involving async and loops are 2:
Option 1: mapping them and getting the promise back:
users.forEach((user) => {
user.dependency= user.getDependency();
});
With this I get similar to the desired outcome in terms of format, only that instead of the resolved thing i get obviously Dependency: Promise { <pending> }, that is, the promises all nested inside each user object of the array. And I don't know how to procceed with it. How to loop through all the user objects and resolve this promises they have inside?
Option 2: return the promises to be resolved in Promise.all
const dependencies = Promise.all(users.map((user) => {
return user.getDependency();
}));
This give me a separated array with all the dependecies I want, but that's it, separated. I get one array with the objects users and another with the objects dependencies.
What is bugging me specially is that I am thinking that there must be a simple straightforward way to do it and I am missing. Anyone have an idea? Much thanks
To build some intuition, here is your forEach example converted with a push in the right direction.
users.forEach((user) => {
user.getDependency().then(dependency => {
user.dependency = dependency
})
})
The above reads "for each user, start off the dependency fetching, then add the dependency to the user once resolved." I don't imagine the above is very useful, however, as you would probably like to be able to do something with the users once all the dependencies have finished fetching. In such a case like that, you could use Promise.all along side .map
const usersWithResolvedDependencies = await Promise.all(
users.map(user => user.getDependency().then(dependency => {
user.dependency = dependency
return user
}))
)
// do stuff with users
I should mention I wrote a library that simplifies async, and could certainly work to simplify this issue as well. The following is equivalent to the above. map and assign are from my library, rubico
const usersWithResolvedDependencies = map(assign({
dependency: user => user.getDependency()
}))(users)
I guess the best way is to do like this. Mapping users, updating every object (better to do this without mutation), and using Promise.all
const pupulatedUsers = await Promise.all(users.map(async (user) => ({
...user,
dependency: await getDependency(user.dependency_id)
// or something like getDependency
// function that fires async operation to get needed data
}));
you're right there is a straightforward way to resolve the promise
as the users variable is not a promise waiting to be resolved, you should try
users.then((userList) => {
// function logic here
})
the userList in the then function should be iterable and contain the desired user list
also you can also handle the rejected part of the promise, if at all the query fails and the promise returns an error:
users.then((error) => {
// error handling
})
you should give this document a glance to get more info about handling promises to further know what goes on in that query when it returns a promise
Perhaps this is a limitation of the language, but I am trying to figure out how I could get away with using a single await keyword for a set of sequential promise resolutions. I am trying to achieve something readable, like the following:
const workoutExercises = await Workout.get(1).workoutExercises;
Workout.get accesses a database, and so returns a Promise. The promise resolves to an instance of Workout, which has a getter method on the instance called workoutExercises, which is also a Promise that resolves to an array of WorkoutExercises.
The above code does not work, and only waits for Workout.get to resolve; doesn't also wait for .workoutExercises to resolve. The following below code examples DO work, but I am trying to achieve a one-liner / more readability:
1:
const workoutExercises = await (await Workout.get(1)).workoutExercises;
2:
const workout = await Workout.get(1);
const workoutExercises = await workout.workoutExercises;
Edit #1
Updated the title and description to clarify that the problem doesn't revolve around the resolution of a Promise chain, but the resolution of a Promise based on the resolution of a preceding Promise.
Workout.get --> <Promise> --> workout --> .workoutExercises --> <Promise> -> desired result
Use .then() on the results of .get(1), and extract workoutExercises. The one liner is very readable.
Example:
(async() => {
const Workout = {
get: () => Promise.resolve({
workoutExercises: Promise.resolve(3)
})
};
const workoutExercises = await Workout.get(1).then(w => w.workoutExercises);
console.log(workoutExercises)
})()
Is it possible to use one await keyword for resolving a set of sequential promises
Because these promises are not actually chained, it is not. You have a promise that resolves to an object that has a property that when you access it has a getter that creates a new promise that resolves to the value you finally want. That's two completely separate promises and you'll have to use await or .then() on each of the two so you can't do this with one single await.
You can use a little intervening code to chain them together and even put them into a helper function:
function getWorkoutExercises(n) {
return Workout.get(n).then(workout => {
// chain this promise to the previous one
return workout.workoutExercises;
});
}
// usage
getWorkoutExercises(1).then(exercises => {
console.log(exercises);
}).catch(err => {
console.log(err);
});
or
// usage
try {
let exercises = await getWorkoutExercises(1);
console.log(exercises);
} catch(e) {
console.log(e);
}
I'm trying to read/write to a file in an async function (example):
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
fs.writeFileAsync(file, 'test');
// Log the contents to console
console.log(fs.readFileAsync(file));
}
But whenever I run it I always get the error:
(node:13480) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): TypeError: Cannot read property 'map' of null
I tried using bluebird by installing it using npm install bluebird in my project directory and adding:
const Bluebird = require('bluebird');
const fs = Bluebird.promisifyAll(require('fs'));
to my index.js (main) file, as well as adding:
const fs = require('fs');
to every file where I wan't to use fs.
I still get the same error and can only narrow down the problem to fs through commenting out stuff.
Any help would be appreciated.
First of all: async functions return a promise. So by definition, you are already using a promise.
Second, there is no fs.writeFileAsync. You are looking for fs.writeFile https://nodejs.org/api/fs.html#fs_fs_writefile_file_data_options_callback
With promises, making use of the power of async functions
const fs = require('fs');
const util = require('util');
// Promisify the fs.writeFile and fs.readFile
const write = util.promisify(fs.writeFile);
const read = util.promisify(fs.readFile);
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
await write(file, 'test');
// Log the contents to console
const contents = await read(file, 'utf8');
console.log(contents);
}
In the above: We used util.promisify to turn the nodejs callback style using functions to promises. Inside an async function, you can use the await keyword to store the resolved contents of a promise to a const/let/var.
Further reading material: https://ponyfoo.com/articles/understanding-javascript-async-await
Without promises, callback-style
const fs = require('fs');
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
fs.writeFile(file, 'test', err => {
if (!err) fs.readFile(file, 'utf8', (err, contents)=> {
console.log(contents);
})
});
}
When I start so code
import Promise from 'bluebird';
const mongodb = Promise.promisifyAll(require('mongodb'));
const MongoClient = mongodb.MongoClient;
MongoClient.connect(url).then((db) => {
return Promise.all([new WorkerService(db)]);
}).spread((workerService) => (
Promise.all([new WorkerRouter(workerService)])
)).spread((workerRouter) => {
app.use('/worker', workerRouter);
}).then(() => {
httpServer.start(config.get('server.port'));
}).catch((err) => {
console.log(err);
httpServer.finish();
});
I see so error
}).spread(function (workerService) {
^
TypeError: MongoClient.connect(...).then(...).spread is not a function
Please help me. What am I doing wrong?
I see several things wrong here:
The root cause here is that MongoClient.connect(...).then(...) is not returning a Bluebird promise, thus, there is no .spread() method. When you promisify an interface with Bluebird, it does NOT change the existing methods at all. Instead, it adds new methods with the "Async" suffix on it.
So, unless you've somehow told Mongo to create only Bluebird promises, then
`MongoClient.connect(...).then(...)`
will just be returning whatever kind of promise Mongo has built in. You will need to use the new promisified methods:
MongoClient.connectAsync(...).then(...).spread(...)
Some other issues I see in your code:
1) Promise.all() expects you to pass it an array of promises. But when you do this: Promise.all([new WorkerRouter(workerService)]), you're passing it an array of a single object which unless that object is a promise itself, is just wrong.
2) In this code:
}).spread((workerService) => (
Promise.all([new WorkerRouter(workerService)])
))
you need to return the resulting promise in order to link it into the chain:
}).spread((workerService) => (
return Promise.all([new WorkerRouter(workerService)])
))
3) But, there's no reason to use Promise.all() on a single element array. If it was a promise, then just return it.
4) It also looks like you're trying to use promises for different steps of synchronous code too. Besides just complicating the heck out of things, there's just no reason to do that as this:
}).spread((workerService) => (
Promise.all([new WorkerRouter(workerService)])
)).spread((workerRouter) => {
app.use('/worker', workerRouter);
}).then(() => {
httpServer.start(config.get('server.port'));
})...
can be combined to this:
)).spread((workerService) => {
app.use('/worker', new WorkerRouter(workerService));
httpServer.start(config.get('server.port'));
})...
And, can probably be condensed even further since new WorkerService(db) probably also doesn't return a promise.
5) And, then we can't really tell why you're even attempting to use .spread() in the first place. It is useful only when you have an array of results that you want to turn into multiple named arguments, but you don't need to have an array of results (since there was no reason to use Promise.all() with a single item and in modern version of node.js, there's really no reason to use .spread() any more because you can use ES6 destructing of an array function argument to accomplish the same thing with the regular .then().
I don't claim to follow exactly what you're trying to do with every line, but you may be able to do this:
import Promise from 'bluebird';
const mongodb = Promise.promisifyAll(require('mongodb'));
const MongoClient = mongodb.MongoClient;
MongoClient.connectAsync(url).then((db) => {
let service = new WorkerService(db);
app.use('/worker', new WorkerRouter(service));
// since httpServer.start() is async and there's no promise returning here,
// this promise chain will not wait for the server to start
httpServer.start(config.get('server.port'));
}).catch((err) => {
console.log(err);
httpServer.finish();
});
I am starting out with Node. Sorry for what probably is a stupid question.
Trying to understand why the below code throws an error: ReferenceError: Promise is not defined
allAccountFixtures: ['account-customer-joe', 'account-partner-sam', 'account-partner-jane', 'account-admin-jill'],
allProductFixtures: ['product-123', 'product-234', 'product-345', 'product-456'],
...
loadBasicFixtures: (Api) => {
return Promise.all([
Support.importRecords(Api.accountsAPI, Support.allAccountFixtures),
Support.importRecords(Api.productsAPI, Support.allProductFixtures)
]);
},
My APIs are defined elsewhere as:
this.accountsAPI = app.service('/api/accounts');
this.productsAPI = app.service('/api/products');
The import function is:
importRecords: (feathersService, fixtureNames) => {
// Wrap in an array if there's only one.
if (!(fixtureNames instanceof Array)) { fixtureNames = [fixtureNames]; }
// Create a separate promise for each JSON fixture to load the JSON from a
// file and send it to feathers.create(). Don't execute yet.
var promises = fixtureNames.map(fixtureName => {
var filePath = `test/fixtures/json/${fixtureName}.json`;
// console.log(`-> Loading JSON fixture: ${filePath}`);
return fs.readFileAsync(filePath, 'utf8')
.then((jsonString) => {
return JSON.parse(jsonString);
}).then((json) => {
return feathersService.create(json);
});
});
// Wrap all fixture loading promises inside a single outer promise that will
// fire when all of the child promises are complete.
return Promise.all(promises);
},
Don't know whether the supplied information is sufficient to advise what is happening. I looked up the concept of a "promise" and that's pretty much it. Perhaps you could point to the right direction. The documentation mentions resolve and reject.
I'll make my comment into an answer since it solved your issue.
Some older versions of node.js do not have promises built-in and to use promises with them requires loading a third party library that adds promise support.
If you upgrade to any 4.x version of node.js or newer, you will have promises built-in to node.js.
You need to import and require Promise
npm install promise --save
Then
var Promise = require('promise');