I have two methods.
The first one reads a file and writes to that file just as plain text. The second one writes a file as a stream.
In order to get this to work I have had to add fs twice in require.
const fs = require('fs').promises;
const fs2 = require('fs');
I'm trying to understand the difference and why I need this twice. But it sems that fs with out the promise doesn't have the ability to use createWriteStream and the one without the .promises doesnt have the ability to writeFile
/**
* Serializes credentials to a file compatible with GoogleAUth.fromJSON.
*
* #param {OAuth2Client} client
* #return {Promise<void>}
*/
async function saveCredentials(client) {
const content = await fs.readFile(CREDENTIALS_PATH);
const keys = JSON.parse(content);
const key = keys.installed || keys.web;
const payload = JSON.stringify({
type: 'authorized_user',
client_id: key.client_id,
client_secret: key.client_secret,
refresh_token: client.credentials.refresh_token,
});
await fs.writeFile(TOKEN_PATH, payload);
}
The second one writes to a file as a stream
/**
* Download file
* #param {OAuth2Client} authClient An authorized OAuth2 client.
*/
async function downloadFile(authClient) {
const service = google.drive({version: 'v3', auth: authClient});
const fileStream = fs2.createWriteStream("test.txt")
fileId = FILEID;
try {
const file = await service.files.get({
fileId: fileId,
alt: 'media',
}, {
responseType: "stream"
},
(err, { data }) =>
data
.on('end', () => console.log('onCompleted'))
.on('error', (err) => console.log('onError', err))
.pipe(fileStream)
);
} catch (err) {
// TODO(developer) - Handle error
throw err;
}
}
Note this does work, I am just trying to wrap my head around Node.js.
fs.promises contains a subset of the interface for what's on fs, but with promise-based interfaces instead of plain callback-style interfaces.
Some things which don't translate well to promises or don't have a natural promise-based interface such as fs.createReadStream() are only available on fs. Note that fs.createReadStream() returns a stream and uses events on the stream, not plain callbacks (which don't translate well to promises). As such, it's interface remains the same on fs and is not duplicated on fs.promises.
Many things are available in either with different interfaces:
fs.writeFile(filename, data, callback); // plain callback interface
or
await fs.promises.writeFile(filename, data) // promise interface
Most of the time, I can use only the fs.promises interface and do:
const fsp = require('fs').promises;
But sometimes, you need both and I would do this:
const fs = require('fs');
const fsp = fs.promises;
Keep in mind fs.promises is not a complete replacement for fs. It's an alternate (promise-based) interface for some (but not all) of the methods in the fs module.
Other interfaces such as fsp.open() have been enhanced and converted to promises in the fs.promises interface where it now returns a promise that resolves to an object-oriented fileHandle object whereas fs.open() just accepts a callback that will be passed a file descriptor.
So, my mode of operation is to look in the fs.promises interface for what I'm doing. If it's there, I use it there and use promises with it.
If it's not there, then go back to the fs interface for what I need.
I would advise you to NOT write code that uses the symbol fs for fs.promises. That will confuse people reading or working on your code because they are likely to think that a symbol named fs is the fs interface. That's why I use fs for the fs interface and fsp for the fs.promises interface in my code.
Related
I need to sequentially launch external scripts with such a condition that the following code is launched only after the completion of the entire contents of the external script, including promises. I use shell.js, but perhaps there are other tools. I need to run the script without importing into the "parent".
External script code (external.js):
const fs = require("fs");
const util = require("util");
const write = util.promisify(fs.writeFile);
(async () => {
await write("Hello.txt", "Hello");
})();
The "parent" code from which it is called:
const fs = require("fs");
const util = require("util");
const shell = require("shelljs");
const read = util.promisify(fs.readFile);
(async () => {
await shell.exec("node external.js", { async: true });
const data = await read('Hello.txt');
// do something with data...
})();
Please tell me if it is possible to implement this task, and how can this be done? Thanks for attention!
If external.js performs a file operation, then the node external.js process will only exit after the file operation has completed. Therefore
await util.promisify(child_process.exec)("node external.js");
should fulfill your requirement.
Is there any way to make AWS SSM getparameters sync?
Requirement :
The secret key, id stored in the SSM store should be accessible on the server up.
Using Node and express in the backend as a reverse proxy, so we have a constant.js file, it stores all the API URLs, paths, etc.
constats.js file
const api1url = 'http://example.com/path1'
const api1_secretkey = 'secret_key'
..................
module.export = {api1url,api1_secretkey}
So we wanted to call the ssm stuff in here before setting the const variables
const SSM = require('aws-sdk/clients/ssm');
const ssm = new SSM();
const params = {
Names: ['/secret_key_api_1', '/secret_key_api_2'],
WithDecryption: true,
};
const parameterList = await ssm.getParameters(params).promise();
I understand that await cant be without async, I just wanted to convey what I am expecting, without it being an async call or whether it be in the callback of getparameters.
const api1url = 'http://example.com/path1'
const api1_secretkey = parameterList.Parameter[1].Value
But since it is an async call we are not able to wait for it,tried doing it in a separate async function and then returning the data but since the function is still async having difficulties.
Not exactly issue wrt to an async function, the async function works as expected, but I want it to be synced, the AWS get parameter function is async. because of this I am not able to do something like const api1_secretkey = process.env.secret_key , this value will be obtained when js file is executed line by line. but since i am waiting for the secret key to come from async func by the time the line by line occurs const api1_secretkey = parameterList.Parameter[1].Value , the promise is still not resolved so undefined will sit. Even if i make a .then on the get params i can make this const api1_secretkey = parameterList.Parameter[1].Value after promise resolved. but since the export module is already done the other file carries undefined in the secret key even after it is changed on promise resolve.
I am new in the JS world, I am creating a query cache and I decide to use redis to cache the information, but I want to know if there is a way to use async/await keywords on the get function of redis.
const redis = require('redis');
const redisUrl = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisUrl);
client.set('colors',JSON.stringify({red: 'rojo'}))
client.get('colors', (err, value) => {
this.values = JSON.parse(value)
})
I want to know if I can use the await keyword instead of a callback function in the get function.
You can use util node package to promisify the get function of the client redis.
const util = require('util');
client.get = util.promisify(client.get);
const redis = require('redis');
const redisUrl = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisUrl);
client.set('colors',JSON.stringify({red: 'rojo'}))
const value = await client.get('colors')
With the util package i modified the get function to return a promise.
This is from redis package npm official documentation
Promises - You can also use node_redis with promises by promisifying node_redis with bluebird as in:
var redis = require('redis');
bluebird.promisifyAll(redis.RedisClient.prototype);
bluebird.promisifyAll(redis.Multi.prototype);
It'll add a Async to all node_redis functions (e.g. return client.getAsync().then())
// We expect a value 'foo': 'bar' to be present
// So instead of writing client.get('foo', cb); you have to write:
return client.getAsync('foo').then(function(res) {
console.log(res); // => 'bar'
});
// Using multi with promises looks like:
return client.multi().get('foo').execAsync().then(function(res) {
console.log(res); // => 'bar'
});
This example uses bluebird promisify read more here
So after you promsified a get to 'getAsync' you can use it in async await
so in your case
const value = await client.getAsync('colors');
For TypeScript users both util.promisify and bluebird.promisifyAll aren't ideal due to lack of type support.
The most elegant in TypeScript seems to be handy-redis, which comes with promise support and first-class TypeScript bindings. The types are generated directly from the official Redis documentation, i.e., should be very accurate.
I use require("fs").promises just to avoid to use callback function.
But now, I also want to use fs.createReadstream to attach a file with POST request.
How can I do this?
Or what alter createReadstream in this case?
Or should I use require("fs")?
So by using const fs = require('fs').promises; you're only gaining access to the promise version of the fs module. According to spec, there is no equivalent createReadStream entry in the File System Promises API. If you want that functionality, you'll need to store a reference to it in addition to the promisified version of fs.
I'd encourage anyone reading this to use the following at the top of your file to include both the promises api and ability to createReadStreams.
const fs = require('fs').promises;
const createReadStream = require('fs').createReadStream;
Your creation of readstreams will look like this (note no longer includes a prepended fs.):
createReadStream('/your/path/here');
Equally important to note:
According to spec, you'll eventually want to use the following instead (disclaimer, current out of box node can't do this without certain flags/dependences)
import { createReadStream } from 'fs';
I do it like this:
import { promises as fs, createReadStream } from "fs"
await fs.mkdir("path...");
const buff = createReadStream("path ...")
Your question is a bit broad, but I can point you to some helpful package. It doesn't cover nearly all functions, but the package fs-extra automatically makes a lot of fs functions return a promise, I definitely suggest that. Then you can always just use the regular fs at the same time. As for the fs.createReadStream() you'll probably be wanting to just wrap what you need in a new Promise().
You can "promisify" it like this:
function streamAsPromise(stream) {
return new Promise((resolve, reject) => {
let data = "";
stream.on("data", chunk => data += chunk);
stream.on("end", () => resolve(data));
stream.on("error", error => reject(error));
});
}
const text = await streamAsPromise(createReadStream('file.txt'));
Notice the await will wait until the whole file is read before resolving the promise, if the file is very big might be a problem.
I'm trying to read/write to a file in an async function (example):
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
fs.writeFileAsync(file, 'test');
// Log the contents to console
console.log(fs.readFileAsync(file));
}
But whenever I run it I always get the error:
(node:13480) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): TypeError: Cannot read property 'map' of null
I tried using bluebird by installing it using npm install bluebird in my project directory and adding:
const Bluebird = require('bluebird');
const fs = Bluebird.promisifyAll(require('fs'));
to my index.js (main) file, as well as adding:
const fs = require('fs');
to every file where I wan't to use fs.
I still get the same error and can only narrow down the problem to fs through commenting out stuff.
Any help would be appreciated.
First of all: async functions return a promise. So by definition, you are already using a promise.
Second, there is no fs.writeFileAsync. You are looking for fs.writeFile https://nodejs.org/api/fs.html#fs_fs_writefile_file_data_options_callback
With promises, making use of the power of async functions
const fs = require('fs');
const util = require('util');
// Promisify the fs.writeFile and fs.readFile
const write = util.promisify(fs.writeFile);
const read = util.promisify(fs.readFile);
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
await write(file, 'test');
// Log the contents to console
const contents = await read(file, 'utf8');
console.log(contents);
}
In the above: We used util.promisify to turn the nodejs callback style using functions to promises. Inside an async function, you can use the await keyword to store the resolved contents of a promise to a const/let/var.
Further reading material: https://ponyfoo.com/articles/understanding-javascript-async-await
Without promises, callback-style
const fs = require('fs');
async readWrite() {
// Create a variable representing the path to a .txt
const file = 'file.txt';
// Write "test" to the file
fs.writeFile(file, 'test', err => {
if (!err) fs.readFile(file, 'utf8', (err, contents)=> {
console.log(contents);
})
});
}