I use require("fs").promises just to avoid to use callback function.
But now, I also want to use fs.createReadstream to attach a file with POST request.
How can I do this?
Or what alter createReadstream in this case?
Or should I use require("fs")?
So by using const fs = require('fs').promises; you're only gaining access to the promise version of the fs module. According to spec, there is no equivalent createReadStream entry in the File System Promises API. If you want that functionality, you'll need to store a reference to it in addition to the promisified version of fs.
I'd encourage anyone reading this to use the following at the top of your file to include both the promises api and ability to createReadStreams.
const fs = require('fs').promises;
const createReadStream = require('fs').createReadStream;
Your creation of readstreams will look like this (note no longer includes a prepended fs.):
createReadStream('/your/path/here');
Equally important to note:
According to spec, you'll eventually want to use the following instead (disclaimer, current out of box node can't do this without certain flags/dependences)
import { createReadStream } from 'fs';
I do it like this:
import { promises as fs, createReadStream } from "fs"
await fs.mkdir("path...");
const buff = createReadStream("path ...")
Your question is a bit broad, but I can point you to some helpful package. It doesn't cover nearly all functions, but the package fs-extra automatically makes a lot of fs functions return a promise, I definitely suggest that. Then you can always just use the regular fs at the same time. As for the fs.createReadStream() you'll probably be wanting to just wrap what you need in a new Promise().
You can "promisify" it like this:
function streamAsPromise(stream) {
return new Promise((resolve, reject) => {
let data = "";
stream.on("data", chunk => data += chunk);
stream.on("end", () => resolve(data));
stream.on("error", error => reject(error));
});
}
const text = await streamAsPromise(createReadStream('file.txt'));
Notice the await will wait until the whole file is read before resolving the promise, if the file is very big might be a problem.
Related
I have two methods.
The first one reads a file and writes to that file just as plain text. The second one writes a file as a stream.
In order to get this to work I have had to add fs twice in require.
const fs = require('fs').promises;
const fs2 = require('fs');
I'm trying to understand the difference and why I need this twice. But it sems that fs with out the promise doesn't have the ability to use createWriteStream and the one without the .promises doesnt have the ability to writeFile
/**
* Serializes credentials to a file compatible with GoogleAUth.fromJSON.
*
* #param {OAuth2Client} client
* #return {Promise<void>}
*/
async function saveCredentials(client) {
const content = await fs.readFile(CREDENTIALS_PATH);
const keys = JSON.parse(content);
const key = keys.installed || keys.web;
const payload = JSON.stringify({
type: 'authorized_user',
client_id: key.client_id,
client_secret: key.client_secret,
refresh_token: client.credentials.refresh_token,
});
await fs.writeFile(TOKEN_PATH, payload);
}
The second one writes to a file as a stream
/**
* Download file
* #param {OAuth2Client} authClient An authorized OAuth2 client.
*/
async function downloadFile(authClient) {
const service = google.drive({version: 'v3', auth: authClient});
const fileStream = fs2.createWriteStream("test.txt")
fileId = FILEID;
try {
const file = await service.files.get({
fileId: fileId,
alt: 'media',
}, {
responseType: "stream"
},
(err, { data }) =>
data
.on('end', () => console.log('onCompleted'))
.on('error', (err) => console.log('onError', err))
.pipe(fileStream)
);
} catch (err) {
// TODO(developer) - Handle error
throw err;
}
}
Note this does work, I am just trying to wrap my head around Node.js.
fs.promises contains a subset of the interface for what's on fs, but with promise-based interfaces instead of plain callback-style interfaces.
Some things which don't translate well to promises or don't have a natural promise-based interface such as fs.createReadStream() are only available on fs. Note that fs.createReadStream() returns a stream and uses events on the stream, not plain callbacks (which don't translate well to promises). As such, it's interface remains the same on fs and is not duplicated on fs.promises.
Many things are available in either with different interfaces:
fs.writeFile(filename, data, callback); // plain callback interface
or
await fs.promises.writeFile(filename, data) // promise interface
Most of the time, I can use only the fs.promises interface and do:
const fsp = require('fs').promises;
But sometimes, you need both and I would do this:
const fs = require('fs');
const fsp = fs.promises;
Keep in mind fs.promises is not a complete replacement for fs. It's an alternate (promise-based) interface for some (but not all) of the methods in the fs module.
Other interfaces such as fsp.open() have been enhanced and converted to promises in the fs.promises interface where it now returns a promise that resolves to an object-oriented fileHandle object whereas fs.open() just accepts a callback that will be passed a file descriptor.
So, my mode of operation is to look in the fs.promises interface for what I'm doing. If it's there, I use it there and use promises with it.
If it's not there, then go back to the fs interface for what I need.
I would advise you to NOT write code that uses the symbol fs for fs.promises. That will confuse people reading or working on your code because they are likely to think that a symbol named fs is the fs interface. That's why I use fs for the fs interface and fsp for the fs.promises interface in my code.
I want to crawl the app info based on app id (e.g. com.instagram.android) in Google Play.
I used the npm package as the crawler: https://www.npmjs.com/package/google-play-scraper
I don't have any javascript experience before, I don't know how to modify it for my purpose.
For now, I cannot use that for crawling a list of app IDs, and save the returned results as a *.txt for further analysis.
I have tried their sample code, and it runs well. But I could only see the returned results by .then(console.log, console.log). How could I save the results to a .txt file???
var gplay = require('google-play-scraper');
gplay.app({appId: 'com.dxco.pandavszombies'})
.then(console.log, console.log);
I am stuck here, could anyone help me on this issue? I have tried many different ways to solve this problem, but they all didn't work... I am really confused about how to save the value returned to .txt file
I tried to save all the potential variables into the .txt files, only the info like:
[object]
[object promise]
are saved.
I tried to save it like this:
var first = gplay.app({appId: 'com.instagram.android'});
app_result = first.then((result) => console.log(result));
fs.writeFile('App_Result.txt', app_result, (err) =>{
if (err) throw err;
console.log('App_Result saved!');
});
I understand that this may be caused by the strategy of Promise, which I need to wait for the Promise resolved and then the results could be returned. But I don't know how to figure out it since I am really new for the javascript.
First: JavaScript by default prints objects very unhelpfully, something like [Object object] or similar. If you need more information, try replacing console.log with data => JSON.stringify(data).
Second, due to the nature of Promises, you'll have to chain another .then onto your existing code. Something like
const gplay = require('google-play-scraper');
const fs = require('fs');
const file = fs.createWriteStream('App_Result.txt');
gplay.app({appId: 'com.dxco.pandavszombies'})
.then(result => JSON.stringify(result))
.then(text => file.write(text));
Your app_result will not equal the result of the promise but rather the promise itself.
You also have to convert the object to a string using JSON.stringify before you can write it to a file.
Try writing an async function like this.
const myFunction = async () => {
const app_result = await gplay.app({appId: 'com.instagram.android'});
fs.writeFile('App_Result.txt', JSON.stringify(app_result))
}
Out of curiosity, i want to know if there is any difference between the two.
readFileSync:
function parseFile(filePath) {
let data = fs.readFileSync(filePath);
}
readFile with promisify:
const readFilePromise = promisify(fs.readFile);
async function parseFile(filePath) {
let data = await readFilePromise(filePath);
}
If you need some context, im trying to read a bunch of files in a folder, replace a lot of values in each one, and write it again.
I don`t know if there is any difference in using Asyncronous or Synchronous code for these actions.
Full code:
function parseFile(filePath) {
let data = fs.readFileSync(filePath);
let originalData = data.toString();
let newData = replaceAll(originalData);
return fs.writeFileSync(filePath, newData);
}
function readFiles(dirPath) {
let dir = path.join(__dirname, dirPath);
let files = fs.readdirSync(dir); // gives all the files
files.map(file => parseFile(path.join(dir, file)));
}
function replaceAll(text) {
text = text.replace(/a/g, 'b');
return text;
}
readFiles('/files');
There's a big difference between the async and synchronous code. Whether that difference matters depends on what you are trying to do. Your javascript is singe threaded, so while you are reading a potentially large file synchronously with fs.readFileSync you can't do anything else such as respond to incoming requests.
If you are running a busy server this can cause big problems because requests queue up while you are reading the file and you may never catch up.
With the async method the file read happens outside your code and it calls your code back when it's done. While it's doing this your code is free to respond to other requests.
If you are just trying to read a local file and it doesn't matter if the thread blocks, then you can use either.
I want to make a function that get the reslt of fingerprint2.js
Fingerprint2 is a Modern & flexible browser fingerprinting library http://valve.github.io/fingerprintjs2/
Usage:
new Fingerprint2().get(function(result, components){
console.log(result); //a hash, representing your device fingerprint
console.log(components); // an array of FP components
});
whatever try i did to get result of Fingerprint2 outside of new Fingerprint2().get(function(result, components){ was failed.
like Global vars and cookie because Fingerprint2().get(...) is asynchronous
Can it be written like a function to get fingerprint2 result?
for example:
var secure = getmefingerprint2();
Leverage with ES2017 feature async/await, you can use Fingerprint2.getPromise() like this:
(async () => {
const components = await Fingerprint2.getPromise();
const values = components.map(component => component.value);
const murmur = Fingerprint2.x64hash128(values.join(""), 31);
console.log('fingerprint:', murmur);
)()
See get and getPromise in Fingerprint2 Doc
This should be a comment but its a bit long.
Even if it were possible, you would be bypassing the published api, meaning you would have to maintain a fork of the original code. You would also need to invoke the functionality synchronously - and fingerprintjs2 runs asynchronously for good and obvious reasons.
You seem to be asking about an XY problem
How you should sole it depends on what you intend to do with the fingerprint after it has been captured.
You can't make async code act completely synchronous. However, you can use async/await, if your target browser has support, but it's not universally supported. Also, it only looks synchronous inside the async function
The basic idea is to return a promise, then await it inside an async function:
const getmefingerprint2 = async () => {
const secure = await (new Promise(resolve => {
new Fingerprint2().get((result, components) => resolve(result) )
}))
// do things with secure, whatever you return is thenable
return secure
}
that function could be called like this (because of Promises):
getmefingerprint2().then(result => {
// do stuff with result
})
but also, inside the async function, you could treat secure like you got it synchronously.
If you really wanted to make your async code act more sync (might be useful for other async code, too, if you hate async), you could wrap all your code in an async function, then use await to get async stuff:
const getFingerprint = () => new Promise(resolve => {
new Fingerprint2().get((result, components) => resolve(result) )
})
const main = async () => {
// do some of your app here
const secure = await getFingerprint()
// do more stuff here
}
main()
Or as an IIFE:
(async() => {
// do some of your app here
const secure = await getFingerprint()
// do more stuff here
})()
These are just kinda hacky workarounds that allow you to escape the burden of async code, which is maybe worth just getting to know, as it will make a better app. If you restructure your code to only have the things that depend on secure inside the callback, you'll get better performance, unblocked UI, and a more dynamic flow that is easy enough to reason about, once you get used to it.
I am starting out with Node. Sorry for what probably is a stupid question.
Trying to understand why the below code throws an error: ReferenceError: Promise is not defined
allAccountFixtures: ['account-customer-joe', 'account-partner-sam', 'account-partner-jane', 'account-admin-jill'],
allProductFixtures: ['product-123', 'product-234', 'product-345', 'product-456'],
...
loadBasicFixtures: (Api) => {
return Promise.all([
Support.importRecords(Api.accountsAPI, Support.allAccountFixtures),
Support.importRecords(Api.productsAPI, Support.allProductFixtures)
]);
},
My APIs are defined elsewhere as:
this.accountsAPI = app.service('/api/accounts');
this.productsAPI = app.service('/api/products');
The import function is:
importRecords: (feathersService, fixtureNames) => {
// Wrap in an array if there's only one.
if (!(fixtureNames instanceof Array)) { fixtureNames = [fixtureNames]; }
// Create a separate promise for each JSON fixture to load the JSON from a
// file and send it to feathers.create(). Don't execute yet.
var promises = fixtureNames.map(fixtureName => {
var filePath = `test/fixtures/json/${fixtureName}.json`;
// console.log(`-> Loading JSON fixture: ${filePath}`);
return fs.readFileAsync(filePath, 'utf8')
.then((jsonString) => {
return JSON.parse(jsonString);
}).then((json) => {
return feathersService.create(json);
});
});
// Wrap all fixture loading promises inside a single outer promise that will
// fire when all of the child promises are complete.
return Promise.all(promises);
},
Don't know whether the supplied information is sufficient to advise what is happening. I looked up the concept of a "promise" and that's pretty much it. Perhaps you could point to the right direction. The documentation mentions resolve and reject.
I'll make my comment into an answer since it solved your issue.
Some older versions of node.js do not have promises built-in and to use promises with them requires loading a third party library that adds promise support.
If you upgrade to any 4.x version of node.js or newer, you will have promises built-in to node.js.
You need to import and require Promise
npm install promise --save
Then
var Promise = require('promise');