I asked myself a question,
I can read files (csv mainly) on a cloud platform but when it's a zip I just get a bunch of:
j�\lȜ��&��3+xT��J��=��y��7���vu� {d�T���?��!�
Which is normal, so I wonder if there is a way to put that in a variable and unzip it using a lib or something like that.
Thanks for your time
you should use npm install node-stream-zip
const StreamZip = require('node-stream-zip');
const zip = new StreamZip({
file: 'archive.zip',
storeEntries: true
});
and get the info like this
zip.on('ready', () => {
console.log('Entries read: ' + zip.entriesCount);
for (const entry of Object.values(zip.entries())) {
const desc = entry.isDirectory ? 'directory' : `${entry.size} bytes`;
console.log(`Entry ${entry.name}: ${desc}`);
}
// Do not forget to close the file once you're done
zip.close()
});
Hope it helps :-)
You should use jszip npm package. This allows you to quickly read zip files.
Example:
var fs = require("fs");
var JSZip = require("jszip");
// read a zip file
fs.readFile("project.zip", function(err, data) {
if (err) throw err;
JSZip.loadAsync(data).then(function (zip) {
files = Object.keys(zip.files);
console.log(files);
});
});
To read the contents of a file in the zip archive you can use the following.
// read a zip file
fs.readFile("project.zip", function(err, data) {
if (err) throw err;
JSZip.loadAsync(data).then(function (zip) {
// Read the contents of the 'Hello.txt' file
zip.file("Hello.txt").async("string").then(function (data) {
// data is "Hello World!"
console.log(data);
});
});
});
and to download the zip file from the server:
request('yourserverurl/helloworld.zip')
.pipe(fs.createWriteStream('helloworld.zip'))
.on('close', function () {
console.log('File written!');
});
Related
I am using just a single Node package, basic-ftp to try and download a TXT file and write the contents to the console. Further down the line I will be editing the text so will need to use fs. Just struggling to work with the output from createWriteStream from within the FTP program.
Can anyone help me write a TXT file to the /tmp/ file within AWS Lambda and then the correct syntax to open and edit the file after createWriteStream has been used?
var fs = require('fs');
const ftp = require("basic-ftp")
var path = require('path');
exports.handler = (event, context, callback) => {
var fullPath = "/home/example/public_html/_uploads/15_1_5c653e6f6780f.txt"; // File Name FULL PATH -------
const extension = path.extname(fullPath); // Used to calculate filenames below
const wooFileName = path.basename(fullPath, extension); // Uploaded filename with no path or extension eg. filename
const myFileNameWithExtension = path.basename(fullPath); // Uploaded filename with the file extension eg. filename.txt
const FileNameWithExtension = path.basename(fullPath); // Uploaded filename with the file extension eg. filename.txt
example()
async function example() {
const client = new ftp.Client()
client.ftp.verbose = true
try {
await client.access({
host: "XXXX",
user: "XXXX",
password: "XXXX",
//secure: true
})
await client.download(fs.createWriteStream('./tmp/' + myFileNameWithExtension), myFileNameWithExtension)
}
catch(err) {
console.log(err)
}
client.close()
}
//Read the content from the /tmp directory to check it's empty
fs.readdir("/tmp/", function (err, data) {
console.log(data);
console.log('Contents of AWS Lambda /tmp/ directory');
});
/*
downloadedFile = fs.readFile('./tmp/' + myFileNameWithExtension)
console.log(downloadedFile)
console.log("Raw text:\n" + downloadedFile.Body.toString('ascii'));
*/
}
Pretty sure your fs.createWriteStream() has to use an absolute path to /tmp in Lambdas. Your actual working directory is var/task not /.
Also, if you're using fs.createWriteStream() you'll need to wait for the finish event before reading from the file. Somethign like this...
async function example() {
var finalData = '';
const client = new ftp.Client()
client.ftp.verbose = true
try {
await client.access({
host: "XXXX",
user: "XXXX",
password: "XXXX",
//secure: true
})
let writeStream = fs.createWriteStream('/tmp/' + myFileNameWithExtension);
await client.download(writeStream, myFileNameWithExtension)
await finalData = (()=>{
return new Promise((resolve, reject)=> {
writeStream
.on('finish', ()=>{
fs.readFile("/tmp/"+myFileNameWithExtension, function (err, data) {
if (err) {
reject(err)
} else {
console.log('Contents of AWS Lambda /tmp/ directory', data);
resolve(data);
}
});
})
.on('error', (err)=> {
console.log(err);
reject(err);
})
})
})();
}
catch(err) {
console.log(err)
}
client.close();
return finalData;
}
You'll also need to access the file using fs.readFile(). What you were using fs.readdir() gives you a list of files in the directory, not the file's contents.
If you want to used readdir() you could do it like this, but as you can see it is redundant in your case. To handle errors I would suggest just handling the error event in the initial createWriteStream() instead of adding this extra overhead (added to previous example)...
writeStream
.on('finish', ()=>{
fs.readdir('/tmp',(err, files)=> {
let saved = files.find(file => file === myFileNameWithExtension);
fs.readFile("/tmp/"+saved, function (err, data) {
if (err) throw new Error();
console.log(data);
console.log('Contents of AWS Lambda /tmp/ directory');
});
})
})
.on('error', (err)=> {
console.log(err);
throw new Error();
})
NOTE: Please log out the result of saved, I can't remember if the files array is absolute of relative paths.
I am trying to get get the folder path from user's selection and for each file I read it and return the data. However upon getting the file I cannot read the data for some reason that I have't been able to understand yet. The directory I am trying to read and render do exist. I have referred to the other similar posts as well.
readFolder() {
dialog.showOpenDialog({ properties: ['openDirectory'] }, (dirFiles) => {
console.log(dirFiles);
if (dirFiles === undefined) {
console.log('No file ');
return;
}
const pathName = dirFiles[0];
fs.readdir(pathName, (err, files) => {
files.forEach(file => {
fs.readFile(file, 'utf-8', (err, data) => {
if (err) {
console.log(`something went wrong ${err}`);
} else {
console.log(data);
}
});
});
});
});
}
readdir returns filenames "a", "b", "c" etc. You want pathName + '/' + file for "/path/to/file/a"
The mistake I made was the fact that I hadn't realised the return values of 'file' which are just the names of the files as strings but not paths. Assinging let filePath =${pathName}/${file}; and reading it onwards solved the problem.
I'm trying with this code below, I can write file on server and the file has content but, when I download the file, it's blank.
var file = fs.createWriteStream('./tmp/text.txt');
file.on('error', function(err) { console.log(err) });
rows.forEach(function(v) {
file.write(v + "\r\n");
productExport.DeleteProduct(v.ProductId, function(err){
if(err) throw err;
});
});
file.end();
var file = "./tmp/text.txt";
res.download(file); // Set disposition and send it.
How can I download the file with all the content?
The way your code is structured is incorrect for what you're trying to do. Primarily, your issue is that you're responding with res.download() before the file stream is done being written to. Additionally, you have var file being initialized twice in the same scope, this isn't correct either.
var file = "./tmp/text.txt";
var writeStream = fs.createWriteStream(file);
writeStream.on('error', err => console.log );
writeStream.on('finish', () => {
return res.download(file); // Set disposition and send it.
});
rows.forEach((v) => {
writeStream.write(v + "\r\n");
productExport.DeleteProduct(v.ProductId, function(err){
if(err) throw err;
});
});
writeStream.end();
If you're confused by this, and perhaps async processing in general, this is the defacto answer on SO for understanding async in Node.js
Writing data to a file through I/O is an asynchronous operation. You have to wait for the WriteStream to complete before you can download the file.
var file = fs.createWriteStream('./tmp/text.txt');
file.on('error', function(err) { console.log(err) });
rows.forEach(function(v) {
file.write(v + "\r\n");
productExport.DeleteProduct(v.ProductId, function(err){
if(err) throw err;
});
});
file.end();
file.on('finish', function() {
var file = "./tmp/text.txt";
res.download(file); // Set disposition and send it.
});
Extra Information:
FileSystem#createWriteStream return WriteStream object.
From the document of WriteStream it stats there are 6 events [close, drain, error, finish, pipe, unpipe].
in the node.js world, you should always expect to use callback or seek for complete/finish event when there are I/O operations.
Electron version: 1.3.3
Operating system: Ubuntu 14.04
I want to save a XML object into a .xml file with Electron. I try this:
const {dialog} = require("electron").remote;
dialog.showSaveDialog(myObj)
A new windows is opening, I fill the name of the file but nothing has been saving.
it's recommended to use returned path from dialog.showSaveDialog to get filepath in new versions of electron: (which is result.filePath in the below code)
filename = dialog.showSaveDialog({}
).then(result => {
filename = result.filePath;
if (filename === undefined) {
alert('the user clicked the btn but didn\'t created a file');
return;
}
fs.writeFile(filename, content, (err) => {
if (err) {
alert('an error ocurred with file creation ' + err.message);
return
}
alert('WE CREATED YOUR FILE SUCCESFULLY');
})
alert('we End');
}).catch(err => {
alert(err)
})
The showSaveDialog() API does not save the file for you. You must use the returned path and use Node to save your file.
const {dialog} = require('electron').remote;
const fs = require('fs');
dialog.showSaveDialog({}).then((result) => {
fs.writeFile(result.filePath, MyFileData, (err) => {
// file saved or err
});
}).catch((err) => {
// err
});
I'm using the excellent Request library for downloading files in Node for a small command line tool I'm working on. Request works perfectly for pulling in a single file, no problems at all, but it's not working for ZIPs.
For example, I'm trying to download the Twitter Bootstrap archive, which is at the URL:
http://twitter.github.com/bootstrap/assets/bootstrap.zip
The relevant part of the code is:
var fileUrl = "http://twitter.github.com/bootstrap/assets/bootstrap.zip";
var output = "bootstrap.zip";
request(fileUrl, function(err, resp, body) {
if(err) throw err;
fs.writeFile(output, body, function(err) {
console.log("file written!");
}
}
I've tried setting the encoding to "binary" too but no luck. The actual zip is ~74KB, but when downloaded through the above code it's ~134KB and on double clicking in Finder to extract it, I get the error:
Unable to extract "bootstrap" into "nodetest" (Error 21 - Is a directory)
I get the feeling this is an encoding issue but not sure where to go from here.
Yes, the problem is with encoding. When you wait for the whole transfer to finish body is coerced to a string by default. You can tell request to give you a Buffer instead by setting the encoding option to null:
var fileUrl = "http://twitter.github.com/bootstrap/assets/bootstrap.zip";
var output = "bootstrap.zip";
request({url: fileUrl, encoding: null}, function(err, resp, body) {
if(err) throw err;
fs.writeFile(output, body, function(err) {
console.log("file written!");
});
});
Another more elegant solution is to use pipe() to point the response to a file writable stream:
request('http://twitter.github.com/bootstrap/assets/bootstrap.zip')
.pipe(fs.createWriteStream('bootstrap.zip'))
.on('close', function () {
console.log('File written!');
});
A one liner always wins :)
pipe() returns the destination stream (the WriteStream in this case), so you can listen to its close event to get notified when the file was written.
I was searching about a function which request a zip and extract it without create any file inside my server, here is my TypeScript function, it use JSZIP module and Request:
let bufs : any = [];
let buf : Uint8Array;
request
.get(url)
.on('end', () => {
buf = Buffer.concat(bufs);
JSZip.loadAsync(buf).then((zip) => {
// zip.files contains a list of file
// chheck JSZip documentation
// Example of getting a text file : zip.file("bla.txt").async("text").then....
}).catch((error) => {
console.log(error);
});
})
.on('error', (error) => {
console.log(error);
})
.on('data', (d) => {
bufs.push(d);
})