There are some way to check file is corrupted with node.js?
I tried many File System methods, like fs.readFile, fs.open abd fs.access but all them returns ok status, and I'm sure my file is corrupted in my tests.
To be more clear, my objective is to check if PDF is readable (not only check if can be generated), if can be opened. I damaged the file here to test.
You could try to parse it with a tool like this and confirm if it was successful.
To expand on that a little, here's some example code lifted from the link:
let fs = require('fs'),
PDFParser = require("pdf2json");
let pdfParser = new PDFParser();
pdfParser.on("pdfParser_dataError", errData => console.error(errData.parserError) );
pdfParser.on("pdfParser_dataReady", pdfData => {
fs.writeFile("./pdf2json/test/F1040EZ.json", JSON.stringify(pdfData));
});
pdfParser.loadPDF("./pdf2json/test/pdf/fd/form/F1040EZ.pdf");
Actually, u can use another npm to check file corruption of pdf.
npm i pdf-parse
const pdfParser = require('pdf-parse')
try {
let bufferData = fs.readFileSync(`${target}/corrupted.pdf`)
pdfParser(bufferData).then((data) => {
// do something with data
}).catch((error) => {
console.log(error.message)
})
} catch (error) {
console.log(error)
}
For corrupted file the error might seem like:
Warning: Indexing all PDF objects
Invalid PDF structure
Related
I'm looking for the best way to send image files to my server using Apollo Express, and Node.
Getting the information there doesn't seem to be an issue, I convert the object into a string but can't find out how to convert it back to a regular file object to store away.
What I have so far;
JS - let buffer = await toBase64(file);
Through Apollo server..
Node - let buffer = Buffer.from(args.image, 'base64');
This gives me a Buffer. I'm unsure how to proceed with NodeJS to convert this back to a file object.
Thanks
I hope this will be helpfull for you
const file = new File([
new Blob(["decoded_base64_String"])
], "output_file_name");
You can use one of the various write or writeFile methods which accept a Buffer.
const fs = require("fs");
let buffer = Buffer.from(
"iVBORw0KGgoAAAANSUhEUgAAAAgAAAAGCAIAAABxZ0isAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAAQSURBVBhXY/iPAwygxP//AAjcj3EdtT3BAAAAAElFTkSuQmCC",
"base64"
);
fs.writeFile("pic.png", buffer, (err) => {
if (err) throw err;
console.log("The file has been saved!");
});
There are a lot of solutions that are based on the fetch api or the XMLHttpRequest, but they return CORS or same-origin-policy errors.
The File/Filereader API works out of the box , but only for files chosen by the user via a input file (because that is the only way to import them as a File obj)
Is there a way to do something simple and minimal like
const myfile = new File('relative/path/to/file') //just use a path
const fr = new FileReader();
fr.readAsText(myfile);
Thanks
Try the following JS, this will use fs to read the file and if it exists it will turn it into a string and output to console. You can change it up to however you'd like.
var fs = require('fs');
fs.readFile('test.txt', 'utf8', function(err, data) {
if (err) {
return console.log(err);
}
console.log(data);
});
Second time posting on StackOverflow so I apologize for any mistakes.
Please bear with me.
Same with the title; How do you read contents of a discord attachment let's say a .txt file and print the contents?
I have tried with fs but unfortunately failed and I have also searched the documentation but failed also.
Ideas?
You can't use the fs module for this as it only deals with local files. When you upload a file to the Discord server, it gets uploaded to a CDN and all you can do is grab the URL of this file from the MessageAttachment using the url property.
If you need to get a file from the web, you can fetch it from a URL using the built-in https module, or you can install one from npm, like the one I used below, node-fetch.
To install node-fetch, run npm i node-fetch in your root folder.
Check out the working code below, it works fine with text files:
const { Client } = require('discord.js');
const fetch = require('node-fetch');
const client = new Client();
client.on('message', async (message) => {
if (message.author.bot) return;
// get the file's URL
const file = message.attachments.first()?.url;
if (!file) return console.log('No attached file found');
try {
message.channel.send('Reading the file! Fetching data...');
// fetch the file from the external URL
const response = await fetch(file);
// if there was an error send a message with the status
if (!response.ok)
return message.channel.send(
'There was an error with fetching the file:',
response.statusText,
);
// take the response stream and read it to completion
const text = await response.text();
if (text) {
message.channel.send(`\`\`\`${text}\`\`\``);
}
} catch (error) {
console.log(error);
}
});
reply to #Andryxa, maybe you can use this with external APIs like a transcription service in case of audio files or to send requests to already created bots from services like dialogflow to replies to the messages
I am trying to understand how stream work. I have a readableStream which i am pipeing to a writableStream.
However, i am getting this error in the console when i do so. Failed to execute 'pipeTo' on 'ReadableStream': Illegal invocation
I am making a call to fetch and pass the ReadableStream using PipeTo to a WritableStream
below is the relevant part of my code.
let writeStream = fs.createWriteStream(`exports/${exportParams.key}`, {flags:'a'});
writeStream.on('end', function(){
console.log('file downloaded');
});
const fetchData = await exportPopup.evaluate(async (fetchUrl, writeStream) => {
const stream = await fetch(fetchUrl);
stream.body.pipeTo(writeStream);
}, fetchUrl, writeStream);
Any help to fix this would be really great, Thanks.
Are you using node-fetch or are you on electron? I faced the same problem trying to download files from electron. It looks like streams from the browser aren't compatible with the streams used by fs. What I finally do is.
async function downloadFile(filename, url) {
const file = fs.createWriteStream(filename);
res = await fetch(url);
buffer = await res.arrayBuffer()
file.write(Buffer.from(buffer));
file.close();
}
I don't know if this is the most efficient way of doing this but it works.
I'm creating an app with Electron. I was need to sent file to server with Socket.IO, so I installed socket.io-stream module. I tested on browser, it works well. But do same in Electron, it always fails with
TypeError: Invalid non-string/buffer chunk
This is server side code:
ss(socket).on('/user/update/profile', (stream, data) => {
const filename = path.basename(data.name);
const ws = fs.createWriteStream(`userdata/profile/${filename}`);
stream.on('error', (e) => {
console.log('Error found:');
console.log(e);
});
stream.on('drain', (e) => {
console.log('drain');
});
stream.on('data', () => {
console.log('data');
});
stream.on('close', () => {
console.log('close');
});
stream.pipe(ws);
//ss(socket).emit('/user/update/profile', {});
});
And this is client side code:
var file = ev.target.files[0];
var stream = ss.createStream();
ss(socket).emit('/user/update/profile', stream, {
email: this.props.user.email,
name: file.name,
size: file.size
});
var blobStream = ss.createBlobReadStream(file);
var size = 0;
blobStream.on('data', (chunk) => {
size += chunk.length;
console.log(`${size} / ${file.size}`);
});
blobStream.pipe(stream);
code is quite simple, just from the example in module's introduction page on NPM. I already said that it worked as browser. You see that I logged every file uploading progress on the console. Using Electron, sending file seems to work because it logged every data size, but on server side, it fails.
I found similar issue with NW.js and that guy solved his problem with his own way but that didn't worked for me.
It will be very appreciate help me how should I do.
The error means your are passing a non-string (e.g. an object) where a string is expected. I cannot see where that might be offhand but since it works in the Browser but not Electron it might be that the underlying Javascript engines have a different tolerance for the issue. Can you run the same in a node debugger? Does one of the streams complain or break oddly? Can you narrow down which line this is coming from?