Develop Reference

TypeError: fs.readFileSync is not a function in Next.js

In a JSON file, I save data for a bot in total.json.
{ "guilds": 3, "users": 21 }
In my index.tsx, I would like to put this data in the page, so I try this:
import fs from 'fs';
function stats() {
const botcount = JSON.parse(fs.readFileSync(`../util/total.json`, { encoding: 'utf8' }));
const userscount = botcount.users;
console.log(userscount);
return userscount;
}
In the terminal, the function correctly returned the number (21), but in my page, I found this error:
TypeError: fs__WEBPACK_IMPORTED_MODULE_6___default(...).readFileSync is not a function

You can only use fs module in Node js NOT in browser. To access JSON data from a file in Nextjs you can use axios or fetch. Here is an example with axios
import axios from 'axios';
async function stats() {
var {data} = await axios.get("http://localhost:8888/utils/total.json");//Change this to your url
const botcount = JSON.parse(data)
const userscount = botcount.users;
console.log(userscount);
return userscount;
}

As #JaivBhup already mentioned, you can't use fs since it's not browser compatible.
A better approach IMO is to use a backend and fetch data from there (axios is a great package for this). If you don't have a backend of some kind, you should consider using the Next.js api routes.
You can use it as if you have been using Node.js!
See the docs or this could also be useful for you.
// File: pages/api/my-json.js
import fs from 'fs'
import path from 'path'
export default (req, res) => {
// Let's say your json is in /public/assets/my-json.json
const filePath = path.resolve('./public', 'assets', 'my-json.json');
const json = fs.readFileSync(filePath);
res.statusCode = 200
res.json(json);
}
The important part is path.resolve(...), which instructs vercel to include the scanned path in the serverless lambda. The shown code works to read images (or other files from your fs) both locally and remotely on vercel!
I tweaked it a little bit, so it loads the json file instead of filenames.

How to save big object to file nodejs?

I have a big object that I need to send from server (nodejs) to client.
But every time I try send I get "invalid string length" error.
And it's ok, because the object is really big. That's why I'd like to save it to file and then send the file to client.
I don't know the depth of the object. The object itself is and octree.
I don't have any code of saving an object to file, because every time I think about this it leads me to stringify the object, that latter leads to "invalid string length".
Here is a screenshot of the object. Every q(n) key has the same recursive structure as result key.
Thanks!

Firstly try to save in a JSON file and send JSON file directly to client-side
const fs = require('fs');
fs.writeFileSync("file_name.json", data);
res.header("Content-Type",'application/json');
res.sendFile(path.join(__dirname, 'file_name.json'));

A good solution to handling large data transfer between server and client is to stream the data.
Pipe the result to the client like so.
const fs = require('fs');
const fileStream = fs.createReadStream("path_to_your_file.json");
res.writeHead(206, {'Content-Type': 'application/json'})
fileStream.pipe(response)
or follow this blog which stringifies each element at a time and concatenates them.
I will suggest you stream it though.

You could try something like this, I'm unsure if Express would throw the same length error by using res.json.
// Assuming you're using Express and data is valid JSON.
res.json(data);

node fs read file from given URL or GCS

When run bellow code it's give error, Reading file from directory working perfect but when pass url it's give file not found error. I've check fs.statSync accept url.
const stat = fs.statSync('http://techslides.com/demos/sample-videos/small.mp4');
Error: ENOENT: no such file or directory, stat 'http://techslides.com/demos/sample-videos/small.mp4'

fs.statSync() can take a URL, but ONLY if that URL is a file:// URL.
It is not clear what you would want to do if the argument was actually an http:// URL. You could check to see if it was not a file URL and then attempt to fetch the contents of the URL to see if it exists using a library such as got().
But, fetching data from another server with http will not be synchronous so you will have to change the design of your function to return a promise instead of a synchronous API.

That's because its hosted on a web-server, you need to send a HTTP GET to fetch it locally.
Install the axios package and issue a HTTP GET request to fetch the remote resource from the web-server.
npm install --save axios
Here's a program of the general idea
const fs = require('fs');
const axios = require('axios');
const { promisify } = require('util');
const writeFilePromise = promisify(fs.writeFile);
(async () => {
const url = 'http://techslides.com/demos/sample-videos/small.mp4';
const response = await axios.get(url);
if (response.data) {
await writeFilePromise('small.mp4', response.data);
}
})();

How to store byte array into local file in JavaScript?

I was working with openssl certificate.
So what I want to achieve is I download the certificate from my API, it returns me bytes arrays. For example:
0�
g0�
' *�H��
��
�
0�
And I try to write these into a file, and use some other function to read and convert to PEM file so I can access it with another API. (The initial format of this cert is PFX).
I've checked and if I download the certificate via Postman, via the 'Send and Download' button I get something like this:
0‚
g0‚
' *†H†÷
 ‚
‚
0‚
It is slightly different than what I direct wrote into a file. How can I convert from that to this? I faced error from the next step is because this is not a valid PFX file. The error from the other API reads as below:
DecodeToAsn:
ASN.1 length cannot be more than 4 bytes in definite long-form.
This error typically occurs when trying to decode data that is not ASN.1
A common cause is when decrypting ASN.1 data with an invalid password,
which results in garbage data. An attempt is made to decode the garbage bytes
as ASN.1, and this error occurs...
--DecodeToAsn
Failed to decode PFX ASN.1 for integrity verification.
So how can I store bytes arrays into local file correctly? Or is there any way to convert the byte arrays to what I have via Postman?
As of now I only write the bytearray directly to the file, below are the codes:
async function downloadCertificate() {
try {
let pfx = new chilkat.Pfx();
let downloadedPfxByteArray = await Api.downloadCertificate(id);
let pfxFileLocation = `${process.cwd()}\\media\\CERTFILE.pfx`;
fs.writeFileSync(pfxFileLocation, downloadedPfxByteArray);
pfx.LoadPfxFile(pfxFileLocation, 'password');
let strPem = pfx.ToPem();
console.log(strPem);
return pemValue;
} catch (error) {
console.log(`READ PFX FAIL ! ${error}`);
}
}
Thanks for reading and appreciates if anyone could help!

#!/usr/bin/env node
const fetch = require('node-fetch');
const fs = require('fs').promises;
async function main() {
await fs.writeFile(
'/tmp/so-58603015.pfx',
Buffer.from(await (await fetch('…url…')).arrayBuffer())
);
}
main();

Sending a file to the client from Node.js with Express

I have a unique situation in terms of difficulty.
I need to send HTML to the server, have the server convert the HTML to a PDF, send that PDF back to the client, and then download the PDF using client-side code.
I have to do it this way because I'm using client-side routing, so the only way I can access my endpoint that should perform this action is via a GET Request with Ajax or Fetch from client-side JavaScript. I am aware of res.sendFile(), but that attempts to render the file in the browser - I don't want that - rather, I want to be able to use client-side code to download the file.
Is it possible, then, to send a PDF file from temporary storage on the server down to the client, allowing client-side code to do whatever it wants to the file thereafter - in my case, downloading it?
I don't believe I have to provide any code because this is more of a theoretical question.

My issue stemmed from the fact that I could not just use res.sendFile() or res.download() from Express because the route was not being accessed by the browser URL bar, rather, my application uses client-side routing, and thus I had to make an HTTP GET Request via Fetch or XMLHttpRequest.
The second issue is that I needed to build the PDF file on the server based on an HTML string sent from the client - so again, I need to make a GET Request sending along a request body.
My solution, then, using Fetch, was to make the Get Request from the client:
fetch('/route' , {
method: 'GET',
body: 'My HTML String'
});
On the server, I have my code that converts the HTML string to a PDF, using the HTML-PDF Node module, and then, I convert that file to a Base64 String, setting the MIME Type and appending data:application/pdf;base64,.
app.get('/route', (req, res) => {
// Use req.body to build and save PDF to temp storage (os.tempdir())
// ...
fs.readFile('./myPDF.pdf', (err, data) => {
if (err) res.status(500).send(err);
res.contentType('application/pdf')
.send(`data:application/pdf;base64,${new Buffer.from(data).toString('base64')}`);
});
});
Back on the client, I have my aforementioned Fetch Request, meaning I just need to tack on the promise to get the response:
fetch('/route', {
method: 'POST',
body: 'My HTML String' // Would define object and stringify.
})
.then(res => res.text())
.then(base64String => {
// Now I just need to download the base64String as a PDF.
});
To make the download, I dynamically create an anchor tag, set its href attribute to the Base64 String in the response from the server, give it a title, and then programmatically click it:
const anchorTag = document.createElement('a');
anchorTag.href = base64String;
anchorTag.download = "My PDF File.pdf";
anchorTag.click();
So, all together and on the client:
fetch('/route', {
method: 'POST',
body: 'My HTML String' // Would define object and stringify.
})
.then(res => res.text())
.then(base64String => {
const anchorTag = document.createElement('a');
anchorTag.href = base64String;
anchorTag.download = "My PDF File.pdf";
anchorTag.click();
});
The solution for using an anchor tag to trigger the download came from another StackOverflow answer. It's also important to note that Base64 Encoding is not very efficient. Better solutions exist, but for my purposes, Base64 will work fine.
It is also imperative to note that Base64 Encoding is precisely that - an Encoding Scheme, not, I repeat, not an Encryption Scheme. So if your PDF files contain privileged information, you would likely want to add token authentication to the endpoint and encrypt the file.