When run bellow code it's give error, Reading file from directory working perfect but when pass url it's give file not found error. I've check fs.statSync accept url.
const stat = fs.statSync('http://techslides.com/demos/sample-videos/small.mp4');
Error: ENOENT: no such file or directory, stat 'http://techslides.com/demos/sample-videos/small.mp4'
fs.statSync() can take a URL, but ONLY if that URL is a file:// URL.
It is not clear what you would want to do if the argument was actually an http:// URL. You could check to see if it was not a file URL and then attempt to fetch the contents of the URL to see if it exists using a library such as got().
But, fetching data from another server with http will not be synchronous so you will have to change the design of your function to return a promise instead of a synchronous API.
That's because its hosted on a web-server, you need to send a HTTP GET to fetch it locally.
Install the axios package and issue a HTTP GET request to fetch the remote resource from the web-server.
npm install --save axios
Here's a program of the general idea
const fs = require('fs');
const axios = require('axios');
const { promisify } = require('util');
const writeFilePromise = promisify(fs.writeFile);
(async () => {
const url = 'http://techslides.com/demos/sample-videos/small.mp4';
const response = await axios.get(url);
if (response.data) {
await writeFilePromise('small.mp4', response.data);
}
})();
Related
In a JSON file, I save data for a bot in total.json.
{ "guilds": 3, "users": 21 }
In my index.tsx, I would like to put this data in the page, so I try this:
import fs from 'fs';
function stats() {
const botcount = JSON.parse(fs.readFileSync(`../util/total.json`, { encoding: 'utf8' }));
const userscount = botcount.users;
console.log(userscount);
return userscount;
}
In the terminal, the function correctly returned the number (21), but in my page, I found this error:
TypeError: fs__WEBPACK_IMPORTED_MODULE_6___default(...).readFileSync is not a function
You can only use fs module in Node js NOT in browser. To access JSON data from a file in Nextjs you can use axios or fetch. Here is an example with axios
import axios from 'axios';
async function stats() {
var {data} = await axios.get("http://localhost:8888/utils/total.json");//Change this to your url
const botcount = JSON.parse(data)
const userscount = botcount.users;
console.log(userscount);
return userscount;
}
As #JaivBhup already mentioned, you can't use fs since it's not browser compatible.
A better approach IMO is to use a backend and fetch data from there (axios is a great package for this). If you don't have a backend of some kind, you should consider using the Next.js api routes.
You can use it as if you have been using Node.js!
See the docs or this could also be useful for you.
// File: pages/api/my-json.js
import fs from 'fs'
import path from 'path'
export default (req, res) => {
// Let's say your json is in /public/assets/my-json.json
const filePath = path.resolve('./public', 'assets', 'my-json.json');
const json = fs.readFileSync(filePath);
res.statusCode = 200
res.json(json);
}
The important part is path.resolve(...), which instructs vercel to include the scanned path in the serverless lambda. The shown code works to read images (or other files from your fs) both locally and remotely on vercel!
I tweaked it a little bit, so it loads the json file instead of filenames.
Second time posting on StackOverflow so I apologize for any mistakes.
Please bear with me.
Same with the title; How do you read contents of a discord attachment let's say a .txt file and print the contents?
I have tried with fs but unfortunately failed and I have also searched the documentation but failed also.
Ideas?
You can't use the fs module for this as it only deals with local files. When you upload a file to the Discord server, it gets uploaded to a CDN and all you can do is grab the URL of this file from the MessageAttachment using the url property.
If you need to get a file from the web, you can fetch it from a URL using the built-in https module, or you can install one from npm, like the one I used below, node-fetch.
To install node-fetch, run npm i node-fetch in your root folder.
Check out the working code below, it works fine with text files:
const { Client } = require('discord.js');
const fetch = require('node-fetch');
const client = new Client();
client.on('message', async (message) => {
if (message.author.bot) return;
// get the file's URL
const file = message.attachments.first()?.url;
if (!file) return console.log('No attached file found');
try {
message.channel.send('Reading the file! Fetching data...');
// fetch the file from the external URL
const response = await fetch(file);
// if there was an error send a message with the status
if (!response.ok)
return message.channel.send(
'There was an error with fetching the file:',
response.statusText,
);
// take the response stream and read it to completion
const text = await response.text();
if (text) {
message.channel.send(`\`\`\`${text}\`\`\``);
}
} catch (error) {
console.log(error);
}
});
reply to #Andryxa, maybe you can use this with external APIs like a transcription service in case of audio files or to send requests to already created bots from services like dialogflow to replies to the messages
How to save and write data acquired by request or fetch as executable?
For example I use
require('request').get('https://github.com/harujisaku/mini-player/raw/master/mini-player.exe', async (e, r, b) => {
require('fs').writeFileSync(path+'test.exe', b);
});
But instead of working .exe file I get a broken file. How to save and write file from server data (from github as example) correctly, so .exe file would not break?
This might help you, credits to 'Michelle Tilley' from : https://stackoverflow.com/a/11944984/5203821
const http = require('http');
const fs = require('fs');
const file = fs.createWriteStream("mini-player.exe");
const request = http.get("https://github.com/harujisaku/mini-player/raw/master/mini-player.exe", function(response) {
response.pipe(file);
});
I have to choose one of these languages:
Python (with Selenium or any suggestion)
Javascript (with node with any module)
Bash (with curl for example)
To do the following:
Make a request to an API (Scrapy cloud) and get some value, in my case I just need the id of the response:
{"count": 1, "id": "195457/7/19", "width": 32, "height": 27, "started_time": "2017-06-22T08:20:26", "total": 1, "status": "ok"}
And then make another request with the id to download that provides a download to a file with a CSV/JSON format.
What I tried:
Python:
With Selenium (Firefox driver) open and get the id, it works fine but when I try to download the file with the next API request it asks me for what I want to do with the file (download or open with...). So, as I have to interact with the dialog it is not viable.
Javascript:
I found a module to download files but it is just to download files as images from image web URLs and not for download a file (like the Linux wget command).
Bash:
With curl it works but I can just get the whole response and then I cannot get the id value so I cant continue with what I want. Also I tried to download de file of the second step and this works fine with a simple curl -o myfile.csv URL
Any help would be appreciated. Thanks for reading!
Here is a node version. Its quite broad but the 2 main functions are the callApi and downloadFile.
I dont know the structure of your API url so for me now I have mocked some simple ones - change to what you need.
You will need to npm install request and update the variables to match your API.
index.js
const request = require('request');
const http = require('http');
//const https = require('https'); maybe required
const fs = require('fs');
const apiEndPoint = 'http://scrapycloud?someparam=';
const fileName = 'data.csv';
const assetEndPoint = 'http://assetUrl?id=';
// This will call your api and get the asset id then calls the downloadFile function.
function callApi(assetId, callback) {
request(apiEndPoint + assetId, function (error, response, body) {
if (error) {
return callback(error);
}
const info = JSON.parse(body);
const assetId = info.id;
downloadFile(assetId, callback);
});
}
// This function creates a writeSteam to save a file to your local machine, performs a http request to the assets and pipes it back into the write stream
function downloadFile(assetId, callback) {
var file = fs.createWriteStream(fileName);
//use the following line if your requests needs to be https
//var request = https.get(assetEndPoint + assetId, function (response) {
var request = http.get(assetEndPoint + assetId, function (response) {
response.pipe(file);
file.on('finish', function () {
file.close(callback);
});
}).on('error', function (err) {
fs.unlink(dest);
if (callback) callback(err.message);
});
}
// Called when everything is finished or an error
function complete(err) {
if (err) {
return console.log(err);
}
console.log('file downloaded');
}
// Starts the process, pass it an id and a callback
callApi('123131', complete);
I'm creating an Electron application and I want to stream an image to a file (so basically download it).
I want to use the native Fetch API because the request module would be a big overhead.
But there is no pipe method on the response, so I can't do something like
fetch('https://imageurl.jpg')
.then(response => response.pipe(fs.createWriteStream('image.jpg')));
So how can I combine fetch and fs.createWriteStream?
I got it working. I made a function which transforms the response into a readable stream.
const responseToReadable = response => {
const reader = response.body.getReader();
const rs = new Readable();
rs._read = async () => {
const result = await reader.read();
if(!result.done){
rs.push(Buffer.from(result.value));
}else{
rs.push(null);
return;
}
};
return rs;
};
So with it, I can do
fetch('https://imageurl.jpg')
.then(response => responseToReadable(response).pipe(fs.createWriteStream('image.jpg')));
Fetch is not really able to work with nodejs Streams out of the box, because the Stream API in the browser differs from the one nodejs provides, i.e. you can not pipe a browser stream into a nodejs stream or vice versa.
The electron-fetch module seems to solve that for you. Or you can look at this answer: https://stackoverflow.com/a/32545850/2016129 to have a way of downloading files without the need of nodeIntegration.
There is also needle, a smaller alternative to the bulkier request, which of course supports Streams.
I guess today the answer is with nodejs 18+
node -e 'fetch("https://github.com/stealify").then(response => stream.Readable.fromWeb(response.body).pipe(fs.createWriteStream("./github.com_stealify.html")))'
in the above example we use the -e flag it tells nodejs to execute our cli code we download the page of a interristing Project here and save it as ./github.com_stealify.html in the current working dir the below code shows the same inside a nodejs .mjs file for convinience
Cli example using CommonJS
node -e 'fetch("https://github.com/stealify").then(({body:s}) =>
stream.Readable.fromWeb(s).pipe(fs.createWriteStream("./github.com_stealify.html")))'
fetch.cjs
fetch("https://github.com/stealify").then(({body:s}) =>
require("node:stream").Readable.fromWeb(s)
.pipe(require("node:fs").createWriteStream("./github.com_stealify.html")));
Cli example using ESM
node --input-type module -e 'stream.Readable.fromWeb(
(await fetch("https://github.com/stealify")).body)
.pipe(fs.createWriteStream("./github.com_stealify.html"))'
fetch_tla_no_tli.mjs
(await import("node:stream")).Readable.fromWeb(
(await fetch("https://github.com/stealify")).body).pipe(
(await import("node:fs")).createWriteStream("./github.com_stealify.html"));
fetch.mjs
import stream from 'node:stream';
import fs from 'node:fs';
stream.Readable
.fromWeb((await fetch("https://github.com/stealify")).body)
.pipe(fs.createWriteStream("./github.com_stealify.html"));
see: https://nodejs.org/api/stream.html#streamreadablefromwebreadablestream-options
Update i would not use this method when dealing with files
this is the correct usage as fs.promises supports all forms of iterators equal to the stream/consumers api
node -e 'fetch("https://github.com/stealify").then(({ body }) =>
fs.promises.writeFile("./github.com_stealify.html", body)))'