Develop Reference

Is there a way to store locally pdf files from an api response?

I've been trying to save pdf files locally from an api response. Here´s what i´ve tried so far.
const config = {
headers: {
Authorization: `Bearer ${token}`,
responseType: 'blob',
},
};
const response = await axios.get(
apiURL,
config,
);
fs.writeFile(
`filename.pdf`,
response.data,
(err) => {
if (err) return console.log(err);
else console.log('file saved succsefully');
},
);
I don't get any errors, but it doesn't write the file either.
Here is what i get from the response.data

responseType should not be in the headers config.
You could also write the response as a stream to the file using responseType: "stream".
const config = {
headers: {
Authorization: `Bearer ${token}`,
},
responseType: 'stream'
};
const response = await axios.get(apiURL, config)
const fileStream = fs.createWriteStream('filename.pdf')
response.data.pipe(fileStream)

Blob is not a valid data type for fs.writeFile.
Try the following:
const config = {
headers: {
Authorization: `Bearer ${token}`
},
responseType: 'blob'
};
const response = await axios.get(
url,
config
);
const arrayBuffer = await response.data.arrayBuffer();
const buffer = Buffer.from(arrayBuffer);
const filePath = `./${response.headers['content-disposition'].split('=')[1]}.pdf`;
await fs.writeFile(filePath, buffer).then(() => {
console.log('file saved succsefully');
}).catch((err) => console.log(err));

Nodejs .Unable to send oauth v1 params in get request with axios

I wanted to make a request to ADP with autho1.0a
I was able to make successful requests as I wanted in postman but not through my application.
postman screenshot
npm module used
similar post
Code I tried
Part:1 Signature generation
const crypto = require('crypto')
const OAuth = require('oauth-1.0a')
const oauthObj = {};
function hash_function_sha1(base_string, key) {
return crypto
.createHmac('sha1', key)
.update(base_string)
.digest('base64')
}
oauthObj.getSignature = async payload => {
const { consumerKey,consumerSecret,apiUrl,method} = payload;
const oauth = OAuth({
consumer: { key: `${consumerKey}`, secret: `${consumerSecret}` },
signature_method: 'HMAC-SHA1',
hash_function: hash_function_sha1,
});
const request_data = {
url: `${apiUrl}`,
method: `${method}`
}
const token = {}
// return oauth.toHeader(oauth.authorize(request_data, token));
console.log('header string-----',oauth.toHeader(oauth.authorize(request_data, token)));
return oauth.authorize(request_data, token);
}
module.exports = oauthObj;
Part 2 : Axios Call
let oauthData=`oauth_consumer_key=${consumerKey}&oauth_signature_method=HMAC-SHA1&oauth_timestamp=${oauthTimestamp}&oauth_nonce=${oauthNonce}&oauth_version=1.0&oauth_signature=${oauthSignature}= HTTP/1.1`;
const eventData = await axios({
url:`${apiUrl}?${oauthData}`,
// url:`${apiUrl}?${oauthHeader.Authorization}`,
method:'GET',
headers:{
// ...oauthHeader,
'Authorization':'OAuth',
'Accept': 'application/json',
// "Authorization": `'OAuth oauth_consumer_key="${consumerKey}", oauth_nonce="${oauthNonce}", oauth_signature="${oauthSignature}", oauth_signature_method="HMAC-SHA1", oauth_timestamp="${oauthTimestamp}", oauth_version="1.0"`
}
});
Expected Result:
{
"code": "Gone",
"message": "Event with token 954c183f-26e0-4f9e-b452-c089aaf9842f has already been consumed."
}
Receiving error:
response: {
status: 401,
statusText: 'Unauthorized',
headers: {
What might have gone wrong ?

Try using request node package oauth option
request.get(`${apiUrl}?${oauthData}`, {
oauth: {
consumer_key: '..',
consumer_secret: '..',
},
headers: {
Accept: 'application/json'
},
}, function (err, res, body) {
console.log(body);
})

Read file from Amazon S3 with nodeJS and post it to another server with axios

Im trying to read a file (image) from amazon S3 and post it to another server with multipart/form.
let imageParams = { Bucket: 'my-bucket', Key: 'imageName.jpg' };
let formData = new FormData();
formData.append('file', s3.getObject(imageParams).createReadStream());
let apiResponse = await api.post("/fileUpload", formData,
{ params: { token: process.env.API_TOKEN } },
{ headers: {'Content-Type': 'multipart/form-data' } } );
But im not managing it to work, it returns me:
Error: Request failed with status code 415
maybe im misunderstanding how the createReadStream() works?

Use concat for pipe the stream. Otherwise form data send only the first chunk of stream, and the server don't know how to handle it.
For example
const axios = require('axios');
const {S3} = require('aws-sdk');
const FormData = require('form-data');
const s3 = new S3();
var concat = require('concat-stream')
const api = axios.default.create({
baseURL: 'http://example.com',
})
const readStream = s3.getObject({Bucket: 'bucket', Key: 'file'}).createReadStream();
readStream.pipe(concat(filebuffer => {
const formData = new FormData();
formData.append('file', filebuffer);
formData.getLength((err, length) => {
console.log(length)
const headers = formData.getHeaders({'content-length': length})
console.log({headers})
return api({
method: 'post',
url: "/upload",
headers: headers,
data: formData
})
})
}))

Increasing maxContentLength and maxBodyLength in Axios

I am using "Axios" to call a WCF method that takes as parameter file information and content.
The file is read and sent as a base64 encoded string.
My issue is that when the file size exceeds a certain limit, AXIOS throws an exception: "Error: Request body larger than maxBodyLength limit".
I looked up the issue and found that all solutions suggest increasing the maxContentLength / maxBodyLength parameters in the AXIOS configuration object, but did not succeed.
Find Below an implemented test case in node.js:
var axios = require('axios');
var fs = require('fs');
var path = require('path')
var util = require('util')
let readfile = util.promisify(fs.readFile)
async function sendData(url,data) {
let params = data
let resp = await axios({
method: 'post',
url: url,
data: JSON.stringify(params),
headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' }
// maxContentLength: 100000000,
// maxBodyLength: 1000000000
}).catch(err => {
throw err;
})
return resp;
}
async function ReadFile(filepath) {
try{
let res = await readfile(filepath,'base64')
let filename = path.basename(filepath).split('.').slice(0, -1).join('.')
let ext = path.extname(filepath)
return {data:res,fext:ext,fname:filename}
let x = 1
}
catch(err)
{
throw err
}
}
(async () => {
try {
let img = await ReadFile('Files/1.pdf')
let res = await sendData('http://183.183.183.242/EMREngineEA/EMRWS.svc/web/EMR_TestUploadImg',img)
console.log(res)
}
catch (ex) {
console.log(ex)
}
}
)();
In my case, the pdf file is 20 MB, upon running, an error is thrown.
"Error: Request body larger than maxBodyLength limit"
I tried to setting the maxContentLength: 100000000, maxBodyLength: 1000000000
as presented above, but did not succeed.
Your help is appreciated.

The maxBodyLength seems to work for me in this simple test, I upload data to a local Express server. If I try to upload more than the maxBodyLength I get the same error you're getting. So I suspect there's something more, like a redirect happening in your case that's triggering the error.
There is an issue logged for axios here that seems to reference the problem, it suggests setting maxContentLength to Infinity (as the other commenter suggests).
e.g.
maxContentLength: Infinity,
maxBodyLength: Infinity
Test code below:
const axios = require("axios");
function generateRandomData(size) {
const a = Array.from({length: size}, (v, k) => Math.floor(Math.random()*100));
return { data: a, id: 1 };
}
async function uploadData(url, size) {
let params = generateRandomData(size);
let stringData = JSON.stringify(params);
console.log(`uploadData: Uploading ${stringData.length} byte(s)..`);
let resp = await axios({
method: 'post',
url: url,
data: stringData,
headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' },
maxContentLength: 100000000,
maxBodyLength: 1000000000
}).catch(err => {
throw err;
})
console.log("uploadData: response:", resp.data);
return resp;
}
uploadData("http://localhost:8080/upload", 10000000);
Corresponding server code:
const express = require("express");
const port = 8080;
const app = express();
const bodyParser = require('body-parser')
app.use(bodyParser.json({limit: '50mb'}));
app.post('/upload', (req, res, next) => {
console.log("/upload: Received data: body length: ", req.headers['content-length']);
res.json( { status: 'ok', bytesReceived: req.headers['content-length']});
})
app.listen(port);
console.log(`Serving at http://localhost:${port}`);

OAuth "unsupported_grant_type" Discord API

I'm trying to make the discord OAuth work. In the doc, it is necessary to generate a code, it works very well this step but after it is to generate the token. It asks to make a POST request with the right parameters but it always brings me the error: {"error":"unsupported_grant_type"}
My code:
app.get('/discord/callback', async function (req, res) {
if (req.query.code === undefined || req.query.code == '') return next();
const response = await fetch("https://discordapp.com/api/v6/auth2/token", {
method: 'POST',
headers: {
"Content-type": "application/x-www-form-urlencoded"
},
data: {
client_id: process.env.CLIENT_ID,
client_secret: process.env.CLIENT_SECRET,
code: req.query.code,
redirect_uri: redirect,
grant_type: "authorization_code",
scope: "identify"
}
});
const json = await response.json();
debug('%O', json);
res.send(json);
});
Doc:
def exchange_code(code):
data = {
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET,
'grant_type': 'authorization_code',
'code': code,
'redirect_uri': REDIRECT_URI,
'scope': 'identify email connections'
}
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
r = requests.post('%s/oauth2/token' % API_ENDPOINT, data, headers)
r.raise_for_status()
return r.json()
Thanks for your help

Your headers are:
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
which means that it also expects the data as form data and NOT json.
So this should work:
app.get('/discord/callback', async function (req, res) {
if (req.query.code === undefined || req.query.code == '') return next();
const params = new URLSearchParams();
params.append('client_id', process.env.CLIENT_ID);
params.append('client_secret', process.env.CLIENT_SECRET);
params.append('grant_type', 'authorization_code');
params.append('code', code);
params.append('redirect_uri', redirect);
params.append('scope', 'identify');
const response = await fetch("https://discordapp.com/api/v6/auth2/token", {
method: 'POST',
body: params
headers: {
"Content-type": "application/x-www-form-urlencoded"
},
});
const json = await response.json();
debug('%O', json);
res.send(json);
});
You can refer this for better understanding: https://www.npmjs.com/package/node-fetch#post-with-form-parameters

I encountered this issue today as well, and inspired by Aakash Sharma's answer, I build a little utility function(in typescript) that will convert an object to that required format:
export const jsonToUrlParams = (data: Record<string, any>) => {
const params = new URLSearchParams();
for (const key in data) {
params.append(key, `${data[key]}`);
}
return params;
};