Develop Reference

Not able to send form data using axios in Nodejs?

I am trying to make POST request with axios but getting 400 bad request error message. I tried same request in POSTMAN but it worked correctly any idea what I am missing?
Code:
const TWITTER_UPLOAD_MEDIA_BASE_URL = 'https://upload.twitter.com/1.1/media/upload.json';
const url = "https://ik.imagekit.io/XXXXXXXXXXXX/test-upload_XXXXXXXXXXX.png";
const { data } = await axios.get(url);
const form = new FormData();
form.append('media', data);
const token = {
key: oauth_access_token,
secret: oauth_access_token_secret
};
const oauth = OAuth({
consumer: {
key: process.env.TWITTER_CONSUMER_KEY,
secret: process.env.TWITTER_CONSUMER_SECRET
},
signature_method: 'HMAC-SHA1',
hash_function: (baseString, key) => crypto.createHmac('sha1', key).update(baseString).digest('base64')
});
const authHeader = oauth.toHeader(oauth.authorize({
url: `${TWITTER_UPLOAD_MEDIA_BASE_URL}?media_category=tweet_image`,
method: 'POST'
}, token));
const result = await axios.post(`${TWITTER_UPLOAD_MEDIA_BASE_URL}?media_category=tweet_image`, form, {
headers: {
Authorization: authHeader["Authorization"],
'Content-type': "multipart/form-data",
"Content-Transfer-Encoding": "base64"
}
});
I also tried:
const url = "https://ik.imagekit.io/XXXXXXXXXXXX/test-upload_XXXXXXXXXXX.png";
const image = await axios.get(url, { responseType: 'arraybuffer' });
const raw = new Buffer.from(image.data).toString('base64');
const base64Image = "data:" + image.headers["content-type"] + ";base64," + raw;
const token = {
key: oauth_access_token,
secret: oauth_access_token_secret
};
const oauth = OAuth({
consumer: {
key: process.env.TWITTER_CONSUMER_KEY,
secret: process.env.TWITTER_CONSUMER_SECRET
},
signature_method: 'HMAC-SHA1',
hash_function: (baseString, key) => crypto.createHmac('sha1', key).update(baseString).digest('base64')
});
const authHeader = oauth.toHeader(oauth.authorize({
url: `${TWITTER_UPLOAD_MEDIA_BASE_URL}?media_category=tweet_image`,
method: 'POST'
}, token));
const result = await axios.post(`${TWITTER_UPLOAD_MEDIA_BASE_URL}?media_category=tweet_image`, {media_data: base64Image}, {
headers: {
Authorization: authHeader["Authorization"],
'Content-type': "multipart/form-data",
"Content-Transfer-Encoding": "base64"
}
});
Bot snippets not working.
Till line 27 (first code snippet) everything is correct. Issue is from line 28 with axios and form content
POSTMAN screenshots of successful request:
I think axios has got some serious issues:
https://github.com/mzabriskie/axios/issues/789
https://github.com/axios/axios/issues/1006
https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/api-reference/post-media-upload
Twitter media upload guide: https://developer.twitter.com/en/docs/twitter-api/v1/media/upload-media/uploading-media/media-best-practices

firstly download your image like:
const url = 'https://url-to-your-image'
const { data } = await axios.get(url, {
responseType: 'stream',
})
after that you can append the image directly in formdata
const formData = new FormData()
formData.append('media', stream)
and finally upload
const result = await axios.post(`${TWITTER_UPLOAD_MEDIA_BASE_URL}?
media_category=tweet_image`, formData, {
headers: {
Authorization: authHeader["Authorization"],
'Content-type': "multipart/form-data",
"Content-Transfer-Encoding": "base64"
}
});
If you still get a problem let me know

Convert file binary to FormData server-side using NodeJS

I'm building a web application that requires to me to receive file binaries on a server-side endpoint, convert those binaries to FormData, then POST them to another endpoint. The second endpoint only accepts "multipart/form-data".
I'm using axios to create the requests, and NodeJS on the server.
The files are posted from the client as form data, but I'm unable to manipulate them as such once they reach the server.
How do I compel file binaries to FormData in my server-side scripts?
Client:
const handleImageSubmit = async (attachments) => {
try {
let image
if (attachments.length) {
const formData = new FormData()
for (let i = 0; i < attachments.length; i += 1) {
formData.append(
`files.${attachments[i].name}`,
attachments[i],
attachments[i].name
)
}
const config = {
headers: { 'Content-Type': 'multipart/form-data' },
}
image = await axios.post(
'/api/proxy/upload',
formData,
config
)
}
console.log('Files dispatched to server')
} catch (err) {
console.error(err)
}
}
Server-side:
const upload = async (req, res) => {
try {
const data = new FormData()
// Convert the binaries in req.body into FormData
const myFiles = ...
data.append('files',
myFiles)
const config = {
method: 'post',
url: `${process.env.API_URL}/upload`,
headers: {
Authorization: `Bearer ${req.headers['auth-token']}`,
...data.getHeaders()
},
data: data,
}
const signal = await axios(config)
res.json(signal.data)
} catch (err) {
console.error(err)
res.status(fetchResponse?.status || 500).json(err)
}
}

Node.js Busboy not firing "file" event before "finish" event

I am just trying to upload a file from the browser through Node.js/Next.js to S3, and busboy is not emitting the file event.
Here is my FE code essentially:
Upload.prototype.to = function(options, fn){
// TODO: x-browser
var path;
if (typeof options == 'string') {
path = options;
options = {};
} else {
path = options.path;
}
var self = this;
fn = fn || function(){};
var req = this.req = new XMLHttpRequest;
req.open('POST', path);
req.onload = this.onload.bind(this);
req.onerror = this.onerror.bind(this);
req.upload.onprogress = this.onprogress.bind(this);
req.onreadystatechange = function(){
if (4 == req.readyState) {
var type = req.status / 100 | 0;
if (2 == type) return fn(null, req);
var err = new Error(req.statusText + ': ' + req.response);
err.status = req.status;
fn(err);
}
};
var key, headers = options.headers || {};
for (key in headers) {
req.setRequestHeader(key, headers[key]);
}
var body = new FormData;
body.append(options.name || 'file', this.file);
var data = options.data || {};
for (key in data) {
body.append(key, data[key]);
}
req.send(body);
};
All that's doing is making the request to /api/<path> with the file name for the selected jpeg I am trying to upload. This is what I receive in the server request body:
body: '------WebKitFormBoundaryWbaXO8J6c8aI7Q4B\r\n' +
'Content-Disposition: form-data; name="file"; filename="1-profile.jpg"\r\n' +
'Content-Type: image/jpeg\r\n' +
'\r\n' +
'����\x00\x10JFIF...
The headers include these:
connection: 'keep-alive',
'content-length': '41079',
pragma: 'no-cache',
'cache-control': 'no-cache',
'content-type': 'multipart/form-data; boundary=----WebKitFormBoundaryWbaXO87Kc9aI2Q4B',
accept: '*/*',
origin: 'http://localhost:3000',
My server code looks like this:
import fetchId from '../../../../utils/get-next-id-from-pg'
import Busboy from 'busboy'
import s3 from 'initializers/s3'
export default async function(req, res) {
if (req.method === 'POST') {
const id = await fetchId('image')
return new Promise((resolve, rej) => {
const busboy = new Busboy({ headers: req.headers });
busboy.on('field', function(fieldname, val, fieldnameTruncated, valTruncated, encoding, mimetype) {
console.log('Field [' + fieldname + ']: value: ' + inspect(val));
})
busboy.on('file', function(fieldname, file, filename, encoding, mimetype) {
console.log('params')
const params = {
Bucket: 'mybucket123',
Key: id,
Body: file
}
s3.upload(params, (err, data) => {
console.log('s3', err, data)
res.setHeader('Connection', 'close');
res.status(200).json({ records: [ { id } ]})
resolve()
})
})
busboy.on('finish', function() {
console.log('finish')
})
req.pipe(busboy);
})
} else {
res.writeHead(405, { 'content-type': 'text/plain' });
res.end("Method not allowed. Send a POST request.");
return;
}
}
It logs finish and that's it, it doesn't log either file or field. What am I missing here? It doesn't log params inside of the on('file') handler, or any of the s3 stuff. I am starting to dig into the busboy source code but is there any better way? What am I doing wrong, or what is a library that works with Next.js?
I am using Next.js and dotenv, could that be causing weird issues?

Hooray I figured it out, add this to the Next.js route:
export const config = {
api: {
bodyParser: false,
},
}
https://nextjs.org/docs/api-routes/api-middlewares#custom-config

Is there a way to store locally pdf files from an api response?

I've been trying to save pdf files locally from an api response. Here´s what i´ve tried so far.
const config = {
headers: {
Authorization: `Bearer ${token}`,
responseType: 'blob',
},
};
const response = await axios.get(
apiURL,
config,
);
fs.writeFile(
`filename.pdf`,
response.data,
(err) => {
if (err) return console.log(err);
else console.log('file saved succsefully');
},
);
I don't get any errors, but it doesn't write the file either.
Here is what i get from the response.data

responseType should not be in the headers config.
You could also write the response as a stream to the file using responseType: "stream".
const config = {
headers: {
Authorization: `Bearer ${token}`,
},
responseType: 'stream'
};
const response = await axios.get(apiURL, config)
const fileStream = fs.createWriteStream('filename.pdf')
response.data.pipe(fileStream)

Blob is not a valid data type for fs.writeFile.
Try the following:
const config = {
headers: {
Authorization: `Bearer ${token}`
},
responseType: 'blob'
};
const response = await axios.get(
url,
config
);
const arrayBuffer = await response.data.arrayBuffer();
const buffer = Buffer.from(arrayBuffer);
const filePath = `./${response.headers['content-disposition'].split('=')[1]}.pdf`;
await fs.writeFile(filePath, buffer).then(() => {
console.log('file saved succsefully');
}).catch((err) => console.log(err));

PDF is blank when opened

I've been at this for hours and I'm not getting anywhere. I'm trying to download a PDF file from my server. I'm currently running this on localhost. When I open the PDF the PDF is a white blank page. Can someone please let me know what I'm missing. I've checked the PDF saved on the server and it is fine.
App - React, Express.
NPM - Axios, file-saver, html-pdf
Path - Front end React
.then(() =>
axios.get(
'http://localhost:3000/retailers/fetch/pdf',
{
headers: {
'Content-type': 'application/json',
'x-auth-token': this.props.token
}
},
{ responseType: 'blob' }
)
)
.then((res) => {
console.log('dd', res);
const pdfBlob = new Blob([res.data], { type: 'application/pdf' });
saveAs(pdfBlob, 'newPdf.pdf');
});
Path - Backend express
const options = {
root: path.join(__dirname, '../'),
dotfiles: 'deny',
headers: {
'x-timestamp': Date.now(),
'x-sent': true
}
};
var fileName = 'test.pdf';
res.sendFile(fileName, options, function (err) {
if (err) {
console.log('err', err);
next(err);
}
});

So the answer is simple. responseType: 'blob' was in the wrong place.
axios.get(
'http://localhost:3000/retailers/fetch/pdf',
{
headers: {
'Content-type': 'application/json',
'x-auth-token': this.props.token
},
responseType: 'blob'
}
)