Develop Reference

Send file with form-data and axios

I am trying to send a video to a videosite, I am able to upload the video using the REST api and postman, so I know the api works as intended. Now I want to do exatcly the same request using axios. I have code that looks like the example on how to use form-data and axios:
const form = new FormData();
const stream = fs.createReadStream(PATH_TO_FILE);
form.append('image', stream);
// In Node.js environment you need to set boundary in the header field 'Content-Type' by calling method `getHeaders`
const formHeaders = form.getHeaders();
axios.post('http://example.com', form, {
headers: {
...formHeaders,
},
})
.then(response => response)
.catch(error => error)
I get the error that data: 'Content-Length is required'
Any ideas?

May be I got your questions wrong , you want to add Content-Length in the header.
I can see you are uploading video stream. So first you have to calculate the data chunk length.
('Content-Length', File.getSize(stream))
Reference: Can I stream a file upload to S3 without a content-length header?
You can make the post request as multi-part type : 'Content-Type': 'multipart/form-data'.
It is preferable way to send large data to server.
You can check this link : How do I set multipart in axios with react?
If I got your question wrong , plese comment or reply . Thanks

The solution to my problem was to set Content-Length accordingly:
"Content-Length": fs.statSync(filePath)['size']

I think the best way to handle this is to actually use the FormData's own method:
const headers = { 'content-length': formData.getLengthSync(), ...formData.getHeaders() }
This will be more accurate because it includes any other data you may add.
To expound, if you are using a ReadStream, you must use the async function instead.
const { promisify } = require('util')
const getLength = promisify(formData.getLength.bind(formData))
const contentLength = await getLength()
const headers = { 'content-length': contentLength, ...formData.getHeaders() }

How to extract/parse the binary content from a Multipart response in NodeJS?

I want to extract the binary content from a multipart response that I have as follows -
I do not have access to any request object (Express/NodeJS). I just have this response object which I want to parse and extract the binary part of it.
EDIT -
To make things more clear -
I have an endpoint (Not a traditional Express app but FaaS) - called "/fileUpload"
This endpoint accepts all kind of requests - GET, POST, PUT (FaaS).
When I make a POST/PUT request with a file - testImage.PNG in the form data, I receive the content in the request object. FaaS runtime provides the req object as a function handler's parameter.
Now, when I print - request.body of the incoming POST request, I get the content as show in the image above.
If I try to upload this binary content directly to S3, it leads to corruption of the image which I believe is due to the presence of additional content in the multipart response like - Content-Disposition...
What I would like to do is, parse the additional multipart content like the Content-Type header and multipart boundary etc, so that I get only the image binary.

To extract the binary content, we need to remove/parse the header and boundary sections of it. There exists a library - parse-multipart which makes it easy to do so.
Example -
Assuming request header contains a content-type header.
var multipart = require('parse-multipart');
// The following section is the express endpoint/ FaaS handler
{
let header = req.headers["content-type"]
let boundary = header.split(" ")[1]
boundary = header.split("=")[1]
let body = event.data;
//console.log("Boundary - "+ boundary)
let parts = multipart.Parse(body, boundary);
for (var i = 0; i < parts.length; i++) {
var part = parts[i];
console.log(part)
var params = {
Bucket: s3bucket,
Key: part.file,
Body: part.data,
ACL: 'public-read',
Metadata: {
'Content-Type': part.type
}
};
//Upload the data to aws
const data = await s3.putObject(params).promise()
}
return parts
}

I struggled for many days with this problem. The solution was in the API GATE WAY > configuration > Binary types add multipart/form-data.

Observation on parse-multipart library: Be warned, this does not provide the 'name' directive of the multi-part body.
The documentation is very sketchy, reporting of error, really is not there.
Apparently valid multi-part body is not parsed and the reason would not be made clear by the library.
For more compliant processing, the libraries listed in the following URL could be considered:
https://www.npmjs.com/package/body-parser

How do I do a POST request using the Google Chrome Puppeteer library?

Good Day All,
I'm trying to do a POST request using the puppeteer headless chrome library. I can't seem to get the below code to work.
// Get csrf token
let token = await page.evaluate(() => document.querySelector('[name="CSRFToken"]').value);
let postResponse = await page.evaluate(async(token, cookies) => {
let response = fetch("/loyalty/points", {
method : 'POST',
cookie : cookies,
postData : 'CSRFToken=' + token,
}).then(response => response.text()).catch(error => console.log(error));
return response;
});
console.log('Final response');
console.log(postResponse);
I keep on getting an error that the CSRF token has not been set or that the cookie is invalid.
My question is, am I using the correct method in puppeteer to do a POST? If so, is there any way for me to do some debugging that I can see the actual POST request that was sent?
I appreciate any advice or help. Thanks

You are not creating a request body: hence the error. The postData attribute you set on the request object is not any known attribute, so it won't be set on the request either, meaning that the server will never see your CSRF token. You should look into the MDN docs on fetch().
I believe you should be all good by simply replacing postData with body, but it's hard to know without access to your endpoint. For all we know it might require special headers.
Given that you only post normal form data (which is implied by your key=value code), I would also start using the FormData objects provided by your browser to avoid manual coding of implementation details.
const formData = new FormData();
formData.append("CSRFToken", token);
const response = fetch("/loyalty/points", {
method : 'POST',
cookie : cookies,
body : formData,
headers : {
'cookie' : cookies,
/* other headers you need, possibly content-type (see below) */
},
}).then(response => response.text()).catch(error => console.log(error));
return response;
});
Caveat: using the FormData API will always set the content-type of the data to multipart/form-data. If your server for some reason doesn't support that encoding, and you need to use application/x-www-form-urlencoded (see here for difference),
you can't blindly change the Content-Type: you also need to url encode the content.
For debugging I would simply use a normal Chrome instance to see this. You should be able to run the code there and see the network requests in DevTools (where it would be immediately noticeable that you POST an empty request).

Upload a file with PUT/POST method on POSTMAN

I am trying to upload a file with POSTMAN to this URL
http://localhost:3000/bucket/test/files/
And should got result in my controller there :
put(request, response, args) {
//HERE IN THE REQUEST.BODY
console.log(request.body)
let fileManager = request.modules.VMFile;
let mimeTypes = ['image/jpeg', 'image/png', 'image/gif', 'image/x-icon', ' video/mpeg', 'text/html', 'video/x-msvideo', 'application/msword', 'application/pdf', 'application/vnd.ms-powerpoint', 'application/x-rar-compressed'];
let maxFileSize = 4 * 1024 * 1024;
fileManager.initUpload(mimeTypes, maxFileSize);
fileManager.receive((files) => {
fileManager.forEachFileContent(files, (file, content) => {
minioClient.putObject(request.body.bucket, request.body.name, content, file.size, file.mimetype, function (err, etag) {
response.setData("File uploaded").apply();
return console.log(err, etag)
})
});
fileManager.clearFilesFromTmp(files);
});
}
In POSTMAN I got this :
With nothing on headers but I could only PUT (or POST, I tried to change my route with POST but same issue) the name and bucket field... I got nothing on my files field...

While using Postman especially when you test file upload ensure that,
in Headers:
The Content-type field has been set as multipart/form-data in Headers.
in Body:
form-data option should remain as default.
Choose File option instead of text from the dropdown on the right side.
Type File in the text box where the placeholder is key.

You might be doing it right but sometimes POSTMAN does not work well. I wrote an API to accept both text and file.
While invoking service from Postman.
I set Content-Type as "application/json" and Accept as "application/json".
In body I pass the text and file
It was not working, I tried multiple times. I shut down postman and my laptop.
Woke up the next morning hit again and it worked. Below is the image of the working request.

Not all of David's answer worked for me when using express-fileupload. In short, I don't require the Content-Type header when using express-fileupload.
Note, these steps are for npm / express-fileupload / Postman
Ensure the header Content-Type is disabled. Setting this forces req.files to be undefined.
As per Davids answer, use form-data and set the key to whatever you require, and upload the file.
If you console.log(req.files.YOUR_KEY) in your express app, you should have an object with the uploaded file, in my case req.files.file.
Here's what postman should look like (once again, disable the Content-Type header):
Here's the output within the console when using console.log(req.files):

How to send enctype as multipart/form-data in jquery POST

I know there are many questions about encoding forms as multipart/form-data, but it seems as if most are related to uploading files. I am not interested in uploading files; nor do I have an actual form. I simply have a button which calls the following function on click:
$.post("http://html5.validator.nu/",
{
headers: { 'Content-Type': 'multipart/form-data' }, //this doesn't work !
content:"<!DOCTYPE html>"//the value of content doesn't matter at the moment
},
function(data){
print(data);
});
The AJAX request executes, but the response from validator.nu (the server) is this:
application/x-www-form-urlencoded not supported. Please use multipart/form-data
How can I encode the form using multipart/form-data without actually having a form (just in query)? Is there a line or two that I can add to the request that will do this?
Thanks

"multipart/form-data" isn't as important to the httpd as it is to the UA. Your browser sees enctype="multipart/form-data" and does special handling, the most significant part of which is in the way it shapes the MIME request it sends on to the httpd.
Here are the most relevant parts of an actual multipart/form-data HTTP request (captured using requestb.in):
Content-Length: 533
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryEknFTYZrogVNxemQ
------WebKitFormBoundaryEknFTYZrogVNxemQ
Content-Disposition: form-data; name="foo"; filename="foo.png"
Content-Type: image/png
PNG
[PNG data elided]
------WebKitFormBoundaryEknFTYZrogVNxemQ
Content-Disposition: form-data; name="Submit"
Submit
------WebKitFormBoundaryEknFTYZrogVNxemQ--
All of that being said, constructing these MIME parts can be a bit of a PITA, so if you can find a library to do that for you (such as the one suggested by #Innovation) it's probably worth looking into.

I had the same response while accessing validator.nu
with the jQuery.post or $.ajax command and decided
not to use jQuery after exhausting the various
possible ajax parameter configurations. It appears
that validator.nu API is more picky than other API
about the format that it receives from jQuery.
The NON jQuery javascript commands that worked for
me were FormData and Append.
It is also important to note the API requirements
available at:
github dot com/validator/validator/wiki
The full code can be found at:
https://github.com/VividVenturesLLC/validatorNu-JSwebServiceAPI
// Following code formatted to fit in the
// response text area, some lines may fail
// syntax check!
//Initialize some variables
//the entire script is written around the API of validator.nu webservice
var validate = 'https://validator.nu';
// developer.mozilla.org/en-US/docs/Web/API/FormData/FormData
var formData = new FormData();
// developer.mozilla.org/en-US/docs/Web/API/FormData/append
formData.append('charset','UTF-8');
formData.append('out','json');
formData.append('laxtype','yes');
formData.append('parser','html5');
// developer.mozilla.org/en-US/docs/Web/Guide/Using_FormData_Objects
// JavaScript file-like object...
var blob = new Blob([currentPageMarkup], { type: "text/html"});
formData.append("content", blob);
var oXHR = new XMLHttpRequest();
var validationTestResults = null;//expect response from validate webservice
oXHR.onreadystatechange = function(){
if (oXHR.readyState===1){//OPENED
console.log("<------- Opened communication at: " + validate +
" in script: test_valid_html.js -------------->");
}
if (oXHR.readyState===2){//HEADERS_RECEIVED
console.log("<------- Received AJAX headers from: " + validate +
" in script: test_valid_html.js -------------->");
}
if (oXHR.readyState===4){//done
console.log('oXHR.response: ' + oXHR.response);
//requested a response in JSON format
validationTestResults = JSON.parse(oXHR.response);
displayMessages(validationTestResults);//custom display function
var vtrStatus = getStatus(validationTestResults);
/* put id=validation_log in a tag in the web page under
test to output the results because
remember all the variables lose scope when
leaving this function
*/
//document.getElementById("validation_log").innerHTML=vtrStatus;
console.log("HTML5 validation status: " + vtrStatus);// this is the message you want
console.log("<------- Ending AJAX call in script: test_valid_html.js -------------->");
}
};//end onreadystate change
oXHR.open("POST", validate);
oXHR.send(formData);