I am trying to upload a photo file to an S3 bucket using the Fetch API. I'm getting 400 Bad Request on the POST when trying to upload the photo. I am getting the presigned post url and the file details correctly but I believe the way I am formatting the formData is incorrect.
I'm using an html file input that uses onchange to run a javascript function handlePhoto.
The html is
<input type="file" onchange="handlePhoto()" id="file_input"/>
and javascript function is
function handlePhoto(){
const file = document.getElementById('file_input').files[0]
let formData = new FormData()
fetch("/v1/photos/get_presigned_post/" , {
method: "GET"
})
.then(response => response.json())
.then(s3Result => {
const { url, fields } = s3Result;
Object.keys(s3Result.fields).forEach(key => {
formData.append(key, s3Result.fields[key]);
});
formData.append('acl', 'public-read');
formData.append('Content-Type', file.type);
formData.append("file", file);
fetch(url, {
method: "POST",
body: formData,
headers: {
"Content-Type": "multipart/form-data"
}
})
});
}
Any help would be greatly appreciated.
Assuming you have a presigned url you could use something like this:
interface UploadIAToS3Params {
url: string;
file: File;
}
export function uploadToS3(params: UploadIAToS3Params) {
const { file, url } = params;
return fetch(url, { body: file, mode: "cors", method: "PUT" });
}
check the url in s3Result, for me it was wrong link and it doesn't include my bucketname. url must be like this to work:
https://{bucket_name}.s3.amazonaws.com
Your URL might not be right,
From the documentation, hostnames should look something like this.
"s3.Region.amazonaws.com".
These two links might help
https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAPI.html
and,
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTForms.html
You don't need to use a FormData, just pass the file with a content-type 'octet-stream'.
acl must be used when generating the presign URL.
fetch(url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': 'application/octet-stream' }
})
Related
I have an API endpoint that lets the client post their csv to our server then post it to someone else server. I have done our server part which save uploaded file to our server, but I can't get the other part done. I keep getting error { message: 'File not found', code: 400 } which may mean the file never reach the server. I'm using axios as an agent, does anyone know how to get this done? Thanks.
// file = uploaded file
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
Update
As axios doc states as below and the API I'm trying to call requires a file
// data is the data to be sent as the request body
// Only applicable for request methods 'PUT', 'POST', and 'PATCH'
// When no transformRequest is set, must be of one of the following types:
// - string, plain object, ArrayBuffer, ArrayBufferView, URLSearchParams
// - Browser only: FormData, File, Blob
// - Node only: Stream, Buffer
Is there any way to make axios send a file as a whole? Thanks.
The 2 oldest answers did not work for me. This, however, did the trick:
const FormData = require('form-data'); // npm install --save form-data
const form = new FormData();
form.append('file', fs.createReadStream(file.path));
const request_config = {
headers: {
'Authorization': `Bearer ${access_token}`,
...form.getHeaders()
}
};
return axios.post(url, form, request_config);
form.getHeaders() returns an Object with the content-type as well as the boundary.
For example:
{ "content-type": "multipart/form-data; boundary=-------------------0123456789" }
I'm thinking the createReadStream is your issue because its async. try this.
Since createReadStream extends the event emitter, we can "listen" for when it finishes/ends.
var newFile = fs.createReadStream(file.path);
// personally I'd function out the inner body here and just call
// to the function and pass in the newFile
newFile.on('end', function() {
const form_data = new FormData();
form_data.append("file", newFile, "filename.ext");
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
});
This is what you really need:
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios
.post(url, form_data, request_config);
In my case, fs.createReadStream(file.path) did not work.
I had to use buffer instead.
const form = new FormData();
form.append('file', fs.readFileSync(filePath), fileName);
const config = {
headers: {
Authorization: `Bearer ${auth.access_token}`,
...form.getHeaders(),
},
};
axios.post(api, form.getBuffer(), config);
I have made an interceptor you can connect to axios to handle this case in node: axios-form-data. Any feedback would be welcome.
npm i axios-form-data
example:
import axiosFormData from 'axios-form-data';
import axios from 'axios';
// connect axiosFormData interceptor to axios
axios.interceptors.request.use(axiosFormData);
// send request with a file in it, it automatically becomes form-data
const response = await axios.request({
method: 'POST',
url: 'http://httpbin.org/post',
data: {
nonfile: 'Non-file value',
// if there is at least one streamable value, the interceptor wraps the data into FormData
file: createReadStream('somefile'),
},
});
// response should show "files" with file content, "form" with other values
// and multipart/form-data with random boundary as request header
console.log(response.data);
I had a same issue, I had a "pdf-creator-service" for generate PDF document from html.
I use mustache template engine for create HTML document - https://www.npmjs.com/package/mustache
Mustache.render function returns html as a string what do I need to do to pass it to the pdf-generator-service ? So lets see my suggestion bellow
//...
async function getPdfDoc(props: {foo: string, bar: string}): Promise<Buffer> {
const temlateFile = readFileSync(joinPath(process.cwd(), 'file.html'))
mustache.render(temlateFile, props)
const readableStream = this.getReadableStreamFromString(htmlString)
const formData = new FormData() // from 'form-data'
formData.append('file', options.file, { filename: options.fileName })
const formHeaders = formData.getHeaders()
return await axios.send<Buffer>(
{
method: 'POST',
url: 'https://pdf-generator-service-url/pdf',
data: formData,
headers: {
...formHeaders,
},
responseType: 'arraybuffer', // ! important
},
)
}
getReadableStreamFromString(str: string): Readable {
const bufferHtmlString = Buffer.from(str)
const readableStream = new Readable() // from 'stream'
readableStream._read = () => null // workaround error
readableStream.push(bufferHtmlString)
readableStream.push(null) // mark end of stream
return readableStream
}
For anyone who wants to upload files from their local filesystem (actually from anywhere with the right streams architecture) with axios and doesn't want to use any external packages (like form-data).
Just create a readable stream and plug it right into axios request function like so:
await axios.put(
url,
fs.createReadStream(path_to_file)
)
Axios accepts data argument of type Stream in node context.
Works fine for me at least in Node v.16.13.1 and with axios v.0.27.2
I have a scenario that I need to upload an zip file.
In the zip file, there are lots of image files which will upload to AWS S3.
Because of the large amount of files in that zipfile, I want to get the information of upload process. In my opinion, the way I can get information is by using streaming response. Once server uploaded a file, respon a json to client.
Every time I upload a image to S3, I want to response a json object like the example bellow.
example for json streaming response:
{
"file_name": "imgae1.jpg",
"s3_url": "http://s3.url/key/to/file",
"other_key": "key for this uploaded file"
}
I'm trying to achieve this approach by using vue(cdn version) + axios(cdn version).
The code bellow which is how I upload my zip file.
function upload() {
var file = document.querySelector("#upload_file")
if (file.files.length <= 0) return
var formData = new FormData();
formData.append("file", file.files[0]);
formData.append("form_data", "form_data");
axios({
method: 'post',
url: "http://127.0.0.1:8000/",
headers: {
'Content-Type': 'multipart/form-data'
},
responseType: 'stream',
data: formData
}).then(function (response) {
if (response.status >= 200 && response.status < 300) {
alert("All images uploaded!")
}
})
}
but those examples I found are using axios npm package which I can't use.
Is there any recommend method or any resources that I can search?
Thanks for helping!
You can try using fetch instead like this:
fetch("http://example.url", {
method: "POST",
body: formData,
mode: "no-cors",
header: {
"Content-Type": "multipart/form-data",
},
}).then((response) => {
a = response.clone();
a.json().then((data) => {
//console.log('data', data)
});
});
I have an API endpoint that lets the client post their csv to our server then post it to someone else server. I have done our server part which save uploaded file to our server, but I can't get the other part done. I keep getting error { message: 'File not found', code: 400 } which may mean the file never reach the server. I'm using axios as an agent, does anyone know how to get this done? Thanks.
// file = uploaded file
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
Update
As axios doc states as below and the API I'm trying to call requires a file
// data is the data to be sent as the request body
// Only applicable for request methods 'PUT', 'POST', and 'PATCH'
// When no transformRequest is set, must be of one of the following types:
// - string, plain object, ArrayBuffer, ArrayBufferView, URLSearchParams
// - Browser only: FormData, File, Blob
// - Node only: Stream, Buffer
Is there any way to make axios send a file as a whole? Thanks.
The 2 oldest answers did not work for me. This, however, did the trick:
const FormData = require('form-data'); // npm install --save form-data
const form = new FormData();
form.append('file', fs.createReadStream(file.path));
const request_config = {
headers: {
'Authorization': `Bearer ${access_token}`,
...form.getHeaders()
}
};
return axios.post(url, form, request_config);
form.getHeaders() returns an Object with the content-type as well as the boundary.
For example:
{ "content-type": "multipart/form-data; boundary=-------------------0123456789" }
I'm thinking the createReadStream is your issue because its async. try this.
Since createReadStream extends the event emitter, we can "listen" for when it finishes/ends.
var newFile = fs.createReadStream(file.path);
// personally I'd function out the inner body here and just call
// to the function and pass in the newFile
newFile.on('end', function() {
const form_data = new FormData();
form_data.append("file", newFile, "filename.ext");
const request_config = {
method: "post",
url: url,
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios(request_config);
});
This is what you really need:
const form_data = new FormData();
form_data.append("file", fs.createReadStream(file.path));
const request_config = {
headers: {
"Authorization": "Bearer " + access_token,
"Content-Type": "multipart/form-data"
},
data: form_data
};
return axios
.post(url, form_data, request_config);
In my case, fs.createReadStream(file.path) did not work.
I had to use buffer instead.
const form = new FormData();
form.append('file', fs.readFileSync(filePath), fileName);
const config = {
headers: {
Authorization: `Bearer ${auth.access_token}`,
...form.getHeaders(),
},
};
axios.post(api, form.getBuffer(), config);
I have made an interceptor you can connect to axios to handle this case in node: axios-form-data. Any feedback would be welcome.
npm i axios-form-data
example:
import axiosFormData from 'axios-form-data';
import axios from 'axios';
// connect axiosFormData interceptor to axios
axios.interceptors.request.use(axiosFormData);
// send request with a file in it, it automatically becomes form-data
const response = await axios.request({
method: 'POST',
url: 'http://httpbin.org/post',
data: {
nonfile: 'Non-file value',
// if there is at least one streamable value, the interceptor wraps the data into FormData
file: createReadStream('somefile'),
},
});
// response should show "files" with file content, "form" with other values
// and multipart/form-data with random boundary as request header
console.log(response.data);
I had a same issue, I had a "pdf-creator-service" for generate PDF document from html.
I use mustache template engine for create HTML document - https://www.npmjs.com/package/mustache
Mustache.render function returns html as a string what do I need to do to pass it to the pdf-generator-service ? So lets see my suggestion bellow
//...
async function getPdfDoc(props: {foo: string, bar: string}): Promise<Buffer> {
const temlateFile = readFileSync(joinPath(process.cwd(), 'file.html'))
mustache.render(temlateFile, props)
const readableStream = this.getReadableStreamFromString(htmlString)
const formData = new FormData() // from 'form-data'
formData.append('file', options.file, { filename: options.fileName })
const formHeaders = formData.getHeaders()
return await axios.send<Buffer>(
{
method: 'POST',
url: 'https://pdf-generator-service-url/pdf',
data: formData,
headers: {
...formHeaders,
},
responseType: 'arraybuffer', // ! important
},
)
}
getReadableStreamFromString(str: string): Readable {
const bufferHtmlString = Buffer.from(str)
const readableStream = new Readable() // from 'stream'
readableStream._read = () => null // workaround error
readableStream.push(bufferHtmlString)
readableStream.push(null) // mark end of stream
return readableStream
}
For anyone who wants to upload files from their local filesystem (actually from anywhere with the right streams architecture) with axios and doesn't want to use any external packages (like form-data).
Just create a readable stream and plug it right into axios request function like so:
await axios.put(
url,
fs.createReadStream(path_to_file)
)
Axios accepts data argument of type Stream in node context.
Works fine for me at least in Node v.16.13.1 and with axios v.0.27.2
I am able to upload an image perfectly fine using the native fetch POST:
let formData = new FormData();
formData.append('file', event.target.files[0]);
console.log(event.target.files[0]);
fetch('http://localhost:8080/file/upload', {
method: 'POST',
headers:{
'Authorization': 'Bearer ' + JWT
},
body:formData
}).then(response => console.log(response));
However, when I try this using Angular's HttpClient, the request fails since the 'Content-Type': 'multipart/form-data' is not added.
My Angular code:
The file where I'm calling the service:
this.fileSaverService.uploadImage(event.target.files[0]).subscribe(
(data)=>{
console.log(data);
},
(error) => {
console.log(error);
}
);
The fileSaverService:
uploadImage(fileToUpload) {
const endpoint = 'http://localhost:8080/file/upload';
const formData: FormData = new FormData();
formData.append('file', fileToUpload);
return this.api
.post(endpoint, formData, { headers: {'Content-Type': 'multipart/form-data'} });
}
The this.api:
post(endpoint: string, body: any, reqOpts: any = {}) {
reqOpts.headers = this.handleHeaders(reqOpts.headers);
return this.http.post(endpoint, JSON.stringify(body), reqOpts);
}
If I add the header manually when using HttpClient, I get this header without the boundary:
However the native fetch adds this header automatically with the boundary if I don't specify a header and this works perfectly:
What are my options here?
There are two problems with the code in your question:
By setting Content-Type to multipart/form-data yourself, the serialisation of your FormData will not result in the correct header with boundaries being set behind the scenes.
In your post function, you have JSON.stringify(body), which is converting your FormData object into a JSON string. After doing this, you're simply attempting to POST a string rather than a complete FormData object.
I use fetch to upload images in react,my code is like below:
let formData = new FormData();
let file = {uri: imgdata, type: 'multipart/form-data', name: '2_resources.jpg'};
formData.append("name", "name");
formData.append("mobile", "18381307123");
formData.append("content", "123654");
formData.append("resources", file,"2_resources.jpg");//mind this line
fetch(Config.report,
{
mode: 'cors',
method: "POST",
body: formData
})
.then(res => res.json())
.then((data) => {
console.log(data)
}
).catch((err) => {
console.log(err)
}
);
but,run it I get the request :
I have look for FormData API document on https://developer.mozilla.org/en-US/docs/Web/API/FormData
it have write below:
and the fetch used is :"isomorphic-fetch": "^2.2.1"
what should I do to use fetch upload images? thanks.
You are adding the file to the FormData object incorrectly. isomorphic-fetch wraps github's fetch polyfill. Looking at their docs shows that it should be:
handleFileUpload(event) {
var formData = new FormData();
formData.append('fileName', event.target.files[0]);
fetch('http://your-domain/upload',
{
mode: 'cors',
method: 'POST',
body: formData
}).then(function (response) {
console.log('response: ', response);
});
}
The field name on the server will be whatever you use as the key when appending formData. In the above example it will be 'fileName'. I also don't think you need to set the content-type and file name yourself. It should get set automatically.
In the above example, 'event' is the event fired from the input tag when the user upload the file:
<input type="file" onChange={this.handleFileUpload}/>