Develop Reference

Uploading Image to AWS presigned post URL using axios

I am trying to upload an image to an S3 bucket using a presigned URL generated using boto3 on Python. I have been using the example python code that was provided in the documentation and was successful (the image got correctly uploaded with the correct Content-Type). However, when trying to do this in Javascript for the purposes of our frontend application, I am really struggling to get it to work.
Here's the example dictionary returned by the backend:
{
"fields": {
"AWSAccessKeyId": "AKIAYS3VM3EBIFL7FKE5",
"key": "posts/623255a762fd9bdfbd13f91a",
"policy": "<very long string>",
"signature": "Qvc/sGBHk0uzirzIfR1YmE2kFlo="
},
"url": "https://hotspot-storage.s3.amazonaws.com/"
}
Here is the functioning Python code:
response = <json response object>
object_name = 'playground/example_profile_group.png'
response['fields']['Content-Type'] = "image/png"
# Demonstrate how another Python program can use the presigned URL to upload a file
with open(object_name, 'rb') as f:
files = {'file': (object_name, f)}
http_response = requests.post(response['url'], data=response['fields'], files=files)
# If successful, returns HTTP status code 204
print(http_response)
print(http_response.text)
Here is the non-functioning Javascript code:
const data = response.data;
let payload = data.fields;
payload['Content-Type'] = 'image/jpeg';
const file = {
uri: previewPath,
name: previewPath,
type: 'image/jpeg',
};
payload.file = file;
const url = data.url;
console.log(payload, "MY PAYLOAD")
axios({
method: 'post',
headers: {'Content-Type': 'multipart/form-data'},
url: url,
data: payload,
})
.then(function (response) {
console.log(response.data, 'uploaded');
const data = response.data;
})
.catch(function (error) {
console.log(
'error uploading image',
error.response.data,
);
});
})
.catch(function (error) {
console.log(
'error getting media link',
error.response.data,
);
});
This is the error that keeps getting returned:
error uploading image <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data.</Message><RequestId>Q0ES6P4QP75YVVED</RequestId><HostId>eowLxSJQD1xP1EfHPnzGSJzXVGpPjurIMhkdwAD22JMvi9zRoFGg6Bq+mnUt/Lu7DNPY80iBDMc=</HostId></Error>
I have been stuck on this for an absurd amount of time, and cannot tell what I am doing wrong. Any help would be very much appreciated.

In order to send a multipart/form-data request body, you'll need to use a FormData instance instead of a JavaScript object.
For example
const { url, fields } = response.data;
const payload = new FormData();
payload.append("file", file); // this is the file blob, eg from <input type="file">
payload.append("Content-Type", "image/jpeg");
// add all the other fields
Object.entries(fields).forEach(([ key, val ]) => {
payload.append(key, val);
});
// No need to manually set content-type header, your browser knows what to do
const { data: result } = await axios.post(url, payload);
console.log("uploaded", result);

multipart/form-data not being automatically set with axios in React Native

When attempting to upload a file to Amazon S3 using axios, I have been encountering a very strange issue. Normally, in a web browser, when FormData has binary data in it, the Content-Type header automatically gets set to multipart/form-data; boundary=<some random string>. However, I have been completely unable to achieve that in React Native (testing on an iOS device). The Content-Type is automatically set to application/json, and thus not being detected as a correctly formatted body when uploading to Amazon S3. I have tried specifying a blob in the file parameter in FormData instead of the URI to the file as well to no avail. I have appended my code below, any advice would be very much appreciated.
const uploadFileToS3 = (
presignedPostData,
file) => {
// create a form obj
const formData = new FormData();
// append the fields in presignedPostData in formData
Object.keys(presignedPostData.fields).forEach(
key => {
formData.append(
key,
presignedPostData.fields[key],
);
},
);
// append the file and uplaod
const getBlob = async () => {
const img_url = previewPath;
let result = await fetch(img_url);
const blob = await result.blob();
formData.append('Content-Type', 'image/jpeg');
formData.append('file', {
uri: previewPath,
type: 'image/jpeg',
name: 'test.jpeg',
});
console.log(formData, 'wild');
// post the data on the s3 url
axios
.post(presignedPostData.url, formData)
.then(function (response) {
console.log(response);
})
.catch(function (error) {
console.log(error.response);
});
};
getBlob();
};

Issue with sending FormData from backend

I have a component which processes and uploads images. Currently I process the image on my backend and then send it to my frontend and then upload it from there. I would like to do everything on my backend. The only issue is that the upload endpoint requires FormData() object. I found an npm package form-data which I'm using on my backend now, but I'm still getting error.
This is how it currently works:
// frontend logic:
const data = await uploadImage(img);
const file = new File([Buffer.from(data)], `img-${i}.webp`, {
type: "image/webp",
});
const formData = new FormData();
formData.append("path", "images");
formData.append("files", file, file.name);
await axios
.post("http://localhost:1338/api/upload", formData, {
headers: { authorization: `Bearer ${jwtToken}` },
})
.then(({ data }) => {
console.log(data);
})
.catch(console.log);
//
//
// backend logic:
const data = await processImage(img.url);
return data;
This is what im trying to do:
// frontend logic:
const data = await uploadImage(img);
//
//
// backend logic:
const data = await processImage(img.url);
const formData = new FormData();
formData.append("path", "images");
formData.append("files", data, "file.name");
await axios
.post("http://localhost:1338/api/upload", formData, {
headers: { authorization: `Bearer ${process.env.JWT_TOKEN}` },
})
.then(({ data }) => {
console.log(data);
})
.catch(console.log); // I get error: 413 Payload Too Large
I'm trying to do it with the same image which works with the first method. Perhaps I need to create a new File(), but I couldn't find any npm packages which worked for that. What should I do to get this working?

send blob to python flask and then save it

So I'm trying to make a website that record your voice, the problem is that when I send to a flask server the blob file or the blob url, my flask python code says that is no content while it is, how can I send the blob, so the server can save it as a file.
mediaRecorder.addEventListener("stop", () => {
const audioBlob = new Blob(audioChunks, { type: "audio/wav" })
const audioUrl = URL.createObjectURL(audioBlob);
const audio = new Audio(audioUrl);
audio.play();
var data = new FormData()
data.append('file', audioUrl)
fetch('http://127.0.0.1:5000/receive', {
method: 'POST',
body: data
}).then(response => response.json()
).then(json => {
console.log(json)
});
and my python flask code:
#app.route("/receive", methods=['post'])
def form():
files = request.files
file = files.get('file')
print(file)
with open(os.path.abspath(f'backend/audios/{file}'), 'wb') as f:
f.write(file.content)
response = jsonify("File received and saved!")
response.headers.add('Access-Control-Allow-Origin', '*')
return response
is there a way to do it? send record blob file, download it into python?

The problem is in this line:
data.append('file', audioUrl)
you don't use FormData.append the right way.
it should be:
data.append('file', audioBlob , 'file')
See documentation: https://developer.mozilla.org/en-US/docs/Web/API/FormData/append

Download and upload image without saving to disk

Using Node.js, I am trying to get an image from a URL and upload that image to another service without saving image to disk. I have the following code that works when saving the file to disk and using fs to create a readablestream. But as I am doing this as a cron job on a read-only file system (webtask.io) I'd want to achieve the same result without saving the file to disk temporarily. Shouldn't that be possible?
request(image.Url)
.pipe(
fs
.createWriteStream(image.Id)
.on('finish', () => {
client.assets
.upload('image', fs.createReadStream(image.Id))
.then(imageAsset => {
resolve(imageAsset)
})
})
)
Do you have any suggestions of how to achieve this without saving the file to disk? The upload client will take the following
client.asset.upload(type: 'file' | image', body: File | Blob | Buffer | NodeStream, options = {}): Promise<AssetDocument>
Thanks!

How about passing the buffer down to the upload function? Since as per your statement it'll accept a buffer.
As a side note... This will keep it in memory for the duration of the method execution, so if you call this numerous times you might run out of resources.
request.get(url, function (res) {
var data = [];
res.on('data', function(chunk) {
data.push(chunk);
}).on('end', function() {
var buffer = Buffer.concat(data);
// Pass the buffer
client.asset.upload(type: 'buffer', body: buffer);
});
});

I tried some various libraries and it turns out that node-fetch provides a way to return a buffer. So this code works:
fetch(image.Url)
.then(res => res.buffer())
.then(buffer => client.assets
.upload('image', buffer, {filename: image.Id}))
.then(imageAsset => {
resolve(imageAsset)
})

well I know it has been a few years since the question was originally asked, but I have encountered this problem now, and since I didn't find an answer with a comprehensive example I made one myself.
i'm assuming that the file path is a valid URL and that the end of it is the file name, I need to pass an apikey to this API endpoint, and a successful upload sends me back a response with a token.
I'm using node-fetch and form-data as dependencies.
const fetch = require('node-fetch');
const FormData = require('form-data');
const secretKey = 'secretKey';
const downloadAndUploadFile = async (filePath) => {
const fileName = new URL(filePath).pathname.split("/").pop();
const endpoint = `the-upload-endpoint-url`;
const formData = new FormData();
let jsonResponse = null;
try {
const download = await fetch(filePath);
const buffer = await download.buffer();
if (!buffer) {
console.log('file not found', filePath);
return null;
}
formData.append('file', buffer, fileName);
const response = await fetch(endpoint, {
method: 'POST', body: formData, headers: {
...formData.getHeaders(),
"Authorization": `Bearer ${secretKey}`,
},
});
jsonResponse = await response.json();
} catch (error) {
console.log('error on file upload', error);
}
return jsonResponse ? jsonResponse.token : null;
}