Develop Reference

Upload file to s3 using presigned post url in the server

TDLR: Using s3 presigned post url to upload file to s3. Works fine on the browser but fails on the server.
I have a simple lambda function that generates presigned post url that can be consumed either in the browser or in the server.
During testing I noticed that the upload works fine one the browser but fails if I try to upload a file from a server even tho the code is identical.
The error i get is:
You must provide the Content-Length HTTP header
Detailed error:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>MissingContentLength</Code>
<Message>You must provide the Content-Length HTTP header.</Message>
<RequestId>JP75YMFARK0G3X5Z</RequestId>
<HostId>toHsKmxmVYYAtac94cQoy8wXoregKG3PNBm97c3gQewEmKxLggcumTAP882T/pJNWx/lxRgH98A=</HostId>
</Error>
Request failed with status code 411
I checked online and found many threads about this issue but unfortunately not a single suggestion helped me.
Code I am running in the server
const axios = require('axios');
const { createReadStream, readFileSync } = require('fs');
const FormData = require('form-data');
const getPostPresignedUrl = async () => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: 'test-2.jpg',
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
const file = createReadStream('./test-1.jpg');
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const init = async () => {
const { fields, url } = await getPostPresignedUrl();
await uploadFileToS3(fields, url);
};
init();
Code I am running in the browser:
const form = document.getElementById('form');
const input = document.getElementById('file');
const getPostPresignedUrl = async (name) => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: name,
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url, file) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const handleSubmit = async (e) => {
e.preventDefault();
const file = input.files[0];
const data = await getPostPresignedUrl(file.name);
await uploadFileToS3(data.fields, data.url, file);
};
form.onsubmit = handleSubmit;

passing binary data to rest-api

I want to upload a thumbnail to viemo api , and based on Vimeo doc, I must include the thumbnail file as binary data in the request body, I have to upload the thumbnail from the client side "fontend" and will send the request from the rest-api "backend" the problem is my rest-api can't receive the binary data, properly because I'm using express.js, is there any way to handle the binary data in the backend.
the steps as following:
client side sent thumbnail data => backend receive the data through the request body and send it to => endpoint
client side request
const handleSubmit = (event) => {
event.preventDefault();
const formData = new FormData();
formData.append('selectedFile', new Blob([selectedFile], { type: 'application/json' }));
formData.append('uploadLink', uploadLink);
const headers = {
'Content-Type': 'application/json',
Accept: 'application/vnd.vimeo.*+json;version=3.4',
};
try {
axios
.post(`${backendPostPath}/thumbnail-upload`, formData, {
headers,
})
.then((response) => {
applyThumbnial();
console.log(`${uploadLink}link for upload`);
});
} catch (error) {
console.log(error);
}
};
backend request can't receive the body request of frontend post as binary data,
const ThumbnailUpload = async (req, res) => {
const { uploadLink } = req.body;
const { selectedFile } = req.body;
console.log(uploadLink);
const clientServerOptions = {
uri: `${uploadLink}`,
encoding: null,
body: JSON.stringify({
name: uploadLink,
file: selectedFile,
}),
method: 'PUT',
headers: {
'Content-Type': 'application/json',
Accept: 'application/vnd.vimeo.*+json;version=3.4',
Authorization: getVimeoAuthorization(),
},
};
request(clientServerOptions, function (error, response) {
if (error) {
res.send(error);
} else {
const body = JSON.parse(response.body);
res.send(body);
}
});
};
is there any way to make the backend request body fetch the binary data, as I getting the data as "undefined"

Sorry for late update, I solved this issue by using the same put request form the client side, as the put request don't require Vimeo access token, so you can use the same put request i mentioned above, and remove authentication from the header, like following
const handleSubmit = (event) => {
event.preventDefault();
const formData = new FormData();
formData.append('selectedFile', new Blob([selectedFile], { type: 'image/jpg, image/png' }));
// formData.append('uploadLink', uploadLink);
const headers = {
'Content-Type': 'image/jpg, image/png',
Accept: 'application/vnd.vimeo.*+json;version=3.4',
};
try {
axios
.put(`${uploadLink}`, formData, {
headers,
})
.then((response) => {
console.log(`${uploadLink}link for upload`);
});
} catch (error) {
console.log(error);
}
};

Uploading file from local file system in node giving 400 error

I am trying to upload a file to an API using multipart/form-data.
I am able to successfully do this using insomnia as follows:
Headers are set to Content-Type multipart/form-data and the basic auth is also set.
When I try to replicate this in my code I receive a 400 error.
Here is what I have at the moment:
const URL = 'https://api.app.uploadplace.com/v1/';
const token = 'randomkey';
async function readFile(path: string) {
return new Promise((resolve, reject) => {
fs.readFile(path, 'utf-8', function (err, data) {
if (err) {
reject(err);
}
resolve(data);
});
});
}
export const Upload = async (path: string) => {
const FormData = require('form-data');
const form = new FormData();
let file = await readFile(path);
form.append('file', file);
const config = {
headers: {
'content-type': 'multipart/form-data'
},
auth: {
username: token,
password: ''
}
}
await axios.post(URL, form, config).then(() => {
console.log('success')
}).catch(err => {
console.log(err);
})
};

The problem is that you are appending the file contents as a string to your form data.
Instead, append the file readable stream...
form.append("file", fs.createReadStream(path));
const config = {
headers: form.getHeaders() // this includes correct mime boundary tokens
auth: {
username: token,
password: ''
}
}
See https://github.com/axios/axios#form-data

Copy images from one folder to another in digitalocean spaces

I am working with digital ocean spaces and I have uploaded the images on it in a temporary folder. Now I want to move that image from temporary folder to permanent folder. I have searched the things from all over but got nothing much satisfying. Is it possible to do so ?
and if yes please help me with the javascript code.
First I generated the signed url and with the help of that signed url I uploaded the image on digitalocean spaces. Following is the code for generating signed url and uploading images.
const getSignedUrl = async () => {
const body = {
fileName: 'temp/' + file.name,
fileType: file.type,
}
const response = await fetch(`${API_URL}/presigned_url`, {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' }
})
const { signedUrl } = await response.json()
return signedUrl
}
const uploadFile = async signedUrl => {
const res = await fetch(signedUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': file.type,
'x-amz-acl': 'public-read',
}
})
return res
}
please help me how can I move my image from temporary folder to permanent folder.

So finally after searching I got the answer,
From the front end side I call an API to copy my image
const copyFile = async (file) => {
try {
const body = {
fileName: file.name
}
const res = await fetch(`${API_URL}/copy_file`, {
method: 'PUT',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' }
})
return res
} catch (error) {
console.log(error);
}
}
And in the backend side the API I made is
app.put('/copy_file', (req, res) => {
const fileName = req.body.fileName
console.log("body", fileName);
const params = {
Bucket: config.spaces.spaceName,
CopySource: `/bucketName/temp/${fileName}`,
Key: `original/${fileName}`,
}
spaces.copyObject(params, function (err, data) {
if (err) {
console.log("Error", err)
// console.log(err, err.stack); // an error occurred
} else {
res.json({ data })
}
});
});
This will copy you image in original folder

Post action API with object parameter within the URL

I've got an API where some of the parameters need to be given within the URL.
Example of how my api url looks like: https://www.server.com/api/actions/execute?auth_type=apikey&data={"Name": "name","Email" : "email"}
What my code looks like right now
register = async () => {
let data = {"Name":this.state.name, "Email":this.state.email}
data = JSON.stringify(data)
let URL = 'https://www.server.com/api/actions/execute?auth_type=apikey&data=';
fetch(URL, {
method: 'POST',
headers: new Headers({
'Content-Type': 'application/json'
}),
body: data
})
.then((response) => response.text())
.then((responseText) => {
alert(responseText);
})
.catch((error) => {
console.error(error);
});
}
The response I get on my device:
{"code":"succes","details":{"userMessage":["java.lang.Object#2e56000c"],"output_type":void","id:"20620000000018001"},"message":"function executed succesfully"}
This is alle working fine when I test it in postman but I can't get it to work within React-Native. I've tried stuff like 'Content-Type':'application/x-www-form-urlencoded' already.

First install the package axios from the url https://www.npmjs.com/package/react-native-axios
Then create two service for handling get and post request so that you can reuse them
GetService.js
import axios from 'axios';
let constant = {
baseurl:'https://www.sampleurl.com/'
};
let config = {
headers: {
'Content-Type': 'multipart/form-data',
'Accept': 'application/json'
}
};
export const GetService = (data,Path,jwtKey) => {
if(jwtKey != ''){
axios.defaults.headers.common['Authorization'] = 'Bearer '+jwtKey;
}
try{
return axios.get(
constant.baseUrl+'api/'+Path,
data,
config
);
}catch(error){
console.warn(error);
}
}
PostService.js
import axios from 'axios';
let constant = {
baseurl:'https://www.sampleurl.com/'
};
let config = {
headers: {
'Content-Type': 'multipart/form-data',
'Accept': 'application/json'
}
};
export const PostService = (data,Path,jwtKey) => {
if(jwtKey != ''){
axios.defaults.headers.common['Authorization'] = 'Bearer '+jwtKey;
}
try{
return axios.post(
constant.baseUrl+'api/'+Path,
data,
config
);
}catch(error){
console.warn(error);
}
}
Sample code for using get and post services is given below
import { PostService } from './PostService';
import { GetService } from './GetService';
let uploadData = new FormData();
uploadData.append('key1', this.state.value1);
uploadData.append('key2', this.state.value2);
//uploadData.append('uploads', { type: data.mime, uri: data.path, name: "samples" });
let jwtKey = ''; // Authentication key can be added here
PostService(uploadData, 'postUser.php', jwtKey).then((resp) => {
this.setState({ uploading: false });
// resp.data will contain json data from server
}).catch(err => {
// handle error here
});
GetService({}, 'getUser.php?uid='+uid, jwtKey).then((resp) => {
// resp.data will contain json data from server
}).catch(err => {
// handle error here
});

If you need to pass parameters via URL you should use GET, if you use POST then the parameters should be passed in the body