Develop Reference

Upload file to s3 using presigned post url in the server

TDLR: Using s3 presigned post url to upload file to s3. Works fine on the browser but fails on the server.
I have a simple lambda function that generates presigned post url that can be consumed either in the browser or in the server.
During testing I noticed that the upload works fine one the browser but fails if I try to upload a file from a server even tho the code is identical.
The error i get is:
You must provide the Content-Length HTTP header
Detailed error:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>MissingContentLength</Code>
<Message>You must provide the Content-Length HTTP header.</Message>
<RequestId>JP75YMFARK0G3X5Z</RequestId>
<HostId>toHsKmxmVYYAtac94cQoy8wXoregKG3PNBm97c3gQewEmKxLggcumTAP882T/pJNWx/lxRgH98A=</HostId>
</Error>
Request failed with status code 411
I checked online and found many threads about this issue but unfortunately not a single suggestion helped me.
Code I am running in the server
const axios = require('axios');
const { createReadStream, readFileSync } = require('fs');
const FormData = require('form-data');
const getPostPresignedUrl = async () => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: 'test-2.jpg',
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
const file = createReadStream('./test-1.jpg');
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const init = async () => {
const { fields, url } = await getPostPresignedUrl();
await uploadFileToS3(fields, url);
};
init();
Code I am running in the browser:
const form = document.getElementById('form');
const input = document.getElementById('file');
const getPostPresignedUrl = async (name) => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: name,
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url, file) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const handleSubmit = async (e) => {
e.preventDefault();
const file = input.files[0];
const data = await getPostPresignedUrl(file.name);
await uploadFileToS3(data.fields, data.url, file);
};
form.onsubmit = handleSubmit;

Upload files VueJS to NodeJS backend by axios

I would like to upload a file in vuejs to my nodejs server. However, I can't receive my file in the backend. Someone could help me. I have tried several things but I still can't get it. If anyone can help me I would be very grateful :-)
VueJS Upload Page Code
<input id="input-file" name="input-file" type="file" ref="file" #change="newFile($event)"/>
const formData = new FormData();
formData.append("file", document.getElementById("input-file"))
const createUserResponse = await RequestManager.executePostRequest("/users/upload", formData, {
headers: {
'Content-Type': 'multipart/form-data'
}
});
RequestManager executePostRequest Function
static async executePostRequest(url: string, params: any, specialConfig?: AxiosRequestConfig) {
const postToken = Utils.buildHmacSha256Signature(params);
let headers: AxiosRequestHeaders = {
"x-access-token": config.server.backendAccessToken,
}
if (RequestManager.token) {
headers = {
"x-access-token": config.server.backendAccessToken,
"x-token-data": RequestManager.token,
}
}
if (specialConfig && specialConfig.headers) {
headers = Utils.mergeObjects(headers, specialConfig.headers) as AxiosRequestHeaders;
}
const instance = axios.create({
baseURL: config.server.host,
headers: headers
});
console.log(headers);
const paramsPost = {
data: params,
token: postToken
}
return new Promise<any>((resolve, reject) => {
instance
.post(url, paramsPost)
.then(response => {
resolve(response.data);
})
.catch(error => {
reject(error);
});
});
}
Backend NodeJS
UserRouter.get("/upload", (req, res) => {
console.log(req.files);
res.send("ok");
});

Azure function able to read file on local but not working on remote

I would download file on local the create a stream to send it via an api call
const blobServiceClient = BlobServiceClient.fromConnectionString(
process.env.CONNEXION_STRING
);
const containerClient = blobServiceClient.getContainerClient(
params.containerName
);
const blobClient = containerClient.getBlobClient(process.env.FILE_LOCATION); // get file from storage
let blobData;
var defaultFile = path.join(params.baseDir, `${params.reportName}.pbix`); // use path module
let stream;
try {
blobData = await blobClient.downloadToFile(defaultFile);
console.log(blobData);
stream = fs.createReadStream(defaultFile);
} catch (error) {
params.context.log(error);
console.log(error);
}
var options = {
method: "POST",
url: `https://api.powerbi.com/v1.0/myorg/groups/${params.groupId}/imports?datasetDisplayName=${params.reportName}`,
headers: {
"Content-Type": "multipart/form-data",
Authorization: `Bearer ${params.accessToken} `,
},
formData: {
"": {
value: stream,
options: {
filename: `${params.reportName}.pbix`,
contentType: null,
},
},
},
};
//check if file keep in mem
return new Promise(function (resolve, reject) {
request(options, function (error, response) {
if (error) {
params.context.log(error);
reject(error);
} else {
params.context.log(response);
resolve(response.body);
}
fs.unlinkSync(defaultFile);
});
});
In local it's working like a charm but when I deploy my function on server I get this error.
[Error: ENOENT: no such file or directory, open 'D:\home\site\wwwroot\importPbix\exampleName.pbix'
I found this post having same issue , that's why I user path module and passed __dirname to function params.baseDir.

Redux Thunk - Change state before REST request

I am quite new to Redux Thunk and have an issue that I want to update a contract with a 'FileList' (file appendix), but if I use JSON.stringify the file will have a 0 value. If I convert the file to Base64 this problem is solved, but the PUT request is performed before the file is converted.
I searched a lot about Redux Thunk and think it might be some issue with Dispatch, I tried quite a lot and didn't become much wiser. Most of the things that I tried returned: "Actions must be plain objects. Use custom middleware for async actions."
Would appreciate some help or some search suggestions..
ps. contract.answers[0].answer[0] is the file. This acquires some refactoring, but first it needs to work.
const toBase64 = (file) => new Promise((resolve, reject) => {
const reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = () => resolve(reader.result);
reader.onerror = (error) => reject(error);
});
export function updateContract(contract) {
const base64File = toBase64(contract.answers[0].answer[0]);
base64File.then((value) => {
contract.answers[0].answer[0] = value; //Set file as base64
});
return {
type: SAVE,
fetchConfig: {
uri: contract._links.self,
method: 'PUT',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(contract), // Does not handle files
failureHandler(error) {
const {
details,
status,
} = error;
// If the contract was invalid, throw form errors:
if (status.code === 400 && details) {
// Map the question ids to fields:
throw new SubmissionError(Object.keys(details).reduce(
(acc, questionId) => {
acc[`question${questionId}`] = details[questionId];
return acc;
},
{},
));
}
return {
type: SAVE_FAILURE,
error,
};
},
successHandler(json) {
return {
type: SAVE_SUCCESS,
data: json,
};
},
},
};
}
Kind regards,
Gust de Backer

This happen because toBase64 return a Promise and itself is async, so in your case is necessary encapsule inside a new then.
export function updateContract(contract) {
const base64File = toBase64(contract.answers[0].answer[0]);
base64File.then((value) => {
contract.answers[0].answer[0] = value; //Set file as base64
});
return (dispatch) => {
base64File.then(() => dispatch({
type: SAVE,
fetchConfig: {
uri: contract._links.self,
method: 'PUT',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(contract), // Does not handle files
failureHandler(error) {
const {
details,
status,
} = error;
// If the contract was invalid, throw form errors:
if (status.code === 400 && details) {
// Map the question ids to fields:
throw new SubmissionError(Object.keys(details).reduce(
(acc, questionId) => {
acc[`question${questionId}`] = details[questionId];
return acc;
}, {},
));
}
return {
type: SAVE_FAILURE,
error,
};
},
successHandler(json) {
return {
type: SAVE_SUCCESS,
data: json,
};
},
},
}))
};
}
Yes, the redux accept a function as return, that function receive a dispatch on params, you can use it to dispatch the request after convert is ready :)

sending empty files to server React-Native

I am trying to send JSON files to Server using NodeJS with multer and i am able to send files but the files are empty.
I am using React-native-File-System to loop through all the files present in the folder
I am not getting any errors and the file upload logs is also showing as UPLOAD COMPLETED but the files are empty
I have tried to send it with form-data but still no luck
var RNFS = require('react-native-fs');
var path = RNFS.DocumentDirectoryPath + '/toBeSynced';
RNFS.readDir(path)
.then((success) => {
success.forEach(function (element) {
var fileName = element.name
var filePath = element.path
const options = {
url: 'http://192.168.1.15:3333/SurveyJsonFiles/GetFiles',
path: filePath,
name: fileName,
field: 'files',
method: 'POST',
type: 'multipart',
headers: {
'content-type': 'multipart/form-data',
},
//Below are options only supported on Android
notification: {
enabled: true
}
}
Upload.startUpload(options).then((uploadId) => {
console.log('Upload started')
Upload.addListener('progress', uploadId, (data) => {
console.log(`Progress: ${data.progress}%`)
})
Upload.addListener('error', uploadId, (data) => {
console.log(`Error: ${data.error}%`)
})
Upload.addListener('cancelled', uploadId, (data) => {
console.log(`cancelled: ${data.error}%`)
})
Upload.addListener('completed', uploadId, (data) => {
// data includes responseCode: number and responseBody: Object
console.log('Completed!')
})
}).catch((err) => {
console.log('Upload error!', err)
})
});
})
.catch((err) => {
console.log(err.message);
});
}

Your path is not an absolute file item path.
if you have lots of file, add this code
var files = [
{
name: 'test1',
filename: 'test1.w4a',
filepath: RNFS.DocumentDirectoryPath + '/test1.w4a',
filetype: 'audio/x-m4a'
}, {
name: 'test2',
filename: 'test2.w4a',
filepath: RNFS.DocumentDirectoryPath + '/test2.w4a',
filetype: 'audio/x-m4a'
}
];
if you have only one file, add like this.
var path = RNFS.DocumentDirectoryPath + '/toBeSynced/test.txt';
or
var path = RNFS.DocumentDirectoryPath + '/test.txt';
You can look at these.
https://github.com/itinance/react-native-fs#examples
https://github.com/itinance/react-native-fs#constants

The error was on the server side as nodemon was restarting server when it was getting new file
const data = new FormData();
data.append('files', {
uri: filePath,
type: 'multipart/form-data',
name: fileName,
});
const config = {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'multipart/form-data',
},
body: data,
};
fetch(uploadUrl, config)
.then((checkStatusAndGetJSONResponse) => {
console.log(checkStatusAndGetJSONResponse);
}).catch((err) => {
console.log(err)
});