Develop Reference

Google app script upload image to Amazon S3 using signature and policy return body is not well-formed multipart/form-data

I'm creating an automation tool to upload my images to Amazon S3 with a given policy, signature and other parameters. I can perform this request easily on Postman or Insomnia but it seems really hard to get it done with code. I've tried some python code too, but it doesn't work
uploadImage(name, imageBlob, token) {
name += imageBlob.getName()
let data = {
'name': name,
'key': token.key,
'Filename': token.key,
"success_action_status": "201",
'AWSAccessKeyId': token.AWSAccessKeyId,
'policy': token.policy,
'signature': token.signature,
'file': imageBlob
}
// let boundary = "---1011100000110100";
// let rawdata = "";
// for (let i in data) {
// rawdata += "--" + boundary + "\r\n";
// rawdata += "Content-Disposition: form-data; name=\"" + i + "\"; \r\n\r\n" + data[i] + "\r\n";
// }
// rawdata += "--" + boundary + "--\r\n";
// console.log(data.key)
const url = 'https://s3.amazonaws.com/someExPath/'
const headers = {
'content-type': "multipart/form-data",
}
const opt = {
contentType: 'multipart/form-data',
headers: headers,
method: 'post',
payload: data,
followRedirects: true,
muteHttpExceptions: true
}
let resp = UrlFetchApp.fetch(url, opt)
console.log(resp)
console.log(resp.text)
return name
}
Here's my python code
import requests
url = "https://s3.amazonaws.com/samplePath/"
payload={
'name': 'demo1.jpg',
'key': 'tmp/sampleIds=/${filename}',
'Filename': 'tmp/sampleIds=/${filename}',
'success_action_status': '201',
'AWSAccessKeyId': 'AKIAIDO4Q6WXHCGAMHIA',
'policy': 'sample polici',
'signature': 'sample sig'
}
files=[
('file',('demo1.jpg',open('demo1.jpg','rb'),'image/jpeg'))
]
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36',
'content-type': 'multipart/form-data'
}
resp = requests.post(url, headers=headers, data=payload, files=files)
The google app script returns: The body of your POST request is not well-formed multipart/form-data.
Python script even can't finish the request, it got a ConnectionError (check out the picture on the link below)
I've tried to pass it as a raw body but still doesn't work. The closest one I got is replacing the imageBlob by Utilities.base64Encode(imageBlob.getBytes()) then put it on the raw text body (check the comment code), I got the success code but when recheck it's an invalid image. Here is the doc that I found: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingHTTPPOST.html
You can check out my success Postman request and my python code here:
https://drive.google.com/drive/folders/1Jb0-OctmXDy_rDD-Jk_N_QyrADHKamB2?usp=sharing
Thanks,

I removed the 'content-type' and it works for both python script and Google app script

axios does not return the full data

axios.get(downloadUrl, Object.assign({}, this.tokens[token_nr].config, {responseType: stream}))
.then((response) => {
console.log("HEADERS", response.headers)
console.log(typeof response.data, "RESPONSE LENGTH", response.data.length)
const decryptedBuffer = encryptionService.decrypt(Buffer.from(response.data), infos);
resolve(decryptedBuffer);
})
This axios request should give the data of a mp3 file. I previously had it via the request package which gives a binary Buffer (using the encoding: null option) and I can use it in the encryptionService.decrypt() function.
In the response.headers I can see it gives the same content-length as it would with the request package. But when I print the length of response.data it's shorter. I tried both ArrayBuffer and stream as my ResponseType. Also leaving the ResponseType option out does not help. What should I do to get the full content.
Some logs: (not all headers)
HEADERS {
'accept-ranges': 'bytes',
'cache-control': 'public',
'content-type': 'audio/mpeg',
'content-length': '14175084',
connection: 'close'
}
string RESPONSE LENGTH 13495410
CONFIG HEADERS {
headers: {
Accept: 'application/json, text/plain, */*',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36',
'cache-control': 'max-age=0',
'accept-language': 'en-US,en;q=0.9,en-US;q=0.8,en;q=0.7',
'accept-charset': 'utf-8,ISO-8859-1;q=0.8,*;q=0.7',
cookie: 'arl=XXXX',
Connection: 'keep-alive',
'Keep-Alive': 'timeout=1500, max=100',
'Content-Type': 'text/plain;charset=UTF-8'
},
}

When creating request try passing following headers Connection, Keep-Alive. Sometimes it close the connection before fully receiving the response
var axioRequest = await axios.create({
baseURL: url,
headers: {
Connection: 'keep-alive',
'Keep-Alive': 'timeout=1500, max=100'
}
});

It was resolved with this answer:
https://disjoint.ca/til/2017/09/20/how-to-download-a-binary-file-using-axios/
I missed the Content-Type header and the {ResponseType: 'arraybuffer'}

No file was submitted when try to send generated PDF file from React Js to Django-Rest Post request

Trying to generate pdf file in react js and then sending it to the django rest backend.
I have successfully created the pdf file using jsPDF and html2canvas, but now unable to send to the rest api, whenever I submit it gives me response "No file was submitted".I have checked django-rest api's and its working fine, the pdf is not going to the rest api's.Here's my below code:
genPDF=(evt)=>{
evt.preventDefault();
html2canvas(document.getElementById("pdfid")).then(canvas=>{
let img=canvas.toDataURL('img/png');
let doc=new JsPDF();
doc.addImage(img,'JPEG',30,30);
//doc.save('test.pdf');
let file=doc;
const formdata=new FormData();
formdata.append("file",file);
this.postpdf(formdata)
});
};
postpdf=(payload)=>{
fetch(`http://127.0.0.1:8000/chauffeur/pdf_upload/`,
{
method: 'POST',
body: JSON.stringify(payload),
headers: {
'Content-Type': 'application/json'
}
}
).then(response => response.json()).catch(error => console.error('Error:', error));
};
Request Headers
Content-Type: multipart/form-data; boundary=----
WebKitFormBoundaryEueEwtpzbquHU6Tb
Origin: http://localhost:3000
Referer: http://localhost:3000/contractinvoice
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/73.0.3683.86 Safari/537.36
Response Headers
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: http://localhost:3000
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 76
Content-Type: application/json
Date: Wed, 10 Apr 2019 05:44:49 GMT
Server: WSGIServer/0.2 CPython/3.5.2
Vary: Accept, Cookie, Origin
X-Frame-Options: SAMEORIGIN
I think I am sending the file in a wrong but can't sort it what's the problem,need for suggestions.Thanks

You have error here:
'Content-Type': 'application/json'
If you want to send file, you should use multipart/form-data

why do you want to convert payload to JSON.stringify()... payload is not json...
try this...
postpdf=(payload)=>{
fetch(`http://127.0.0.1:8000/chauffeur/pdf_upload/`,
{
method: 'POST',
body: payload,
headers: {
'Content-Type': 'multipart/form-data'
}
}
).then(response => response.json()).catch(error => console.error('Error:', error));
};

How to use the node.js 'request' library with this http request?

I was trying to make a simple request to site. it should get html text, but it gets ' '
NPM module here: github.com/request/request
Code:
var fs = require('fs');
var request = require('request');
var options = {
url:'https://sample.site/phpLoaders/getInventory/getInventory.php',
encoding : 'utf8',
gzip : true,
forever: true,
headers: {
'Host': 'sample.site',
'Connection': 'keep-alive',
'Content-Length': '58',
'Cache-Control': 'max-age=0',
'Accept': '*/*',
'Origin': 'https://csgosell.com',
'X-Requested-With': 'XMLHttpRequest',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'Referer': 'https://sample.site/',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4',
'Cookie': 'my-cookies from browser'
},
form: {
stage:'bot',
steamId:76561198284997423,
hasBonus:false,
coins:0
}
};
request.post(options,
function(error, response, body){
console.log(response.statusCode);
if (!error) {
fs.writeFileSync('site.html', body);
}
else{
console.log(error);
}
}
);
Chrome request: https://i.stack.imgur.com/zKQo5.png
Nodejs request:https://i.stack.imgur.com/yH9U3.png
the difference is in headers:
:authority:csgosell.com
:method:POST :path:/phpLoaders/getInventory/getInventory.php :scheme:https
after some googling, I anderstood that it is http2, and tried to put it inow another agent's options, but nothing changed.
var spdy = require('spdy');
var agent = spdy.createAgent({
host: 'sample.site',
port: 443,
spdy: {
ssl: true,
}
}).once('error', function (err) {
this.emit(err);
});
options.agent = agent;

To answer your question i will copy/paste a part of my code that enable you to receive a post request from your frontend application(angularJS) to your backend application (NodeJS), and another function that enable you to do the inverse send a post request from nodeJS to another application (that might consume it):
1) receive a request send from angularJS or whatever inside your nodeJS app
//Import the necessary libraries/declare the necessary objects
var express = require("express");
var myParser = require("body-parser");
var app = express();
// we will need the following imports for the inverse operation
var https = require('https')
var querystring = require('querystring')
// we need these variables for the post request:
var Vorname ;
var Name ;
var e_mail ;
var Strasse ;
app.use(myParser.urlencoded({extended : true}));
// the post request is send from http://localhost:8080/yourpath
app.post("/yourpath", function(request, response ) {
// test the post request
if (!request.body) return res.sendStatus(400);
// fill the variables with the user data
Vorname =request.body.Vorname;
Name =request.body.Name;
e_mail =request.body.e_mail;
Strasse =request.body.Strasse;
response.status(200).send(request.body.title);
});
2) Do the inverse send a POST request from a nodeJS application to another application
function sendPostRequest()
{
// prepare the data that we are going to send to anymotion
var jsonData = querystring.stringify({
"Land": "Land",
"Vorname": "Vorname",
"Name": "Name",
"Strasse": Strasse,
});
var post_options = {
host: 'achref.gassoumi.de',
port: '443',
method: 'POST',
path: '/api/mAPI',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': jsonData.length
}
};
// request object
var post_req = https.request(post_options, function(res) {
var result = '';
res.on('data', function (chunk) {
result += chunk;
console.log(result);
});
res.on('end', function () {
// show the result in the console : the thrown result in response of our post request
console.log(result);
});
res.on('error', function (err) {
// show possible error while receiving the result of our post request
console.log(err);
})
});
post_req.on('error', function (err) {
// show error if the post request is not succeed
console.log(err);
});
// post the data
post_req.write(jsonData);
post_req.end();
// ps : I used a https post request , you could use http if you want but you have to change the imported library and some stuffs in the code
}
So finally , I hope this answer will helps anyone who is looking on how to get a post request in node JS and how to send a Post request from nodeJS application.
For further details about how to receive a post request please read the npm documentation for body-parser library : npm official website documentation

Why does my nodejs pdf upload code return Status 415 "unsupported Media Type"

Hey so I am using the Node.js framework Hapi.js to make a simple file upload api. After I receive the file I will convert the pdf file to a png. I have been looking for some Hapi.js docs on uploading files but have had no luck. After I receive the file I want to use the Node file system to read it and then pass it to a command line tool.
Here is my route for the conversion:
server.route({
method: 'POST',
path: '/convert',
config: {
payload: {
output: 'stream',
parse: true,
allow: ['application/json', 'image/jpeg', 'multipart/form-data','application/pdf']
//allow:'application/json'
},
handler:function (request, reply) {
console.log(request.raw.req);
var data = request.payload;
if (data.file) {
var name = data.file.hapi.filename;
console.log(name);
var path = __dirname + "/uploads/" + name;
console.log(path);
var file = fs.createWriteStream(path);
file.on('error', function (err) {
console.error(err)
});
data.file.pipe(file);
data.file.on('end', function (err) {
var ret = {
filename: data.file.hapi.filename,
headers: data.file.hapi.headers
}
console.log(JSON.stringify(ret));
reply(JSON.stringify(ret));
});
data,file.on('data',function(err){
console.log('data');
});
}
}
}
});
I have a simple form with method POST and the action set to my /convert route.
Anyone see where my error is? I have not used hapi.js before this.
Here is the header in my post:
headers:
{ host: 'localhost:8000',
connection: 'keep-alive',
'content-length': '31',
'cache-control': 'max-age=0',
accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
origin: 'http://localhost:8000',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/37.0.2062.120 Chrome/37.0.2062.120 Safari/537.36',
'content-type': 'application/x-www-form-urlencoded',
referer: 'http://localhost:8000/convert',
'accept-encoding': 'gzip,deflate',
'accept-language': 'en-US,en;q=0.8'
}

As you can see in the headers, the Content-Type header is set to application/x-www-form-urlencoded:
'content-type': 'application/x-www-form-urlencoded'
However it is not in the list of allowed content types:
allow: ['application/json', 'image/jpeg', 'multipart/form-data','application/pdf']
That's why hapi responds with 415. Instead of validating the type of the request, validate the type of the file in the payload:
validate: {
payload: {
fileUpload: Joi.object({
headers: Joi.object({
'content-type': Joi.string().valid(['application/pdf']).required(),
}).unknown().required()
}).unknown()
}
}