Develop Reference

Upload file to s3 using presigned post url in the server

TDLR: Using s3 presigned post url to upload file to s3. Works fine on the browser but fails on the server.
I have a simple lambda function that generates presigned post url that can be consumed either in the browser or in the server.
During testing I noticed that the upload works fine one the browser but fails if I try to upload a file from a server even tho the code is identical.
The error i get is:
You must provide the Content-Length HTTP header
Detailed error:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>MissingContentLength</Code>
<Message>You must provide the Content-Length HTTP header.</Message>
<RequestId>JP75YMFARK0G3X5Z</RequestId>
<HostId>toHsKmxmVYYAtac94cQoy8wXoregKG3PNBm97c3gQewEmKxLggcumTAP882T/pJNWx/lxRgH98A=</HostId>
</Error>
Request failed with status code 411
I checked online and found many threads about this issue but unfortunately not a single suggestion helped me.
Code I am running in the server
const axios = require('axios');
const { createReadStream, readFileSync } = require('fs');
const FormData = require('form-data');
const getPostPresignedUrl = async () => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: 'test-2.jpg',
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
const file = createReadStream('./test-1.jpg');
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const init = async () => {
const { fields, url } = await getPostPresignedUrl();
await uploadFileToS3(fields, url);
};
init();
Code I am running in the browser:
const form = document.getElementById('form');
const input = document.getElementById('file');
const getPostPresignedUrl = async (name) => {
var config = {
method: 'post',
url: LAMBDA_GET_URL,
headers: {
'Content-Type': 'application/json',
},
data: JSON.stringify({
key: name,
fileType: 'image/jpeg',
}),
};
const {
data: { data },
} = await axios(config);
return data;
};
const uploadFileToS3 = async (fields, url, file) => {
const formData = new FormData();
Object.entries(fields).map(([key, value]) => {
formData.append(key, value);
});
formData.append('file', file);
try {
const { data } = await axios({
url,
method: 'post',
headers: {
'Content-Type': 'multipart/form-data',
},
data: formData,
});
} catch (error) {
if (error instanceof axios.AxiosError) {
console.log(error.response.data);
}
console.log(error.message);
}
};
const handleSubmit = async (e) => {
e.preventDefault();
const file = input.files[0];
const data = await getPostPresignedUrl(file.name);
await uploadFileToS3(data.fields, data.url, file);
};
form.onsubmit = handleSubmit;

Unable to invoke "btoa" and "item.slice" method in my script for retrieving the playlist

Whenever I am trying to invoke the "btoa" method, I am not able to use this within my script. I created a variable to store the client id: client_secret in base64. The id and secrets are being retrieved from the ".env" file.
I have also tried to use the Buffer method, but unable to use this as well. I am getting the error "invalid from" in Buffer.
can someone help me?
Please look at the full code,
const client_id = process.env.SPOTIFY_CLIENT_ID;
const client_secret = process.env.SPOTIFY_CLIENT_SECRET;
const refresh_token = process.env.SPOTIFY_REFRESH_TOKEN;
const basic = btoa(`${client_id}:${client_secret}`);
const NOW_PLAYING_ENDPOINT = `https://api.spotify.com/v1/me/player/currently-playing`;
const TOP_TRACKS_ENDPOINT = `https://api.spotify.com/v1/me/top/tracks`;
const TOKEN_ENDPOINT = `https://accounts.spotify.com/api/token`;
const getAccessToken = async () => {
const response = await fetch(TOKEN_ENDPOINT, {
method: 'POST',
headers: {
Authorization: `Basic ${basic}`,
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
grant_type: 'refresh_token',
refresh_token
})
});
return response.json();
};
export const getNowPlaying = async () => {
const { access_token } = await getAccessToken();
return fetch(NOW_PLAYING_ENDPOINT, {
headers: {
Authorization: `Bearer ${access_token}`
}
});
};
export const getTopTracks = async () => {
const { access_token } = await getAccessToken();
return fetch(TOP_TRACKS_ENDPOINT, {
headers: {
Authorization: `Bearer ${access_token}`
}
});
};
Using the above script I am trying to embed the customized Spotify play on my site. This wrapper is intended to display the top track as well.
Also, whenever I am trying to run the wrapper used to display the top tracks, it displays the following error,
Full code for displaying the top tracks:
import { type NextRequest } from 'next/server';
import { getTopTracks } from 'lib/spotify';
export const config = {
runtime: 'experimental-edge'
};
export default async function handler(req: NextRequest) {
const response = await getTopTracks();
const { items } = await response.json();
const tracks = items.slice(0, 10).map((track) => ({
artist: track.artists.map((_artist) => _artist.name).join(', '),
songUrl: track.external_urls.spotify,
title: track.name
}));
return new Response(JSON.stringify({ tracks }), {
status: 200,
headers: {
'content-type': 'application/json',
'cache-control': 'public, s-maxage=86400, stale-while-revalidate=43200'
}
});
}

The problem is that you misspelled the Bytes to ASCII function, it is btoa, not btao.
If you are looking to do it the other way around, spell it atob.

how to write a axios post call correctly?

I was always using plain JS und fetch call to handle API, they are working fine, now I want to replace all fetch with axios, so far the axios get calls are all working, but the axios post call are not, please guide:
the original fetch post from client side is like this, i need to send 3 values to server
const postTrip = async(location = '', daysToGo = '', notes ='') => {
const res = await fetch('http://localhost:7777/addData', {
method: 'POST',
mode: "cors",
cache: "no-cache",
credentials: "same-origin",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({location, daysToGo, notes}),
})
if (res.status >= 400 && res.status < 600) {
throw new Error("Bad response from server");
}
updateUI(daysToGo);
}
from server side
app.post('/addData', async (req, res) => {
try {
const city = req.body.location;
const dayLength = req.body.daysToGo;
const memo = req.body.notes;
let geo = await getGeo(city);
let weather = await getWeather(geo.lat, geo.lng, dayLength);
let image = await getImage(city);
const newEntry = {
geo,
weather,
image,
memo
}
projectData = newEntry;
res.status(201).send(projectData);
now I try to change client fetch post to axios post like this
const postTrip = async(location = '', daysToGo = '', notes ='') => {
const res = await axios('http://localhost:7777/addData', {
method: 'POST',
mode: "cors",
cache: "no-cache",
credentials: "same-origin",
headers: {
"Content-Type": "application/json"
},
data: JSON.stringify({location, daysToGo, notes}),
})
if (res.status >= 400 && res.status < 600) {
throw new Error("Bad response from server");
}
updateUI(daysToGo);
}
and change the server side like this:
app.post('/addData', async (req, res) => {
try {
const city = req.data.location;
const dayLength = req.data.daysToGo;
const memo = req.data.notes;
let geo = await getGeo(city);
let weather = await getWeather(geo.lat, geo.lng, dayLength);
let image = await getImage(city);
const newEntry = {
geo,
weather,
image,
memo
}
projectData = newEntry;
res.status(201).send(projectData);
it dosen't work, the server didn't get the three values from front, where is wrong?

With axios you don't have to stringify the body.
const postTrip = async(location = '', daysToGo = '', notes ='') => {
const res = await axios('http://localhost:7777/addData', {
method: 'POST',
mode: "cors",
cache: "no-cache",
credentials: "same-origin",
headers: {
"Content-Type": "application/json"
},
data: {location, daysToGo, notes},
})
if (res.status >= 400 && res.status < 600) {
throw new Error("Bad response from server");
}
updateUI(daysToGo);
}

my post request get response as https but does not return to a new page (javascript)

I'm send a post request and I get the response like this
" [Symbol(Response internals)]: {
url: 'https://login.somenewloginpage'}"
and what I want to do is I want to open a new page via that url but it does not direct to the new page.
const login= () => async () => {
const api = `somePostRequest`
fetch(api, {
method: 'post',
headers: {
'Content-Type': 'application/x-www-form-url-encoded',
Accept: 'application/json',
},
})
.then(function(res) {
return res //maybe I should do something in this part...
})
.then(data => console.log(data));
};

Here's how to use fetch() with async/await syntax :
const login= () => async () => {
const api = `somePostRequest`;
const response = await fetch(api, {
method: 'post',
headers: {
'Content-Type': 'application/x-www-form-url-encoded',
Accept: 'application/json',
},
});
const data = await response.json(); // { url: 'https://login.somenewloginpage'}
window.location.replace(data.url); // <-- Redirection
};

How to get response from https.request in node outside of function [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 2 years ago.
I am trying to do something that I just can't wrap my head around, I have tried doing something with promises but still get stuck. I tried reading through this and still am stuck How do I return the response from an asynchronous call?
What I need to do is run the following code and get the body outside of the req, so I can check if it is succesful and send a response in a lambda with a 200 and a message from the body. I don't want to return the 200 status inside the function because I need to also check if a fetch request is succesful before sending the 200 status with a body back.
Hopefully someone can help with this
let statusTrue
const req = https.request(options, function(res) {
res.setEncoding("utf8")
res.on("data", function(body) {
console.log(`Body: ${body}`)
statusTrue = body.status
})
})
if (statusTrue) {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: true }),
}
} else {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: false }),
}
}
Update Heres my code with oswins answer, the full code for context. Right now the function fires after the handler finishes so I never am able to send a proper response back.
const sgMail = require("#sendgrid/mail")
require("dotenv").config()
const { SENDGRID_API_KEY, SENDGRID_TO_EMAIL } = process.env
const URL = require("url")
const https = require("https")
const fetch = require("node-fetch")
exports.handler = async (event, context) => {
try {
//console.log(JSON.parse(event.body))
//*** send off to google to verify captcha */
const body = JSON.parse(event.body)
let secret = "fdsf"
const fetchUrl = `https://www.google.com/recaptcha/api/siteverify?secret=${secret}&response=${body.captcha}&remoteip=${event.headers["client-ip"]}`
let isItTrue
await fetch(fetchUrl, {
method: "POST",
body: JSON.stringify({ message: "hello world" }),
headers: { "Content-Type": "application/json" },
})
.then(response => response.json())
.then(data => {
isItTrue = data.success
})
.catch(error => {
console.error("Error:", error)
})
//console.log(isItTrue)
//*** end */
//*** Running Form Sends Now if Captcha Valid */
if (isItTrue) {
//*** Zapier Send */
const webhook_url = URL.parse(
"https://hooks.zapier.com/hooks/catch/fds"
)
const options = {
hostname: webhook_url.hostname,
path: webhook_url.pathname,
method: "POST",
headers: { "Content-Type": "application/json" },
}
// Set up webhook request
const req = https.request(options, function(res) {
res.setEncoding("utf8")
res.on("data", function(body) {
console.log(`Body: ${body}`)
sendResponse(body.status)
})
})
// Handle webhook request error
req.on("error", function(e) {
const errorMessage = `[ERROR] Problem with request: ${e.message}`
console.log(errorMessage)
callback(e.message, {
statusCode: 400,
body: errorMessage,
})
})
// Send form data to webhook request and end request
req.end(JSON.stringify(body.data))
//*** End */
//console.log(zapierStatus)
const sendResponse = statusTrue => {
if (statusTrue === "success") {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: true }),
}
} else {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: false }),
}
}
}
} else {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ captcha: false }),
}
}
//*** end */
} catch (err) {
return { statusCode: 500, body: err.toString() }
}
}

Maybe wrapping statements outside the function into another function will help ;-)
const req = https.request(options, function (res) {
res.setEncoding("utf8")
res.on("data", function (body) {
console.log(`Body: ${body}`)
statusTrue = body.status
sendResponse(statusTrue)
})
})
function sendResponse(statusTrue) {
if (statusTrue) {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: true }),
}
} else {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: false }),
}
}
}

You can wrap your https.request call in a promise like this:
const makeRequest = function(options) {
return new Promise(function(resolve, reject) {
const req = https.request(options, function(res) {
res.setEncoding("utf8")
res.on("error", reject)
res.on("data", function(body) {
console.log(`Body: ${body}`)
resolve(body.status)
})
})
}
Then later you can do:
makeRequest({/*options*/})
.then(function(statusTrue) {
if (statusTrue) {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: true }),
}
} else {
return {
statusCode: 200,
headers: {
"Access-Control-Allow-Origin": "*",
},
body: JSON.stringify({ email: false }),
}
}
})
.then(function(ret) {
console.log(ret)
})
.catch(function(err) {
/* handle error */
})