my api is not taking data from client sided code to the /api enpoint I have created on the localhost of the webserver on port 3000. How do I get it to take the server sided code to take the post request and console.log(request); correctly if it Page started
(index):23 POST https://goldengates.club:3000/api net::ERR_CONNECTION_TIMED_OUT
(anonymous) # (index):23
goldengates.club/:1 Uncaught (in promise) TypeError: Failed to fetch
Client Sided Code:
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<script>
console.log("Page started");
const num1=1;
const num2=2;
const data=35;//{num1,num2};
const options =
{
method: 'POST',
headers:
{
"Content-Type": "application/json"
},
body: JSON.stringify(data)
};
fetch('https://goldengates.club:3000/api'/*'https://goldengates.club/Cypher-Network/fetchTest.js'*/,options);
</script>
</body>
</html>
server sided code:
const express = require('express');
const app = express();
app.listen(3000,()=>console.log('listening on port 3000'));
app.use(express.static('public'));
app.post('/api',(request,response)=>
{
response.setHeader("Content-Type", "application/json");response.send({ statusCode:200, customMessage : 'Sucess', status : true })
console.log(request);
response.end();
});
the function fetch returns a promise, in order to use it, you need to call the then method, like so:
// backend code
app.post('/example', (req, res) => res.json({ data: 'abc' }));
// frontend code
fetch('/example', { method: 'post' }).then(r => r.json()).then(d => alert(d.data));
I think the way you show in your code, it will just store the function of how to compute the result of promise, not execute it and produce actual result (data fetched and passed to alert). With the Promise.then method, you can pass data the way you were doing, with Content-Type: application/json header and body: JSON.stringify(object).
Related
I am using expressJS on the back end to make a very simple API since I am a beginner. I am sending a request to the back end from the front end and I expect the front end to receive a response. This works fine until I change the nodejs for it to make a second request before sending the original response back to the client. The process looks something like:
Front end sends a POST request
back end receives request, then:
makes its own POST request to a source
waits for this data to come back, then:
sends back a response to the original request from the front end including the data gotten from the second request.
This process works fine when I remove the few lines of code which send the second request, but when the NodeJs back end makes this second request, I get a 404 error returned to the front end - and this error does not come from the second request.
Here is the code:
front end:
function post() {
return new Promise(() => {
$.ajax("URL of my nodejs backend", {
method: "POST",
cache: false,
data: {
action: "test-https"
},
}).then(response => {
console.log(response);
});
});
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.0/jquery.min.js"></script>
nodejs backend (only the bits needed for this question)
const express = require("express");
//const $ = require("./djax.js");
const https = require('https');
const app = express();
app.post("/", (req, res, next) => {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
const body = [];
req.on("data", (chunk) => {
body.push(chunk);
});
req.on("end", () => {
const parsedBody = Buffer.concat(body).toString();
//res.status(200).send("bod" + parsedBody);
// Now parsedBody will be like a query string: key1=val1&key2=val2
const queryObject = new URLSearchParams(parsedBody);
parseRequest(queryObject, res);
//console.log(parsedBody);
});
//console.log(body);
next();
});
function parseRequest(queryParameters, response) {
// Here, queryParameters is a QueryParams object holding the body of the request
// sendResponseFunc is the function which sends back the response for this
// current request.
// Now, we have access to the body of the request and we can use this
// to call the neccessary functions and logic, after which
// send a response back to the front-end via the second
// parameter
const action = queryParameters.get("action");
switch(action.toLowerCase()) {
// ... other cases ...
case "test-https":
sendHttpsRequest(response);
break;
default:
response.status(200).send("Error: unknown action:'" + action.toLowerCase() + "'");
break;
}
}
function sendHttpsRequest(response) {
const postData = JSON.stringify({
works: true
});
const postOpts = {
host: "httpbin.org", // This is a test-server. Not mine.
path: "/post",
method: "POST",
headers: {
"Content-Type": "application/json",
"Content-Length": Buffer.byteLength(postData)
}
};
const newReq = https.request(postOpts, result => {
result.setEncoding("utf8");
res.on("data", chunk => {
console.log("Response" + chunk);
response.status(200).send("Request made from NodeJS end came back " + chunk);
});
});
newReq.write(postData);
newReq.end();
}
When I change the sendHttpsRequest function so that it does NOT make a request, like so:
function sendHttpsRequest(response) {
response.status(200).send("Hi");
}
... then the front-end receives the response and there is no error. There is a 500 internal server error only when the second request is made - when the function sendHttpsRequest is like in the penultimate snippet. I have tried to fix this for two days but I have no idea why this error is happening.
How can I make this second request from the NodeJS server and send back the contents of that without causing the 500 error?
You need to have error handling. You cannot expect that external requests will succeed all the time, for that reason you have to have res.on("error", ...) to respond the client appropriately.
However, I don't see a special case why you are using a data listener to collect payload chunks, it can be simplified very much.
Here is a very simple working example for you
// Backend
const express = require('express');
const axios = require('axios');
const cors = require('cors');
const app = express();
const port = 3000;
app.use(express.json()); // Accepts JSON as a payload
app.use(cors());
app.post('/', (req, res) => {
axios
.get('EXTERNAL URL')
.then((response) => {
console.log('Received payload', req.body);
// Handle response
res.json({data: response.data}).status(201);
})
.catch((error) => {
// Handle error
res.json({
message: error.message,
code: 422})
.status(422);
});
});
app.listen(port, '0.0.0.0', () => console.log(`Started at //127.0.0.1:${port}`));
Required dependencies are
ExpressJS cors package
Axios HTTP client
Axios can be used in browsers as well
Here is your jQuery Ajax request which sends JSON payload instead of FormData
// FE jQuery
function post() {
return new Promise((resolve, reject) => {
$.ajax('http://127.0.0.1:3000', {
contentType: 'application/json; charset=utf-8',
dataType: 'json',
method: 'POST',
data: JSON.stringify({
action: 'test-https',
}),
success: (data) => resolve(data),
error: (err) => reject(err),
});
});
}
post().then(console.log).catch(console.error);
I am trying to send a GET request and receive a response on my client side (react).
On my express server:
app.get('/', function (req, res) {
res.send('Hey, I came from the express server');
})
When I use postman to sent the request I receive a good answer:
So I don't think the problem is with the server.
A problem:
When I try to send the request using react like this:
const getData = () => {
fetch("http://localhost:8081", {
method: "GET",
mode: 'no-cors'
}).then(response => console.log(response));
}
For some reason the response status is 0 and not 200 (I am receiving status code of 200 when checking this request with postman and also on the chrome network tab).
In addition, if I try to print response.body it will be null.
The response:
body: (...)
bodyUsed: false
headers: Headers {}
ok: false
redirected: false
status: 0
statusText: ""
type: "opaque"
url: ""
[[Prototype]]: Response
What am I doing wrong here?
You can add a proxy property to your package.json:
"proxy": "http://localhost:8081",
And then just use / as your fetch endpoint.
fetch('/'...
You need to transfer your response to an text or json like:
const getData = () => {
fetch("http://localhost:8081", {
method: "GET",
mode: 'no-cors'
}).then(response => response.json()).then(data => console.log(data))
}
Besides .json() there are even more methods. Its everything written down in the docs. You just need to google it:
https://developer.mozilla.org/en-US/docs/Web/API/Response#methods
I made a node server with the three packages express, body-parser and cors.
I am trying to access data from client-side in my app.js with an async/await function that has a post request with the required data that I want to fetch
here's the fetch request and the post request in app.js, I am trying to pass the data [temperature, date, userResponse] via the postData function in app.js:
//post routes
const postData=async function postData(url = 'http://api.openweathermap.org/data/2.5/weather?zip=', data = {}) {
const response = await fetch(url, {
method: 'POST',
mode: 'cors',
cache: 'no-cache',
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json'
},
redirect: 'follow',
referrerPolicy: 'no-referrer',
body: JSON.stringify(data)
});
return await response.json();
}
postData('/add', {temperature: '1', date: '2', userResponse: '2'});
}
and here's the server side code where I made an add route as post route:
// Setup empty JS object to act as endpoint for all routes
projectData = {};
// Require Express to run server and routes
const express = require('express');
// Start up an instance of app
const app = express();
/* Middleware*/
//Here we are configuring express to use body-parser as middle-ware.
const bodyParser = require('body-parser')
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
// Cors for cross origin allowance
const cors = require('cors');
app.use(cors());
// Initialize the main project folder
app.use(express.static('website'));
const port = 3000;
// Setup Server
const server=app.listen(port, ()=>{console.log(`running on localhost: ${port}`)});
app.get('/all', sendData);
function sendData (request, response) {
response.send(projectData);
};
// TODO-ROUTES!
const data=[]
app.post('/add', function(req,res){
data.push(req.body)
console.log(data)
})
here's what I got when I ran the project on localhost:3000
I want the the parameters temperature, date and userResponse to appear in the console
the errors at line 35 related to the post request are :
app.js:35 POST http://localhost:3000/add net::ERR_EMPTY_RESPONSE
app.js:48 Uncaught (in promise) TypeError: Failed to fetch
never mind the errors at the two above lines related to the get request
in vs code I can see this(the temperature, date and userResponse appear in the terminal but don't appear in the localhost:3000 console in the browser):
I just need the client-side to pass data to server-side dynamically and not just pass the response in the server-side
You are not sending any response back from the server. Send some response as below
app.post('/add', function(req,res){
data.push(req.body)
console.log(data)
return res.send({}); //return whatever response you need to send
})
The API I am trying to consume is from a third party and it blocks Cross-Origin, so I can not consume it with jquery, nor javascript ... So I had to create a script in nodejs to consume this API ...
I would like to know how do I consume this data that nodejs got from the API to my frontend, using javascript?
Remembering that this nodejs is in a separate file from my frontend and running from another server.
var request = require("../../node_modules/request");
var options = { method: 'GET',
url: 'https://....apimanagement.us2.hana.ondemand.com/bot/v1/...',
qs: { Shop: '..'', PeriodoAte: '...' },
headers:
{ 'postman-token': '822e513f-da5e-4a0b-b403-1dd8fa46e86f',
'cache-control': 'no-cache',
authorization: 'Basic .........',
apikey: '....',
'content-type': 'application/json' },
json: true };
request(options, function (error, response, body) {
console.log('error:', error); // Print the error if one occurred
console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
console.log('body:', body); // Print the HTML for the Google homepage.
});
One path would be:
pick a node framework (Express, Hapi, ...) lets pick Hapi for this example
read the getting started guide https://hapijs.com/tutorials/getting-started
learn about promises or go through callback hell
end up with something like this for testing
'use strict';
const Hapi = require('hapi');
const rp = require('request-promise');
const server = Hapi.server({
port: 3000,
host: 'localhost'
});
server.route({
method: 'GET',
path: '/',
handler: (request, h) => {
const options = {
method: 'GET',
uri: 'https://./..',
qs: {Shop: '..', PeriodoAte: '...'},
headers: {
'postman-token': '822e513f-da5e-4a0b-b403-1dd8fa46e86f',
'cache-control': 'no-cache',
authorization: 'Basic .........',
apikey: '...',
'content-type': 'application/json'
},
json: true
};
return rp(options).catch(e => {
console.log(`api call failed ${e}`);
return 'fail';
})
}
});
const init = async () => {
await server.start();
console.log(`Server running at: ${server.info.uri}`);
};
process.on('unhandledRejection', (err) => {
console.log(err);
process.exit(1);
});
init();
now you can start it with node and visit 'localhost:3000/'
read more, try more
read about node production mode and hosting
Something like that, hope it helps a bit
EDIT: //
to consume on client just use for example jquery to fetch the route served with the code above
Here is some client example as requested in the comments:
If you are serving your html from another location, not the hapi api, you will need to enable cors in the HapiJS route.
server.route({
method: 'GET',
path: '/',
config: {
cors: {
origin: ['*'],
additionalHeaders: ['cache-control', 'x-requested-with']
}
},
Then one way to consume this endpoint would be jquery
<!doctype html>
<html>
<head>
<script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
</head>
<body>
<input type="button" id="Button" value="Fetch"/>
<div id='main'></div>
<script type="text/javascript">
$(document).ready(function () {
$('#Button').click(() => {
$.ajax({
url: "http://localhost:3000/info", success: (result) => {
$("#main").text(result);
}
});
});
});
</script>
</body>
</html>
You need a web server listening for requests in order to communicate your frontend with your backend (Node.js). You could use Express if you want something simple. You could also use a JavaScript framework, there are many (Sails.js would be a good option to get started fast).
If you can't install Node.js in your server, another option would be using AWS Lambda to quickly create an API that you can consume from your frontend through an HTTP request.
I want to make GET request to scrape some data thru a proxy server that is randomly generated using the gimmeproxy.com free API.
I am able to get the proxy ip/port and am using
'https-proxy-agent' to setup the agent with the proxy data.
Whenever I try to call any website I always get
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>405 Method Not Allowed</title>
</head><body>
<h1>Method Not Allowed</h1>
<p>The requested method CONNECT is not allowed for the URL
/index.html.en.backup.</p>
</body></html>
Here is my node script:
const request = require('request'), HttpsProxyAgent = require('https-proxy-agent');
generateRandomProxy(function(proxy){
var agent = new HttpsProxyAgent({
proxyHost: proxy.proxyHost,
proxyPort: proxy.proxyPort
});
request({
uri: "http://example.com",
method: "GET",
agent: agent,
timeout: 5000,
}, function(error, response, body) {
console.log(body);
});
})
function generateRandomProxy(cb){
request.get(' https://gimmeproxy.com/api/getProxy?get=true&cookies=true&country=US',{json:true},function(err,res){
if(!err){cb({
proxyHost: res.body.ip,
proxyPort: res.body.port
})}
else{console.log('problem obtaining proxy')}
})
}
So my question: How can I route my request thru the proxy and then get a returned body that is valid?
As you see now I keep getting the 405 Method Not Allowed
Thank you for any assistance.
Edit: Just found some GimmeProxy wrapper for Node.js: gimmeproxy-request.
It claims to automatically re-route requests through another proxy when one fails.
With this module code would look like this:
const setup = require('gimmeproxy-request').setup;
const request = require('gimmeproxy-request').request;
setup({
api_key: 'your api key',
query: 'get=true&cookies=true&country=US&supportsHttps=true&maxCheckPeriod=1800&minSpeed=10', // additional gimmeproxy query parameters
retries: 5, // max retries before fail
test: (body, response) => body.indexOf('captcha') === -1 && response.statusCode === 200 // test function
});
request('https://example.com', {
timeout: 10000 // additional request parameters, see https://github.com/request/request
},
function(err, res, body) {
console.log('err', err)
console.log('res', res)
console.log('body', body)
process.exit()
});
I guess the issue is that you sometimes get not an https proxy from Gimmeproxy, while 'https-proxy-agent' expects https proxy only.
To fix it, use the proxy-agent package of the same author and pass curl field of GimmeProxy response. It will select correct proxy agent implementation.
The following code works for me:
const request = require('request'), ProxyAgent = require('proxy-agent');
generateRandomProxy(function(proxy){
console.log(proxy);
var agent = new ProxyAgent(proxy.curl);
request({
uri: "https://example.com",
method: "GET",
agent: agent,
timeout: 5000,
}, function(error, response, body) {
console.log(error);
console.log(body);
});
})
function generateRandomProxy(cb){
request.get('https://gimmeproxy.com/api/getProxy?get=true&cookies=true&country=US&supportsHttps=true&maxCheckPeriod=1800&minSpeed=10',{json:true},function(err,res){
if(!err){cb(res.body)}
else{console.log('problem obtaining proxy')}
})
}
Note: If you want to call https websites, you should query for proxies with https support using supportsHttps=true parameter. Also it makes sense to query for fresh proxies with maxCheckPeriod=1800 parameter. Setting minSpeed=10 also helps:
https://gimmeproxy.com/api/getProxy?get=true&cookies=true&country=US&supportsHttps=true&maxCheckPeriod=1800&minSpeed=10