Node.- issues when trying to request behind https proxy - javascript

I am trying to download an image from a server through an https proxy, please help.
My code:
var request = require('request');
request({
url: url,
proxy: proxy
}, function (err, res, imgBuffer) {
console.log(err)
console.log(res)
})
The error:
Error: tunneling socket could not be established, cause=write EPROTO 101057795:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:openssl\ssl\s23_clnt.c:827:
I can provide any additional info needed, I have already tried a lot.

I'm thinking you need to change the config file in npm.
Check out this tutorial: https://www.jhipster.tech/configuring-a-corporate-proxy/
Go down to the section titled "NPM configuration."

Related

Retrieve JSON from HTTP POST request

I have the following function (essentially taken straight from the answer to another SO question):
function makePostRequest(requestURL, postData) {
request(
{
url: requestURL,
method: "POST",
json: true,
body: postData
},
function(error, response, body) {
console.log(response);
});
}
When I call it with the right requestURL, I successfully reach this route:
router.post("/batchAddUsers", function(req, res) {
console.log("Reached batchAddUsers");
});
I've been trying to retrieve the postData I sent with the request to no avail. Both req.params and req.body are {}. I haven't got the faintest clue how to refer to the object containing body passed in the request.
I read the whole console.log(req) and found nothing useful. I've done this kind of stuff before, except the request was made by a form and req.body worked like a charm. Now that I'm doing the request "manually", it doesn't work anymore. The whole thing is running on Node.js.
What am I doing wrong?
I think you did not insttalled body-parser
To handle HTTP POST request in Express.js version 4 and above, you need to install middleware module called body-parser.
body-parser extract the entire body portion of an incoming request stream and exposes it on req.body .
The middleware was a part of Express.js earlier but now you have to install it separately.
This body-parser module parses the JSON, buffer, string and url encoded data submitted using HTTP POST request. Install body-parser using NPM as shown below.
npm install body-parser --save

Access Control Origin Header error using Axios

I'm making an API call using Axios in a React Web app. However, I'm getting this error in Chrome:
XMLHttpRequest cannot load
https://example.restdb.io/rest/mock-data. No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://localhost:8080' is therefore not allowed
access.
{
axios
.get("https://example.restdb.io/rest/mock-data", {
headers: {
"x-apikey": "API_KEY",
},
responseType: "json",
})
.then((response) => {
this.setState({ tableData: response.data });
});
}
I have also read several answers on Stack Overflow about the same issue, titled Access-Control-Allow-Origin but still couldn't figure out how to solve this. I don't want to use an extension in Chrome or use a temporary hack to solve this. Please suggest the standard way of solving the above issue.
After trying out few answers I have tried with this,
headers: {
'x-apikey': '59a7ad19f5a9fa0808f11931',
'Access-Control-Allow-Origin' : '*',
'Access-Control-Allow-Methods':'GET,PUT,POST,DELETE,PATCH,OPTIONS',
},
Now I get the error as,
Request header field Access-Control-Allow-Origin is not
allowed by Access-Control-Allow-Headers in preflight response
I'll have a go at this complicated subject.
What is origin?
The origin itself is the name of a host (scheme, hostname, and port) i.g. https://www.google.com or could be a locally opened file file:// etc.. It is where something (i.g. a web page) originated from. When you open your web browser and go to https://www.google.com, the origin of the web page that is displayed to you is https://www.google.com. You can see this in Chrome Dev Tools under Security:
The same applies for if you open a local HTML file via your file explorer (which is not served via a server):
What has this got to do with CORS issues?
When you open your browser and go to https://website.example, that website will have the origin of https://website.example. This website will most likely only fetch images, icons, js files and do API calls towards https://website.example, basically it is calling the same server as it was served from. It is doing calls to the same origin.
If you open your web browser and open a local HTML file and in that HTML file there is JavaScript which wants to do a request to Google for example, you get the following error:
The same-origin policy tells the browser to block cross-origin requests. In this instance origin null is trying to do a request to https://www.google.com (a cross-origin request). The browser will not allow this because of the CORS Policy which is set and that policy is that cross-origin requests is not allowed.
Same applies for if my page was served from a server on localhost:
Localhost server example
If we host our own localhost API server running on localhost:3000 with the following code:
const express = require('express')
const app = express()
app.use(express.static('public'))
app.get('/hello', function (req, res) {
// res.header("Access-Control-Allow-Origin", "*");
res.send('Hello World');
})
app.listen(3000, () => {
console.log('alive');
})
And open a HTML file (that does a request to the localhost:3000 server) directory from the file explorer the following error will happen:
Since the web page was not served from the localhost server on localhost:3000 and via the file explorer the origin is not the same as the server API origin, hence a cross-origin request is being attempted. The browser is stopping this attempt due to CORS Policy.
But if we uncomment the commented line:
const express = require('express')
const app = express()
app.use(express.static('public'))
app.get('/hello', function (req, res) {
res.header("Access-Control-Allow-Origin", "*");
res.send('Hello World');
})
app.listen(3000, () => {
console.log('alive');
})
And now try again:
It works, because the server which sends the HTTP response included now a header stating that it is OK for cross-origin requests to happen to the server, this means the browser will let it happen, hence no error.
Just to be clear, CORS policies are security features of modern day browsers, to protect people from harmful and malicious code.
How to fix things (One of the following)
Serve the page from the same origin as where the requests you are making reside (same host).
Allow the server to receive cross-origin requests by explicitly stating it in the response headers.
If using a reverse proxy such as Nginx, configure Nginx to send response headers that allow CORS.
Don't use a browser. Use cURL for example, it doesn't care about CORS Policies like browsers do and will get you what you want.
Example flow
Following is taken from: Cross-Origin Resource Sharing (CORS)
Remember, the same-origin policy tells the browser to block
cross-origin requests. When you want to get a public resource from a
different origin, the resource-providing server needs to tell the
browser "This origin where the request is coming from can access my
resource". The browser remembers that and allows cross-origin resource
sharing.
Step 1: client (browser) request When the browser is making a cross-origin request, the browser adds an Origin header with the
current origin (scheme, host, and port).
Step 2: server response On the server side, when a server sees this header, and wants to allow access, it needs to add an
Access-Control-Allow-Origin header to the response specifying the
requesting origin (or * to allow any origin.)
Step 3: browser receives response When the browser sees this response with an appropriate Access-Control-Allow-Origin header, the
browser allows the response data to be shared with the client site.
More links
Here is another good answer, more detailed as to what is happening: https://stackoverflow.com/a/10636765/1137669
If your backend support CORS, you probably need to add to your request this header:
headers: {"Access-Control-Allow-Origin": "*"}
[Update] Access-Control-Allow-Origin is a response header - so in order to enable CORS - you need to add this header to the response from your server.
But for the most cases better solution would be configuring the reverse proxy, so that your server would be able to redirect requests from the frontend to backend, without enabling CORS.
You can find documentation about CORS mechanism here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
I had a similar problem and I found that in my case the withCredentials: true in the request was activating the CORS check while issuing the same in the header would avoid the check:
Reason: expected ‘true’ in CORS header ‘Access-Control-Allow-Credentials’
Do not use
withCredentials: true
but set
'Access-Control-Allow-Credentials':true
in the headers.
For Spring Boot - React js apps I added #CrossOrigin annotation on the controller and it works:
#CrossOrigin(origins = {"http://localhost:3000"})
#RestController
#RequestMapping("/api")
But take care to add localhost correct => 'http://localhost:3000', not with '/' at the end => 'http://localhost:3000/', this was my problem.
I had the same error. I solved it by installing CORS in my backend using npm i cors. You'll then need to add this to your code:
const cors = require('cors');
app.use(cors());
This fixed it for me; now I can post my forms using AJAX and without needing to add any customized headers.
For any one who used cors package change
const cors = require('cors');
app.use(cors());
to
const cors = require('cors');
app.use(cors({credentials: true, origin: 'http://localhost:5003'}));
change http://localhost:5003 to your client domain
Using the Access-Control-Allow-Origin header to the request won't help you in that case while this header can only be used on the response...
To make it work you should probably add this header to your response.You can also try to add the header crossorigin:true to your request.
First of all, CORS is definitely a server-side problem and not client-side but I was more than sure that server code was correct in my case since other apps were working using the same server on different domains. The solution for this described in more details in other answers.
My problem started when I started using axios with my custom instance. In my case, it was a very specific problem when we use a baseURL in axios instance and then try to make GET or POST calls from anywhere, axios adds a slash / between baseURL and request URL. This makes sense too, but it was the hidden problem. My Laravel server was redirecting to remove the trailing slash which was causing this problem.
In general, the pre-flight OPTIONS request doesn't like redirects. If your server is redirecting with 301 status code, it might be cached at different levels. So, definitely check for that and avoid it.
After a long time of trying to figure out how CORS works. I tried many way to fix it in my FE and BE code. Some ways CORS errors appearance, some ways the server didn't receive body from client, and other errors...
And finally got this way. I'm hoping this can help someone:
BE code (NodeJS + Express)
var express = require("express");
const cors = require("cors");
var app = express();
app.use(
cors({
origin: "*",
})
);
app.use(function (req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
next();
});
// your routers and codes
My FE code (JS):
fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Connection: 'Keep-Alive',
Authorization: `Bearer test`,
},
body: JSON.stringify(data),
});
I imagine everyone knows what cors is and what it is for.
In a simple way and for example if you use nodejs and express for the management, enable it is like this
Dependency:
https://www.npmjs.com/package/cors
app.use (
cors ({
origin: "*",
... more
})
);
And for the problem of browser requests locally, it is only to install this extension of google chrome.
Name: Allow CORS: Access-Control-Allow-Origin
https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf?hl=es
This allows you to enable and disable cros in local, and problem solved.
npm i cors
const app = require('express')()
app.use(cors())
Above code worked for me.
You can create a new instance of axios with a custom config, and then use this new configured instance,
create a file with axios-configure.js, add this sharable exported method and use this preconfigured import, rather importing axios directly like we use traditionally,
import axios from 'axios';
import baseUrl from './data-service';
const app = axios.create({
baseURL: baseUrl,
headers: {
'Access-Control-Allow-Origin': '*',
'Content-Type': 'application/json',
},
withCredentials: true
})
export default app;
use this exported function like,
import axios from '../YOUR_DIRECTORY/axios-configure';
axios.get();// wont throw cors
dont import axios from axios;
then use axios.get() it will dont throw cors worked for us,
NOTE this solution will work for them who facing CORS at local environment as local starts at 5000 and backend at 8080, but in production, build gets deployed from java 8080 no CORS in productions (Facing CORS at only local environment)
As I understand the problem is that request is sent from localhost:3000 to localhost:8080 and browser rejects such requests as CORS. So solution was to create proxy
My solution was :
import proxy from 'http-proxy-middleware'
app.use('/api/**', proxy({ target: "http://localhost:8080" }));
$ npm install cors
After installing cors from npm add the code below to your node app file. It solved my problem.
var express = require('express')
var cors = require('cors')
var app = express()
app.use(cors())
I had a similar problem when I tried to create the React Axios instance.
I resolved it using the below approach.
const instance = axios.create({
baseURL: "https://jsonplaceholder.typicode.com/",
withCredentials: false,
headers: {
'Access-Control-Allow-Origin' : '*',
'Access-Control-Allow-Methods':'GET,PUT,POST,DELETE,PATCH,OPTIONS',
}
});
try it proxy
package.json add code:
"proxy":"https://localhost:port"
and restart npm enjoy
same code
const instance = axios.create({
baseURL: "/api/list",
});
You can use cors proxy in some specific cases - https://cors.sh
In node js(backend), Use cors npm module
$ npm install cors
Then add these lines to support Access-Control-Allow-Origin,
const express = require('express')
const app = express()
app.use(cors())
app.get('/products/:id', cors(), function (req, res, next) {
res.json({msg: 'This is CORS-enabled for a Single Route'});
});
You can achieve the same, without requiring any external module
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
},
"proxy": "http://localhost:8080",
"devDependencies": {
use proxy in package.json

How do I open a websocket connection to cloud from location behind corporate proxy

I have a nodejs websocket server running in the cloud to which I can make a secured web socket connection from any location outside of corporate firewall.
Now, how can I provide proxy information while creating a websocket connection from a resource behind corporate proxy (on-premise)? I am using this nodejs module from https://github.com/websockets/ws.
I get EHOSTUNREACH error when I execute following code. Please note there is no vpn connection between cloud resource and the on premise resource.
**var wss = new WebSocket('wss://cloudserver:443', {
rejectUnauthorized: false
});**
With a sample code like below, I am able to make http connection to cloud from a resource behind corporate proxy using proxy information , but can't figure out how to make it work with the web sockets.
**var Http = require('http');
var req = Http.request({
host: 'proxy',
port: 8080,
method: 'GET',
path: 'http://cnn.com/' // full URL as path
}, function (res) {
res.on('data', function (data) {
console.log(data.toString());
});
});
req.end()
**
I also looked at following http://blog.vanamco.com/proxy-requests-in-node-js/ and it seems to have what I need, but as I am new to nodejs I am somewhat lost here.
Well EHOSTUNREACH implies that no network route can be found to the IP address that DNS provides for the targeted server.
This can either be a DNS error from the local DNS server (wrong IP address) or a routing error (wrong or missing route to that IP address) or possibly there is purposely no route to the target address (private network or corp network blocking things).
You could check the DNS side of things by seeing if you get the same IP address for that host from both work and home. A simple ping hostname will show you the IP address. If the target server supports ping requests, it will also tell you if the host is reachable. You could also examine tracert hostname from both home and work and see where things get lost.
I was able to connect to my cloud instance by providing proxy information using websocket-node (https://github.com/theturtle32/WebSocket-Node/)
module as below. The actual url is replaced with 'xxx' for security reasons. Thanks everyone who tried to help.
var fs = require('fs');
var client = new WebSocketClient();
var tunnel = require('tunnel');
var net = require('net');
var tunnelingAgent = tunnel.httpsOverHttp({
rejectUnauthorized: false,
proxy: {
host: 'xxxxxxxx',
port: 8080,
}
});
var requestOptions = {
agent: tunnelingAgent,
rejectUnauthorized: false,
strictSSL: false,
requestCert: false,
};
var headers = {
};
client.connect('wss://xxxxxx',null, 'xxxxxx', headers, requestOptions);

Monitor changes of remote file without downloading it with nodejs

Is there a possibility to monitor changes of file on some server without downloading it?
I read about chokidar module but I cant find anything about my issue
I believe there is the way to watch some headers or smth like this
Maybe here is someone who have solved similar issue?
You can check Last-Modified and/or E-tag http headers.
var http = require('http');
var options = {method: 'HEAD', host: 'stackoverflow.com', port: 80, path: '/'};
var req = http.request(options, function(res) {
console.log(res.headers);
}
);
req.end();
Yes you can "watch" files using NodeJS.
How to do it:
There is several NodeJS modules allowing that. Take a look at watchr for example. You can "watch" a file or a directory using it.

How to authenticate a request with htpasswd in Node.js

I am trying to fetch a rss feed from our staging site and at present it has htpasswd security on it.
I have tied using the format:
http://username:password#url.com
This works on the browser but when I try to do this with nodejs it fails.
Could you tell me what is the right way to do this with node.js.
app.js
var request = require('request');
request.get('http://url.com', callback).auth('username', 'password', false);
function callback(err, response, body) {
console.log(body);
}
terminal:
npm install request
node app.js
document: https://github.com/mikeal/request

Categories

Resources