How to solve CORS error while fetching an external API? - javascript

I'm developing a web app in Angular 10 that works as follows:
I'm dealing with CORS issue. I do not have permission to add code to the server I'm fetching.
I want to be able to:
Fetch the website
Parse the result, and put it in my database
I'm aiming to deploy the solution on an Apache server.
Here is the CORS error I'm dealing with:
Blocking a Cross-Origin Request: The "Same Origin" policy does not
allow viewing the remote resource located at
https://wwwfrance1.CENSORED.eu.com/api/?apikey=CENSORED.
Reason: "Access-Control-Allow-Origin" CORS header is missing. Status
code: 200.
Here is what i've tried:
Using MOSIF mozilla extension (works, but not sustainable for deployment, and for some reason, when I'm ignoring the CORS security, I cannot post on my DB any more)
Adding a header in my fetching request, such as:
/******API SEACH****/
/***Global Update***/
private updateClients() {
let xmlRequestPromise = fetch('https://wwwfrance1.CENSORED.eu.com/api/?apikey=CENSORED&service=list_clients', {
method: 'GET',
headers: {
'Access-Control-Allow-Origin': '*',
}
})
.then(async response => this.clients = this.regexSearchClient(await response.text()))
return xmlRequestPromise
}
But that doesn't work either. I've verified that the header appears in the request.
How to proceed?

What is CORS ?
Cross-origin resource sharing is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served. From wiki
In simple terms only an internal webserver can send Requests which are potentially dangerous to it's web server, and requests from other server's are simply blocked.
But few HTTP requests are allowed ,Few of the allowed methods are GET, HEAD, POST.
How do I resolve the issue ?
Apparently in this circumstance you cannot send a fetch request to a web server having CORS header. Instead you can do a GET request to the web server as a web server having CORS allows HTTP GET requests.
Note - If you send a GET request to a web server using angular in your browser it wouldn't work as browser's convert GET requests into fetch requests and fetch requests aren't allowed from a web server with CORS. Instead send a GET request from a webserver/local machine rather than a browser.

Create your own server and make a route which fetches that API. From your Angular application fetch that route on your server.

You have to use a package as a middleware. If you are using nodejs-framework expressjs.At first, you have to run npm install cors -force.Then add the code that is given bellow:-
const cors=require('cors')
app.use(cors({origin:true}))

Related

Inclusion of header causes requests to not be send

I have react application where I want to add a request-id header to my requests, such that the frontend can tell the backend to undo a specific request. So requests (using superagent) are something like this:
let result = request(method, endpoint);
result = result.set("Accept", "application/json").set("Request-Id", getRequestId());
And when I add the ".set("Request-Id", getRequestId())" I get the error below.
I can see that I can send requests with postman with the request-ID header and I can see that the loadbalancer does not receive any requests other than options calls. CORS is enabled and exposing all headers for all origins.
Does anybody have ideas for what might be wrong? I'm quite new to frontend development.
The answer was I was that my corporate computer that has hardcoded in restrictions in the browser for not allowing custom headers. So I went in and found a standard header that in conjunction with the url could be used for the id so https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Date in my case.
Another evidence for this is that I could make post anything on my corporate computer on Facebook. Since the Facebook application uses custom headers.

API request blocked by CORS

I'm trying to use the g-trends API in my vuejs application running from localhost, but keep running into issues of my requests getting blocked due to CORS restrictions: No 'Access-Control-Allow-Origin' header is present on the requested resource.
I know how to fix the issue when performing a fetch request, but not in a scenario where I don't actually see the request URL like when using an API. Is there a way to fix this?
const { ExploreTrendRequest } = require('g-trends');
const explorer = new ExploreTrendRequest();
explorer.past5Years()
.addKeyword("keyword")
.download().then(csv => {
console.log(csv)
})
I usually use a proxy server for avoiding this situation, your app asks your server to do that request for you so the communication with the API is managed by your server, when your server receives the answer it forwards it to your app and there won´t be any CORS problem.

POSTing to external API throws CORS but it works from Postman

I am using the imgur api to upload images via a node js app.
I am converting images to base64 strings and sending them via Postman works great.
I use node-fetch to make api calls.
const fetch = require('node-fetch')
...
async uploadImage(base64image) {
try {
const url = 'https://api.imgur.com/3/image'
const res = await fetch(url,
{
method: 'POST',
body: { image: base64image },
headers: {
'content-type': 'application/json',
'Authorization': 'Client-ID [my-client-id]',
'Access-Control-Allow-Headers': 'Content-Type, Authorization, Access-Control-Allow-Headers',
'Access-Control-Allow-Methods': 'POST',
}
}
)
console.log(res)
} catch(err) {
console.log(err)
}
}
Error:
Access to fetch at 'https://api.imgur.com/3/image' from origin 'http://localhost:3000' has been blocked by CORS policy: Request header field Access-Control-Allow-Headers is not allowed by Access-Control-Allow-Headers in preflight response.
I have tried many 'Access-Control-Allow-xxx' headers but none of them worked..
I assume it must be something simple that I am missing. I have been stuck on this for hours please help me.
Browser restricts HTTP requests to be at the same domain as your web page, so you won't be able to hit imgur api directly from the browser without running into CORS issue.
I am converting images to base64 strings and sending them via Postman
works great.
That's because Postman is not a browser, so is not limited by CORS policy.
I have tried many 'Access-Control-Allow-xxx' headers but none of them
worked..
These headers must be returned by the server in response - in your case by the imgur server. You can't set them in the request from browser, so it'll never work.
Error: Access to fetch at 'https://api.imgur.com/3/image' from origin
'http://localhost:3000' has been blocked by CORS policy: Request
header field Access-Control-Allow-Headers is not allowed by
Access-Control-Allow-Headers in preflight response.
Possible solutions to your problem:
If you have access to the backend api you can set the "Access-Control-Allow-Origin" header on the server and let your app access the api - but as you won't have access to the imgur server - you probably can't do that.
Disable CORS in the browser - you can use a plugin like: https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi?hl=en. This workaound should be fine for development. The plugin will disable your CORS settings and you will be able to hit imgur apis.
The third solution is using a proxy. You can setup a small node server using express. You will then hit your own node server, which in turn will hit the imgur api. As node server is not a browser environment, it won't have any CORS issue and you will be able to access imgur API that way. This is also the reason you were able to hit the API from Postman without any issues. As Postman is not a browser environment, it's not limited by CORS policy.
That's because Access-Control-Allow-Headers, Access-Control-Allow-Methods are the headers that is used by the server. The server appends the header by a middleware.
Now, imagine in the server(in this below example an express server) with CORS enabled this kind of (default) headers are getting set:
app.use(function (req, res, next) {
res.header('Access-Control-Allow-Origin', '*');
res.header('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE,OPTIONS');
res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization, Content-Length, X-Requested-With, Accept');
});
And you are sending Access-Control-Allow-Headers from the client side, and server sees that as a header that is not whitelisted.
So, in headers just use these:
headers: {
'content-type': 'application/json',
'Authorization': 'Client-ID [my-client-id]'
}
It should work fine.
Btw, I think it is working with postman because:
Postman cannot set certain headers if you don't install that tiny postman capture extension.
Browser security stops the cross origin requests. If you disable the chrome security it will do any CORS request just fine.
Also, according to this:
I believe this might likely be that Chrome does not support
localhost to go through the Access-Control-Allow-Origin -- see
Chrome issue
To have Chrome send Access-Control-Allow-Origin in the header, just
alias your localhost in your /etc/hosts file to some other domain,
like:
127.0.0.1 localhost yourdomain.com
Then if you'd access your script using yourdomain.com instead of
localhost, the call should succeed.
Note: I don't think the content type should be application/json it should be like image/jpeg or something. Or maybe don't include that header if it doesn't work.
I have some few observations in my own app that helped me solve this issue. I have a node app as a backend api service and a VueJS built front end. I set my node app with cors with a list of endpoints that are allowed to access my node app. Working on my local machine doesn't give me any errors until I upload it to my server.
here are my environments
Local Environment
Nodejs: 12.16.1
OS: Windows 10
DB: MySQL
NodeJS Server Framework: ExpressJS
Upload Module: Multer
Production Environment
Nodejs: 12.19.0
OS: Ubuntu 20.0.4
DB: MySQL
NodeJS Server Framework: ExpressJS
Upload Module: Multer
nginx
Here are my observations based on my production built app.
When I upload a form with image [500kb and above] [post or put], the cors error shows up but less than that, it all went fine.
If I use form data to send data to the server, I see 2 requests in my network tab, the OPTIONS and the actual request.
The actual request failed but I saw that my content-length is very high which leads me to the conclusion that my request is rejected due to the large amount of data that the client sent which my server may have limited. I know that may be misleading but the solution I did works so I don't know why cors issue is popping up even though the data limit is the issue.
MY SOLUTION:
In my nginx config file, I increased my client_max_body_size to 100M. I believe that nginx has a default of 1MB
Open /etc/nginx/sites-available/your-server-file where your-server-file can be like www.example.com or default.
Add the following line inside the server block. You can set it to any amount you want other than 100M.
server {
client_max_body_size 100M;
...
}
type in sudo systemctl restart nginx to restart nginx.
type in sudo nginx -t to check if change is successful.
Reload your app if you are using pm2 and you are done.
According to this article I used this command in linux and SOME OF(!) cross-origins fixed.
google-chrome --disable-web-security --user-data-dir="/tmp/YOUR_TEMPORARY_PATH"
This will not work if you pass headers from frontend. CORS policy is enabled by browsers. Browser blocks the response when they don't found the headers in response.
Possible Solutions:
You can pass the headers in response (If you have the access of backend or ask the API provider for this)
You can setup a middleware to resolve this.
You can get information from here

how to call IBM Watson services from javascript

I am implementing a virtual agent using IBM Watson services. My application is developed using Jquery, Angular JS & Java.Currently i am calling the watson services from middle layer that is java. But i want to avoid that and call directly from javascript.When i call from javascript using XML Http request, i am getting CORS error.How to solve this?
Below is my code:
var username = "uid";
var password = "pwd";
var xhr = new XMLHttpRequest();
xhr.open('GET', 'url');
//xhr.withCredentials = true;
xhr.setRequestHeader("Access-Control-Allow-Headers", "Access-Control-Allow-Origin,Content-Type, application/json, Authorization");
xhr.setRequestHeader("Access-Control-Allow-Origin", "*");
xhr.setRequestHeader('Access-Control-Allow-Credentials', '*');
xhr.setRequestHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, PATCH, DELETE');
xhr.setRequestHeader('Content-Type', undefined);
xhr.setRequestHeader('Authorization', 'Basic ' + btoa(username + " " + password));
xhr.send('"query":"hi"');
The IBM Watson services don’t yet support getting cross-origin requests from browser-based apps.
See the answer at Can't access IBM Watson API locally due to CORS on a Rails/AJAX App:
We don't support CORS, we are working on it but in your case Visual Recognition is not supported yet.
That implies some of the services support CORS but I guess the one you’ve tried isn’t one of them.
So other than what you say you’re doing now (accessing the services from your server-side Java layer instead), your only option to get at the services from JavaScript code running in a web app is, either set up your own server-side proxy with https://github.com/Rob--W/cors-anywhere or such, or send your requests through an open CORS proxy like https://cors-anywhere.herokuapp.com/ (though it’s unlikely you’ll want to do that in the case where your requests include any kind of authentication token that you don’t want to expose to the operator of a third-party proxy service).
The way such proxies works is, instead of using https://gateway.watsonplatform.net/some/api as the request URL that specify in your client-side JavaScript code, you instead specify the proxy URL, like https://cors-anywhere.herokuapp.com/https://gateway.watsonplatform.net/some/api, and the proxy sends the actual request to the service, gets back the response, and adds the needed Access-Control-Allow-Origin response header and other headers to it and passes it on.
So that response with the CORS headers included is what the browser sees.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS has more details about how CORS works, but the main thing to know is that the browser is the CORS enforcement point. So in the case with the Watson services, the browser will actually get the response from the Watson API—you will be able to use devtools in the browser to see the response—but the browser will expose the response to your client-side JavaScript code only if the response includes the Access-Control-Allow-Origin response header to indicate the server that sent the response has opted in to receiving cross-origin requests from client-side JavaScript running in web apps.
So that’s why, regardless, all the xhr.setRequestHeader("Access-Control-Allow- lines in your XHR code snippet above need to just be removed—because Access-Control-Allow-* headers are response headers, not request headers; sending them in a request to a server has no effect on CORS, because as noted above, the browser’s the CORS enforcement point, not the server.
So it’s not the case that the server receives some request from a browser and says, OK I see this request has the right headers, so I’ll allow it. Instead the server allows all requests from browsers, just as it allows all requests from non-browser tools like your Java code or curl or Postman or whatever (as long as they are authenticated of course) and sends a response.
The difference is, when a non-browser-based app receives a response, it doesn’t refuse to let you access the response if it lacks the Access-Control-Allow-Origin header. But the browser does refuse to let your client-side JavaScript web-app code access the response if it lacks that.
You might also want to look at some of the Watson SDK's available on GitHub.
Some Watson services support CORS, others do not. However, when accessing over CORS, you must use an Auth Token rather than a username/password combination*.
This is a partial list of which services support CORS: https://github.com/watson-developer-cloud/node-sdk/tree/master/examples/webpack#important-notes
Here are a couple of examples using the Node.js SDK:
Webpack: https://github.com/watson-developer-cloud/node-sdk/tree/master/examples/webpack
Browserify: https://github.com/watson-developer-cloud/node-sdk/tree/master/examples/browserify
And, a whole host of examples with the Speech JavaScript SDK:
https://watson-speech.mybluemix.net/
* There are a couple of services that use API keys rather than username/password combinations. In that case, you can use the API key directly from client-side code if the service supports CORS.
take a look at this tutorial on IBM developerWorks on using Watson's Question and Answer service -
http://www.ibm.com/developerworks/cloud/library/cl-watson-qaapi-app/index.html#N10229

Execute route if only the local Angular App calls it true the controller

I have a few API routes that only the app itself (controller.js) should have access to. Is there a way to use an IP address (possibly insecure because of spoofing) to create a restriction of who uses this part of the api?
Server size (server.js)
app.get("/api/specs",function(req,res){
// Only the app should have access to it, not external entities
res.json({used:getUsed()});
});
Client side (controller.js)
$http.get('/api/specs').success(function(specs,code){
console.log(specs);
});
By default your browser doesn't allow you to make Cross-site HTTP requests because are subject of the same origin policy.
Note:
In particular, this meant that a web application using XMLHttpRequest
could only make HTTP requests to the domain it was loaded from, and
not to other domains.
Which it means in your case that only the js in the same domain of your api can have access to them.
What if I want to extend the use of the API to other domains?
Well in this case you have to setup in your backend api the Access-Control-Allow-Origin header.
Some eg:
// Cross-site HTTP requests from http://siteA.com
Access-Control-Allow-Origin: http://siteA.com
// Cross-site HTTP requests from all
Access-Control-Allow-Origin: *
If you want to debug this behaviour you can just open firebug and check in networks the headers of your requests.
References:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy
I suppose that using a GET parameter is the easiest way.
Node.js(express)
app.get("/api/specs",function(req,res){
var queryParam = url.parse(req.url,true).query
if(queryParam.whois == "yourUniqueName"){
res.send("okay");
}else{
res.status(404);
res.send("NG");
}
});
angular
$http.get("/api/specs", {
params: { whois: "yourUniqueName" }
});
define names something unique for each APIs.
and set the server returns response only when a client sends correct name.
server returns 404 except you pass yourUniqueName as query parameter whois.
/api/specs?whois=yourUniqueName
okay
.
/api/specs?whois=otherName
/api/specs?otherParam=something
/api/specs
NG

Categories

Resources