I'm in pursue of solution how to fetch data from API "PVGIS" (https://re.jrc.ec.europa.eu/api/) using vanilla javascript and process data for futher calcualtions
when accessing api from browser js script i get an "CORS policy error"
when accessing API from standalone node.js script i can fetch a data
here is fetch function from node that outputs data correctly
async function fetchData() {
const data = await fetch('https://re.jrc.ec.europa.eu/api/PVcalc?lat=45&lon=8&peakpower=1&loss=14&outputformat=json')
.then(response => response.json())
.then(json => {
console.log(json.outputs.monthly)
})
}
as i understand, i should be building a backend server side with api request to pvgis and then connecting it with my front js side to process the data ... is this a correct path?
Can you give me some details how to get going ?
here is the GUI of this api
https://re.jrc.ec.europa.eu/pvg_tools/en/tools.html
and API documentation
https://joint-research-centre.ec.europa.eu/pvgis-online-tool/getting-started-pvgis/api-non-interactive-service_en
As the website says, they are not making this API's available for use in browser
Warning: access to PVGIS APIs via AJAX is not allowed. Please, do not ask for changes in our CORS policy since these requests will be rejected by the system administrators.
They basically have CORS policy which blocks any other origin to make request. Cors is basically browser security mechanism. But as you mentioned you can access it through script since its not in browser.
Also as they mentioned, they have rate limiters so you can not call api every second.
Way to go here would be to have backend script on nodeJS which calls api, gets data and saves it in your database and then you would have another api endpoint, that queries this data from database. Also add some cron job, which would schedule request to get newer data from api every hour or so, so your db data does not go stale
Related
i have one empty json file. I want to add json data to my json file when button is clicked. but it gives error 405. how can I do?
error:
POST net::ERR_ABORTED 405 (Method Not Allowed)
code:
var data={
method:"POST",
body:{
userId:1,
title:"sample title",
body:"sample body"
},
headers:new Headers({
'content-type':'application/json',
'dataType': 'json'
})
}
fetch("anaveri.json",data).
then(res=>{
console.log(res);
}).
catch(error=>{
console.log(error);
});
The browser isn't able to directly write data to the server's file system.
It would be a horrible security problem if it could. Google's homepage would get overwritten with some different bit of trolling every few seconds!
The browser can send an HTTP request to a URL. The server then needs to use server side programming to process that request.
You need to pick a programming language that your server supports (or change servers to one that supports your server-side language of choice) and write a webservice that takes the data from the request and stores it.
You could have it write directly to a JSON file, but that risks "fun" problems with concurrent writes so it is more typical to store the data in a database and have another webservice generate the JSON on demand.
You should consider adding some sort of tests (e.g. password authentication and data validation) to control who can insert new data and what sort of data they can insert to avoid the aforementioned vandalism problem.
Client-side scripting isn't allowed to change files on the server or the local file system for security reasons. Depending on what you're trying to achive, you need to do one of these things:
Send your data to your server via POST and your server does the saving
Create the file contents blob and download it
Use the browser's FileSystem API instead of the client one's
405 (Method Not Allowed) means that the resource you're querying (in this case, your json file) does not implement the method you're trying to use on it. In order to add information to your json file, you need to have some sort of a backend logic that implements a RESTful API, so that you can issue requests using JavaScript - you can't just do it with JavaScript alone.
I'm working on small project for fun that I started as a Node script, using axios to make a GET call to an API. I got it working and decided I wanted to move it to a normal script to be used in an html file, so I moved it to an html file between <script></script> tags and made some changes to make it a bit cleaner with ES6 which as far as I know shouldn't have broken anything, but now I'm getting the error (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). From what I've gathered, this is something that the server has to set so there's not much I can do. I'm just not sure why it worked with Node?
Here's the gist of my code:
NodeJS:
axios('url_here')
.then(data => {/* Some processing here */})
Standalone JS in HTML file:
const data = await (await fetch('url_here')).json();
/* Same processing here as what I did in Node */
CORS checks are a browser security feature. That's the reason it works in Node.js but not in a browser.
"The CORS mechanism supports secure cross-origin requests and data transfers between browsers and servers. Modern browsers use CORS in APIs such as XMLHttpRequest or Fetch to mitigate the risks of cross-origin HTTP requests." MDN
If you need a GET call to server then you need to provide this in axios call like
axios.get('url')
OR
axios({method: 'get', url: 'your_url'})
I have react application where I want to add a request-id header to my requests, such that the frontend can tell the backend to undo a specific request. So requests (using superagent) are something like this:
let result = request(method, endpoint);
result = result.set("Accept", "application/json").set("Request-Id", getRequestId());
And when I add the ".set("Request-Id", getRequestId())" I get the error below.
I can see that I can send requests with postman with the request-ID header and I can see that the loadbalancer does not receive any requests other than options calls. CORS is enabled and exposing all headers for all origins.
Does anybody have ideas for what might be wrong? I'm quite new to frontend development.
The answer was I was that my corporate computer that has hardcoded in restrictions in the browser for not allowing custom headers. So I went in and found a standard header that in conjunction with the url could be used for the id so https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Date in my case.
Another evidence for this is that I could make post anything on my corporate computer on Facebook. Since the Facebook application uses custom headers.
I need to fetch some data from ApiGateway endpoint and then based on response store this data in database.
I created simple Lambda function that just fetch data from ApiGateway endpoint and print it in console. My first Lambda function did not have any VPC configuration and fetch operation worked like a charm.
const fetch = require('node-fetch');
exports.handler = async () => {
const data = await fetch("https://<<ag-api-key>>.execute-api.us-east-1.amazonaws.com/v1/data");
const response = await data.json();
console.log(data, response);
}
As I need to store data received from endpoint into database which run under VPC I decided to put Lambda in same VPC(this vpc has configured Internet Gateways and other stuff to have access to internet).
As a result fetch operation start to fail with 403 response code and {"message":"Forbidden"} response body.
Api Gateway resource does not have any custom domain configuration and maintained by other team so I do not have direct access to its configuration
May be anyone can suggest how I can fix this
Security Groups, check if port 443 is open
Check your CORS Setting on API Gateway.
Try Hitting the API Gateway with Postman/Fiddler, or any other testing tool to make sure your API Gateway is online and you can get the response you want.
If you are using a Private API-Gateway (Sounds like you are using public looking at the URL) you will need some header data and different URL. I can guide you through it if needed. I would avoid private API gateway if I were you.
Let me know if any of that helps. I have ran into that issue many times in different situations.
Since last couple of hours when I try to connect api.ai using javascript I get the following error .
"Failed to load https://api.api.ai/v1/query?v=20150910: No 'Access-Control-Allow-Origin' header is present on the requested resource."
I try to send request to Api.AI using javascript by following way:-
const client = new ApiAi.ApiAiClient({
accessToken: "*******************"
});
const promise = client.textRequest(query);
promise
.then(handleResponse)
.catch(handleError);
function handleResponse(serverResponse) {}
You can use browser Extension for this problem.
In chrome
If you use PostMan or other service except browser you don't see this error or you can use nginx proxy to get result.
I too have started facing this issue from few hours, it looks like people at DialogFlow have changed the CORS restrictions.
Right now, only solution is to use a middle proxy server / cloud function which will call the api.ai url on behalf of browser and send the result back to browser.
Found the issue and answer here.
I guess you are calling the API from the browser. It is not a great option here as anyone can see your access code.
You can try the following solutions:
Call the API from the back-end - not front-end.
Use fetch API and disable CORS.