I'm creating an Express route that calls the GitHub API with a ?callback=foo pattern added to the endpoint so that it will return the Link headers which I'll need to parse out the Link: header because it contains the link that I'll have to call to get the next page of the response.
The problem is that the response has the expected pattern, but when I try to create a function to tease out the meta and data portions of the function, they turn up undefined.
My code:
app.get('/populate', function(req, res, next) {
console.log('/populate route hit');
var token = "<something>";
var options = {
url: 'https://api.github.com/users?callback=resp',
headers: {
'User-Agent': 'Our-App',
'Authorization': 'token '+ token
}
};
api(options) // 'api' is request-promise module, makes http requests
.then(function(response) {
console.log(response); // Note 1
function resp(res) {
var meta = res.meta;
var data = res.data;
console.log('meta ', meta); // Note 2
console.log('data ', data);
}
resp(response);
Note 1: The response looks like:
/**/resp({"meta":{"X-RateLimit-Limit":"5000","X-RateLimit-Remaining":"4993",
"X-RateLimit-Reset":"1435297775","X-OAuth-Scopes":"public_repo, user:email",
"X-Accepted-OAuth-Scopes":"repo","Cache-Control":"private, max-age=60, s-maxage=60",
"Vary":"Accept, Authorization, Cookie, X-GitHub-OTP",
"ETag":"\"0cbbd180648a54f839a237b0302025db\"",
"X-GitHub-Media-Type":"github.v3; format=json",
"Link":[["https://api.github.com/users?callback=resp&since=46",
{"rel":"next"}],["https://api.github.com/users{?since}",
{"rel":"first"}]],"status":200},"data":[{"login":"mojombo","id":1,
...etc etc...
}]})
The response looks like it's been JSON.stringified but when I JSON.parse(response) it returns an error. I don't know how to access the deeply-embedded Link: headers and even the data, which looks like JSON, too.
Note 2 The res.meta and res.data log as undefined.
The response isn't JSON, it's JSONP. JSONP is a cross-domain mechanism for retrieving data. You don't use XHR (e.g., app.get) to request JSONP, you use a script tag. (Because XHR is limited by the Same Origin Policy; script tags aren't.)
If your call retrieving that data via XHR works, it means cross-domain XHR calls are allowed in your situation (the server supports Cross-Origin Resource Sharing with your page's origin, and the browser supports CORS). You can get JSON instead of JSONP by removing the ?callback=resp in the URL.
Related
I know that I can use a service worker to intercept outgoing fetch operations, and even generate custom responses to them, e.g.
self.addEventListener('fetch', (event) => {
if (/\.jpg$/.test(event.request.url)) {
event.respondWith(
fetch('/images/anotherimage.jpg'));
}
});
However, what if I want to intercept the response to a given fetch request before that response is passed back to the page? Is this doable?
To be clear, I do not in any way want to modify the request itself - I just want to access the response to it.
I'm assuming that you mean you'd like to add logic to a service worker so that it requests a resource, and then modifies the response from the network, resulting in a response that's passed back to the page that is a mix of what you'd get from the network and what the service worker added.
If so, the answer is yes, you can do that for same-origin responses, and for cross-origin responses when CORS is used. (You can't modify opaque responses, which is what you get when making a cross-origin request without using CORS.)
Here's an example of a fetch handler that responds to requests for an hypothetical /api endpoint that returns JSON my making the request to /api, and then adding in an additional field to the API response before returning the response to the page.
async function modifyAPIResponse(request) {
const apiResponse = await fetch(request);
const json = await apiResponse.json();
json.extraField = 'set by fetch handler';
return new Response(JSON.stringify(json), {
// Ensure that the Content-Type: and other headers are set.
headers: apiResponse.headers,
});
}
self.addEventListener('fetch', (event) => {
const url = new URL(event.request.url);
if (url.pathname === '/api') {
event.respondWith(modifyAPIResponse(event.request));
}
});
I'm simply trying to send some urlencoded parameters via a GET request using fetch. I'm just trying to print the parameters using Express at the moment, like so:
app.get('/api', function (req, res) {
console.log(req.body);
res.sendStatus(200);
return;
});
This works just fine in Postman using a GET request and x-www-form-urlencoded key-value pairs. The webserver will print all the key-value pairs just fine.
But when I try and use fetch to do the exact same thing I get nothing but problems. I've tried two different methods:
fetch(`http://localhost:3000/api?user=test&password=123`, {
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
}
});
The request does go through using this method, but the webserver only prints {} - an empty object.
var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/x-www-form-urlencoded");
var urlencoded = new URLSearchParams();
urlencoded.append("user", "test");
urlencoded.append("password", "123");
var requestOptions = {
method: 'GET',
headers: myHeaders,
body: urlencoded,
};
fetch("localhost:3000/api", requestOptions)
.then(response => response.text())
.then(result => console.log(result))
.catch(error => console.log('error', error));
The request does not go through using this method, and the browser gives me the error TypeError: Window.fetch: HEAD or GET Request cannot have a body.
This code was generated using the request that works in Postman using the generate code snippets option.
What am I doing wrong?
The parameters in this URL:
http://localhost:3000/api?user=test&password=123
are in the query string, not in the body and thus the content-type does not apply to them - they are properly encoded to be in a URL. In Express, you would access these with req.query. You should see a value for req.query.user and req.query.password in your Exprss request handler.
Note, it is not recommended that you send user credentials in a URL like this because URLs are often present in log files at your ISP, at the recipient server, in proxies, in your browser history, etc... User credentials like this should be sent in POST request over https where the credentials would go encoded in the body (where it won't be logged or saved by intermediaries).
The fetch error is accurate. GET requests do not have a body sent with them. That would be for POST or PUT requests. A GET request is a "get" request for a resource that you specify only with a URL.
You're confusing request body with a query string.
Your second request (you don't need a Content-Type for it)
fetch("http://localhost:3000/api?user=test&password=123");
would be handled by the following Express function:
app.get('/api', function (req, res) {
console.log(req.query); // Note that query, not body is used.
res.sendStatus(200);
return;
});
You can access fields from the query object as req.query.user && req.query.password.
As for having a request body in a GET request: while RFC doesn't explicitly fordbid it, it requires server to not change response based on the contents of the body, i.e. the body in GET has no meaning in the standard, so JS HTTP APIs (both fetch & XmlHttpRequest) deny it.
firstly if you are trying to get some data from your API or others API you should do GET request in order to get your desired data from server for example, if you want to get a specific things like a user or something else you can pass your data in GET request URL using query string or route params.
secondly, if you want to authenticate and send your credentials to the server its not recommended to use GET request as i said earlier GET request simply is for fetching some data from server, so if you want to send your credential or anything else you are better off using POST request to send data to the server and you can't do POST request in the browser, so you have to use something like postman or insomnia in order to send your POST request to the server. i hope it could help you to solve your issue.
I am working on an internal web application at work. In IE10 the requests work fine, but in Chrome all the AJAX requests (which there are many) are sent using OPTIONS instead of whatever defined method I give it. Technically my requests are "cross domain." The site is served on localhost:6120 and the service I'm making AJAX requests to is on 57124. This closed jquery bug defines the issue, but not a real fix.
What can I do to use the proper http method in ajax requests?
Edit:
This is in the document load of every page:
jQuery.support.cors = true;
And every AJAX is built similarly:
var url = 'http://localhost:57124/My/Rest/Call';
$.ajax({
url: url,
dataType: "json",
data: json,
async: true,
cache: false,
timeout: 30000,
headers: { "x-li-format": "json", "X-UserName": userName },
success: function (data) {
// my success stuff
},
error: function (request, status, error) {
// my error stuff
},
type: "POST"
});
Chrome is preflighting the request to look for CORS headers. If the request is acceptable, it will then send the real request. If you're doing this cross-domain, you will simply have to deal with it or else find a way to make the request non-cross-domain. This is why the jQuery bug was closed as won't-fix. This is by design.
Unlike simple requests (discussed above), "preflighted" requests first
send an HTTP request by the OPTIONS method to the resource on the
other domain, in order to determine whether the actual request is safe
to send. Cross-site requests are preflighted like this since they may
have implications to user data. In particular, a request is
preflighted if:
It uses methods other than GET, HEAD or POST. Also, if POST is used to send request data with a Content-Type other than
application/x-www-form-urlencoded, multipart/form-data, or text/plain,
e.g. if the POST request sends an XML payload to the server using
application/xml or text/xml, then the request is preflighted.
It sets custom headers in the request (e.g. the request uses a header such as X-PINGOTHER)
Based on the fact that the request isn't sent on the default port 80/443 this Ajax call is automatically considered a cross-origin resource (CORS) request, which in other words means that the request automatically issues an OPTIONS request which checks for CORS headers on the server's/servlet's side.
This happens even if you set
crossOrigin: false;
or even if you ommit it.
The reason is simply that localhost != localhost:57124. Try sending it only to localhost without the port - it will fail, because the requested target won't be reachable, however notice that if the domain names are equal the request is sent without the OPTIONS request before POST.
I agree with Kevin B, the bug report says it all. It sounds like you are trying to make cross-domain ajax calls. If you're not familiar with the same origin policy you can start here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Same_origin_policy_for_JavaScript.
If this is not intended to be a cross-domain ajax call, try making your target url relative and see if the problem goes away. If you're really desperate look into the JSONP, but beware, mayhem lurks. There really isn't much more we can do to help you.
If it is possible pass the params through regular GET/POST with a different name and let your server side code handles it.
I had a similar issue with my own proxy to bypass CORS and I got the same error of POST->OPTION in Chrome. It was the Authorization header in my case ("x-li-format" and "X-UserName" here in your case.) I ended up passing it in a dummy format (e.g. AuthorizatinJack in GET) and I changed the code for my proxy to turn that into a header when making the call to the destination. Here it is in PHP:
if (isset($_GET['AuthorizationJack'])) {
$request_headers[] = "Authorization: Basic ".$_GET['AuthorizationJack'];
}
In my case I'm calling an API hosted by AWS (API Gateway). The error happened when I tried to call the API from a domain other than the API own domain. Since I'm the API owner I enabled CORS for the test environment, as described in the Amazon Documentation.
In production this error will not happen, since the request and the api will be in the same domain.
I hope it helps!
As answered by #Dark Falcon, I simply dealt with it.
In my case, I am using node.js server, and creating a session if it does not exist. Since the OPTIONS method does not have the session details in it, it ended up creating a new session for every POST method request.
So in my app routine to create-session-if-not-exist, I just added a check to see if method is OPTIONS, and if so, just skip session creating part:
app.use(function(req, res, next) {
if (req.method !== "OPTIONS") {
if (req.session && req.session.id) {
// Session exists
next();
}else{
// Create session
next();
}
} else {
// If request method is OPTIONS, just skip this part and move to the next method.
next();
}
}
"preflighted" requests first send an HTTP request by the OPTIONS method to the resource on the other domain, in order to determine whether the actual request is safe to send. Cross-site requests
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Consider using axios
axios.get( url,
{ headers: {"Content-Type": "application/json"} } ).then( res => {
if(res.data.error) {
} else {
doAnything( res.data )
}
}).catch(function (error) {
doAnythingError(error)
});
I had this issue using fetch and axios worked perfectly.
I've encountered a very similar issue. I spent almost half a day to understand why everything works correctly in Firefox and fails in Chrome. In my case it was because of duplicated (or maybe mistyped) fields in my request header.
Use fetch instead of XHR,then the request will not be prelighted even it's cross-domained.
$.ajax({
url: '###',
contentType: 'text/plain; charset=utf-8',
async: false,
xhrFields: {
withCredentials: true,
crossDomain: true,
Authorization: "Bearer ...."
},
method: 'POST',
data: JSON.stringify( request ),
success: function (data) {
console.log(data);
}
});
the contentType: 'text/plain; charset=utf-8', or just contentType: 'text/plain', works for me!
regards!!
I'm trying to use window.fetch() to get json from the server, but can't get the data from the response.
I have this code:
let url =
'https://api.flightstats.com/flex/schedules/rest/v1/json/from/SYD/to/MEL/departing/2016/3/28?appId=f4b1b6c9&appKey=59bd8f0274f2ae88aebd2c1db7794f7f';
let request = new Request (url, {
method: 'GET',
mode: 'no-cors'
});
fetch(request)
.then(function(response){
console.log(response)
});
It seems that this request is successfull, I see status 200
and response body with json in network tab - status and response. But in console.log I dont see json object - console log image
I cant understand why I dont see json in console.log
The host site you are requesting from does not appear to support CORS. As such, you can't use fetch() to make a cross origin request and get the data back. If, you change your fetch() request to mode: 'cors', the debug console will show that the host site does not offer CORS headers to allow the browser to show you the result of the request.
When you are using mode: 'no-cors', the browser is hiding the result from you (because you don't have permission to see it) and you can see the response is tagged as opaque.
In a little poking around on the api.flightstats.com site, I did see that it supports JSONP which will allow you to work around the lack of CORS support issue and successfully complete a cross origin request.
For simplicity of showing that it can work, I used jQuery to just prove that a JSONP request can be made. Here's that code in a working snippet. Note I changed the URL from /json/ to /jsonp/ and specific dataType: "jsonp" in the jQuery request. This causes jQuery to add the callback=xxxxx query parameter and to fetch the response via that corresponding script (the JSONP method).
var url =
'https://api.flightstats.com/flex/schedules/rest/v1/jsonp/from/SYD/to/MEL/departing/2016/3/28?appId=f4b1b6c9&appKey=59bd8f0274f2ae88aebd2c1db7794f7f';
$.ajax(url, {dataType: "jsonp"}).then(function(response) {
log(response);
}, function(err) {
log("$.ajax() error:")
log(err);
})
<script src="http://files.the-friend-family.com/log.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
If you take a look at the documentation of the Fetch API; you'll notice that the API offers various methods to extract the data:
arrayBuffer()
blob()
json()
text()
formData()
Assuming the response is valid JSON (which I've noticed it doesn't seem to appear), you can use the response.json() function to retrieve the response data. This also uses a Promise mechanism, as for everything with the Fetch API.
response.json().then(function(data) {
console.log(data);
});
I'am learning AngularJs and I've tried to write a very basic script sending an http request to Ebay public API, I've signed up and got my API keys, I've read the docs several times and wrote this basic code :
$scope.getQueryUrl = function () {
// Some unrelated code ...
$scope.queryUrl["Ebay"] = "http://svcs.sandbox.ebay.com/services/search/FindingService/v1?OPERATION-NAME=findItemsByKeywords&SERVICE-NAME=FindingService&SERVICE-VERSION=1.0.0&GLOBAL-ID=EBAY-US&SECURITY-APPNAME="+dataAuth.EbayKeyApi+"&RESPONSE-DATA-FORMAT=XML&keywords="+$scope.qtext ;
};
$scope.sendRequest = function () {
$scope.getQueryUrl(); // Gets the query url after adding all the parameters
alert($scope.queryUrl.Ebay);
$http.get($scope.queryUrl["Ebay"]).then(
function(response){
alert("success" + response.data );
},
function(response){
alert("error" + response.statusCode );
});
};
How this code should work :
It should create a formated Ebay query url, send it through HTTP GET request and sending back the response .
Note : $scope.qtext & dataAuth.EbayKeyApi are already assigned with their respective values .
What's the problem:
The problem is that using this Angularjs script, the code doesn't work, the alert "Error" is shown, and the response.statusCode is undefined .
But when I copy the formatted Ebay query link in Firefox it works perfectly and the XML response is shown .
The formatted Ebay query was generated using the script provided .
I think it's a header related problem .
$http has some default headers defined. $http sends Json payload and accepts Json as the response by default. Since you are dealing with XML you have to explicitly specify the accepted response type as XML using the header:
Accept: application/xml
Please use the following function with appropriate headers and you should get the response. Also, please look into any Cross Origin Resource Sharing (CORS) restrictions on the ebay API.
function getRequest(url) {
$http({
method: "GET",
url: url,
headers: {
'Content-Type': 'application/xml, text/xml',
'Accept': 'application/xml, text/plain, * / *'
}
})
.then(function (response) {
alert(response.data);
},
function (error) {
alert (error);
});
}
Thank you,
Soma.