I found the answer to this question. It is resolved.
Narrative: I am accessing a Python API, a set of methodCalls on top of SimpleXMLRPCServer. Server responds to browser GET request with a html page, "web_interface.html". The HTML page is a very simple script that sends a XHR POST request of xml params to the XMLRPC server. Server responds to XHR POST with headers but empty document. Server responds to cURL with correct data. Why is JavaScript not getting any readable data in response from server?
| web_interface.html |
<!DOCTYPE html>
<html>
<head>
<script>
var xrequest = '<?xml version="1.0?"><methodCall><methodName>helloWorld<methodName><params><param><firstWord><string>hello</string><firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>';
function hello() {
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (this.readyState == XMLHttpRequest.DONE) {
alert(this.responseText);
alert(this.status);
alert(this.response);
}
}
xhr.open('POST', '/', true);
xhr.setRequestHeader("Authorization", "Basic " + "aGVsbG8=" + ":" + "dGVzdA==");
xhr.send(xrequest);
}
</script>
</head>
<body>
<div>
<h2 id="msgoutput">HelloWorld API Test</h2>
<button type="button" onclick="hello(); return false;">SAY HELLO!</button>
</div>
</body>
</html>
Note: Clicking the button produces the alert dialogs. Status dialog shows "200" while Text and response dialogs are null.
| Mozilla Inspector Data & Headers |
POST Raw Data:
<?xml version="1.0?"><methodCall><methodName>helloWorld<methodName><params><param><firstWord><string>hello</string><firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>
Response Headers:
Content-Length 332
Content-Type text/html
Date Sat, 10 Dec 2016 23:51:21 GMT
Server BaseHTTP/0.3 Python/2.7.12
Request Headers:
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-US,en;q=0.5
AuthorizationBasic aGVsbG8=:dGVzdA== (not my actual creds, swapped fakes)
Connection keep-alive
Content-Length 218
Content-Type text/plain;charset=UTF-8
DNT1
Hostlocalhost:8442
Referer http://localhost:8442/
User-Agent Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0
| Test with cURL |
:~$ curl -i --data '<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>' http://username:password#localhost:8442
HTTP/1.0 200 OK
Server: BaseHTTP/0.3 Python/2.7.12
Date: Sun, 11 Dec 2016 00:00:13 GMT
Content-type: text/html
Content-length: 137
<?xml version='1.0'?>
<methodResponse>
<params>
<param>
<value><string>hello-world</string></value>
</param>
</params>
</methodResponse>
Note: No problem, cURL returns the XML response as text. I pointed cURL at a netcat socket to see exactly what it is sending to the XMLRPC server. Here's what netcat shows when cURL hits:
| cURL POST Data |
POST / HTTP/1.1
Host: localhost:8442
Authorization: Basic YWRtaW46Z2liYmVyc2g=
User-Agent: curl/7.47.0
Accept: */*
Content-Length: 220
Content-Type: application/x-www-form-urlencoded
<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>
It's not CORS. Already tested a GET request to xhr.responseText with same browser on same machine. Setup is using same host, same port, same directory for both the GET page and the XHR POST XMLRPC request.
What am I missing?
The Python SimpleXMLRPCServer had some code hacked into it to handle cookies. Problem is, the code couldn't handle the cookie objects when a browser makes connection. This code threw an error on each invocation. It was stopping the server from sending the response text to the browser. I learned this after writing a Python snippet to output debug information to a file. I removed the cookie code and then the response XML was successfully sent from the server to the browser.
I also discovered that sending the username/password from the XHR Send statement would throw an authentication error from the server. The server is expecting credentials to be in the POST header as Authentication: Basic base64 [btoa(username:password)].
The content-type header was not the culprit in this case. Now content-type is set to text/xml on both client and server.
Here is the modified JavaScript code that works:
<!DOCTYPE html>
<html>
<head>
<script>
var xrequest = '<?xml version="1.0"?><methodCall><methodName>helloWorld</methodName><params><param><firstWord><string>hello</string></firstWord></param><param><secondWord><string>world</string></secondWord></param></params></methodCall>';
function hello() {
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
if (this.readyState == XMLHttpRequest.DONE) {
}
}
xhr.open("POST", "/", true);
xhr.setRequestHeader("Authorization", "Basic " + btoa("admin" + ":" + "1234"));
xhr.setRequestHeader("Content-Type", "text/xml")
xhr.send(xrequest);
}
</script>
</head>
<body>
<div>
<h2 id="msgoutput">HelloWorld API Test</h2>
<button type="button" onclick="hello(); return false;">SAY HELLO!</button>
</div>
</body>
</html>
Related
I am trying to get an OAuth2 Token via XHR-Request in TypeScript like mentioned on this side(https://developer.paypal.com/docs/integration/direct/make-your-first-call/). My code so far:
var clientID = <clientID>;
var secret = <mySecret>;
var oReq = new XMLHttpRequest();
oReq.open("POST", "https://api.sandbox.paypal.com/v1/oauth2/token", true);
oReq.setRequestHeader('grant_type', "client_credentials");
oReq.setRequestHeader('Username', this.clientID);
oReq.setRequestHeader('Password', this.secret);
oReq.setRequestHeader('Content-type', "application/x-www-form-urlencoded");
oReq.setRequestHeader('Accept', "application/json");
oReq.setRequestHeader('Accept-Language', "en_US");
oReq.onreadystatechange = function () {
if (oReq.readyState === 4) {
if (oReq.status === 200) {
console.log(oReq.responseText);
} else {
console.log("Error: " + oReq.status);
}
}
console.log("Status: " + oReq.status);
};
console.log("Status: " + oReq.status);
oReq.send();
Sadly I keep getting 401 as response. I already tried the curl command with the same clientID and secret, which worked for me. Can someone tell me whats wrong with me code?
The request you’re getting the 401 response for isn’t your POST request but instead the CORS preflight OPTIONS request that the browser automatically does on its own.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Preflighted_requests has details about what triggers browsers to do a preflight but what it comes down to in this specific case is that you’re adding grant_type, Username, and Password request headers to the request in order to do the authentication needed—but your browser makes the preflight OPTIONS request without those headers (because the purpose of the preflight is to ask the server to indicate whether it’s OK with receiving requests that include those request headers).
And so what happens is the following (reproduced using curl just for illustration purposes):
$ curl -X OPTIONS -i -H "Origin: http://example.com" \
'https://api.sandbox.paypal.com/v1/oauth2/token'
HTTP/1.1 401 Unauthorized
Date: Wed, 09 Aug 2017 09:08:32 GMT
Server: Apache
paypal-debug-id: f8963606c5654
Access-Control-Allow-Origin: http://example.com
Access-Control-Expose-Headers: False
HTTP_X_PP_AZ_LOCATOR: sandbox.slc
Paypal-Debug-Id: f8963606c5654
Set-Cookie: X-PP-SILOVER=name%3DSANDBOX3.API.1%26silo_version%3D1880%26app%3Dapiplatformproxyserv%26TIME%3D282167897%26HTTP_X_PP_AZ_LOCATOR%3Dsandbox.slc; Expires=Wed, 09 Aug 2017 09:38:32 GMT; domain=.paypal.com; path=/; Secure; HttpOnly
Set-Cookie: X-PP-SILOVER=; Expires=Thu, 01 Jan 1970 00:00:01 GMT
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=ISO-8859-1
That is, the https://api.sandbox.paypal.com/v1/oauth2/token endpoint apparently requires authentication even for OPTIONS requests. Because it does, there’s no way you can make POST requests to that endpoint directly from your frontend JavaScript code running in a browser.
So you’ll instead need to make the request from your backend code.
We're making an XHR request with the following headers (I've simplified a bit):
POST http://localhost:9001/login
Host: localhost:9001
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: application/json, text/plain, */*
Content-Type: application/json;charset=utf-8
Content-Length: 67
Then our server responds like this (again simplified):
Status code: 200 OK
Cache-Control: no-cache, no-store
Connection: close
Content-Length: 0
Date: Mon, 27 Feb 2017 17:19:53 GMT
Server: WildFly/9
Set-Cookie: JSESSIONID=123; path=/
There's no payload in the response. Note the Content-Length: 0. But Firefox still tries to parse it as XML. And outputs the following error to the console:
XML Parsing Error: no root element found
Location: http://localhost:9001/login
Line Number 1, Column 1
Note, the server doesn't send a content-type header. And according to RFC 7231 it only has to send a content-type header when there is actual content.
So is this a bug in Firefox or is my research faulty?
Reproducing it yourself
I've written a small server and client to reproduce the problem.
server.js (start with node ./server.js):
const fs = require('fs'), http = require('http');
const server = http.createServer(function (request, response) {
if (request.method === 'POST') {
// send empty response
response.end();
return;
}
// send file content
fs.readFile('.' + request.url, function (error, content) {
response.writeHead(200, { 'Content-Type': request.url === '/index.html' ? 'text/html' : 'text/javascript' });
response.end(content);
});
}).listen(8080);
index.html
<script src="client.js"></script>
client.js
var xhr = new XMLHttpRequest();
xhr.open('POST', 'http://localhost:8080/login');
xhr.send();
When opening the URL http://localhost:8080/index.html in Firefox, there will be an error in the JavaScript console.
Firefox Version 51.0.1
There's no payload in the response. Note the Content-Length: 0
That means there is a payload, and that payload is 0 bytes in size. Consider the difference between null and "" as strings. What you have here is the equivalent of "" when you want the equivalent of null.
Set the status code to 204 rather than 200 to indicate you aren't sending an entity (and remove the content-type, since there's no content-type with no entity).
(For a long time Firefox would still log an error for this case, but thankfully this is finally fixed. Even when it did log the error it would still continue to run any scripts correctly).
Judging from the POST request it seems that you are using XHR/JS to send the request.
So the problem is likely in the code that is processing the result.
(Also, the Content-Type in the request is incorrect, there's no charset parameter on application/json)
I'm trying to send a JSON string from a single HTML (this file is not served by cherrypy) file via Javascript to a cherrpy server.
This is my minimal cherrypy example (followed the "dealing with json" part)
import cherrypy
class HelloJson(object):
#cherrypy.expose
#cherrypy.tools.json_in()
def default(self):
data = cherrypy.request.json
print(data)
return "Hello world!"
if __name__ == '__main__':
cherrypy.config.update({'server.socket_port':1234})
cherrypy.quickstart(HelloJson())
Sending a JSON string via python works gently
>>> requests.post('http://localhost:1234', json=json.dumps({'Hello': 'Json'}))
<Response [200]>
>>>
The cherrypy output prints the json string too
20:59 $ ./HelloJson.py
[24/Aug/2015:20:59:34] ENGINE Listening for SIGTERM.
[24/Aug/2015:20:59:34] ENGINE Listening for SIGUSR1.
[24/Aug/2015:20:59:34] ENGINE Listening for SIGHUP.
[24/Aug/2015:20:59:34] ENGINE Bus STARTING
CherryPy Checker:
The Application mounted at '' has an empty config.
[24/Aug/2015:20:59:34] ENGINE Started monitor thread '_TimeoutMonitor'.
[24/Aug/2015:20:59:34] ENGINE Started monitor thread 'Autoreloader'.
[24/Aug/2015:20:59:34] ENGINE Serving on http://127.0.0.1:1234
[24/Aug/2015:20:59:34] ENGINE Bus STARTED
{"Hello": "Json"}
127.0.0.1 - - [24/Aug/2015:21:00:17] "POST / HTTP/1.1" 200 12 "" "python-requests/2.7.0 CPython/3.4.3 Linux/4.1.5-1-ARCH"
So my single HTML file looks like this
<html>
<head>
<script>
function makeRequest()
{
var insertJSON = { "my_key": "my_value" };
var xmlhttp = new XMLHttpRequest(); // new HttpRequest instance
xmlhttp.open("POST", "http://localhost:1234");
xmlhttp.setRequestHeader("Content-Type", "application/json;charset=UTF-8");
xmlhttp.send(JSON.stringify(insertJSON));
}
</script>
</head>
<body>
<form name="frm1" id="yourTextBox" onsubmit="makeRequest()">
<input type="submit" value="Submit">
</form>
</body>
</html>
But this results in an error AttributeError: 'Request' object has no attribute 'json'
[24/Aug/2015:21:10:36] HTTP
Request Headers:
CONNECTION: keep-alive
ACCEPT-LANGUAGE: en-US,en;q=0.5
ACCESS-CONTROL-REQUEST-HEADERS: content-type
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
ACCESS-CONTROL-REQUEST-METHOD: POST
ACCEPT-ENCODING: gzip, deflate
PRAGMA: no-cache
CACHE-CONTROL: no-cache
HOST: localhost:1234
Remote-Addr: 127.0.0.1
ORIGIN: null
[24/Aug/2015:21:10:36] HTTP Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/cherrypy/_cprequest.py", line 670, in respond
response.body = self.handler()
File "/usr/lib/python3.4/site-packages/cherrypy/lib/encoding.py", line 217, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python3.4/site-packages/cherrypy/_cpdispatch.py", line 61, in __call__
return self.callable(*self.args, **self.kwargs)
File "./HelloJson.py", line 15, in default
data = cherrypy.request.json
File "/usr/lib/python3.4/site-packages/cherrypy/__init__.py", line 224, in __getattr__
return getattr(child, name)
AttributeError: 'Request' object has no attribute 'json'
127.0.0.1 - - [24/Aug/2015:21:10:36] "OPTIONS / HTTP/1.1" 500 1515 "" "Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
I've no idea what I'm doing wrong.
The OPTIONS request you see is a CORS preflight request, which obviously isn't a JSON request and you see the error. Because you open your file from file:// protocol (or another host), and CherryPy is serving on http://127.0.0.1:1234 you do a cross-domain request, which is subject to Same-Origin Policy.
The simplest way to solve this is to also serve the HTML file by CherryPy (Static content serving). The hard way is to provide proper CORS headers to allow cross domain requests (see this answer)
I agree with saaj's response that the browser sends a CORS preflight request which can be handled in the following manner:
import cherrypy
class HelloJson(object):
#cherrypy.expose
#cherrypy.tools.json_in()
def POST(self):
data = cherrypy.request.json
print(data)
return "Hello world!"
def OPTIONS(self):
cherrypy.response.headers["Access-Control-Allow-Methods"] = "POST, OPTIONS"
cherrypy.response.headers["Access-Control-Allow-Credentials"] = "true"
cherrypy.response.headers["Access-Control-Max-Age"] = "86400"
cherrypy.response.headers[
"Access-Control-Allow-Headers"] = "X-Mobile, Authorization, Origin, X-Requested-With, Content-Type, Accept"
cherrypy.response.headers["Content-Type"] = "application/json; charset=utf-8"
return ''
if __name__ == '__main__':
cherrypy.config.update({
'server.socket_port':1234,
'request.dispatch': cherrypy.dispatch.MethodDispatcher()
})
cherrypy.quickstart(HelloJson(), '/')
This would work because now you have enabled your API backend to listen to browser's OPTIONS call which tells the browser, what are the allowed methods and origins. cherrypy.dispatch.MethodDispatcher() enables you to treat your class as a method dispatcher and thus you can use the class to serve RESTFUL API.
I'm trying to send an HTTP request via TCP sockets.
But I'm not getting any response from www.google.com at all. No idea what I'm doing wrong.
Here is the code:
var client, net, raw_request;
net = require('net');
raw_request = "GET http://www.google.com/ HTTP/1.1\nUser-Agent: Mozilla 5.0\nhost: www.google.com\nCookie: \ncontent-length: 0\nConnection: keep-alive";
client = new net.Socket();
client.connect(80, "www.google.com", function() {
console.log("Sending request");
return client.write(raw_request);
});
client.on("data", function(data) {
console.log("Data");
return console.log(data);
});
Hope someone can help me.
Just to clarify... the requst was missing two ending newlines and all newlines had to be in the format of /r/n.
Thanks everyone! :)
If you have google chrome installed you can see the exact get request that is sent to google. This is how mine looks like:
GET https://www.google.com/ HTTP/1.1
:host: www.google.com
accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
accept-encoding: gzip,deflate,sdch
accept-language: en-US,en;q=0.8
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17
:path: /
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
:version: HTTP/1.1
cache-control: max-age=0
cookie: <lots of chars here>
:scheme: https
x-chrome-variations: CMq1yQEIjLbJAQiYtskBCKW2yQEIp7bJAQiptskBCLa2yQEI14PKAQ==
:method: GET
At a first view I can see that chrome is sending the request to https://www.google.com and you are sending to http://www.google.com
Another thing is that you are using "\n" and you need to use "\r\n", and the request has to end with "\r\n\r\n".
If you still can't get any response try using http://77.214.52.152/ instead of http://google.com.
GET /
The way you wrote it, you are trying get something like http://www.google.com/http://www.google.com/
And you need two new-lines at the end, so the web-server knows you're done. That's why you're getting nothing back -- it's still waiting for you.
Always try these things with telnet first:
$ telnet www.google.com 80
GET http://www.google.com/ HTTP
HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Content-Length: 925
Date: Sat, 19 Jan 2013 19:02:06 GMT
Server: GFE/2.0
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 400 (Bad Request)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}
</style>
<a href=//www.google.com/><img src=//www.google.com/images/errors/logo_sm.gif alt=Google></a>
<p><b>400.</b> <ins>That’s an error.</ins>
<p>Your client has issued a malformed or illegal request. <ins>That’s all we know.</ins>
Connection closed by foreign host.
I'm submitting a HTML form to REST(eXist db) web service using POST method.A normal submission is giving 400 bad request
Here is my HTML code
<html>
<script type="text/javascript">
/* function createXMLHttpRequest()
{
if( typeof XMLHttpRequest == "undefined" )
XMLHttpRequest = function()
{
try
{
return new ActiveXObject("Msxml2.XMLHTTP.6.0")
}
catch(e) {}
try
{
return new ActiveXObject("Msxml2.XMLHTTP.3.0")
}
catch(e) {}
try
{
return new ActiveXObject("Msxml2.XMLHTTP")
}
catch(e) {}
try
{
return new ActiveXObject("Microsoft.XMLHTTP")
}
catch(e) {}
throw new Error( "This browser does not support XMLHttpRequest." )
};
return new XMLHttpRequest();
}
var AJAX = createXMLHttpRequest();*/
function submitForm()
{
//AJAX.open("POST",'http://localhost:8899/exist/rest/db/xql/sample.xq');
// AJAX.send(document.form.xmlData.value);
document.form.submit();
};
</script>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
</head>
<body>
<form name='form' action="http://localhost:8899/exist/rest/db/xql/sample.xq" enctype="text/plain" method="post">
<input type="text" name="xmlData"/>
<input type="button" value="Submit" onclick="submitForm()";>
</form>
</body>
</html>
The commented code is to send POST request using AJAX.
I captured the http header request and response for form submit and AJAX submit
These are the request headers:
HTML form submit header:
(Request-Line) POST /exist/rest/db/xql/sample.xq HTTP/1.1
Host localhost:8899
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20100101 Firefox/12.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Connection keep-alive
Content-Type text/plain
Content-Length 26
AJAX request header:
(Request-Line) POST /exist/rest/db/xql/sample.xq HTTP/1.1
Host localhost:8899
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20100101 Firefox/12.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Connection keep-alive
Content-Length 16
Content-Type text/plain; charset=UTF-8
Origin null
Pragma no-cache
Cache-Control no-cache
Im not getting what's wrong in my code .
Im working on this for 2 days but i din't find any solution.
Please look into this and provide a solution.
Thanks in advance.
I'm pretty sure it's because you're sending only the value in your data.
You need to send a name = value pair.
Your code sumbits data to the server as it should be. There must be some problem with your server side code.
Quoting from checkupdown.com about error 400
400 errors in the HTTP cycle
1.Any client (e.g. your Web browser or our CheckUpDown robot) goes through the following cycle:
2.Obtain an IP address from the IP name of the site (the site URL without the leading 'http://'). This lookup (conversion of IP name to IP address) is provided by domain name servers (DNSs).
3.Open an IP socket connection to that IP address.
4.Write an HTTP data stream through that socket.
5.Receive an HTTP data stream back from the Web server in response. This data stream contains status codes whose values are determined by the HTTP protocol. Parse this data stream for status codes and other useful information.
This error occurs in the final step above when the client receives an HTTP status code it recognises as '400'.
Does your target accept POST requests, or only GET?
But you aren't sending any parameters with the Ajax POST?
The Ajax code should look something like this:
var xmlData=encodeURIComponent(document.getElementById("xmlData").value);
var parameters="xmlData="+xmlData;
AJAX.open("POST", "'http://localhost:8899/exist/rest/db/xql/sample.xq", true)
AJAX.setRequestHeader("Content-type", "application/x-www-form-urlencoded")
AJAX.send(parameters)