what is this request response format? - javascript

i'm trying to scrape a website but the thing is that i get this as a response (most likely javascript ? ). I am using request in order to send requests to the website.
Is it possible to convert it to a parsable format ? I thought of using jQuery to send request an accept a certain response format.But as of right now i'm mainly trying to identify the type of response i'm getting. Thank you.
The response i get :

Likely gzip
https://en.wikipedia.org/wiki/List_of_file_signatures
Click F12 and look at the response in a browser.
HTML is often gzipped and if you are not using a client that unzips it, then you will get the raw zip
How to understand a website is using gzip compression?

Related

How can you access the HTTP response from a server using client-side JavaScript?

I'm trying to do client-side processing of some data sent in a server's HTTP response.
Here's what I'm working with: I have a web application that sends commands to a backend simulation engine, which then sends back a bunch of data/results in the response body. I want to be able to access this response using JavaScript (note..not making a new response, but simply accessing the data already sent from the server).
Right now, I am able to do this via a kludgy hack of sorts:
var responseText = "{{response}}";
This is using Django's template system, where I have already pre-formatted the template context variable "response" to contain a pre-formatted string representation of a csv file (i.e., proper unicode separators, etc).
This results in a huge string being transmitted to the page. Right now, this supports my immediate goal of making this data available for download as a csv, but it doesn't really support more sophisticated tasks. Also, I'm not sure if it will scale well when my string is, say, 2 MB as opposed to less than 1 KB.
I'd like to have this response data stored more elegantly, perhaps as part of the DOM or maybe in a cache (?) [not familiar with this].
The ideal way to do this is to not load the csv on document load, either as a javascript variable or as part of the DOM. Why would you want to load a 2MB data every time to the user when his intention may not be to download the csv everytime?
I suggest creating a controller/action for downloading the csv and get it on click of the download button.

text/html output when requesting JSONP

I have been playing around with the jQuery library the last week or two.
Very handy! I am now playing with the AJAX requests to retrieve things such as the weather, current downloads and more, which have been going well so far!
I have now tried to connect up to my ISP to get my current data usage (peak, off peak etc).
When I use Chrome, I can manually type the variables into the URL and have the required JSON code show in the browser. The issue is, that it seems to return text/html instead of application/json.
When you go into developer tools, it shows text/html. This make it difficult for me to retrieve the data from my home server using AJAX and JSONP. See here for a failed query (but you can still see the text/html output, which is in a JSON format! Failed JSON Query on ISP
My question is, how could I get this data from the server URL, then make it into JSON that jQuery can read?
When I try the .load , $.get functions I run into Cross Origin Issues...
EDIT:Here is the PDF documentation for the API (Download at the bottom of the page)
Notice that I need to append certain values (user / pass / token). My ultimate aim is to have my JS read these values and store them.
The issue is, that it seems to return text/html instead of application/json.
That's a serverside issue. Go and file a bug report.
This make it difficult for me to retrieve the data
Not by itself. You should be able to override the settings how responses are parsed, e.g. in jQuery by using the datatype parameter.
using AJAX and JSONP
Notice that you cannot use JSONP, as it is not supported by that API (judging from the docs and a simple ?callback=test try). If you want support for that, file a bug report against the service provider.
When I try the .load, $.get functions I run into Cross Origin Issues...
Yes. They don't send CORS headers either. I suspect that this API is only used internally, and by devices that are not subject to a same-origin policy.
how could I get this data from the server URL, then make it into JSON that jQuery can read?
Use a proxy on your own server (that runs in the same domain as your app). It can also fix that content-type header.
For more details see also Ways to circumvent the same-origin policy, though most of the methods require cooperation of the service provider (to implement serverside features).
If i understand you correctly You ask for a certain value and it gives you a string. For most API's in the world they send a string that you have to parse into JSON or some language code. I would suggest looking at Parsing JSON Strings link. It explains how to take well formated strings and parse them into JSON readable objects.
var obj = jQuery.parseJSON( '{ "name": "John" }' );
alert( obj.name === "John" );
if you go on further and start using php take a look that Parsing JSON Strings with PHP
EDIT:
use .done() method to grab text from other pages after AJAX call.
$.ajax(...).done(function(html){
//do what you want with the html from the other page
var object = $.parseJSON(html)
}

P2P Ajax image transfer

I am using Ajax to retrieve images from a remote server. First I try this directly using the URL of the remote server - the returned image is a string(since that's how Ajax communicates). I use the Javascript fromCharCode and CharCodeAt to convert the data back to binary and then the window.btoa() to display it. This works. Then I want to transfer this image through an overlay network (P2P). I intercept the Ajax request, transfer it to the server through the P2P network and then retrieve the response in []byte array. But now I need to know to what type of string I should convert the byte array before I feed it back to the calling Ajax client. If I use Base64 or simply convert the byte array to string it does not display the image correctly.
Anyone has tried working with something like this before?
I will appreciate any feedback very much. Thanks
Javascript doesn't have different kinds of strings.
The desired character set will be the same one the web page is encoded in, ideally UTF-8.
Have you compared the response sent by the P2P server to the response sent by the original server? Is there some kind of wrapper that's missing, or perhaps an important MIMEtype difference?

Is it possible to send non POST data with javascript?

I'm trying to submit a postscript print job directly to printer on port 9100. I tried submitting a form directly to the IP and port, but it includes a lot of header information which obviously messes it up.
Is there a way to do this with jQuery or AJAX (or some other term I don't know about)?
You can't do it with Javascript, it'll only do HTTP requests (e.g. POST/GET), which means you get the full HTTP headers included.
Once WebSockets get more widespread, you could use those and send arbitrary data without the HTTP overhead/payload, but at present, that's only in 'bleeding edge' browsers.
This means you're stuck using a Flash or Java applet at present.
You can create a proxy php script which will accept your POST data from the form, format this data and send it to the printer
If you'd like to submit data to this script in background - please see my answer to the following post:
JavaScript: How do I create JSONP?

Read/write json from js to file on server (not server app)

I'm working with a .js client and have and object that I need to write out to a file on the server. A couple of questions - file i/o with JavaScript is new to me... I was planning on using jquery and json. I'm using java serverside. I don't have a problem reading what I get back from my servlet, but the file i/o is killing me! A couple of questions:
I can open a file I generated myself via the .js with an $.ajax call, but it's not handling my json syntax (I tried both an $.getJson and $.ajax - handwritten json, so I might (probably) are doing something wrong with it). I used firebug's console and it looks ok...
How can I write my object to a file on the server?
Then, when I want to read it, what do I need to do to process it? Right now I'm using a jsonFilter function (uses JSON.parse if that's available, otherwise eval) to process data that I'm getting from the servlet.
The object I'm writing isn't simple, but it's not super complex either. There's an array that contains an array, but that shouldn't make a difference if the software is both reading/writing it.
Thanks for any help! I'm at a loss - tried alot of different things.
You can open a file located on the server via ajax by querying the file and loading it into a JSON object. You might want to LINT your JSON
You can not write to an object on the server via the client. This is a severe security breach.
Common practice is to change the JSON data and then send it via ajax to server-side code. The server will then do the file IO.
Yes using JSON.parse otherwise eval is indeed correct. I would recommend json2.js
The data should be fine as long as it passes JSONLint.
Your main issue is that it's impossible to write to the server from the client. Get the client to load the data through ajax change it and then query the server to update the file.
js don't have i/o property;
you should use ajax or http request to send message to server,and tell server to do de i/o action...

Categories

Resources