I'm building a web app, and need to be able to encode user-generated data as concisely as possible before transmitting it to my server. In the past I've used Flash, and it had a very neat system where for any class that you want to serialize, you could write a pair of functions that would describe exactly how to serialize the data. For example:
out.writeShort(session);
out.writeUnsignedInt(itemID);
out.writeObject(arbitraryData);
out.writeShort(score);
You would have to write an equivalent function to read bytes from the serialized data and build the class from it.
Once data is serialized it could be encoded into a Base64 string for safe network transmission to the server.
I can't figure out how to do this in Javascript? JSON is nice and easy but it's incredibly wasteful, sending all object key/value pairs, and unless I'm mistaken everything is encoded as a string? So the value false is encoded as the string "false"?
Any advice on how to implement this in Javascript would be greatly appreciated! Use of libraries is fine so long as they work both on Node and in browser.
Look at this answer. You can use BSON format (Binary JSON) and it doesn't have those features of JSON you mentioned.
Related
In Python, when sending a POST request to an API endpoint, we need to encode() the data, that encodes it using UTF-8 and bytes type is assigned to the resulting Object. Why exactly do we need to encode it before sending?
What if we send it without encoding it? What would happen if we send it as a Unicode string instead of converting it to UTF-8?
In JavaScript, we do something similar like JSON.stringify(). Does this serve the same purpose as encode()? Why exactly do we need to use stringify() in a POST request?
I know the overall working of character encodings, but I cannot find any article that explains it deeply as to why exactly we use JSON.stringify()/JSON.parse() in JS and encode()/decode() in Python. Can we not send data without using these functions? Can anyone explain by comparing these four functions?
I was testing performance of my node.js server and I detected that this small code last too much time:
socket.emit("a",{a1:something,a2:veryBigArray});
Problem is that my array is very big and socket.io must encode it to JSON.
My array is something like:
veryBigArray=[{x:0,y:0},{x:0,y:1},{x:1,y:1},{x:1,y:2}.......];
I am interested mainly in server performance. I want server can continue in work as soon as possible.
Before sending array to client, I generate array, so I don't have problem to completely change structure of sended data. Maybe array could be somehow compressed. I read about ArrayBuffer;
What is the best (fastest for sever) way of sending big array of coordinates to client (browser) using Socket.io?
If you must encode the array to JSON then I don't think an ArrayBuffer will help. However, if you know the precise data structure of the array to be sent and can predict it server-side, you can come up with your own efficient encoding/decoding schema.
In your example, your data is simply an array of x and y value pairs where each value is an integer. An extremely simple but possibly fruitful approach would be to strip the data of the predictable (unnecessary) x and y keys and encode it as a simple CSV (comma-separated value) string.
For example:
[{x:0,y:0},{x:0,y:1},{x:1,y:1},{x:1,y:2}]
would encode as the string:
0,0,0,1,1,1,1,2
The most efficient approach however would probably be to abandon socket.io for your own custom websocket interface that can somehow send binary data directly instead of encoding it as JSON. JSON will inherently be radically inefficient for sending a large dataset compared to sending encoded binary.
Edit: It looks like socket.io can send binary data so I would explore that along with some kind of efficient encoding/decoding scheme tailored to your dataset.
I have been working with JSONP on my GWT application. When my server sends a json string, I can get it in the form of a JavaScriptObject on the client side.
My problem is my json has complicated structures: using maps, nests with a lot of different keys. That is a big pain to extract data (I may have to write few hundred functions for all keys to extract data one by one and some complicated codes to fill maps).
I am considering few solutions:
Encode and send whole json strings as normal strings to client (as a value of a simple json string). Just worry my encoded strings may be few time longer than the original ones and may easily exceed the limit of 2k long
Convert back a JavaScriptObject into a pure string (similar to one I sent from the server)
After having a pure string I will parse it using some json parsers / methods to the structures I feel convenient.
My questions:
1) How to convert back a JavaScriptObject object into a pure / original json string?
2) Any idea about solutions?
Many thanks
1) Convert JavaScriptObject to JSON: JsonUtils.stringify(yourJSO)
Convert JSON to JavaScriptObject: JsonUtils.safeEval(jsonString);
2) Did you think about using AutoBeans?? Check out the GWT page
Do you really need to encode JSON.stringify when saving in cookie?
I tested it out and IE8+ and chrome works just fine without encoding (encodeURIComponent) the actual data. Problem with encoding is you are limited to 4096 bytes, and a lot of the items in JSON will be encoded causing a larger increase in byte size.
I would be saving variables like this, doesn't seem like encoding is really required.
(only 345 bytes)
{
"s":
"p":"1",
"c":"nameofsomething",
"i":"56456,54115,878451,651451,65156,878941,5165165,54545,22115,874845",
"t":"1407515818100",
"gcid":"CPOa-ZTbpL4CFWdo7AodTnQA3A",
"k":"54154154"
}
When I extract the cookie, everything is preserved. I know it is best practice to do encoding, but saving the bytes and keeping the cookie clean would be better.
Encoding would look like this (545 Bytes) about 200 increase because of the encoding
%7B%22s%22%3A%7B%22p%22%3A%221%22%2C%22c%22%3A%22nameofsomething%22%2C%22i%22%3A%2256456%2C54115%2C878451%2C651451%2C65156%2C878941%2C5165165%2C54545%2C22115%2C874845%22%2C%22t%22%3A%221407515818100%22%2C%22gcid%22%3A%22CPOa-ZTbpL4CFWdo7AodTnQA3A%22%2C%22k%22%3A%2254154154%22%7D%2C%22o%22%3A%7B%7D%7D
You are asking about two different things here:
JSON ENCODING
If you are trying to store simple data, then no, JSON encoding is not required. You can manually store key and value pairs in a cookie with no difficulty. Cookies are very good at this.
If, on the other hand, you have a complex object like your example then JSON encoding is the best way. If you don't use JSON encoding, you'll need some other way to handle key/value encoding, and by the time you end up handling tested objects you'll have just created a poor version of JSON. So, use JSON and save the headaches.
encodeURIComponent
Not needed. A cookie is not a URI, nor is it a URI component.
Other
The JSON encoding will handle any unicode encoding that needs done.
Right off the bat I'm having a hard time imagining any encoding that needs to be done that wouldn't be handled by JSON.
From this site, I've learned that ASP.NET script services accepting JSON actually require them to be serialized JSON strings (see: "JSON, objects, and strings: oh my!" section of the link). Is there a quick and easy way to serialize them for ASP.NET AJAX consumption on the client side instead of trying to manually convert a bunch of existing objects to JSON-looking strings?
Thanks in advance!
You can use JSON.stringify() to serialize client-side objects for consumption in ASP.NET's Script Services.
Using that approach, you can map client-side objects to server-side objects very easily. ASP.NET will automatically handle converting the JSON to objects (or even collections of objects) for you.
The article writer is confusing Javascript objects with JSON strings. There is no such thing as a "JSON object".
Naturally if you try to send an object to a web service, it has to be serialised, as the request data can only contain text, not objects. The standard way of serialising data to be posted is URL encoding it, so that is what jQuery does.
There is no JSON serialisation built into Javascript or jQuery. You would have to do the serialising yourself or find a library that does it. Here are some options: Serializing to JSON in jQuery
Also, the data sent in the example is not valid JSON. It looks like this:
"{'fname':'dave', 'lname':'ward'}"
To be valid JSON it should look like this:
'{"fname":"dave", "lname":"ward"}'