Develop Reference

Creating Binary Data Structures in Javascript

I'm building a web app, and need to be able to encode user-generated data as concisely as possible before transmitting it to my server. In the past I've used Flash, and it had a very neat system where for any class that you want to serialize, you could write a pair of functions that would describe exactly how to serialize the data. For example:
out.writeShort(session);
out.writeUnsignedInt(itemID);
out.writeObject(arbitraryData);
out.writeShort(score);
You would have to write an equivalent function to read bytes from the serialized data and build the class from it.
Once data is serialized it could be encoded into a Base64 string for safe network transmission to the server.
I can't figure out how to do this in Javascript? JSON is nice and easy but it's incredibly wasteful, sending all object key/value pairs, and unless I'm mistaken everything is encoded as a string? So the value false is encoded as the string "false"?
Any advice on how to implement this in Javascript would be greatly appreciated! Use of libraries is fine so long as they work both on Node and in browser.

Look at this answer. You can use BSON format (Binary JSON) and it doesn't have those features of JSON you mentioned.

Effective send big array of coordinates using Socket.io

I was testing performance of my node.js server and I detected that this small code last too much time:
socket.emit("a",{a1:something,a2:veryBigArray});
Problem is that my array is very big and socket.io must encode it to JSON.
My array is something like:
veryBigArray=[{x:0,y:0},{x:0,y:1},{x:1,y:1},{x:1,y:2}.......];
I am interested mainly in server performance. I want server can continue in work as soon as possible.
Before sending array to client, I generate array, so I don't have problem to completely change structure of sended data. Maybe array could be somehow compressed. I read about ArrayBuffer;
What is the best (fastest for sever) way of sending big array of coordinates to client (browser) using Socket.io?

If you must encode the array to JSON then I don't think an ArrayBuffer will help. However, if you know the precise data structure of the array to be sent and can predict it server-side, you can come up with your own efficient encoding/decoding schema.
In your example, your data is simply an array of x and y value pairs where each value is an integer. An extremely simple but possibly fruitful approach would be to strip the data of the predictable (unnecessary) x and y keys and encode it as a simple CSV (comma-separated value) string.
For example:
[{x:0,y:0},{x:0,y:1},{x:1,y:1},{x:1,y:2}]
would encode as the string:
0,0,0,1,1,1,1,2
The most efficient approach however would probably be to abandon socket.io for your own custom websocket interface that can somehow send binary data directly instead of encoding it as JSON. JSON will inherently be radically inefficient for sending a large dataset compared to sending encoded binary.
Edit: It looks like socket.io can send binary data so I would explore that along with some kind of efficient encoding/decoding scheme tailored to your dataset.

using regexp on raw binary data

I'm embedding JavaScript in my C++ app (via V8) and I get some raw binary data which I want to pass to JavaScript. Now, in the JavaScript, I plan to do some regular expressions on the data.
When using just the standard JavaScript String object for my data, everything is quite straight-forward. However, as far as I understand it, it uses an UTF16 representation and expects the data to be valid Unicode. But I have arbitrary data (might contain '\0' and other raw data - although it is just text for the most part).
How should I handle this? I searched a bit around and maybe ArrayBuffer or something like this is the object I need to store my raw data. However, I didn't found how to do the usual regular expression methods on that object. (Basically I need RegExp.test and RegExp.exec).
I just checked out the Node.js code and it seems as if they support binary data and just put it into a string via v8::String::NewFromOneByte. See here and here. So that would answer my question (i.e., I can just use String), wouldn't it? Any downsides?
(I still don't see why my question is bad. Please explain the downvote.)

From all my current tests, it seems like it works just as expected with normal String.
You can even specify that in JavaScript directly, e.g.
var s = "\x00\x01\x02\x03"
and regular expressions on that string work like expected.
On the C++ side, if you want to get your binary data into a JS String object:
v8::Local<v8::String> jsBinary(const uint8_t* data, uint32_t len) {
assert(int(len) >= 0);
return String::NewFromOneByte(v8::Isolate::GetCurrent(), data, String::kNormalString, len);
}

Decoding cp1251 to UTF-8 in javascript

How to decode cp-1251 to UTF-8 in javascript?
The cp-1251 is from a datafeed, which required to decode from js client side.
There is no way to change server side output, since it is related to a 3rd party, and due to some reason, I would not use any server side programming to convert the datafeed to become another datafeed.

(Assuming that by "UTF-8" you meant the JS strings in their native encoding...)
Depending on the format your 'cp-1251' data is in and depending on the browsers you need to support, you can choose from:
TextDecoder.decode() API (decodes a sequence of octets from a typed array, like Uint8Array) - if you're using web sockets, you can get an ArrayBuffer out of it to decode.
https://github.com/mathiasbynens/windows-1251 operates on something it calls 'byte strings' (JS Strings consisting of characters like \u00XY, where 0xXY is the encoded byte.
build the decoding table yourself (example)
Note that in most cases (not something as low-level as websockets though) it might be easier to read the data in the correct encoding before it ends up as a JS string (for example, you can force XMLHttpRequest to use a certain encoding even if the server misreports the encoding).

JSON to String on client side for ASP.NET Script Service?

From this site, I've learned that ASP.NET script services accepting JSON actually require them to be serialized JSON strings (see: "JSON, objects, and strings: oh my!" section of the link). Is there a quick and easy way to serialize them for ASP.NET AJAX consumption on the client side instead of trying to manually convert a bunch of existing objects to JSON-looking strings?
Thanks in advance!

You can use JSON.stringify() to serialize client-side objects for consumption in ASP.NET's Script Services.
Using that approach, you can map client-side objects to server-side objects very easily. ASP.NET will automatically handle converting the JSON to objects (or even collections of objects) for you.

The article writer is confusing Javascript objects with JSON strings. There is no such thing as a "JSON object".
Naturally if you try to send an object to a web service, it has to be serialised, as the request data can only contain text, not objects. The standard way of serialising data to be posted is URL encoding it, so that is what jQuery does.
There is no JSON serialisation built into Javascript or jQuery. You would have to do the serialising yourself or find a library that does it. Here are some options: Serializing to JSON in jQuery
Also, the data sent in the example is not valid JSON. It looks like this:
"{'fname':'dave', 'lname':'ward'}"
To be valid JSON it should look like this:
'{"fname":"dave", "lname":"ward"}'