EDIT
I found this:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays
Basically if I have something like this:
struct someStruct {
unsigned long id;
char username[16];
float amountDue;
};
on client side I can do:
var ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = function (e) {
var buffer = e.data;
var data_view = new DataView(buffer);
// ... read the data into the buffer ...
var idView = data_view.getUint32(0);
var usernameView = data_view.getUint32(4);
var amountDueView = data_view.getFloat32(20);
};
The problem is that I want to convert them to normal Javascript objects (numbers, strings etc).
Original question
I would send data via websocket packed using a C struct, and unpack on browser using Javascript.
I know modules exists for node.js, but I can't find nothing client-side.
If you're familiar with Python struct, then you may like structjs. It's my attempt at porting Python struct to javascript. As it is, it's for Node, but a client port should be easy.
You won't have issues with alignment or padding (you can specify those explicitly though) for that structure, but you may need to indicate little-endian (by the '<' in the format string) if that's your flavour. You might do like so (I haven't tested this example in any way):
let struct = require("./struct") // Node specific, you need to wrap it.
let someStruct = struct('<I16sf') // This is your struct definition
let ws = new WebSocket("ws://URI");
ws.binaryType = "arraybuffer";
ws.onmessage = e => {
// Unpack using the structure definition. Unpack takes an ArrayBuffer.
let [id, username, amountDue] = someStruct.unpack(e.data);
// Use data...
};
Ok,
https://www.npmjs.com/package/c-struct looks like what you want.
Good luck!
Ok, after some researching, I finally decided this could not be a good idea:
https://justin.harmonize.fm/development/2013/04/28/a-slower-js-msgpack.html
Shortly: javascript is slow in decoding.
Probably it's just more simple to use JSON and Content-Encoding: gzip, if it does not slow down your web app.
Related
For my Java application I need to be able of, given a JavaScript string, determine the actual arguments passed into a function call.
For example, given this JavaScript string:
const url = "http://foo.bar?q=" + location.href
const method = "GET"
const isAjax = true
let xmlhttp = new XMLHttpRequest();
xmlhttp.open(method, url, isAjax);
I would like to evaluate this JavaScript in order to get this:
xmlhttp.open("GET", "http://foo.bar?q=someurl", true);
Right now I'm using a regex to look for the parts of the JavaScript I'm interested in (in this example it would be the open method of the XMLHttpRequest object), but I need to be able to compute the actual values of the arguments passed to a function, if they are not hardcoded from the call-side.
I have been searching here but what I found was more related to actually executing the JavaScript code rather than looking at its expressions values (more like evaluating it, getting an AST or something like that).
Any ideas on how to accomplish this?
My idea is to add some javascript mocking library like sinon and execute this javascript.
Especially take a look at fake XMLHttpRequest
Javascript code will like this:
let sinon = require('sinon');
let xhr = sinon.useFakeXMLHttpRequest();
let requests = [];
xhr.onCreate = function (xhr) {
requests.push(xhr);
}
const url = "http://foo.bar?q=" + location.href
const method = "GET"
const isAjax = true
let xmlhttp = new XMLHttpRequest();
xmlhttp.open(method, url, isAjax);
console.log(requests[0].url)
Problem
I'm trying to create a simple trustless file sharing application. I'm using EasyRSA (https://github.com/paragonie/EasyRSA) to create a key-pair and then encrypt my data with the public key. I'm sending the private key to my JS wherein I'm using Node-RSA (https://github.com/rzcoder/node-rsa). Here, I try to decrypt using the previously created private key. But this happens:
Error
Uncaught Error: Error during decryption (probably incorrect key). Original error: Error: Incorrect data or key
at NodeRSA.module.exports.NodeRSA.$$decryptKey (drop.js:23265)
at NodeRSA.module.exports.NodeRSA.decrypt (drop.js:23213)
at window.onload (drop.js:22270)
My JS code looks something like this:
var NodeRSA = require('node-rsa');
window.onload = function() {
var key_val = $('#key').val();
var ciphertext = $('#encrypted').val();
var key = new NodeRSA();
key.importKey($('#key').val(), 'pkcs1');
var ciphertext_base = key.encrypt(ciphertext, 'base64');
var decrypted_base = key.decrypt(btoa(ciphertext));
console.log(decrypted);
}
I think this has got something to do with the incompatibility between the formats/key-sizes/algos used to create the keys. If someone could help me 'hack' this and make it work, I would be very grateful to you.
Is it possible passing an object from/to webWorker from/to main thread by reference? I have read here information about transferable objects.
Chrome 13 introduced sending ArrayBuffers to/from a Web Worker using
an algorithm called structured cloning. This allowed the postMessage()
API to accept messages that were not just strings, but complex types
like File, Blob, ArrayBuffer, and JSON objects. Structured cloning is
also supported in later versions of Firefox.
I just want to pass information, not object with methods. Just something like this (but with a lot of information, a few MB, so that main thread does not have to receive a copy of the object):
var test = {
some: "data"
}
Once you have some data in an object (this: {bla:666, color:"red"}) you will have to copy it and there is no way to avoid it. The reason is, that you don't have control over the memory object is stored in, so you can't transfer it. The only memory that can be transferred is memory allocated for transferable objects - typed arrays.
Therefore if you need some data transferred, you must think in advance and use the transferable interface. Also keep in mind that even when object is copied, the transfer speed is very fast.
I wrote a library that converts object to binary data (therefore transferable), but it isn't faster than native transfer, it's way slower actually. The only advantage is that it allows me to transfer unsupported data types (eg. Function).
There Is An Array 2nd Argument To postMessage
Actually yes, it is possible in, (surprise, Surprise!) Chrome 17+ and Firefox 18+ for certain objects (see here).
// Create a 32MB "file" and fill it.
var uInt8Array = new Uint8Array(1024 * 1024 * 32); // 32MB
for (var i = 0; i < uInt8Array.length; ++i) {
uInt8Array[i] = i;
}
worker.postMessage(uInt8Array.buffer, [uInt8Array.buffer]);
You can also apply this to strings by converting the string to and from an array buffer using FastestSmallestTextEncoderDecoder as shown below.
//inside the worker
var encoderInst = new TextEncoder;
function post_string(the_string){
var my_array_buffer = encoderInst.encode(the_string).buffer;
postMessage( my_array_buffer, [my_array_buffer] );
}
Then, to read the arraybuffer as a string:
// var workerInstance = new Worker("/path/to/file.js");
var decoderInst = new TextDecoder;
workerInstance.onmessage = function decode_buffer(evt){
var buffer = evt.data;
var str = decoderInst.decode(buffer);
console.log("From worker: " + str);
return str;
}
Here is a small interactive example of using a Worker to increment each letter of a string.
var incrementWorker = new Worker("data:text/javascript;base64,"+btoa(function(){
// inside the worker
importScripts("https://dl.dropboxusercontent.com/s/r55397ld512etib/Encode" +
"rDecoderTogether.min.js?dl=0");
const decoderInst = new TextDecoder;
self.onmessage = function(evt){
const u8Array = new Uint8Array(evt.data);
for (var i=0, len=u8Array.length|0; i<len; i=i+1|0) {
++u8Array[i];
}
postMessage(decoderInst.decode(u8Array));
};
} .toString().slice("function(){".length, -"}".length)));
const inputElement = document.getElementById("input");
const encoderInst = new TextEncoder;
(inputElement.oninput = function() {
const buffer = encoderInst.encode(inputElement.value).buffer;
incrementWorker.postMessage(buffer, [buffer]); // pass happens HERE
})();
incrementWorker.onmessage = function(evt){
document.getElementById("output").value = evt.data;
};
<script src="https://dl.dropboxusercontent.com/s/r55397ld512etib/EncoderDecoderTogether.min.js?dl=0" type="text/javascript"></script>
Before: <input id="input" type="text" value="abc123 foobar" /><br />
After: <input id="output" type="text" readonly="" />
Sources: Google Developers and MDN
It is not possible. You have to send the object, update it in the worker and then return the updated version to the main thread.
If you want to pass an object just with information, you only need to pass your object as a string
myWorker.postMessage(JSON.stringify(myObject));
parse the object inside your worker
JSON.parse(myObject)
and finally return your updated object to the main thread.
Take a look also to ParallelJs that is library to work easier with web-workers
I'm trying to access a named pipe in a Firefox add-on. My code, based on solution 2 to this CodeProject question, is:
var file = Components.classes["#mozilla.org/file/local;1"].createInstance(Components.interfaces.nsILocalFile);
file.initWithPath("\\\\.\\pipe\\test");
var text = "Some text to be written";
var writer = Components.classes["#mozilla.org/network/file-output-stream;1"].createInstance(Components.interfaces.nsIFileOutputStream);
// Open file for read/write access, and create it if it does not exist.
writer.init (file, 0x04 | 0x08, -1, 0);
writer.write (text, text.length);
writer.flush ();
writer.close ();
When I run this is Firefox Scratchpad, I get:
/*
Exception: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFileOutputStream.init]
#6
*/
Line 6 is the line where I call writer.init.
I've played with passing different flags to writer.init, but no luck. I'm able to write to a normal file path with this code, just not the named pipe path.
I've been searching for more information for most of a day. The only other relevant thing I've found is this Bugzilla bug that mentions the same problem, but it's dated 2009.
Thank you in advance for any suggestions.
Edit: Based on the replies I tried the following:
var encoder = new TextEncoder();
var array = encoder.encode("This is some text");
var path = "\\\\.\\pipe\\test";
Task.spawn(function() {
let pfh = yield OS.File.open(path, {write: true});
yield pfh.write(array);
yield pfh.close();
});
which lets me write to a file (if I change the path accordingly) but does not seem to send anything to the pipe.
Here is the (admittedly crude) code I'm using to read the pipe on the .NET side:
let server = new NamedPipeServerStream ("\\\\.\\pipe\\test", PipeDirection.InOut)
let reader = new StreamReader (server)
do
printfn "Listening..."
server.WaitForConnection ()
printfn "Connected."
while true do
while reader.EndOfStream = false do reader.ReadLine () |> printfn "%s"
And the code I threw together to verify that I could at least write to the pipe in .NET:
let client = new NamedPipeClientStream ("\\\\.\\pipe\\test")
do client.Connect ()
let writer = new StreamWriter (client)
do writer.AutoFlush <- true
// Wait for connection
System.Threading.Thread.Sleep (1000)
let message = "hello"
do message |> writer.WriteLine
Try OS.File:
let encoder = new TextEncoder(); // This encoder can be reused for several writes
let array = encoder.encode("This is some text"); // Convert the text to an array
let promise = OS.File.writeAtomic("\\\\.\\pipe\\test", array, // Write the array atomically to "file.txt", using as temporary
{tmpPath: "file.txt.tmp"}); // buffer "file.txt.tmp".
I'm trying to write a string to a socket (socket is called "response"). Here is the code I have sofar (I'm trying to implement a byte caching proxy...):
var http = require('http');
var sys=require('sys');
var localHash={};
http.createServer(function(request, response) {
var proxy = http.createClient(80, request.headers['host'])
var proxy_request = proxy.request(request.method, request.url, request.headers);
proxy_request.addListener('response', function (proxy_response) {
proxy_response.addListener('data', function(x) {
var responseData=x.toString();
var f=50;
var toTransmit="";
var p=0;
var N=responseData.length;
if(N>f){
p=Math.floor(N/f);
var hash="";
var chunk="";
for(var i=0;i<p;i++){
chunk=responseData.substr(f*i,f);
hash=DJBHash(chunk);
if(localHash[hash]==undefined){
localHash[hash]=chunk;
toTransmit=toTransmit+chunk;
}else{
sys.puts("***hit"+chunk);
toTransmit=toTransmit+chunk;//"***EOH"+hash;
}
}
//remainder:
chunk=responseData.substr(f*p);
hash=DJBHash(chunk);
if(localHash[hash]==undefined){
localHash[hash]=chunk;
toTransmit=toTransmit+chunk;
}else{
toTransmit=toTransmit+chunk;//"***EOH"+hash;
}
}else{
toTransmit=responseData;
}
response.write(new Buffer(toTransmit)); /*error occurs here */
});
proxy_response.addListener('end', function() {
response.end();
});
response.writeHead(proxy_response.statusCode, proxy_response.headers);
});
request.addListener('data', function(chunk) {
sys.puts(chunk);
proxy_request.write(chunk, 'binary');
});
request.addListener('end', function() {
proxy_request.end();
});
}).listen(8080);
function DJBHash(str) {
var hash = 5381;
for(var i = 0; i < str.length; i++) {
hash = (((hash << 5) + hash) + str.charCodeAt(i)) & 0xffffffff;
}
if(hash<-1){
hash=hash*-1;
}
return hash;
}
The trouble is, I keep getting a "content encoding error" in Firefox. It's as if the gizipped content isn't being transmitted properly. I've ensured that "toTransmit" is the same as "x" via console.log(x) and console.log(toTransmit).
It's worth noting that if I replace response.write(new Buffer(toTransmit)) with simply response.write(x), the proxy works as expected, but I need to do some payload analysis and then pass "toTransmit", not "x".
I've also tried to response.write(toTransmit) (i.e. without the conversion to buffer) and I keep getting the same content encoding error.
I'm really stuck. I thought I had this problem fixed by converting the string to a buffer as per another thread (http://stackoverflow.com/questions/7090510/nodejs-content-encoding-error), but I've re-opened a new thread to discuss this new problem I'm experiencing.
I should add that if I open a page via the proxy in Opera, I get gobblydeegook - it's as if the gzipped data gets corrupted.
Any insight greatly appreciated.
Many thanks in advance,
How about this?
var responseData = Buffer.from(x, 'utf8');
from: Convert string to buffer Node
Without digging very deep into your code, it seems to me that you might want to change
var responseData=x.toString();
to
var responseData=x.toString("binary");
and finally
response.write(new Buffer(toTransmit, "binary"));
From the docs:
Pure Javascript is Unicode friendly but not nice to binary data. When
dealing with TCP streams or the file system, it's necessary to handle
octet streams. Node has several strategies for manipulating, creating,
and consuming octet streams.
Raw data is stored in instances of the Buffer class. A Buffer is
similar to an array of integers but corresponds to a raw memory
allocation outside the V8 heap. A Buffer cannot be resized.
So, don't use strings for handling binary data.
Change proxy_request.write(chunk, 'binary'); to proxy_request.write(chunk);.
Omit var responseData=x.toString();, that's a bad idea.
Instead of doing substr on a string, use slice on a buffer.
Instead of doing + with strings, use the "concat" method from the buffertools.
Actually, now new Buffer() is deprecated sence node.js v10+, so better to use
Buffer.from(,)
from
response.write(new Buffer(toTransmit));
do
response.write(Buffer.from(toTransmit,'binary'));