I have a C++ code like this:
extern "C" {
void MyCoolFunction (int** values)
{
int howManyValuesNeeded = 5;
*values = new int[howManyValuesNeeded];
for (int i = 0; i < howManyValuesNeeded; i++) {
(*values)[i] = i;
}
}
}
From C++ it can be used like this:
int *values = NULL;
MyCoolFunction (&values);
// do something with the values
delete[] values;
Of course the real code is much more complicated, but the point is that the function allocates an int array inside, and it decides what the array size will be.
I translated this code with Emscripten, but I don't know how could I access the array allocated inside the function from javascript. (I already know how to use exported functions and pointer parameters with Emscripten generated code, but I don't know how to solve this problem.)
Any ideas?
In Emscripten, memory is stored as a giant array of integers, and pointers are just indexes into that array. Thus, you can pass pointers back and forth between C++ and Javascript just like you do integers. (It sounds like you know how to pass values around, but if not, go here.)
Okay. Now if you create a pointer on the C++ side (as in your code above) and pass it over to Javascript, Emscripten comes with a handful of helper functions to allow you to access that memory. Specifically setValue and getValue.
Thus, if you passed your values variable into JS and you wanted to access index 5, you would be able to do so with something like:
var value5 = getValue(values+(5*4), 'i32');
Where you have to add the index times the number of bytes (5*4) to the pointer, and indicate the type (in this case 32 bit ints) of the array.
You can call the delete from JavaSCript by wrapping it inside another exported function.
extern "C" { ...
void MyCoolFunction (int** values);
void finsih_with_result(int*);
}
void finsih_with_result(int *values) {
delete[] values;
}
Alternatively you may also directly do this on JavaScript side: Module._free(Module.HEAPU32[values_offset/4]) (or something like that; code not tested).
Related
I am wondering if, using C (or C++ or Rust) and javascript, I am able to do CRUD operations to a shared data object. Using the most basic example, here would be an example or each of the operations:
#include <stdio.h>
typedef struct Person {
int age;
char* name;
} Person;
int main(void) {
// init
Person* sharedPersons[100];
int idx=0;
// create
sharedPersons[idx] = (Person*) {12, "Tom"};
// read
printf("{name: %s, age: %d}", sharedPersons[idx]->name, sharedPersons[idx]->age);
// update
sharedPersons[idx]->age = 11;
// delete
sharedPersons[idx] = NULL;
}
Then, I would like to be able to do the exact same thing in Javascript, and both be able to write to the same shared sharedPersons object. How could this be done? Or does the setup need to be something like a 'master-slave' where one just needs to pass back information to the other and the master does all the relevant actions? I'm hoping that there's a way do CRUD on a shared data object in webassembly, and any help would be greatly appreciated.
As a reference: https://rustwasm.github.io/wasm-bindgen/contributing/design/js-objects-in-rust.html
Creating the object
Let's create the object in C and return it:
typedef struct Person {
int age;
char* name;
} Person;
Person *get_persons(void) {
Person* sharedPersons[100];
return sharedPersons;
}
You could also create the object in JS, but it's harder. I'll come back to this later.
In order for JS to get the object, we've defined a function (get_persons) that returns (a pointer to) it. In this case it's an array, but of course it could have been a single object. The thing is, there must be a function that will be called from JS and that provides the object.
Compiling the program
emcc \
-s "SINGLE_FILE=1" \
-s "MODULARIZE=1" \
-s "ALLOW_MEMORY_GROWTH=1" \
-s "EXPORT_NAME=createModule" \
-s "EXPORTED_FUNCTIONS=['_get_persons', '_malloc', '_free']" \
-s "EXPORTED_RUNTIME_METHODS=['cwrap', 'setValue', 'getValue', 'AsciiToString', 'writeStringToMemory']" \
-o myclib.js
person.c
I don't remember why we have a leading underscore in _get_persons, but that's how Emscripten works.
Getting the object in JS
const createModule = require('./myclib');
let myclib;
let Module;
export const myclibRuntime = createModule().then((module) => {
get_persons: Module.cwrap('get_persons', 'number', []),
});
What this does is create a get_persons() JS function that is a wrapper of the C get_persons() function. The return value of the JS function is "number". Emscripten knows that the C get_persons() function returns a pointer, and the wrapper will convert that pointer to a JS number. (Pointers in WASM are 32-bit.)
Manipulating the object in JS
const persons = get_persons();
Module.getValue(persons, 'i32'); // Returns the age of the first person
Module.AsciiToString(Module.getValue(persons + 4, 'i32')); // Name of first person
// Set the second person to be "Alice", age 18
const second_person = persons + 8;
Module.setValue(second_person, 18, 'i32');
const buffer = Module._malloc(6); // Length of "Alice" plus the null terminator
Module.writeStringToMemory("Alice", buffer);
Module.setValue(second_person + 4, buffer, 'i32');
This is a fairly low level way of doing it, although there seems to be an even lower level way. As other people have suggested, there may be higher level tools to help in C++ and Rust.
Creating the object in JS
You can create objects in JS by using _malloc() (and free them with _free()) as we did with the string above, and then pass their pointers to C functions. But, as I said, creating them in C is probably easier. In any case, anything _malloc()ed must eventually be freed (so the string creation above is incomplete). The FinalizationRegistry can help with this.
Yes, this is possible.
WebAssembly stores objects within linear memory, a contiguous array of bytes that the module can read and write to. The host environment (typically JavaScript within the web browser) can also read and write to linear memory, allowing it to access the objects that the WebAssembly modules stores there.
There are two challenges here:
How do you find where your WebAssembly module has stored an object?
How is the object encoded?
You need to ensure that you can read and write these objects from both the WebAssembly module and the JavaScript host.
I'd pick a known memory location, and a known serialisation format and use that to read/write from both sides.
I would compare what I am doing to what JavaScript runtimes already do, yet I'm doing it in JavaScript and Wasm. JavaScript implementations store JavaScript objects and values in actual computer heap memory, yet performing operations such as attempting to read/write out of bounds memory don't actually modify the memory (ex: arrays perform a no-op and return undefined respectively).
I'll give an example of my specific situation:
Let's say that I have an array buffer of 1000 bytes, we'll name the variable memory.
I want to split apart the buffer specifically into Int32Arrays of size 4. Each partition from the ArrayBuffer must do two things:
a) Refer to the original buffer (so that, when the original data is manipulated, the partition will update its values automaticially)
b) Not expose the original buffer (as the partition could then be used to corrupt the other partitions)
I have a function that determines which section is available for usage, we'll call it findPartition. It returns an integer acting as a pointer to a set of available bytes. (like C's malloc)
Each partition is expected to always remain the same type, that is, they will always be Int32Arrays if they start as an Int32Array, and their size will always be constant.
The script operating on the partition may both, write to, and read from, its partitioned array.
Originally, I was thinking that I could just call the Int32Array constructor on my array buffer, simply like so:
const createPartition = () => new Int32Array( memory, findPartition(), 4 );
The problem is that the buffer is exposed, so I could either delete the buffer property.
But... the buffer property is readonly, so delete fails when used on the array.
I then thought that I could make a class to do this:
class Partition {
#source = new Int32Array( memory, findPartition(), 4 );
get 0() { return this.#source[0]; }
set 0(x) { this.#source[0] = x; }
get 1() { return this.#source[1]; }
set 1(x) { this.#source[1] = x; }
get 2() { return this.#source[2]; }
set 2(x) { this.#source[2] = x; }
get 3() { return this.#source[3]; }
set 3(x) { this.#source[3] = x; }
get length() { return 4; }
};
Well, that works, but it's much more verbose, thus harder to maintain later, and, as the partitions are not given direct access to the indexes' values, because they have to go through getters and setters, I feel that performance could be lost.
Ideally, the Int32Array.prototype is also on the object, so I would have to wrap everything, which would be annoying and unmaintainable. If the spec updates the methods of the prototype, then I would have to update the wrappers too.
Does anyone have a better way to segment the array buffer, while maintaining safety between the segments?
Simplest way is to extend chosen typed array like that:
// Seal and freeze hidden Object (TypedArray.prototype) that has methods
// that can leak original buffer if attacked using defineProperty tricks
// Since we can't directly access hidden Object on which `subarray`
// and many other methods defined we use this workaround
// Paranoid: More checks required to make sure that: `subarray` method; 'byteOffset',
// 'byteLength', 'buffer' getters; are not modified beforehand
Object.seal(Int8Array.__proto__.prototype);
Object.freeze(Int8Array.__proto__.prototype);
class customUint32Array extends Uint32Array {
get buffer(){
// copy! viewed array buffer segment
// test if `super.` is faster/slower than `this.` access
return super.buffer.slice(this.byteOffset, this.byteOffset + this.byteLength);
// return super.buffer.slice(super.byteOffset, super.byteOffset + super.byteLength);
}
}
// var customUint32ArrayOverWholeBufferCached = new customUint32Array(memory);
function Partition(){
// test performance of `new` vs `customUint32ArrayOverWholeBufferCached.subarray`
// for fastest array buffer view creation
return new customUint32Array(memory, findPartitionByteOffset(), 4);
// return customUint32ArrayOverWholeBufferCached.subarray(findPartitionIndex(), 4);
}
By the way 'private' class properties in most JS environments are exposed as any other property and will leak original buffer.
Prototype chains forged manually instead of class X extends Y are welcome in comments.
If one will pass my original buffer leak tests, I'll include it here.
Current instance' prototype chain looks something like: customUint32Array.Uint32Array.TypedArray.prototype.Object
Lets say I have a C API as follows:
void get_result_buffer(context* ctx, void** result, size_t* result_size);
Where context is some arbitrary opaque context type holding state. The intended way to call this is
context* ctx = ...;
do_something_with_context(ctx, ...);
void* result_buffer = 0;
size_t result_buffer_size = 0;
get_result_buffer(ctx, &result_buffer, &result_buffer_size);
/* Now result_buffer and result_buffer_size are meaningful and populated with the results of having called `do_something_with_context`. */
The result_buffer is owned by the context object, so the caller doesn't need to free it. Now I'd like to be able to call get_result_buffer from Emscripten. I can easily enough set up cwrap for this, it looks something like:
wrap_get_result_buffer = something.cwrap(
'get_result_buffer',
null,
['number', 'number', 'number']
)
But I'm unclear how I can set things up so that the out parameters "work" in JS. Ideally, at the end, I'd have something that looks like a byte buffer containing a copy of the data pointed to by the result out parameter, with a length as described by the result_size out parameter.
It seems that the values that I pass in need to be allocated somehow, and then I would pass the resulting allocation handle in as the number type parameters, but I have no idea how to do that in the JS/Emscripten layer. Similarly, after the call, I'd expect that those values have now been updated by the transpiled C code, but I'm unclear on how to extract the now populated data into some sort of JS byte array.
Any guidance on how to do this or pointers to example code?
OK. I figured this out. For future emscripteners, you want to do something like the following.
var out_data_ptr = Module._malloc(8)
var out_data_array = new Uint32Array(Module.HEAPU32.buffer, out_data_ptr, 2)
wrap_get_result_buffer(context, out_data_ptr, out_data_ptr + 4)
var response_uint8_array = new Uint8Array(Module.HEAPU8.buffer, out_data_array[0], out_data_array[1])
Module._free(out_data_ptr)
The theory of operations here is that we create a two element array that will store the 'slots' to be filled in by calling get_result_buffer, and then construct a view over that exposing it as two number compatible elements. We then pass those in to our get_result_buffer function as lifted above with cwrap. After that, the heap memory that the context refers to is reachable from those slots, which can then be used to construct a Uint8Array that provides JS level access to the bytes in the result.
In FORTRAN and C++, the address of a specific array element can be passed into a function. For example, in the main routine, WA1 is a work array of size 25 and offset is an integer variable that indicates the offset from the 0-index. Say offset is presently 6.
The declaration of the sub-routine might look like the following:
void Array_Manip1(double* WorkArray1){
. . .
When the sub-routine is called in the main program, the call might look like this:
Array_Manip1(&WA1[offset]);
By doing this, I can index WorkArray1 within the sub-routine starting at the 0-index, but knowing it is actually WA1[6].
e.g. -
for (int i = 0; i < 19; ++i)
WorkArray1[i] = whatever computation is required.
To do this in Javascript, I suppose the full array could be passed in to the sub-routine, plus one more variable to hold the offset. And then within the sub-routine, the offset would have to be added to the array index value.
e. g. -
for (int i = 0; i < 19; ++i){
WorkArray1[offset + i] = whatever computation is required.
But now I am passing one more variable into the sub-routine, and have to add the offset to the array index each time through the loop.
Is there a better way to do this in Javascript?
Is there a way to imitate C++'s ability to pass the address of a specific array element into a function?
The cleanest way would be to splice the array and pass in a subarray from the current index on. That way you still have one reference, and everything stays clean.
But no, arrays in most higher level languages do not allow you to reference a single element and then get back to the array. It is dangerous for a number of reasons on those kinds of languages where the underlying data may not even be stored contiguously. JavaScript is no exception, and you can pass in an array and an index, or a subarray, but you can't pass in a reference to an element in the array and get back to the array after passing it in.
Tim's answer is correct. I just want to add something about the C-like typed arrays: they can be created as a view into an ArrayBuffer, in which case you could create a new view of the same buffer as the larger array but starting at an offset, and pass that, without duplicating the underlying data. Closest you can get to your pointers.
You can sort of do what you want. And in fact, sometimes javascript can only do what you want. It all depends on what data the array contains.
In Javascript, the content of a variable may either be a value or a reference (pointer but without pointer arithmetic). But you have no choice in the matter. Numbers and strings are always values (there are exceptions but none of them apply when passing as function arguments) and everything else are always references.
So to get the behavior you want, simply use an object or array as your value holder instead of a string or number:
var ref_array = [ {value:1}, {value:2}, {value:3} ];
function increment (v_obj) {
v_obj.value ++;
}
var ref = ref_array[1];
increment(ref);
// ref_array will now contain: [{value:1},{value:3},{value:3}]
It's not that simple though. While the object appears to be passed by reference, the reference is however copied when the function is called. What this means is that ref and ref_array[1] and v_obj are three separate variables that point to the same thing.
For example, this wouldn't work:
function replace (obj1, obj2) {
obj1 = obj2;
}
replace(ref_array[1], {value:9});
// ref_array is still: [{value:1},{value:3},{value:3}]
That's because, while obj1 in the function above points to the same object as ref_array[1], it is not really a pointer to ref_array[1] but a separate variable. In C, this would be something like obj1 = &ref_array[1]. So passing an argument passes a copy of the pointer but not the pointer itself.
I was wondering if I could pass a IEnumerable Collection into a Javascript method on the page load. So something like this...
<%# Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<MyAppMVC.Models.ViewModels.News.NewsIndexViewData>" %>
<div id="container">
<%= String.Format("<script type='text/javascript'>testMethod({0})</script>", Model.NewsList) %>
</div>
I realize JS is client side, just didn't know if there was any way possible to do that?
Thanks!
No, you cannot. For several reasons.
1) There is no IEnumerable in JavaScript, as you use it in .NET. There is something similar, but it is implemented completely differently. In .NET, IEnumerable just means the class provides a method, GetEnumerator(), which returns an IEnumerator (which itself only contains Current, MoveNExt, and Reset methods). In JavaScript, when you do a for iteration on an item, you are iterating over the names of its properties.
var myObj = { 'a' = 1, 'b' = 2 };
for (var name in myObj) { alert(name); } // will alert 'a', and 'b'
Even when working with JavaScript arrays, the above loop returns the index of the array element, not the actual member at that index.
2) By doing a String.Format() on your list, you wouldn't have been passing the list as an object to your JavaScript but just the ToString() result of your list. Which probably just returns "System.Collections.Generic.List`1[System.String]"
3) Unless your developing environment explicitly allows it, you can assume that passing arguments from one language into another is not going to work. Much like you can't write JavaScript inside your .NET code, you can't write .NET code in your JavaScript. These languages have different feature sets, different syntax, and are executed with completely different mechanisms - .NET is compiled, and JavaScript (generally, speaking) is interpreted (Compiled vs. Interpreted languages).
What you have to do is transform your data into a format that can be used by JavaScript. Most likely this means converting it into something called JSON. You didn't provide a lot of details as to what exactly Model.NewList is, or what your testMethod() expects as an argument. But for the sake of an example lets assume NewList is a list of strings. In that case, your JSON would look something like this:
{ 'NewList' : ['string1', 'string2', 'string3'] }
The easiest way to convert your .NET data into JSON is to use built-in libraries, such as JavaScriptSerializer:
System.Web.Script.Serialization.JavaScriptSerializer serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
string json = serializer.Serialize(Mdoel.NewList);
You can register the javascript in the code behind in the Page_Load with the following code (Note I am assuming that Model.NewsList is a string enumeration or array):
StringBuilder sb = new StringBuilder();
bool isFirst = true;
//build a comma seperated list.
foreach (string s in Model.NewsList)
{
if (isFirst)
isFirst = false;
else
sb.Append(", ");
sb.Append("'").Append(s.Replace("'", "''")).Append("'");
}
//create the javascript array
string javascript = String.Format(#"var news = [{0}];", sb);
//put the array in the generated page.
Page.ClientScript.RegisterClientScriptBlock(GetType(), "newsList", javascript);
This will put the javascript on the page accessible from other javascript functions.