Develop Reference

Trouble with HarmonyImportSpecifierDependency

I am writing plugin, which inlines some code. And I have troubles with internal webpack plugin
Intro
My aim is to transpile this code
// a.js
export const getFoo = x => x.foo;
// b.js
import {getFoo} from './a.js';
export const func = x => getFoo(x);
to this one
// b.js
export const func = x => x.foo;
Using this comment I implemented such logic
Hook on compiler.hooks.beforeCompile and find all imports of transpiled modules with parser.hooks.importSpecifier and
Then using the same compiler hook, I find all CallExpressions and remember all functions which will be transpiled and their position
In compiler.hooks.compilation in compilation.hooks.buildModule I add template dependency which will replace CallExpressions with MemberExpression
The problem
Webpack has internal plugin HarmonyImportSpecifierDependency. Apparently, it does the same logic replacing
import {getFoo} from './a.js';
export const func = x => x.foo;
with
"use strict";
/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "a", function() { return func; });
/* harmony import */ var _getters_uno__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(0);
const func = x => {
// NEXT LINE IS THE RESULT OF `HarmonyImportSpecifierDependency` WORK
return Object(_getters_uno__WEBPACK_IMPORTED_MODULE_0__[/* getFoo */ "e"])(x);
};
My problem is that both plugins (mine and HarmonyImportSpecifierDependency) remembers the original source and original places of CallExpressions. The result of both plugins is invalid code:
return Object(_getters_uno__WEBPACK_IMPORTED_MODULE_0__[/* getFoo */ "e"])x.foo);
The question
Is there a way how to give HarmonyImportSpecifierDependency the result of my plugin? I've tried to reparse module in compilation phase, but whith no success

I solved my problem with patching replacements object in source in succeedBuild hook
source.replacements = source.replacements
.filter(r => !r.content.includes(`/* ${callExpression.callee.name} */`));
Still wonder if there is a better solution

Webpack doesn't inline a const from another file

I have a file with a few time calculations.
In order to "DRY", at its top there's a
const HOUR_MS = 60 * 60 * 1000
In the transpiled output each HOUR_MS occurrence is inlined with 36e5.
So far, so good ^_^
BUT! If I extract this const to its own file, because I want to reuse it in more places, this no longer works.
Instead, the transpiled output now has references to that const (e.g. r.HOUR_MS) which means it's not as minified as it can be.
Is this a deliberate behavior (which maybe I can suppress with some flag) or is it an optimization oversight? (was about to report it, but their GitHub bug template suggested I'd ask here first)

This phenomenon is problematic, but due to the multiplicity of layers (2) working together to generate output, this feat is not feasible.
This is because import and export behave differently than CommonJS.
No matter what, import and export are by reference (even with primitive variables), whereas CommonJS does not manipulate the exported values (therefore, primitive values won't be referenced).
Proof that import and export are by reference:
1.ts
export let number = 10;
export function increase() {
number++;
}
2.ts
import {number, increase} from './1';
console.log(number);
increase();
console.log(number);
the log will print 10, 11
Layer 1: TypeScript Transpiler
const number = 10;
console.log(number);
Will transpile to:
const number = 10;
console.log(number);
But
export const number = 10;
console.log(number);
Will transpile to: (This is where the problem begins)
Object.defineProperty(exports, "__esModule", { value: true });
exports.number = 10;
console.log(exports.number);
Layer 2: uglify-es
const number = 10;
console.log(number);
Will compress to
console.log(10);
But
Object.defineProperty(exports, "__esModule", { value: true });
exports.number = 10;
console.log(exports.number);
Will compress to: (This is where the problem ends)
Object.defineProperty(exports, "__esModule", { value: !0 });
exports.number = 10;
console.log(exports.number);
And this is why this optimization will never work without any common standard between the layers.

mocha test (and babel) using global variables

I am writing library using es6, transpiling it with babel via webpack and npm.
The problem is, that my lib is dependent on some code, that I can not change but need to use. I don't know how to load var stuff (from the following example) in my tests so that it is visible for the module.
See the example:
external-stuff.js - this one can not be changed and is loaded before my lib is loaded on prod env.
var stuff = {
get some() { return "some"; }
get stuff() { return "stuff"; }
}
some-module.js - this is one of the modules in the library
export class foo {
static get whatever() { return stuff.some; }
static get whichever() { return stuff.stuff; }
}
test
import {foo} from "../src/foo.js";
let assert = require('assert');
describe('foo', function() {
describe('#whatever()', function() {
it("should do some", function () {
assert.equals(foo.whatever(), "some");
});
});
});
If I run this I get "ReferenceError: stuff is not defined"
I already tried to define "stuff" in before() hook, but no success.

In the end I found a solution that's 'good enough'. I am not sure it would be sufficient for some advanced library though.
I have created file called globals.js
var g = typeof(window) === 'undefined' ? global : window;
// Dependencies - add as many global stuff as needed
g.stuff= typeof(stuff) === 'undefined' ? {} : stuff;
And I import this 'es6module' at the beginning of test
import * as globals from "../lib/global/globals"
import {foo} from "../src/foo.js";
And then I use node-import npm module with which I load the global to the tests in beforeEach hook.
beforeEach(function () {
global.stuff = imports.module("lib/global/stuff.js").stuff;
});
This is perfect for unit testing because I can also mock stuff. And its even more awesome because this way I have a place where I 'define' global dependencies. It would be nice to improve on it and make it per es6modul dependencies... and build on it something fancy... ya know.. dependency injection.
complete test
require('node-import'); // +
import * as globals from "../lib/global/globals"; // +
import {foo} from "../src/foo.js";
let assert = require('assert');
describe('foo', function() {
beforeEach(function () { // +
global.stuff = imports.module("lib/global/stuff.js").stuff; // +
}); // +
describe('#whatever()', function() {
it("should do some", function () {
assert.equals(foo.whatever(), "some");
});
});
});

Calling a JavaScript function from C

Is it possible to call a JavaScript function (written over, say, node.js) from C?
(There are plenty of tutorials on calling C/C++ from node.js. But not the other way around.)

I've been working on the same problem recently, and found a tractable solution using QuickJS and esbuild. It's not the prettiest, but it works quite well!
To call JS from C, the general process is:
Get QuickJS and esbuild
esbuild your desired library/script into an ESM format using CommonJS. This will output one big script with all needed dependencies included.
output=/path/to/esbuild/output
npx esbuild --bundle /path/to/original/node-library --format=esm --outfile="$output"
Patch the output of esbuild to make it compatible with QuickJS:
sed -i 's/Function(\"return this\")()/globalThis/g' $output
sed -i 's#export default#//export default#g' $output
Load the script text into an object file using your linker:
ld -r -b binary my_obj_file.o $output
Depending on your compiler, this will automatically create 3 symbols in the object file:
- name_start
- name_end
- name_size
name in this context is automatically generated from the filename you provided as the last argument to ld. It replaces all non-alphanumeric characters with underscores, so my-cool-lib.mjs gives a name of my_cool_lib_mjs.
You can use ld_magic.h (here) for a cross platform way to access this data from your C code.
After the object file is generated, you should see the patched esbuild output if you run strings:
% strings foo_lib_js.o
var __getOwnPropNames = Object.getOwnPropertyNames;
var __commonJS = (cb, mod) => function __require() {
return mod || (0, cb[__getOwnPropNames(cb)[0]])((mod = { exports: {} }).exports, mod), mod.exports;
// src/foo.js
var require_foo = __commonJS({
"src/foo.js"(exports, module) {
function foo(bar, baz) {
return bar + baz;
}
module.exports = foo;
//export default require_foo();
_binary_my_foo_lib_mjs_end
_binary_my_foo_lib_mjs_start
_binary_my_foo_lib_mjs_size
.symtab
.strtab
.shstrtab
.data
Link the object file into your binary:
gcc my_obj_file.o <other object files> -o my_static_binary
You can also link the object file into a shared library, for use in other applications:
gcc -shared -o my_shared_library.so my_obj_file.o <other object files>
The source of this repo shows how to do this with a CMake project.
How to actually call the JS functions
Let's say you have a NodeJS library with a function you want to call from C:
// Let's say this lives in foo.js, and esbuild output goes in my-lib-foo.mjs
function foo(bar, baz) {
return bar + baz
}
module.exports = foo;
esbuild creates a series of require_thing() functions, which can be used to get the underlying thing(param1, param2...) function object which you can make calls with.
A simple loader in QuickJS looks like this:
JSValue commonjs_module_data_to_function(JSContext *ctx, const uint8_t *data, size_t data_length, const char *function_name)
{
JSValue result = JS_UNDEFINED;
char * module_function_name = NULL;
// Make sure you properly free all JSValues created from this procedure
if(data == NULL) {
goto done;
}
/**
* To pull the script objects, including require_thing() etc, into global scope,
* load the patched NodeJS script from the object file embedded in the binary
*/
result = JS_Eval(ctx, data, data_length, "<embed>", JS_EVAL_TYPE_GLOBAL);
if(JS_IsException(result)) {
printf("failed to parse module function '%s'\n", function_name);
goto cleanup_fail;
}
JSValue global = JS_GetGlobalObject(ctx);
/**
* Automatically create the require_thing() function name
*/
asprintf(&module_function_name, "require_%s", function_name);
JSValue module = JS_GetPropertyStr(ctx, global, module_function_name);
if(JS_IsException(module)) {
printf("failed to find %s module function\n", function_name);
goto cleanup_fail;
}
result = JS_Call(ctx, module, global, 0, NULL);
if(JS_IsException(result)) {
goto cleanup_fail;
}
/* don't lose the object we've built by passing over failure case */
goto done;
cleanup_fail:
/* nothing to do, cleanup context elsewhere */
result = JS_UNDEFINED;
done:
free(module_function_name);
return result;
}
If you wanted to, for example, get the foo(bar, baz) function mentioned above, you would write a function like this:
#include <stdio.h>
#include <inttypes.h>
// A simple helper for getting a JSContext
JSContext * easy_context(void)
{
JSRuntime *runtime = JS_NewRuntime();
if(runtime == NULL) {
puts("unable to create JS Runtime");
goto cleanup_content_fail;
}
JSContext *ctx = JS_NewContext(runtime);
if(ctx == NULL) {
puts("unable to create JS context");
goto cleanup_runtime_fail;
}
return ctx;
cleanup_runtime_fail:
free(runtime);
cleanup_content_fail:
return NULL;
}
int call_foo(int bar, int baz)
{
JSContext *ctx = easy_context();
JSValue global = JS_GetGlobalObject(ctx);
/**
* esbuild output was to my-foo-lib.mjs, so symbols will be named with my_foo_lib_mjs
*/
JSValue foo_fn = commonjs_module_data_to_function(
ctx
, _binary_my_foo_lib_mjs_start // gcc/Linux-specific naming
, _binary_my_foo_lib_mjs_size
, "foo"
);
/**
* To create more complex objects as arguments, use
* JS_ParseJSON(ctx, json_str, strlen(json_str), "<input>");
* You can also pass callback functions by loading them just like we loaded foo_fn
*/
JSValue args[] = {
JS_NewInt32(ctx, bar),
JS_NewInt32(ctx, baz)
};
JSValue js_result = JS_Call(ctx
, foo_fn
, global
, sizeof(args)/sizeof(*args)
, args
);
int32_t c_result = -1;
JS_ToInt32(ctx, &c_result, js_result);
return c_result;
}
Check out a minimal example project using CMake here: https://github.com/ijustlovemath/jescx/blob/master/README.md

You could use Emscripten.
Emscripten is an LLVM-to-JavaScript compiler.
It takes LLVM bitcode - which can be generated from C/C++, using llvm-gcc (DragonEgg) or clang, or any other language that can be converted into LLVM - and compiles that into JavaScript, which can be run on the web (or anywhere else JavaScript can run).
Also see this: How to execute Javascript function in C++

How to call a C++ method from javascript

I have a C++ method (which role is killing some processes). She needs 2 parameters : a hostname and a port.
On the other hand, I am developing a web-application (using Nodejs and AngularJS), running on Google Chrome.
When I click on a button through the browser, I would like to be able to call the C++ function, through my app.js file.
I haven't found how to "bind" javascript with C++.
EDIT : I think this link could be very useful
How can I use a C++ library from node.js?

You can use Google's V8. V8 is Google's open source JavaScript engine.
V8 can run standalone, or can be embedded into any C++ application.
http://code.google.com/p/v8/
Following example from github demonstrates, binding a C++ class with Google V8.
v8_wrap_class.cpp - Author is nicholas
/*
* v8_wrap_class.cpp
*
* Created on: 14/01/2013
* Author: nicholas
* License: public domain
*/
#include <v8.h>
#include <cstdio>
#include <string>
#include <stdexcept>
#include <memory>
using namespace v8;
/*
var Simple = function(v)
{
this.value = v;
}
Simple.prototype.func = function()
{
alert(this.value);
}
var obj = new Simple(4);
obj.func();
*/
struct Simple
{
double value;
Simple(double v)
: value(v)
{
fprintf(stderr, "Simple::ctor\n");
}
void func()
{
fprintf(stderr, "Simple::func(%f)\n", value);
}
~Simple()
{
fprintf(stderr, "Simple::dtor\n");
}
};
namespace js
{
/*
* Retrieve the c++ object pointer from the js object
*/
template <typename T>
T* unwrap(const Arguments& args)
{
auto self = args.Holder();
auto wrap = Local<External>::Cast(self->GetInternalField(0));
return static_cast<T*>(wrap->Value());
}
/*
* Construct a new c++ object and wrap it in a js object
*/
template <typename T, typename... Args>
Persistent<Object> make_object(Handle<Object> object, Args&&... args)
{
auto x = new T(std::forward<Args>(args)...);
auto obj = Persistent<Object>::New(object);
obj->SetInternalField(0, External::New(x));
obj.MakeWeak(x, [](Persistent<Value> obj, void* data)
{
auto x = static_cast<T*>(data);
delete x;
obj.Dispose();
obj.Clear();
});
return obj;
}
}
void bind_Simple(Local<Object> global)
{
// Name the class in js
auto name = String::NewSymbol("Simple");
auto tpl = FunctionTemplate::New([&](const Arguments& args) -> Handle<Value>
{
if (!args.IsConstructCall())
return ThrowException(String::New("Cannot call constructor as function"));
HandleScope scope;
// Read and pass constructor arguments
js::make_object<Simple>(args.This(), args[0]->NumberValue());
return args.This();
});
tpl->SetClassName(name);
tpl->InstanceTemplate()->SetInternalFieldCount(1);
auto prototype = tpl->PrototypeTemplate();
// Add object properties to the prototype
// Methods, Properties, etc.
prototype->Set(String::New("func"), FunctionTemplate::New([](const Arguments& args) -> Handle<Value>
{
auto s = js::unwrap<Simple>(args);
s->func();
return {};
})->GetFunction());
auto constructor = Persistent<Function>::New(tpl->GetFunction());
global->Set(name, constructor);
}
int main()
{
std::string js_source = R"(
for(var i = 0; i < 1000; ++i)
{
var s = new Simple(4);
s.value = 5;
s.func();
}
)";
/*
* This code is mostly uninteresting.
* Just run the vm with the script provided.
*/
{
HandleScope handle_scope;
Handle<ObjectTemplate> global_template = ObjectTemplate::New();
Persistent<Context> context = Context::New(0, global_template);
Context::Scope context_scope(context);
auto global = context->Global();
// Wrap the class and bind to the global scope.
bind_Simple(global);
{
HandleScope handle_scope;
TryCatch trycatch;
Local<String> source = String::New(js_source.c_str(), js_source.size());
Local<Script> script = Script::Compile(source);
if (script.IsEmpty())
{
Handle<Value> exception = trycatch.Exception();
String::AsciiValue exception_str(exception);
throw std::runtime_error(*exception_str);
}
Local<Value> result = script->Run();
if (result.IsEmpty())
{
Local<Value> exception = trycatch.Exception();
String::AsciiValue exception_str(exception);
throw std::runtime_error(*exception_str);
}
}
context.Dispose();
context.Clear();
}
// Run the GC until there is nothing to reclaim.
while (!V8::IdleNotification())
;
return 0;
}

This answer gives four appraoches to using C++ in javascript. The methods shown try to keep the original C++ in a standard and simple C++ implementation.
Two methods using WASM and two using SWIG and JRPC are used to give examples and ideas on executing C++ in javascript. In detail, this answer gives one example for executing in the browser and one for executing in nodejs using WASM. The end of this answer also lists two other ways to execute C++ in javascript, one of which calls nodejs from the browser which is slightly more complex but possible using jrpc-oo.
If you want to execute in the browser or nodejs, then you can compile your C++ to WASM and load that module in the browser or nodejs, executing the C++ there. This WASM repo exemplifies how to do that. I will expand the key code in the WASM repo here.
Create some C++ code and declare your WASM binding, for example (from the files include/Test.H and src/Test.C) :
class Test {
public:
void sayHello(){
printf("Hi, my name is test\n");
}
};
#include <emscripten/bind.h>
EMSCRIPTEN_BINDINGS(Test_ex) {
emscripten::class_<Test>("Test")
.function("sayHello", &Test::sayHello)
;
}
Compile that down and you can now run that in nodejs (from the file nodejs/WASMTestNode.js) :
libWASM = require('./libwasmNode.js');
libWASM().then((mod)=>{
libWASM = mod;
let test = new libWASM.Test;
test.sayHello();
});
In the browser you can use the WASM code as well. To do that firs import the WASM library (from the file webcomponent/libWASM.js) :
import modProm from './libwasm.js';
The create your webcomponent and compile your WASM then execute the C++ method Test::sayHello (from the file webcomponent/libWASM.js) :
import { LitElement } from 'lit';
export class LibWASM extends LitElement {
constructor() {
super();
modProm().then((mod)=>{
this.libWASM = mod; // for rendered wasm that delay
this.WASMReady();
})
}
WASMReady(){
console.log('LibWASM.libWASM module compiled and ready to go.')
let test = new this.libWASM.Test;
test.sayHello();
}
}
This code example is implemented in the reference repo.
Alternatively a third way to do this is to only use C++, SWIG and nodejs from this reference repo.
If you want to execute nodejs from the browser or use a a different method of integrating C++ into nodejs, you can also look at this SWIG and jrpc-oo reference for doing the same thing, but not only in nodejs, also from the browser calling nodejs.
There are also other ways to execute C++ form javascript, however the methods demonstrated in this answer are reasonably straightforward and either rely on WASM binding or SWIG abstraction which leaves your original code as standard C++.