Develop Reference

Node.js C++ add-on trying to execute javascript code with `require` function

I'm maintaining a Node.js add-on in C++.
I need to call a SQL parser (a very non-standard variant of SQL, actually) inside an asynchronously called C++ function, but unfortunately (for me) that parser has been implemented in Javascript using some Node.js libraries (the function require is being used); this is an example of the kind of javascript code I need to execute:
require("./util/SqlParser")("SELECT 1 FROM DUAL").getSyntaxTree()
I've tried writing something like this:
v8::Isolate* isolate = v8::Isolate::GetCurrent();
v8::Local<v8::Context> context = isolate->GetCurrentContext();
v8::TryCatch trycatch;
v8::Local<v8::String> source = v8::String::NewFromUtf8(isolate
,
"require(\"./util/SqlParser\")(\"SELECT 1 FROM DUAL\").getSyntaxTree()"
,
v8::NewStringType::kNormal).ToLocalChecked()
;
v8::Local<v8::Script> script = v8::Script::Compile(context, source).ToLocalChecked();
v8::MaybeLocal<v8::Value> result = script->Run(context);
if( result.IsEmpty() )
{
v8::Local<v8::Value> exc = trycatch.Exception();
if( !exc.IsEmpty() )
{
auto msg = exc->ToString();
if( !msg.IsEmpty() )
throw std::string( *v8::String::Utf8Value(msg) );
}
throw "unknown error in called js function";
}
But unfortunately this doesn't work. The net outcome is the error message "ReferenceError: require is not defined": apparently, my context doesn't know anything about Node.js.
I tried to wrap the above expression in a js function parseSqlText, but this function is equally unknown to the script executor (I get the error message "ReferenceError: parseSqlText is not defined").
My question is: is there any way to get around this?
Any help would be very appreciated. I would be very glad to avoid reimplementing that parser in C++... which at present seems the only viable way to do what I need.

First, v8 is not Node.js, Node.js is built on top of v8.
v8 is a javascript engine
Thus Node.js libraries are not carried with v8 by it-self
The N-API is the answer to question. Although, normally it is used to write native C++ plugins for Node.js. The github thread belwo has some examples. Additionally, the node.js docs for N-API have also been linked.
Github Thread
Node N-API Docs
EDIT: It would seem the work has been done already, here in this repository. The writer made it so it appears to abstract all the prep work of v8 and be one line for you.
#include "node_embed.h"
int main(int argc, char** argv) {
node_context *context = nodeSetup(argc, argv);
if (context) {
nodeExecuteString(context, "let foo = 1", "__init__");
nodeExecuteString(context, "foo = 2", "__init__");
nodeExecuteString(context, "console.log(foo)", "__init__");
return nodeTeardown(context);
} else {
return 12;
}
}
The above is from the repository and will allow you to run that snippet of code.

I suggest that you go find an SQL parser that is already written in C++. I found one immediately in a ten-second search on github.com.
In my opinion, there is zero technical justification for bringing the JavaScript language into this scenario, just to be able to "parse SQL." You will not have to "write from scratch" anything at all. It will just be an object – pluck it "off the shelf," instantiate an instance of it, and use it.

problems on learnyounode exercise 5

Hi guys im trying to solve the exercises from nodeschool, i started with learnyounode, people recomende it a lot for beginners, the exercise question is this:
# LEARN YOU THE NODE.JS FOR MUCH WIN!
## FILTERED LS (Exercise 5 of 13)
Create a program that prints a list of files in a given directory,
filtered by the extension of the files. You will be provided a directory
name as the first argument to your program (e.g. '/path/to/dir/') and a
file extension to filter by as the second argument.
For example, if you get 'txt' as the second argument then you will need to
filter the list to only files that end with .txt. Note that the second
argument will not come prefixed with a '.'.
Keep in mind that the first arguments of your program are not the first
values of the process.argv array, as the first two values are reserved for
system info by Node.
The list of files should be printed to the console, one file per line. You
must use asynchronous I/O.
─────────────────────────────────────────────────────────────────────────────
## HINTS
The fs.readdir() method takes a pathname as its first argument and a
callback as its second. The callback signature is:
function callback (err, list) { /* ... */ }
where list is an array of filename strings.
Documentation on the fs module can be found by pointing your browser here:
file://C:\Users\Filipe\AppData\Roaming\npm\node_modules\learnyounode\node_
apidoc\fs.html
You may also find node's path module helpful, particularly the extname
method.
Documentation on the path module can be found by pointing your browser
here:
file://C:\Users\Filipe\AppData\Roaming\npm\node_modules\learnyounode\node_
apidoc\path.html
─────────────────────────────────────────────────────────────────────────────
» To print these instructions again, run: learnyounode print
» To execute your program in a test environment, run: learnyounode run
program.js
» To verify your program, run: learnyounode verify program.js
» For help run: learnyounode help
i tried to solve the exercise doing this:
var fs = require('fs');
fs.readdir(process.argv[2],process.argv[3],function(err,list){
for(var i = 0;i<list.length;i++)
{
var fileParts = list[i].split(".");
if(fileParts[1] == process.argv[3]){
console.log(list[i] + "\n");
}
}
});
why it doesnt work, i dont know what i am doing wrong, if someone can give me a tip to solve this i appreciate a lot, since the solutions on web are quite bit strange.

Hm, I just tested your code and it actually runs fine for me. Verify returns with solved. Not sure what you are experiencing. Btw.: put in at least some error handling. Below my solution:
const fs = require('fs');
const test = '.' + process.argv[3]
fs.readdir(process.argv[2], (err, data) => {
if (err) {
console.error(err);
} else {
data.filter(file => {
if (file.substring(file.length - test.length) === test) {
console.log(file)
}
})
}
});

Why can't I extend everyone's pocket in nowjs?

I'm trying to provide functions in everyone's pocket of nowjs. I'd like to do so by _.extending everyone's pocket, i.e. everyone.now. For some reason which I cannot understand, _.extend fails to properly provide the function at the client side.
This is my current code:
var _ = require("underscore"),
everyone = require("nowjs").initialize(app);
everyone.now.foo = function() {};
_.extend(everyone.now, {
bar: function() {}
});
console.log(everyone.now.foo); // [Function]
console.log(everyone.now.bar); // undefined
On both the server and client sides, I can do now.foo() just fine. On the other hand, now.bar() fails because now.bar is not defined. This is the case on both the client and server sides. I tried to check for existence at the server side, as shown above on the last line. However, this line logs undefined.
Underscore's extend function (obviously) does work on other objects so I guess it has something to do with the "magical namespace" that nowjs uses.
How come extending doesn't work with everyone.now and how can I get it to work?
Edit 2: I digged some more into proxies. It seems like setting a property on a proxy by passing a variable as its name does not work. I removed my first edit because this testcase is more narrowed down.
Why is this not working? Is this a bug? (Most of the times I ask this myself I know it isn't, but this is really making me clueless...)
var proxy = Proxy.create({
get: function(pr, name) {
console.log("get called");
return null;
},
set: function(pr, name, value) {
console.log("set called");
}
});
var key = "foo";
proxy["foo"] = "bar";
proxy[ key ] = "bar";
proxy["foo"];
proxy[ key ];
Log result:
set called
get called
get called
Apparently, proxy[ key ] = "bar"; does not cause set to be called on the proxy. Why is that?

They posted a blog entry on the NowJS website on how to use node-proxy on Windows, instead of the native V8 implementation using the --harmony_proxies flag.
It appeared that the V8 version that Node currently uses contains several bugs with regard to proxies, which were causing the weird behaviour as outlined in the question. node-proxy, however, is a module that enables proxies (the core of the "magical namespace" of NowJS) without those bugs. (The bugs are fixed in a newer version of V8 as well, but that would require a custom build of Node.)
I just couldn't figure out how to build node-proxy on Windows (it's a .node addon; not a pure JavaScript one). In the above blog post they distributed the compiled module, and everything now works like a charm.
To fix:
Download the compiled module
Extract the folder to the node_modules folder and rename it to now
Don't run Node with the proxy flag
Edit: Node 0.7.0 uses V8 3.8.6 which also solves this issue. Just run with the --harmony flag and remove the reference to node-proxy.

Copy to clipboard in Node.js?

Is there a way you can copy to clipboard in Node.js? Any modules or ideas what so ever? I'm using Node.js on a desktop application. Hopefully that clears up why I want it to be able to achieve this.

For OS X:
function pbcopy(data) {
var proc = require('child_process').spawn('pbcopy');
proc.stdin.write(data); proc.stdin.end();
}
write() can take a buffer or a string. The default encoding for a string will be utf-8.

Check out clipboardy. It lets you copy/paste cross-platform. It is more actively maintained than the copy-paste module mentioned in another answer and it fixes many of that module's issues.
const clipboardy = require('clipboardy');
// Copy
clipboardy.writeSync('🦄');
// Paste
clipboardy.readSync();
//🦄

Here's a module that provide copy and paste functions: https://github.com/xavi-/node-copy-paste
When require("copy-paste").global() is executed, two global functions are added:
> copy("hello") // Asynchronously adds "hello" to clipbroad
> Copy complete
> paste() // Synchronously returns clipboard contents
'hello'
Like many of the other answer mentioned, to copy and paste in node you need to call out to an external program. In the case of node-copy-paste, it calls out to pbcopy/pbpaste (for OSX), xclip (for linux), and clip (for windows).
This module was very helpful when I was doing a lot of work in the REPL for a side project. Needless to say, copy-paste is only a command line utility -- it is not meant for server work.

Shortest way in Windows:
const util = require('util');
require('child_process').spawn('clip').stdin.end(util.inspect('content_for_the_clipboard'));

A clipboard is not inherent to an operating system. It's a construct of whatever window system the operating system happens to be running. So if you wanted this to work on X for example, you would need bindings to Xlib and/or XCB. Xlib bindings for node actually exist: https://github.com/mixu/nwm. Although I'm not sure whether it gives you access to the X clipboard, you might end up writing your own. You'll need separate bindings for windows.
edit: If you want to do something hacky, you could also use xclip:
var exec = require('child_process').exec;
var getClipboard = function(func) {
exec('/usr/bin/xclip -o -selection clipboard', function(err, stdout, stderr) {
if (err || stderr) return func(err || new Error(stderr));
func(null, stdout);
});
};
getClipboard(function(err, text) {
if (err) throw err;
console.log(text);
});

I managed to do so by creating a different application which handles this. It's certainly not the best way, but it works.
I'm on Windows and created a VB.NET application:
Module Module1
Sub Main()
Dim text = My.Application.CommandLineArgs(0)
My.Computer.Clipboard.SetText(text)
Console.Write(text) ' will appear on stdout
End Sub
End Module
Then in Node.js, I used child_process.exec to run the VB.NET application, with the data to be copied passed as a command line argument:
require('child_process').exec(
"CopyToClipboard.exe \"test foo bar\"",
function(err, stdout, stderr) {
console.log(stdout); // to confirm the application has been run
}
);

Mac has a native command line pbcopy for this usecase:
require('child_process').exec(
'echo "test foo bar" | pbcopy',
function(err, stdout, stderr) {
console.log(stdout); // to confirm the application has been run
}
);
Same code for Linux but replace pbcopy with Xclip (apt get install xclip)

I saw there wasn't a solution here or anywhere obvious that worked for Windows, so reposting this one found in the depths of page 2 on Google search, which uses the Windows native command line clip, and avoids any potentially messy usage of command line workarounds such as echo foo | clip like suggested above.
var cp = require("child_process");
var child = cp.spawn('clip');
child.stdin.write(result.join("\n"));
child.stdin.end();
Hope this helps someone!

This is how you would do this on node-ffi for windows, this interacts directly with the native clipboard API from windows. (Which means you can also read/write different clipboard formats)
var {
winapi,
user32,
kernel32,
constants
} = require('#kreijstal/winffiapi')
const GMEM_MOVEABLE=0x2;
var stringbuffer=Buffer.from("hello world"+'\0');//You need a null terminating string because C api.
var hmem=kernel32.GlobalAlloc(GMEM_MOVEABLE,stringbuffer.length);
var lptstr = kernel32.GlobalLock(hmem);
stringbuffer.copy(winapi.ref.reinterpret(lptstr, stringbuffer.length));
kernel32.GlobalUnlock(hmem);
if (!user32.OpenClipboard(0)){
kernel32.GlobalLock(hmem);
kernel32.GlobalFree(hmem);
kernel32.GlobalUnlock(hmem);
throw new Error("couldn't open clipboard");
}
user32.EmptyClipboard();
user32.SetClipboardData(constants.clipboardFormats.CF_TEXT, hmem);
user32.CloseClipboard();

check this zeroclipboard
npm install zeroclipboard

node.js execute system command synchronously

I need in node.js function
result = execSync('node -v');
that will synchronously execute the given command line and return all stdout'ed by that command text.
ps. Sync is wrong. I know. Just for personal use.
UPDATE
Now we have mgutz's solution which gives us exit code, but not stdout! Still waiting for a more precise answer.
UPDATE
mgutz updated his answer and the solution is here :)
Also, as dgo.a mentioned, there is stand-alone module exec-sync
UPDATE 2014-07-30
ShellJS lib arrived. Consider this is the best choice for now.
UPDATE 2015-02-10
AT LAST! NodeJS 0.12 supports execSync natively.
See official docs

Node.js (since version 0.12 - so for a while) supports execSync:
child_process.execSync(command[, options])
You can now directly do this:
const execSync = require('child_process').execSync;
code = execSync('node -v');
and it'll do what you expect. (Defaults to pipe the i/o results to the parent process). Note that you can also spawnSync now.

See execSync library.
It's fairly easy to do with node-ffi. I wouldn't recommend for server processes, but for general development utilities it gets things done. Install the library.
npm install node-ffi
Example script:
var FFI = require("node-ffi");
var libc = new FFI.Library(null, {
"system": ["int32", ["string"]]
});
var run = libc.system;
run("echo $USER");
[EDIT Jun 2012: How to get STDOUT]
var lib = ffi.Library(null, {
// FILE* popen(char* cmd, char* mode);
popen: ['pointer', ['string', 'string']],
// void pclose(FILE* fp);
pclose: ['void', [ 'pointer']],
// char* fgets(char* buff, int buff, in)
fgets: ['string', ['string', 'int','pointer']]
});
function execSync(cmd) {
var
buffer = new Buffer(1024),
result = "",
fp = lib.popen(cmd, 'r');
if (!fp) throw new Error('execSync error: '+cmd);
while(lib.fgets(buffer, 1024, fp)) {
result += buffer.readCString();
};
lib.pclose(fp);
return result;
}
console.log(execSync('echo $HOME'));

Use ShellJS module.
exec function without providing callback.
Example:
var version = exec('node -v').output;

There's an excellent module for flow control in node.js called asyncblock. If wrapping the code in a function is OK for your case, the following sample may be considered:
var asyncblock = require('asyncblock');
var exec = require('child_process').exec;
asyncblock(function (flow) {
exec('node -v', flow.add());
result = flow.wait();
console.log(result); // There'll be trailing \n in the output
// Some other jobs
console.log('More results like if it were sync...');
});

Native Node.js solution is:
const {execSync} = require('child_process');
const result = execSync('node -v'); // 👈 this do the trick
Just be aware that some commands returns Buffer instead of string. And if you need string just add encoding to execSync options:
const result = execSync('git rev-parse HEAD', {encoding: 'utf8'});
... and it is also good to have timeout on sync exec:
const result = execSync('git rev-parse HEAD', {encoding: 'utf8', timeout: 10000});

This is not possible in Node.js, both child_process.spawn and child_process.exec were built from the ground up to be async.
For details see: https://github.com/ry/node/blob/master/lib/child_process.js
If you really want to have this blocking, then put everything that needs to happen afterwards in a callback, or build your own queue to handle this in a blocking fashion, I suppose you could use Async.js for this task.
Or, in case you have way too much time to spend, hack around in Node.js it self.

This is the easiest way I found:
exec-Sync:
https://github.com/jeremyfa/node-exec-sync
(Not to be confused with execSync.)
Execute shell command synchronously. Use this for migration scripts, cli programs, but not for regular server code.
Example:
var execSync = require('exec-sync');
var user = execSync('echo $USER');
console.log(user);

Just to add that even though there are few usecases where you should use them, spawnSync / execFileSync / execSync were added to node.js in these commits: https://github.com/joyent/node/compare/d58c206862dc...e8df2676748e

You can achieve this using fibers. For example, using my Common Node library, the code would look like this:
result = require('subprocess').command('node -v');

my way since 5 years is to have 2 lines ;
const { execSync } = require('child_process');
const shell = (cmd) => execSync(cmd, {encoding: 'utf8'});
Then enjoy:
shell('git remote -v')
or
out = shell('ls -l')
.. so on

I get used to implement "synchronous" stuff at the end of the callback function. Not very nice, but it works. If you need to implement a sequence of command line executions you need to wrap exec into some named function and recursively call it.
This pattern seem to be usable for me:
SeqOfExec(someParam);
function SeqOfExec(somepParam) {
// some stuff
// .....
// .....
var execStr = "yourExecString";
child_proc.exec(execStr, function (error, stdout, stderr) {
if (error != null) {
if (stdout) {
throw Error("Smth goes wrong" + error);
} else {
// consider that empty stdout causes
// creation of error object
}
}
// some stuff
// .....
// .....
// you also need some flag which will signal that you
// need to end loop
if (someFlag ) {
// your synch stuff after all execs
// here
// .....
} else {
SeqOfExec(someAnotherParam);
}
});
};

I had a similar problem and I ended up writing a node extension for this. You can check out the git repository. It's open source and free and all that good stuff !
https://github.com/aponxi/npm-execxi
ExecXI is a node extension written in C++ to execute shell commands
one by one, outputting the command's output to the console in
real-time. Optional chained, and unchained ways are present; meaning
that you can choose to stop the script after a command fails
(chained), or you can continue as if nothing has happened !
Usage instructions are in the ReadMe file. Feel free to make pull requests or submit issues!
EDIT: However it doesn't return the stdout yet... Just outputs them in real-time. It does now. Well, I just released it today. Maybe we can build on it.
Anyway, I thought it was worth to mention it.

you can do synchronous shell operations in nodejs like so:
var execSync = function(cmd) {
var exec = require('child_process').exec;
var fs = require('fs');
//for linux use ; instead of &&
//execute your command followed by a simple echo
//to file to indicate process is finished
exec(cmd + " > c:\\stdout.txt && echo done > c:\\sync.txt");
while (true) {
//consider a timeout option to prevent infinite loop
//NOTE: this will max out your cpu too!
try {
var status = fs.readFileSync('c:\\sync.txt', 'utf8');
if (status.trim() == "done") {
var res = fs.readFileSync("c:\\stdout.txt", 'utf8');
fs.unlinkSync("c:\\stdout.txt"); //cleanup temp files
fs.unlinkSync("c:\\sync.txt");
return res;
}
} catch(e) { } //readFileSync will fail until file exists
}
};
//won't return anything, but will take 10 seconds to run
console.log(execSync("sleep 10"));
//assuming there are a lot of files and subdirectories,
//this too may take a while, use your own applicable file path
console.log(execSync("dir /s c:\\usr\\docs\\"));
EDIT - this example is meant for windows environments, adjust for your own linux needs if necessary

I actually had a situation where I needed to run multiple commands one after another from a package.json preinstall script in a way that would work on both Windows and Linux/OSX, so I couldn't rely on a non-core module.
So this is what I came up with:
#cmds.coffee
childproc = require 'child_process'
exports.exec = (cmds) ->
next = ->
if cmds.length > 0
cmd = cmds.shift()
console.log "Running command: #{cmd}"
childproc.exec cmd, (err, stdout, stderr) ->
if err? then console.log err
if stdout? then console.log stdout
if stderr? then console.log stderr
next()
else
console.log "Done executing commands."
console.log "Running the follows commands:"
console.log cmds
next()
You can use it like this:
require('./cmds').exec ['grunt coffee', 'nodeunit test/tls-config.js']
EDIT: as pointed out, this doesn't actually return the output or allow you to use the result of the commands in a Node program. One other idea for that is to use LiveScript backcalls. http://livescript.net/