When I run this in bash:
mkfifo im-a-pipe && node -e '
var fs = require("fs")
var childProcess = require("child_process")
console.log("pre-open")
fs.open("im-a-pipe", "w", function(err, fd){
if(err)
throw err
console.log("opened")
})
console.log("post-open")
childProcess.exec("echo wat")
console.log("YOU CAN NOT SEE MEEE")
'
I expect the following output:
pre-open
post-open
YOU CAN NOT SEE MEEE
But instead node waits after printing the first two lines:
pre-open
post-open
I'm thinking that this probably has something to do with the pipe blocking until something opens the other side, but this behavior surprised me.
Am I missing something with how these functions are supposed to operate?
This may be a bug in your operating system or just an older version of Node. Works for me on Mac with Node 4. A co-worker was able to run it on Debian but not until he upgraded to Node 8.
That is exactly how pipes work (either named or anonymous). The writer will block until there is a reader at the other end, and a reader would block without a writer. You could say that's a fundamental principle.
Here's a little demo you can do on the command-line. You will need two terminal sessions, I'll call them A and B.
On A:
mkfifo mypipe
ls > mypipe
Session A will block. Now on session B:
cat mypipe
That should display the ls output, and unblock session A.
You can also try this by doing the cat first.
Related
Hi guys im trying to solve the exercises from nodeschool, i started with learnyounode, people recomende it a lot for beginners, the exercise question is this:
# LEARN YOU THE NODE.JS FOR MUCH WIN!
## FILTERED LS (Exercise 5 of 13)
Create a program that prints a list of files in a given directory,
filtered by the extension of the files. You will be provided a directory
name as the first argument to your program (e.g. '/path/to/dir/') and a
file extension to filter by as the second argument.
For example, if you get 'txt' as the second argument then you will need to
filter the list to only files that end with .txt. Note that the second
argument will not come prefixed with a '.'.
Keep in mind that the first arguments of your program are not the first
values of the process.argv array, as the first two values are reserved for
system info by Node.
The list of files should be printed to the console, one file per line. You
must use asynchronous I/O.
─────────────────────────────────────────────────────────────────────────────
## HINTS
The fs.readdir() method takes a pathname as its first argument and a
callback as its second. The callback signature is:
function callback (err, list) { /* ... */ }
where list is an array of filename strings.
Documentation on the fs module can be found by pointing your browser here:
file://C:\Users\Filipe\AppData\Roaming\npm\node_modules\learnyounode\node_
apidoc\fs.html
You may also find node's path module helpful, particularly the extname
method.
Documentation on the path module can be found by pointing your browser
here:
file://C:\Users\Filipe\AppData\Roaming\npm\node_modules\learnyounode\node_
apidoc\path.html
─────────────────────────────────────────────────────────────────────────────
» To print these instructions again, run: learnyounode print
» To execute your program in a test environment, run: learnyounode run
program.js
» To verify your program, run: learnyounode verify program.js
» For help run: learnyounode help
i tried to solve the exercise doing this:
var fs = require('fs');
fs.readdir(process.argv[2],process.argv[3],function(err,list){
for(var i = 0;i<list.length;i++)
{
var fileParts = list[i].split(".");
if(fileParts[1] == process.argv[3]){
console.log(list[i] + "\n");
}
}
});
why it doesnt work, i dont know what i am doing wrong, if someone can give me a tip to solve this i appreciate a lot, since the solutions on web are quite bit strange.
Hm, I just tested your code and it actually runs fine for me. Verify returns with solved. Not sure what you are experiencing. Btw.: put in at least some error handling. Below my solution:
const fs = require('fs');
const test = '.' + process.argv[3]
fs.readdir(process.argv[2], (err, data) => {
if (err) {
console.error(err);
} else {
data.filter(file => {
if (file.substring(file.length - test.length) === test) {
console.log(file)
}
})
}
});
I'm pretty new to node.js and it seems fairly easy to use but when it comes to getting a value using the command line and returning that value to be used in another package or .js, it seems harder than I expected.
Long story short, I've used a npm package (akamai-ccu-purge), to enter a file to purge on the akamai network successfully.
I want to make it more dynamic though by prompting the user to enter the file they want purged and then using that in the akamai package.
After making a few tries using var stdin = process.openStdin(); I actually found another npm package called Prompt that seemed to be easier. Both ways seem to have the same problem though.
Node doesn't seem to want to stop for the input. It seems to want to automatically make the purge without waiting for input even though I've called that module first. It actually gets to the point where I should enter the file but it doesn't wait.
I am definitely missing something in my understanding or usage here, what am I doing wrong?
My code so far is:
var purgeUrl = require('./getUrl2');
var PurgerFactory = require('../../node_modules/akamai-ccu-purge/index'); // this is the directory where the index.js of the node module was installed
// area where I placed the authentication tokens
var config = {
clientToken: //my tokens and secrets akamai requires
};
// area where urls are placed. More than one can be listed with comma separated values
var objects = [
purgeUrl // I am trying to pull this from the getUrl2 module
];
// Go for it!
var Purger = PurgerFactory.create(config);
Purger.purgeObjects(objects, function(err, res) {
console.log('------------------------');
console.log('Purge Result:', res.body);
console.log('------------------------');
Purger.checkPurgeStatus(res.body.progressUri, function(err, res) {
console.log('Purge Status', res.body);
console.log('------------------------');
Purger.checkQueueLength(function(err, res) {
console.log('Queue Length', res.body);
console.log('------------------------');
});
});
});
The getUrl2 module looks like this:
var prompt = require('../../node_modules/prompt');
//
// Start the prompt
//
prompt.start();
//
// Get property from the user
//
prompt.get(['newUrl'], function (err, result) {
//
// Log the results.
//
console.log('Command-line input received:');
console.log(' http://example.com/custom/' + result.newUrl);
var purgeUrl = 'http://example.com/custom/' + result.newUrl;
console.log(purgeUrl);
module.exports = purgeUrl;
});
Thanks again for the help!
I would probably just allow getURL2 to expose a method that will be invoked in the main module. For example:
// getURL2
var prompt = require('../../node_modules/prompt');
module.exports = {
start: function(callback) {
prompt.start();
prompt.get(['newUrl'], function (err, result) {
// the callback is defined in your main module
return callback('http://example.com/custom/' + result.newUrl);
});
}
}
Then in your main module:
require('./getUrl2').start(function(purgeURL) {
// do stuff with the purgeURL defined in the other module
});
The implementation may differ, but conceptually, you need to make your second module, which requires some sort of input from the user, happen as a result of that input. Callbacks are a common way to do this (as are Promises). However, as prompt is not necessarily exposing a method that would necessitate a Promise, you can do it with plain old callbacks.
You might also want to search around for articles on writing command line tools (sometimes referenced as CLIs) or command line apps with Node. I found the following article to be helpful when trying to figure this out myself:
http://javascriptplayground.com/blog/2015/03/node-command-line-tool/
Also, the command-line-args module worked well for me (though there's a number of other modules out there to choose from):
https://www.npmjs.com/package/command-line-args
Good luck!
This question already has answers here:
Node.js spawn with colors?
(2 answers)
Closed 7 years ago.
I have a mocha file full of Selenium tests. When I run mocha from the command line like normal, I get this nice formatted and colorized output thanks to the colors module.
This looks great and works wonderfully, but running manually only runs the tests against a single environment. To run the tests in multiple environments in parallel, Sauce Labs (Selenium cloud hosting) recommends spawning mocha child processes. I built this into a Gulp task in our project.
gulp.task('test', function () {
var targets = [
'windows.chrome',
'windows.firefox',
'windows.ie',
'mac.chrome',
'mac.firefox',
'mac.safari',
'linux.chrome',
'linux.firefox'
];
function run_mocha(target, done) {
var env = Object.assign({}, process.env);
env.TARGET = target;
var mocha = exec('mocha ./test/test-runner.js', {env: env}, done);
['stdout', 'stderr'].forEach((stream) =>
mocha[stream].on('data', (data) => process[stream].write(`${target}:${data}`))
);
}
var jobs = targets.map(
(target) => new Promise(
(resolve, reject) => run_mocha(target, resolve)
)
);
return Promise.all(jobs).then(() => {
console.log('ALL SUCCESSFUL');
});
});
This works great but the output completely loses the colorization. It was also injecting superfluous newlines but I was able to fix that by swapping out console.log and console.error for process.stdout.write and process.stderr.write.
You can see that lines printed from gulp are colorized and work fine, but the minute it spawns the child processes any lines printed from there lose their color. It's not the end of the world but now I'm curious what is going on and how to fix it. I've learned a bit about ANSI escape codes but I'm still very confused about what's going on. Any suggestions?
So I found this question:
Node.js spawn with colors?
It looks like it's an issue with Mocha, where it's detecting that its output is not going to be stdout. You'll need to specifically enable colors:
exec('mocha ./test/test-runner.js --colors')
Is there a way you can copy to clipboard in Node.js? Any modules or ideas what so ever? I'm using Node.js on a desktop application. Hopefully that clears up why I want it to be able to achieve this.
For OS X:
function pbcopy(data) {
var proc = require('child_process').spawn('pbcopy');
proc.stdin.write(data); proc.stdin.end();
}
write() can take a buffer or a string. The default encoding for a string will be utf-8.
Check out clipboardy. It lets you copy/paste cross-platform. It is more actively maintained than the copy-paste module mentioned in another answer and it fixes many of that module's issues.
const clipboardy = require('clipboardy');
// Copy
clipboardy.writeSync('🦄');
// Paste
clipboardy.readSync();
//🦄
Here's a module that provide copy and paste functions: https://github.com/xavi-/node-copy-paste
When require("copy-paste").global() is executed, two global functions are added:
> copy("hello") // Asynchronously adds "hello" to clipbroad
> Copy complete
> paste() // Synchronously returns clipboard contents
'hello'
Like many of the other answer mentioned, to copy and paste in node you need to call out to an external program. In the case of node-copy-paste, it calls out to pbcopy/pbpaste (for OSX), xclip (for linux), and clip (for windows).
This module was very helpful when I was doing a lot of work in the REPL for a side project. Needless to say, copy-paste is only a command line utility -- it is not meant for server work.
Shortest way in Windows:
const util = require('util');
require('child_process').spawn('clip').stdin.end(util.inspect('content_for_the_clipboard'));
A clipboard is not inherent to an operating system. It's a construct of whatever window system the operating system happens to be running. So if you wanted this to work on X for example, you would need bindings to Xlib and/or XCB. Xlib bindings for node actually exist: https://github.com/mixu/nwm. Although I'm not sure whether it gives you access to the X clipboard, you might end up writing your own. You'll need separate bindings for windows.
edit: If you want to do something hacky, you could also use xclip:
var exec = require('child_process').exec;
var getClipboard = function(func) {
exec('/usr/bin/xclip -o -selection clipboard', function(err, stdout, stderr) {
if (err || stderr) return func(err || new Error(stderr));
func(null, stdout);
});
};
getClipboard(function(err, text) {
if (err) throw err;
console.log(text);
});
I managed to do so by creating a different application which handles this. It's certainly not the best way, but it works.
I'm on Windows and created a VB.NET application:
Module Module1
Sub Main()
Dim text = My.Application.CommandLineArgs(0)
My.Computer.Clipboard.SetText(text)
Console.Write(text) ' will appear on stdout
End Sub
End Module
Then in Node.js, I used child_process.exec to run the VB.NET application, with the data to be copied passed as a command line argument:
require('child_process').exec(
"CopyToClipboard.exe \"test foo bar\"",
function(err, stdout, stderr) {
console.log(stdout); // to confirm the application has been run
}
);
Mac has a native command line pbcopy for this usecase:
require('child_process').exec(
'echo "test foo bar" | pbcopy',
function(err, stdout, stderr) {
console.log(stdout); // to confirm the application has been run
}
);
Same code for Linux but replace pbcopy with Xclip (apt get install xclip)
I saw there wasn't a solution here or anywhere obvious that worked for Windows, so reposting this one found in the depths of page 2 on Google search, which uses the Windows native command line clip, and avoids any potentially messy usage of command line workarounds such as echo foo | clip like suggested above.
var cp = require("child_process");
var child = cp.spawn('clip');
child.stdin.write(result.join("\n"));
child.stdin.end();
Hope this helps someone!
This is how you would do this on node-ffi for windows, this interacts directly with the native clipboard API from windows. (Which means you can also read/write different clipboard formats)
var {
winapi,
user32,
kernel32,
constants
} = require('#kreijstal/winffiapi')
const GMEM_MOVEABLE=0x2;
var stringbuffer=Buffer.from("hello world"+'\0');//You need a null terminating string because C api.
var hmem=kernel32.GlobalAlloc(GMEM_MOVEABLE,stringbuffer.length);
var lptstr = kernel32.GlobalLock(hmem);
stringbuffer.copy(winapi.ref.reinterpret(lptstr, stringbuffer.length));
kernel32.GlobalUnlock(hmem);
if (!user32.OpenClipboard(0)){
kernel32.GlobalLock(hmem);
kernel32.GlobalFree(hmem);
kernel32.GlobalUnlock(hmem);
throw new Error("couldn't open clipboard");
}
user32.EmptyClipboard();
user32.SetClipboardData(constants.clipboardFormats.CF_TEXT, hmem);
user32.CloseClipboard();
check this zeroclipboard
npm install zeroclipboard
I need in node.js function
result = execSync('node -v');
that will synchronously execute the given command line and return all stdout'ed by that command text.
ps. Sync is wrong. I know. Just for personal use.
UPDATE
Now we have mgutz's solution which gives us exit code, but not stdout! Still waiting for a more precise answer.
UPDATE
mgutz updated his answer and the solution is here :)
Also, as dgo.a mentioned, there is stand-alone module exec-sync
UPDATE 2014-07-30
ShellJS lib arrived. Consider this is the best choice for now.
UPDATE 2015-02-10
AT LAST! NodeJS 0.12 supports execSync natively.
See official docs
Node.js (since version 0.12 - so for a while) supports execSync:
child_process.execSync(command[, options])
You can now directly do this:
const execSync = require('child_process').execSync;
code = execSync('node -v');
and it'll do what you expect. (Defaults to pipe the i/o results to the parent process). Note that you can also spawnSync now.
See execSync library.
It's fairly easy to do with node-ffi. I wouldn't recommend for server processes, but for general development utilities it gets things done. Install the library.
npm install node-ffi
Example script:
var FFI = require("node-ffi");
var libc = new FFI.Library(null, {
"system": ["int32", ["string"]]
});
var run = libc.system;
run("echo $USER");
[EDIT Jun 2012: How to get STDOUT]
var lib = ffi.Library(null, {
// FILE* popen(char* cmd, char* mode);
popen: ['pointer', ['string', 'string']],
// void pclose(FILE* fp);
pclose: ['void', [ 'pointer']],
// char* fgets(char* buff, int buff, in)
fgets: ['string', ['string', 'int','pointer']]
});
function execSync(cmd) {
var
buffer = new Buffer(1024),
result = "",
fp = lib.popen(cmd, 'r');
if (!fp) throw new Error('execSync error: '+cmd);
while(lib.fgets(buffer, 1024, fp)) {
result += buffer.readCString();
};
lib.pclose(fp);
return result;
}
console.log(execSync('echo $HOME'));
Use ShellJS module.
exec function without providing callback.
Example:
var version = exec('node -v').output;
There's an excellent module for flow control in node.js called asyncblock. If wrapping the code in a function is OK for your case, the following sample may be considered:
var asyncblock = require('asyncblock');
var exec = require('child_process').exec;
asyncblock(function (flow) {
exec('node -v', flow.add());
result = flow.wait();
console.log(result); // There'll be trailing \n in the output
// Some other jobs
console.log('More results like if it were sync...');
});
Native Node.js solution is:
const {execSync} = require('child_process');
const result = execSync('node -v'); // 👈 this do the trick
Just be aware that some commands returns Buffer instead of string. And if you need string just add encoding to execSync options:
const result = execSync('git rev-parse HEAD', {encoding: 'utf8'});
... and it is also good to have timeout on sync exec:
const result = execSync('git rev-parse HEAD', {encoding: 'utf8', timeout: 10000});
This is not possible in Node.js, both child_process.spawn and child_process.exec were built from the ground up to be async.
For details see: https://github.com/ry/node/blob/master/lib/child_process.js
If you really want to have this blocking, then put everything that needs to happen afterwards in a callback, or build your own queue to handle this in a blocking fashion, I suppose you could use Async.js for this task.
Or, in case you have way too much time to spend, hack around in Node.js it self.
This is the easiest way I found:
exec-Sync:
https://github.com/jeremyfa/node-exec-sync
(Not to be confused with execSync.)
Execute shell command synchronously. Use this for migration scripts, cli programs, but not for regular server code.
Example:
var execSync = require('exec-sync');
var user = execSync('echo $USER');
console.log(user);
Just to add that even though there are few usecases where you should use them, spawnSync / execFileSync / execSync were added to node.js in these commits: https://github.com/joyent/node/compare/d58c206862dc...e8df2676748e
You can achieve this using fibers. For example, using my Common Node library, the code would look like this:
result = require('subprocess').command('node -v');
my way since 5 years is to have 2 lines ;
const { execSync } = require('child_process');
const shell = (cmd) => execSync(cmd, {encoding: 'utf8'});
Then enjoy:
shell('git remote -v')
or
out = shell('ls -l')
.. so on
I get used to implement "synchronous" stuff at the end of the callback function. Not very nice, but it works. If you need to implement a sequence of command line executions you need to wrap exec into some named function and recursively call it.
This pattern seem to be usable for me:
SeqOfExec(someParam);
function SeqOfExec(somepParam) {
// some stuff
// .....
// .....
var execStr = "yourExecString";
child_proc.exec(execStr, function (error, stdout, stderr) {
if (error != null) {
if (stdout) {
throw Error("Smth goes wrong" + error);
} else {
// consider that empty stdout causes
// creation of error object
}
}
// some stuff
// .....
// .....
// you also need some flag which will signal that you
// need to end loop
if (someFlag ) {
// your synch stuff after all execs
// here
// .....
} else {
SeqOfExec(someAnotherParam);
}
});
};
I had a similar problem and I ended up writing a node extension for this. You can check out the git repository. It's open source and free and all that good stuff !
https://github.com/aponxi/npm-execxi
ExecXI is a node extension written in C++ to execute shell commands
one by one, outputting the command's output to the console in
real-time. Optional chained, and unchained ways are present; meaning
that you can choose to stop the script after a command fails
(chained), or you can continue as if nothing has happened !
Usage instructions are in the ReadMe file. Feel free to make pull requests or submit issues!
EDIT: However it doesn't return the stdout yet... Just outputs them in real-time. It does now. Well, I just released it today. Maybe we can build on it.
Anyway, I thought it was worth to mention it.
you can do synchronous shell operations in nodejs like so:
var execSync = function(cmd) {
var exec = require('child_process').exec;
var fs = require('fs');
//for linux use ; instead of &&
//execute your command followed by a simple echo
//to file to indicate process is finished
exec(cmd + " > c:\\stdout.txt && echo done > c:\\sync.txt");
while (true) {
//consider a timeout option to prevent infinite loop
//NOTE: this will max out your cpu too!
try {
var status = fs.readFileSync('c:\\sync.txt', 'utf8');
if (status.trim() == "done") {
var res = fs.readFileSync("c:\\stdout.txt", 'utf8');
fs.unlinkSync("c:\\stdout.txt"); //cleanup temp files
fs.unlinkSync("c:\\sync.txt");
return res;
}
} catch(e) { } //readFileSync will fail until file exists
}
};
//won't return anything, but will take 10 seconds to run
console.log(execSync("sleep 10"));
//assuming there are a lot of files and subdirectories,
//this too may take a while, use your own applicable file path
console.log(execSync("dir /s c:\\usr\\docs\\"));
EDIT - this example is meant for windows environments, adjust for your own linux needs if necessary
I actually had a situation where I needed to run multiple commands one after another from a package.json preinstall script in a way that would work on both Windows and Linux/OSX, so I couldn't rely on a non-core module.
So this is what I came up with:
#cmds.coffee
childproc = require 'child_process'
exports.exec = (cmds) ->
next = ->
if cmds.length > 0
cmd = cmds.shift()
console.log "Running command: #{cmd}"
childproc.exec cmd, (err, stdout, stderr) ->
if err? then console.log err
if stdout? then console.log stdout
if stderr? then console.log stderr
next()
else
console.log "Done executing commands."
console.log "Running the follows commands:"
console.log cmds
next()
You can use it like this:
require('./cmds').exec ['grunt coffee', 'nodeunit test/tls-config.js']
EDIT: as pointed out, this doesn't actually return the output or allow you to use the result of the commands in a Node program. One other idea for that is to use LiveScript backcalls. http://livescript.net/