Stream output to stdout after spawning `git clone` - javascript

I'm just trying to run git clone from node and stream the output into stdout just like running from shell normally would, but after using child_process.spawn, I can't get the output to pipe into stdout. Currently I'm using:
child = spawn('git', ['clone', url]);
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);
But I only see "Cloning into 'directory'" message and not the remote messages and "Receiving objects...".
What am I doing wrong?

By default, git clone only shows progress when it's running in a terminal. When it's not running in a terminal progress can be enabled with the --progress argument:
Progress status is reported on the standard error stream by default when it is attached to a terminal, unless -q is specified. This flag forces progress status even if the standard error stream is not directed to a terminal.
However I'm not sure that that will do exactly what you expect. The progress output isn't simple output; values change in place. I'm not exactly sure how nicely that will play with child_process.spawn().
Note also that output is to STDERR, not STDOUT.

Related

Docker pull is not giving real time downloaded-size/total-size when executed through Node.js child process sync

When I run docker pull my-image from my terminal this is how my output looks like.
59e69571f6c7: Pull complete
43da27f69c98: Pull complete
d22174e9eddd: Pull complete
cc0ac48a6d21: Downloading 312.3 MB/2.888 GB
b47aa969d5dc: Download complete
When I run a piece of Node.js code
const { exec } = require('child_process');
exec('docker pull my-image', (error, stdout, stderr) => {
if (error) {
console.error('exec error: ${error}');
return;
}
console.log('stdout: ${stdout}');
console.log('stderr: ${stderr}');
});
My output looks like this.
c49bda5ed612: Waiting
43da27f69c98: Verifying Checksum
43da27f69c98: Download complete
d22174e9eddd: Verifying Checksum
d22174e9eddd: Download complete
b47aa969d5dc: Verifying Checksum
I was expecting same output. Mainly the information regarding how much is downloaded and remaining. If I want to progress a progress bar based on the size of the image downloaded, there is no way to do with this output
Please let me know if there is a way to find real-time download size status.
FYI if you download any binary from curl, in Node.js exec command you get the real time bytes downloaded/total data.
The output that you are mentioning only happens if the docker CLI is running somewhere where there is a pseudo-tty allocated. You can force the output to not have a pseudo-tty on the command line by running the command this way:
docker pull my-image | cat
The pole will happen, and then you will get the same sort of output that you mentioned above.
It is likely that the way that you are calling the docker CLI is not allocating the pseudo-tty, causing it to not stream the progress information to standard out.
Try calling the executable with a pseudo-tty allocated to get the desired behavior.

How to start Node.js in specific URL?

When I'm updating (and testing) a specific page in my application, I need to stop the server, start again with node server.js, switch to the browser window, hit F5 and switch back to the terminal to see the output. This takes a lot of time.. :)
Is there a way I can start Node with something like this:
node server.js -url /my_page
so I can directly see the output as if someone hit the page from their browser?
I found this question but I guess it's included in the server code so it needs to be updated each time too. It is a solution, but I wonder if there is a faster way to do this.
Thanks,
Edit:
With curl, I'm getting this error:
$ node server.js & curl 'http://localhost:5000/my_page'
[2] 12824
curl: (7) Failed to connect to localhost port 5000: Connection refused
[1] Terminated node server.js
$ Listening on port 5000
Notice the last line, is it executing curl before the server is started?
node server.js & curl 'http://localhost/my_page'
This starts the server in the background, then immediately uses curl to make an HTTP request.
The server will still be running after this. You can bring it to the foreground (e.g. in order to kill it) with fg. Alternatively just run the two commands (without &) in separate terminals (or screens).
If the server doesn't start fast enough, just add a delay:
node server.js & sleep 1 ; curl 'http://localhost/my_page'

Node.js - Server crashes when started on startup, doesn't crash when manually started

I am running Node.js and Socket.io for online chat.
I have created a file in:
/etc/init/example.conf
it has two lines:
start on startup
exec forever start /var/www/example.com/html/server.js //location of server file.
Whenever I start file upload in chat application, it crashes but instantly restarts.
Whenever I kill node process though, and start it manually - it works fine.
I also can't get any logs or anything from terminal as when it's auto started - it doesn't print me anything to terminal.
I am still new to Node.js and Linux in general.
Node.js is running on Express + Jade.
How do I determine specific cause?
I managed to solve my problem, after a bit of searching around I found out about tail command.
My issue was a bit harder to trace because Node.js was a process started by autostart so when I launched terminal and connected to server, process was just running in background basically and I wouldn't get any output (including exception messages).
Anyway, solution that worked for me was:
I typed
ps aux | grep node //to find PID of node process
then I went to following directory
cd /proc/[pid of running node service]/fd
In fd directory there are few objects you can get output from but if you want to attach and listen to servers output including uncaught exceptions, you need 1.
So:
tail -f 1
that way I was able to cause website to crash and see the output.

CLI debugger for Mocha tests

I want to be able to drop a debugger statement (or something similar) right into my code and get a REPL when I run my tests so I can interact with my code to debug stuff.
I understand that Node (via V8) supports the debugger statement out of the box, but you have to run your code with certain flags (via node debug or node --debug or node --debug-brk -- I don't really understand the differences).
Mocha has these same flags as well, and when I run mocha debug I indeed get a debugger prompt in my terminal:
< Debugger listening on port 5858
connecting to port 5858... ok
break in node_modules/mocha/bin/_mocha:5
3 */
4
> 5 var program = require('commander')
6 , path = require('path')
7 , fs = require('fs')
debug>
But the breakpoint is in Mocha's code, not my breakpoint. This blog post says you're supposed to type run then continue to get to your breakpoint, but when I enter run it tells me "App is already running..." and when I enter continue it tells me "SyntaxError: Illegal continue statement".
It says "listening on port 5858" -- do I need to open a separate connection to that port from somewhere, like a browser or another terminal session?
I've seen some things about node-inspector, but I like to stay in the terminal as much as possible so I prefer a way to do this without opening a browser.
I'm new to JS, so please ELI5 :)
Also, if it's relevant, I'm actually using io.js, not Node proper
When the node/iojs debugger first starts up and connects it always breaks on the first line of whatever the script was that was passed into the interpreter.
Since you're already running your app, you do not need to type run here, but you can just type c to continue execution until your breakpoint has been it.
run is useful if the program terminates while you're in the debugger -- you can start it up again! (You can also do restart to restart the program within the debugger).

Using nodejs's Forever to output console.logs to screen

I just discovered that my nodejs app keeps crashing, so I've used forever app.js to start my app and have it automatically restarted when it crashes.
Problem: Now my app outputs alot of useful information as it runs via console.log and util.log. I used to use screen to run the node app, but now that I'm using forever to run the nodejs app, I can no longer see all the outputs.
Is there any way to see all the output from the nodejs app in realtime?
Directly with forever command :
forever logs app.js -f
It shows in realtime output of your application and forever logs (shows detected changes & restart messages).
You can watch a logfile live by using this shell-command.
tail -f /path/to/logfile
Not sure if this is what you needed.
Simplest way to go is
Run :
forever logs // will give you list of log files
forever logs 0 -f // then just provide the index of log
If you pass the --help flag to Forever you'll see this example:
forever -o out.log -e err.log my-script.js
-o and -e define the stdout and stderr log files.
-o OUTFILE Logs stdout from child script to OUTFILE
-e ERRFILE Logs stderr from child script to ERRFILE
Forever seems a little overkill for development. Give Supervisor a shot and see if you like it better:
npm install -g supervisor
supervisor app.js
I am running nodejs on AWS EC2 instance and this worked for me.
Note: logs are accessible if you are root owner. So first use sudo su then
forever -a -o out.log -e err.log yourServerApp.js
It will tail console log (in case you track event messages from log) and error log. I hope it helps someone.
linux : tail -f /path/to/logfile.log
windows : enter PowerShell -> Get-Content /path/to/logfile.log -Wait -Tail 1000

Categories

Resources