I have an application that runs its tests with Jasmine and WebdriverIO which I would like to automate in CircleCI. I'm new to testing in general so I'm not sure what to do.
Here's what I know:
To run the tests, I invoke npm test
A selenium server is required on port 4444 (which I can start with npm start)
The application should be running on port 80 (which I can serve with another npm command)
When the tests complete, I'm returned to command line, but the other services (on p4444 and p80) are still running
Here's what I don't fully understand:
Locally, these require 3 terminals to run concurrently, is there a way to do this with CircleCI?
If so, how do I tell when the p4444 and p80 are ready to test on, or cancel them when the tests are done?
Is my issue with Docker, or CircleCI?
In order to answer your questions clearly, I'm going to refer to each of your commands as follows:
To run tests you run npm test
To run selenium you run npm start selenium
To run your app you run npm start app
Questions/Answers:
Locally, these require 3 terminals to run concurrently, is there a way to do this with CircleCI?
Yes. you just need to start a process with background set to true
i.e. to start Selenium in the background you can run the following
- run:
name: Start Selenium in background
command: |
npm start selenium
background: true
After starting a process, but before using it, wait for the process to be ready on the given port
- run:
name: Waiting for Selenium server to be ready
command: |
for i in `seq 1 10`;
do
nc -z localhost 4444 && echo Success && exit 0
echo -n .
sleep 1
done
echo Failed waiting for Selenium && exit 1
Note if you replace 4444 in the above command, you can wait for a process on another port
How do I tell when the p4444 and p80 are ready to test on, or cancel them when the tests are done?
Your CircleCi commands might look like this
- run:
name: Start Selenium in background
command: |
npm start selenium
background: true
- run:
name: Start App in background
command: |
npm start app
background: true
- run:
name: Waiting for Selenium server to be ready
command: |
for i in `seq 1 10`;
do
nc -z localhost 4444 && echo Success && exit 0
echo -n .
sleep 1
done
echo Failed waiting for Selenium && exit
- run:
name: Waiting for App server to be ready
command: |
for i in `seq 1 10`;
do
nc -z localhost 80 && echo Success && exit 0
echo -n .
sleep 1
done
echo Failed waiting for Selenium && exit
- run:
name: Run Tests
command: |
npm test
You asked a separate question - How do I cancel the processes on port 4444 and 80 when the tests are done? You don't really need to. When the test job finishes the container will be disposed and the helper apps will stop.
However, if you want to stop those processes in order to run some other job steps, you can run kill commands (I can elaborate if this is unclear)
Is my issue with Docker, or CircleCI?
It looks like it's just an issue in understanding how to run a series of commands in CircleCi
If you follow the steps above you should be able to accomplish your goal.
Related
I am setting up gitlab-runner locally on my mac to be able to run build and test scripts using docker. I have gone through the installation instructions listed on the Gitlab runners page to install the runner locally:
# Download the binary for your system
sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64
# Give it permissions to execute
sudo chmod +x /usr/local/bin/gitlab-runner
# The rest of commands execute as the user who will run the Runner
# Register the Runner (steps below), then run
cd ~
gitlab-runner install
gitlab-runner start
From what I can tell it is installed fine. I then registered a runner like so:
sudo gitlab-runner register --url https://gitlab.com/ --registration-token $REGISTRATION_TOKEN
(I obviously replaced $REGISTRATION_TOKEN with my token). When it prompts me for additional details I have entered the following:
Tags: (I left this blank)
Enter an executor: docker
Enter the default docker image: node:14.0.0
I then get the following message:
Runner registered successfully. Feel free to start it, but if it's
running already the config should be automatically reloaded!
When I then navigate to the root of my project I try and run gitlab-runner run but I get the following error:
Starting multi-runner from /Users/ben/.gitlab-runner/config.toml... builds=0
WARNING: Running in user-mode.
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...
Configuration loaded builds=0
listen_address not defined, metrics & debug endpoints disabled builds=0
[session_server].listen_address not defined, session endpoints disabled builds=0
^CWARNING: [runWait] received stop signal builds=0 stop-signal=interrupt
WARNING: Graceful shutdown not finished properly builds=0 error=received stop signal: interrupt
WARNING: Starting forceful shutdown StopSignal=interrupt builds=0
All workers stopped. Can exit now builds=0
When I look at the config.toml if looks like it may be missing some configuration in there as the error above may suggest? Here is a cat of the entire file:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
I'm not sure why i'm receiving this error message? Does my config look alright? When searching the issue I found another thread that said to just set "Can run untagged jobs" to yes which I have done but it still does not work...
It makes sense that you're having problems with the configuration.
If you read carefully the output, it says that you're running in user-mode, so I suppose you started the runner by using gitlab-runner.
Problem is that you registered your runner using sudo, so you configured the system-mode under /etc/gitlab-runner/. This configuration is loaded when you start gitlab-runner with sudo.
In order to verify that, you will be able to see the registered configuration under /etc/gitlab-runner/, with all the additional sections regarding Docker Runners and so on, instead of the basic configuration that you have under ~/.gitlab-runner/ which I suppose is the one you attached to your question.
I want to run a node application from my raspberry pi. The application is supposed to start on boot.
I have included the following lines in /etc/rc.local (before exit 0) :
cd /home/pi/PPBot
node bot.js > dev/null &
I first navigate to the correct folder and then run the bot from there. However the node application is not running when I reboot my raspberry pi. So it seems that rc.local is not executed or unable to execute the lines I provided.
I am looking for a solution so that the application will run at boot.
save this
[Unit]
Description=Node JS Script Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/node /path/to/hello_env.js
Restart=on-failure
[Install]
WantedBy=multi-user.target
to /etc/systemd/system/ as nodescript.service
sudo systemctl daemon-reload
sudo systemctl start nodescript
if it worked, make it startup on boot
sudo systemctl enable nodescript
The examples I 've seen (like this one) used
cd /home/pi/PPBot
node bot.js < dev/null &
instead of
cd /home/pi/PPBot
node bot.js > dev/null &
Note the < instead of the >. As you can see here, the < redirects the Standard In (stdin), while the > redirects the Standard Output (stdout). And as described here,
< /dev/null is used to instantly send EOF to the program, so that it doesn't wait for input (/dev/null, the null device, is a special file that discards all data written to it, but reports that the write operation succeeded, and provides no data to any process that reads from it, yielding EOF immediately). & is a special type of command separator used to background the preceding process.
Since you havent used < dev/null, maybe your program is stuck and waits for some input. The & at the end makes your program run in the background. Try to not include it and see if your Rasberry Pi is still booting. If not, you know that your programs runs continously and blocks further booting; when you used the & sign, your program just failed in a separate process. In this case changing the > to < should help because your programs doesn't wait for input.
If the above doesn't work, update your question with the specific error message. You can view the boot log by using the command dmesg (display message).
In this case when you
cd /home/pi/PPBot
you are no longer in the root directory so when you run
node bot.js > dev/null &
it is looking for the dev folder at /home/pi/PPBot/dev
You will need to add a leading / to ensure it is accessing /dev
change
node bot.js > dev/null &
to
node bot.js > /dev/null &
I have developed an application using express js. In order the automate the process, I created a jenkins job and the commands used are as follows
ssh "ip address of the server" node app.js "a" "b" "c" "d" "e" &
curl "url of the application"
curl "another url of the application"
Here a,b,c,d,e are the command line arguments which i am passing to my application on starting the server. The problem is each time i have to shutdown the server first and then run the job. How can this whole process be automated and also server should keep on running in background.
Please let me know if anyone want some more clarification on the question asked. Any help is deeply appreciated.
Step1 Get rid of passing args by:
option 1
use npm scripts in package.json with forever
"scripts": {
"start": "forever express/app.js a b c"
},
Command to run your server npm start
option 2
use bat or sh file to restart your app
file.bat content
forever express/app.js a b c
automate.js
var automaton = require('automaton').create();
automaton.run('run', {cmd: 'file.bat'}).pipe(process.stdout);
Command to run your server node automate
Step2 Configure forever code to run npm start or node automate whenever some change is found in package.json or file.bat, respectively
See it in action, clone Git Repo and run node express\index.js or run npm start
Happy Helping!
I have this Javascript Signal server running using nodejs.
But daily it's crashing as a result whole service goes down. I am using following infinite loop to restart the nodejs script if it's crashed or not running. But it's not perfectly working.
Can anyone optimise it or is there any better way to keep the a.js up and running always if suddenly the process was not alive.
#!/bin/bash
while :
do
# 1
videoupload=$(pgrep -f "a.js")
if [ $videoupload ]; then
log1="running a $1 $2"
else
log1="re-launch a $1 $2"
nohup node /var/tmp/signal/a.js 2>&1 | tee -a /var/tmp/signal.log &
fi
echo $log1
sleep 1
done
If you're using the new CentOS 6, a much better way to handle this is to put it in an Upstart script. Upstart monitors all the system daemons and makes sure they stay running. The Upstart config below will also launch your process when the system boots.
edit the file /etc/init/a.conf and put the following config in it. You'll need to sudo to edit as root.
description "a.js"
author "YumYumYum"
# Stanzas
#
# Stanzas control when and how a process is started and stopped
# See a list of stanzas here: http://upstart.ubuntu.com/wiki/Stanzas#respawn
# When to start the service
start on runlevel [2345]
# When to stop the service
stop on runlevel [016]
# Automatically restart process if crashed
respawn
script
echo $$ > /var/run/a.pid;
exec node /var/tmp/signal/a.js
end script
post-stop script
rm -f /var/run/a.pid
end script
Now that you've created an Upstart config for your process you can start it from the command line:
$ sudo service a start
Upstart will monitor your process and will restart it any time it goes down. It also redirects logs to /var/log/upstart/a.log.
I just discovered that my nodejs app keeps crashing, so I've used forever app.js to start my app and have it automatically restarted when it crashes.
Problem: Now my app outputs alot of useful information as it runs via console.log and util.log. I used to use screen to run the node app, but now that I'm using forever to run the nodejs app, I can no longer see all the outputs.
Is there any way to see all the output from the nodejs app in realtime?
Directly with forever command :
forever logs app.js -f
It shows in realtime output of your application and forever logs (shows detected changes & restart messages).
You can watch a logfile live by using this shell-command.
tail -f /path/to/logfile
Not sure if this is what you needed.
Simplest way to go is
Run :
forever logs // will give you list of log files
forever logs 0 -f // then just provide the index of log
If you pass the --help flag to Forever you'll see this example:
forever -o out.log -e err.log my-script.js
-o and -e define the stdout and stderr log files.
-o OUTFILE Logs stdout from child script to OUTFILE
-e ERRFILE Logs stderr from child script to ERRFILE
Forever seems a little overkill for development. Give Supervisor a shot and see if you like it better:
npm install -g supervisor
supervisor app.js
I am running nodejs on AWS EC2 instance and this worked for me.
Note: logs are accessible if you are root owner. So first use sudo su then
forever -a -o out.log -e err.log yourServerApp.js
It will tail console log (in case you track event messages from log) and error log. I hope it helps someone.
linux : tail -f /path/to/logfile.log
windows : enter PowerShell -> Get-Content /path/to/logfile.log -Wait -Tail 1000