I would like to automate the testing sequence of my Cordova app.
I know I can run nom scripts in sequence by specifying a script name, and then specifying a series of scripts separated by semicolons. This is working perfectly for other parts of my dev environment.
For the UI tests, I'm using Webdriver and Appium. This is the series of scripts I would like to automate, but I can't seem to find out what the commands are to get the specific sequence running (specifically regarding the Appium server):
1) Run jest unit tests (this is not a problem)
2) If jest tests do not fail, start the Appium server in a new terminal tab
3) Either wait for a successful signal (not sure what this would be) that the Appium server has started or set a reasonable timeout
4) Run the UI tests with WebDriver/Mocha (this is also not a problem running by itself).
5) Kill the Appium server
Part 3 is my hurdle, specifically how to get the Appium started in a separate terminal tab. Chaining these together would be great.
The goal here is to automate the entire testing series in something like npm run allTheTests so I can leave and get a sandwich, send a text, read a Medium article (etc.) while the tests complete.
Many thanks =)
Related
I started to learn ReactJS yesterday (to be used in my next product), I am willing to set up my dev environment but now I'm stuck with Jest...
Having a bluetooth lightbulb on my desk (already op with scripts etc..), I want to get a red light when my tests launched with jest --watch fail (see create-react-app from FB devs here)
The problem is, I don't know how to run a callback after the tests, it seems like no one ran into this issue on the interwebz, no solution found yet for me.
Update:
I am currently using a log file to grep:
lamp.rb
def ci
if File.readlines("path/jest.log").grep(/failed/).any?
File.truncate('path/jest.log', 0)
fail_jest # This method updates my lightbulb :) (red blink)
end
rescue
puts 'No jest log found :('
end
Launching my jest tests like this: unbuffer npm run test |& tee tmp/jest.log
I am still looking for a better solution !
Thanks for your help
Your problem is not specific to React or Jest. When your run jest tests, you are basically running a Node/npm command and when the tests fail the process exists with an unsuccessful exit code. This is the same mechanism used to make automated CI builds fail when tests don't pass. So I'd suggest you start your research from there and depending on your lightbulb's API, it should be straight forward to make it fire events whenever the process fails regardless of the reason.
I am looking for a way to make karma.js crash when my tests fail so that my build process gets interrupted since that is easier to monitor in a remote building server.
I am using mocha as a reporter and jasmine as the test processor engine.
Is there any option or particular variable in the karma.conf.js file maybe that would allow me to "crash on test fail"
There is no need for a "crash on failure".
Your build process should check the exit code of the Karma process that it launches. If all test pass, the code will be 0. If there is a test failure, the code will be non-zero. (I've just checked and it was 1 when I tried.)
You may be able to get a plugin for karma for your build server that will report the results of your tests.
For example, there's a plugin for karma/teamcity (karma-teamcity-reporter) which allows you to fail the build step, preventing the application from publishing if you have it set up that way.
you can use grunt as automation tool to run your karma test case so that whenever any grunt job(test case) failed it will by default stop execution.
My general problem is how to run unit tests with karma in IE or other browser installed on a Windows machine, while karma is run on a Linux machine, though in the question below I might ask more specific question based on what I've already tried.
Also it may be worth to mention, that I have Windows installed on a Virtual box machine.
I started karma from a terminal, then opened in IE this url: http://10.0.2.2:9876 and saw the karma page in the browser window (though I spotted some quick blink with a red background, I suppose it is related to an error I'll mention further).
After that I tried to run tests with the command: node_modules/karma/bin/karma run config.js in another terminal tab. When I did this I got this error (I get it even when I use Chrome on my Linux host-machine instead IE on the Virtual box guest, so I suppose the error has nothing to do with networking):
You need to include some adapter that implements __karma__.start method!
I googled for this error and found another question: Error: You need to include some adapter that implements __karma__.start method
Supposing that I need to run tests from the same place I started karma-server I tried to redirect the output to another tab:
$node_modules/karma/bin/karma start &> /dev/pts/17 &
$node_modules/karma/bin/karma run config.js
But this did not help. If I try to start server and run tests in a single command, then I do not have time to register IE.
So could anyone please answer any of the next questions:
how to run unit tests with karma in a browser on other machine?
how to get rid of this error You need to include some adapter that implements __karma__.start method!, if the karma server was started in one terminal tab and the run command was issued in another?
how to start the karma server and run tests in separate steps, i.e. issue the karma start and karma run config.js commands instead of karma start config.js?
My regards and sincerest appreciation in advance for any help.
Currently I've found a workaround for this. I start karma with the config file:
karma start config.js
in the config I have singleRun: false.
Then I manually connect my IE instance, and after this I run the tests from another terminal tab. Why this happens I have no idea. The only inconvenience is that I always have browsers' windows open.
I have moved an existing node.js + express project to VS because I prefer the IDE over JetBrains for now (used VS for years, only peeked into Webstorm).
I used NTVS new project->from existing sources and all files were imported successfully.
Afterwards, I opened the project settings of my project and set the node.exe arguments to bin\www, startup file for express.
When I press F5 (debug) I get the console.log messages I have put into the www and app.js files in the opening command prompt, and it looks like the server is running (cannot confirm, I want to debug if everything is working), but the VS debugger directly exits again, it also does not open any page in the browser I selected for debugging.
My node app actually is a REST webservice, so I want to test different URLs with different parameters.
Also, I cannot access the app on the port I specified, though when I directly start it from node.exe I can, even though the command prompt is still open.
(I have NTVS and WebEssentials installed - some operations take a long long time, but I attribute this to NTVS being still an early version.)
Question: how does the Visual Studio debugger stay connected to the node.js application so I can use breakpoints and use any browser then to connect and test different URLs? (Even a breakpoint put on the console.log that gets printed during startup is not being triggered.)
For everyone who asks receives, and the one who searches finds....
(and yes, I did spend a long time searching and trying before posting here..)
Kind of nice to debug node.js server with VS..
hope this helps someone
Edit: The arguments to node.exe can be hard to read in the image. It must be
--debug=<portno>
that is with two dashes (and not just one) to specify the debug port.
Not so much knowledge on expressjs but with a recent release of NTVS 1.0 Alpha, I did find it supports remote debugging which can be also used to debug nodejs app running locally - anyway haven't tried if it works with nodejs app + expressjs but it should.
I followed the step in this video https://youtu.be/-ir9ZB8lUg4 which is
Run your nodejs with node.exe RemoteDebug.js <your_javascript_file>.
RemoteDebug.js has come when you install NTVS.
In Visual Studio, select Debug > Attach to Process
Select Node.js remote debugging for Transport
Enter localhost:5859 for Qualifier
Click Attach
This will put Visual Studio in debugging mode which you can set a breakpoint, do step-in/step-out, very same experience when you use VS to debug .NET app.
Its pretty straight forward with NTVS, you can download required version for your windows from github here
Once you install NTVS, NodeJS project templates will be added
Now, Goto File->New project -> Basic NodeJS Express 3 application (it will be available in javascript project templates)
Now just goto debug and select Start Debugging, add breakpoints where ever required and you can start debugging
I've been evaluating JSTestDriver, and it seems to be working well. I want to use it in a continuous integration environment, and I want to know if multiple projects can be testing against the same JSTestDriver server at the same time.
So my configuration is I have 1 JSTestDriver server with 3 different browsers captured. Can multiple projects run their test suites against that same server? Or is a JSTestDriver server only designed to run one test suite at a time?
I am using JSTestDriver in a continious integration environment, too. Unfortunately, I noted that JSTestDriver gets instable when running for a long time, which means that we need to restart the JSTestDriver server process and reconnect all browser slaved. I found this behavior also described in this post.
Therefore, we are restarting the JSTestDriver before each build to make sure it is working. That means that you can not use one JSTestDriver for all builds because it would might be restarted by one build while the other build wants to run tests on it. Therefore, every build plan in your Continious Integration should have an own instance of JSTestDriver to use, and developers should use their own instances running on their local machine.
The conf file specifies all the files needed to do a test run. The better question might be if your able to run multiple conf files at the same time. Personally I would just include all the files needed to be tested in one conf file. But if that's not an option here are some links.
Check this link out: (Grunt-JsTestDriver)
https://npmjs.org/package/grunt-jstestdriver
https://github.com/rickyclegg/this/blob/master/node_modules/grunt-jstestdriver/README.md