My Application has a lot of calculation being done in JavaScript according to how and when the user acts on the application. The project prints out valuable information (through console calls) as to how this calculation is going on, and so we can easily spot any NaNs creeping in.
We are planning to integrate Selenium (RC with python) to test or project, but if we could get the console output messages in the python test case, we can identify any NaNs or even any miscalculations.
So, is there a way that Selenium can absorb these outputs (preferably in a console-less environment)?
If not, I would like to know if I can divert the console calls, may be by rebinding the console variable to something else, so that selenium can get that output and notify the python side. Or if not console, is there any other way that I can achieve this.
I know selenium has commands like waitForElementPresent etc., but I don't want to show these intermediate calculations on the application, or is it the only way?
Any help appreciated.
Thank you.
There is GetEval() call that returns the result of a JavaScript call to the page. If you have the JavaScript on the page then you can do something like
self.assertEqual(selenium.GetEval("this.browserbot.getUserWindow().functionUnderTest().isNaN();"),"false","There was a NaN detected")
The browserbot access allows you to call the javascript functions on the page and get the result. The isNaN() will return false if you get a decent result
If you are purely testing that the JavaScript functions are performing the correct calculations with the given inputs, I would suggest separating your JavaScript from your page and use a JavaScript testing framework to test the functionality. Testing low level code using Selenium is a lot of unnecessary overhead. If you're going against the fully rendered page, this would require your application to be running to a server, which should not be a dependency of testing raw JavaScript.
We recently converted our application from using jsUnit to use YUI Test and it has been promising so far. We run about 150 tests in both FireFox and IE in less than three minutes. Our testing still isn't ideal - we still test a lot of JavaScript the hard way using Selenium. However, moving some of the UI tests to YUI Test has saved us a lot of time in our Continuous Integration environment.
Related
I am a beginner in in Python3.6 using BeautifulSoup to perform "web-scraping."
Once I have ran a request.get() and prettyify the output I notice that the webpage does not return the values, it would seem to be storing code which would be related to the value.
Here is the link to the webpage in specific:
http://www.tennisabstract.com/cgi-bin/wplayer.cgi?p=AngeliqueKerber&f=r1
I am trying to extract the hand which the player uses in Tennis. Highlighted Yellow from picture below:
Picture of what I am trying to obtain:
I would appreciate feedback concerning the outline of the question, if it is confusing (or non-standard) feedback such as this will help me in the future to ensure I am asking questions appropriately.
There are two options (mostly).
The first one is easier and slower - browser emulation. You just try to use the site as a normal user - with browser. There is a python module for this task - selenium. It uses specific webdriver to use browser. There are plenty of webdrivers available (for example chromedriver to use chrome). Also, there are headless solutions (PhantomJS for example).
The other way is smarter and faster - XMLHttpRequests (XHRs). Basically - site uses some hidden API to get info via JS, and you try to find out how exactly. In most cases you can use Inspect Element toolbox of your browser. Switch to the network tab of it, clear it an try to get results. Then sort it to see only XHRs. It usually returns JSON-based values that are easily converted into a python dictionary using json() method of Response object.
Here's a really great GitHub that someone made on this website, an API practically you can change/edit few things (fork it) and then use it the way you want to.
HERE
It uses Selenium webdriver but it's high quality.
This is part of a project I am working on for work.
I want to automate a Sharepoint site, specifically to pull data out of a database that I and my coworkers only have front-end access to.
I FINALLY managed to get mechanize (in python) to accomplish this using Python-NTLM, and by patching part of it's source code to fix a reoccurring error.
Now, I am at what I would hope is my final roadblock: Part of the form I need to submit seems to be output of a JavaScript function :| and lo and behold... Mechanize does not support javascript. I don't want to emulate the javascript functionality myself in python because I would ideally like a reusable solution...
So, does anyone know how I could evaluate the javascript on the local html I download from sharepoint? I just want to run the javascript somehow (to complete the loading of the page), but without a browser.
I have already looked into selenium, but it's pretty slow for the amount of work I need to get done... I am currently looking into PyV8 to try and evaluate the javascript myself... but surely there must be an app or library (or anything) that can do this??
Well, in the end I came down to the following possible solutions:
Run Chrome headless and collect the html output (thanks to koenp for the link!)
Run PhantomJS, a headless browser with a javascript api
Run HTMLUnit; same thing but for Java
Use Ghost.py, a python-based headless browser (that I haven't seen suggested anyyyywhere for some reason!)
Write a DOM-based javascript interpreter based on Pyv8 (Google v8 javascript engine) and add this to my current "half-solution" with mechanize.
For now, I have decided to use either use Ghost.py or my own modification of the PySide/PyQT Webkit (how ghost works) to evaluate the javascript, as apparently they can run quite fast if you optimize them to not download images and disable the GUI.
Hopefully others will find this list useful!
Well you will need something that both understands the DOM and understand Javascript, so that comes down to a headless browser of some sort. Maybe you can take a look at the selenium webdriver, but I guess you already did that. I don't hink there is an easy way of doing this without running the stuff in an actually browser engine.
I am working on a scrapy app to scrapte some data on a web page
But there is some data loaded by ajax, and thus python just cannot execute that to get the data.
Is there any lib that simulate the behavior of a browser?
For that you'd have to use a full-blown Javascript engine (like Google V8 in Chrome), to get the real functionality of the browser and how it interacts. However, you could possibly get some information by looking up all URLs in the source and doing a request to each, hoping for some valid data. But in overall, you're stuck without a full Javascript engine.
Something like python-spidermonkey. A wrapper to the Javascript engine of Mozilla. However using it might be rather complicated, but that's dependant on your specific application.
You'd basically have to build a browser, but seems Python-people have made it simple. With PyWebkitGtk you'd get the dom and using either python-spidermonkey mentioned before or PyV8 mentioned by Duncan you'd theoretically get the full functionality needed for a browser/webscraper.
The problem is that you don't just have to be able to execute some Javascript (that's easy), you also have to emulate the browser DOM and that's a lot of work.
If you want to be able to run Javascript then you can use PyV8. Install it with easy_install PyV8 and then you can execute any standalone javascript:
>>> import PyV8
>>> ctxt = PyV8.JSContext()
>>> ctxt.enter()
>>> ctxt.eval("(function(a,b) { return [a+b, a*b, a/b, a-b] })(13,29)")
<_PyV8.JSArray object at 0x01F26A30>
>>> list(_)
[42, 377, 0.4482758620689655, -16]
You can also pass in classes defined in Python, so in principle might be able could emulate enough of the DOM for your purposes.
An AJAX request is a normal web request which is executed asynchronously. All you need is the URL which the JavaScript code sends to the server. Use that URL with urllib to get at the same data.
The simplest way to get work done is by using 'PyExecJS'.
https://pypi.python.org/pypi/PyExecJS
PyExecJS is a porting of ExecJS from Ruby. PyExecJS automatically picks the best runtime available to evaluate your JavaScript program, then returns the result to you as a Python object.
I use Macbook and installed node.js so pyexecjs can use node.js javascript runtime.
pip install PyExecJS
Test code:
import execjs
execjs.eval("'red yellow blue'.split(' ')")
Good luck!
A little update for 2020
i reviewed the results recommended by Google Search. Turns out the best choice is #4 and then #1 #2 are deprecated!
#4 https://github.com/sqreen/PyMiniRacer is actually the most straight forward installation.
#3 https://github.com/kovidgoyal/dukpy is based lightweight JS engine. I could not find any limitations compared with v8. No significant benefit in terms of performance either.
Deprecated
#1 https://github.com/sony/v8eval has received very little maintenance for 2 years. takes 20 minutes to build and fails... (filed a bug report for that https://github.com/sony/v8eval/issues/34
#2 https://github.com/doloopwhile/PyExecJS
is discontinued by the owner
I'm writing an automation program for a Web Application. I am accessing the Web Application through a javascript API and have wrapper functions with custom assertions that currently just write output to a table in an HTML page.
Now I need to get the data output into my hudson (https://hudson.dev.java.net/) automation, where I have a lot of flexibility when it comes to arranging, sharing and presenting the results.
When I wrote NUnit tests, the hudson-integration was impeccable. I saw there was a thing called JSUnit, but it is no longer actively maintained(?), so maybe I shouldn't spend too much time learning it?
I have seen that tools like Firebug can output javascript results to a console, though I don't know where to go from there. The console output seems to stay in firefox and come no further.
Any help or tips are most welcome.
Thanks!
/ Jakob
If I understand correctly, you want your Hudson build to run a test of your Web Application which is set up and running somewhere else. (This gets a little harder if you're also building your Web Application and want to set it up for a test run all inside Hudson.)
The easy option: As one of your build steps, retrieve the HTML page with your output and tell Hudson that the page is a build artifact. That way you can look at the test output manually.
Somewhat harder: change your test output (or pass a parameter to specify the format) to match the XML format used by NUnit -- see example XML output. This is a direct link to an XML file and may not display well in your browser; try viewing source or saving as text.
Update: On re-reading your question, it wasn't clear to me whether you were interested solely in Hudson integration (which my original answer assumed), or in other possibilities for testing frameworks.
Depending on what you want to test:
you might look at testing your Web Application with Selenium. I know there's a Hudson plugin for Selenium, but I've also noticed several questions here recently describing problems with Selenium+Hudson. I don't have any experience with the combination myself.
there are lots of javascript testing frameworks with different capabilities.
So, you are using a bunch of javascript libraries in a website. Your javascript code calls the several APIs, but every once in a while after an upgrade, one of the API changes, and your code breaks, without you knowing it.
How do you prevent this from happening?
I'm mostly interested in javascript, but any answer regarding dynamically typed languages would be valuable.
I don't think there's much you can do. You always run a risk when updating any piece of software. The best advice is to:
Read and understand documentation about upgrading
Upgrade in your test environment
TEST
Roll out live when you are happy there are no regressions
You should consider building unit tests using tools such as JsUnit and Selenium. As long as your code passes the tests, you're good to go. If some tests fail, you would quickly identify what needs to be fixed.
As an example of a suite of Selenium tests, you can check the Google Maps API Tests, which you can download and run locally in your browser.
Well there are two options:
Don't upgrade
Retest everything after you upgrade.
There is no way to guarantee that an upgrade won't break something. Even if you have something that could check the underlying API and make sure it still all lines up, you can't be certain that the underlying functionality is the same.