I am a newbie in programming and I have a task here I need to solve. I am trying to get the html source code of a webpage using Java / Webdriver method getPageSource(). Problem is, that page is somehow generated, probably by javascript, so the result I get is html code containing just page skeleton - a table that is empty, not filled by data. But, there is tag like <script type="text/javascript" src="/x/js/main.c0e805a3.js"></script> in the very bottom of that html code.
The question is, how can I force Webdriver to run that Javascript and give me the result - the whole source html with data. I already tried to use this (js.executeScript("window.location = '/x/js/main.c0e805a3.js'");) before calling getPageSource() but not successful.
Any help will be appreciated, thanks!
There are quite a few setups, now, that can run the Java-Script on a web-page. The most well known, I think, is likely Selenium since I think it has been around for a while. Others include karate, Puppeteer, and even an old tool called Rhino. Puppeteer is a Google, Inc. project that uses Java-Script (server-side Java-Script, called Node.js. They don't like us comparing, contrasting libraries here.
I haven't had the time to engage Selenium, yet, but I write HTML parser, search and update code all the time. If your only goal is to load a page whose contents are dynamically "filled in by AJAX calls" - and what I mean by that, you only want the contents of an HTML that would normally see when you visit the sites web-page, and you are not concerned with button presses then the one I have been using for that is called Splash This tool does have the ability to let you invoke Java-Script, but if all you want to do is see the JS on a page dynamically load the table, then, literally, all you have to do is start-the tool, and add one line to your program.
On Google Cloud Platform, these 2 lines will start a Splash Proxy Server. If you are writing your code on AWS (Amazon) or Azure (Microsoft), it would likely be similar. If you are running your code in an office on the local machine, you would have to research how to start it.
Install Docker. Make sure Docker version >= 17 is installed.
Pull the image:
$ sudo docker pull scrapinghub/splash
Start the container:
$ sudo docker run -it -p 8050:8050 --rm scrapinghub/splash
Then, in your code, all you have to do is the following:
// If your original code looked like this:
URL url = new URL("https://en.wikipedia.org/wiki/Christopher_Columbus");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", USER_AGENT);
return new BufferedReader(new InputStreamReader(con.getInputStream()));
Change the first line of code in this example to this, and (theoretically), and dynamically loaded HTML tables that are completed with the onload page events will be automatically loaded before returning the HTML page.
// Add this line to your methods
String splashProxy = "http://localhost:8050/render.html?url=";
URL url = new URL(splashProxy + "https://en.wikipedia.org/wiki/Christopher_Columbus");
For most web-sites, any initial tables that are filled by JS/jQuery/AJAX will be filled in. If you are willing to learn teh Lua Programming Language, you can also start invoking the methods there. It has been pretty convenient for my purposes, since I am not writing web-page testing code (code that simulates user button presses). If that is what you are doing, Selenium is likely worth spending time learning / studying the A.P.I.
I have already existing jmeter scripts(which I cant change) which generates a random number using javaScript to generate Url parameters dynamically. Problem is the javascript itself is printed in the url as parameter instead of the number it shoud generate. (All of this happens when using Spring and maven combo(WebService- JAX-RS))
I need it to be a web Project to simplify the execution of the regression scripts. The script is correct as It runs properly when using the Jmeter UI
Interestingly when I run it using a main method It runs and generates the random number using JavaScript. I have all the jars under a standard download of Jmeter Installation in the classpath. I need help in either getting all files in classpath to the maven build or the dependencies of all to be added to get it working.
The Javascript is as follows :
72AE-442A-93C5-SWID1${
__javaScript(vard=newDate();varday=d.getDate();if(day<10){
day='0'+day;
}varmonth=d.getMonth()+1;if(month<10){
month='0'+month;
}varyear=d.getFullYear();varhour=d.getHours();if(hour<10){
hour='0'+hour;
}varmin=d.getMinutes();if(min<10){
min='0'+min;
}varsec=d.getSeconds();if(sec<10){
sec='0'+sec;
}varmsec=d.getMilliseconds();if(msec<10){
msec='00'+msec;
}elseif(msec<100&&msec>=10){
msec='0'+msec;
}currentDate=min+''+sec+''+msec)
}
If more details are required Please tell me
https://blazemeter.com/blog/5-ways-launch-jmeter-test-without-using-jmeter-gui
Using the Fourth way in the following link but calling the same logic via a Webservice
Thanks
I am developing a B2B that is now in production.
The problem is that when I update the application on IIS
the html files of the template are not updated.
After I clear the cache of the browser everything gets updated.
How can I avoid IIS to cache the pages of the templates?
Or more generally how can I do to always have the most recent server files?
Thanks
Matteo
When you update the files on the html , it means you are applying some patch which eveytime you have your updated number like build number or patch number.
You have to append this patch number or build number at the end of each page.
You have to set this build number in global service and embed at the last of each html template.
For example.
At start:
build_number =1;
your_template ='content.html?'+buildnumber
At second iteration:
build_number =2;
This way you always get updated htmls everytime you apply some changes on server. Usually this build number is final svn build number. Hope you get my point.
I have an update for my website and some Javascript files are changed. But some customers are still getting the old Javascript file and not the new one. They can remove their temporary internet files, but that's not what I want.
Is there something in IIS6 that I can use?
Change the link to the .js file - add a parameter .js?ver=1.0, for example.
Every time you change the .js file, change the parameter - this will ensure clients will be getting the latest version.
You can use "cache busting". A simple solution is to append a version number in the query string, eg: script.js?v=1.1.
Another solution is to dynamically calculate a checksum or hash of the .js file that gets automatically appended to the query string, so everytime you modify the contents of the .js file, a new checksum/hash is generated and appended to the query string.
So I changed some jquery on a page in my application and it works fine etc.
But I just noticed that when I push the code to a different server and run the page on that server I need to hit ctrl + f5 to see the changes reflected. Is there a way when doing a code push to clear the cache on that page / javascript so it loads the latest version not the cached version.
on your referenced javascript file you can add a querystring, like:
<script src="/scripts/myfile.js?v=1" type="text/javascript"></script>
whenever you change the text in the file update the query string.
You need to take a look at your js files content expiry.
One way to solve this problem is to append a dummy query string parameter to all your *.js urls and set it to the build number of your app or file modified time.
/Scripts/file.js?v=1.0.12345.0
This will ensure that each build will use new urls and you can set their content expiry to never.