We try to scrape a website using Selenium, JAVA, eg. try to change product color at https://www.ergodyne.com/shax-6054-pop-up-tent-sidewall-kit-includes-10ftx10ft
How to find out/check when the in-site JS has changed the HTML upon the arrival of the AJAX response ?
How big should be delay after AJAX/XHR response and JS replaced HTML ?
The AJAZ queries are quite complicated to try to get data from them...
Link for JSON response: https://jsonformatter.org/298d31
Using org.openqa.selenium.support.ui.ExpectedConditions and org.openqa.selenium.support.ui.WebDriverWait. See https://www.guru99.com/implicit-explicit-waits-selenium.html
Code:
package tests;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
public class IgorSavinking {
public static String userDir = System.getProperty("user.dir");
public static String chromedriverPath = userDir + "\\resources\\chromedriver.exe";
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver", chromedriverPath);
ChromeOptions options = new ChromeOptions();
options.addArguments("--ignore-certificate-errors");
options.addArguments("--start-maximized");
options.addArguments("--disable-notifications");
WebDriver driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://www.ergodyne.com/shax-6054-pop-up-tent-sidewall-kit-includes-10ftx10ft");
driver.findElement(By.id("cookie-policy-accept")).click();
String colorBefore = driver.findElement(By.tagName("legend")).getText().substring(5);
System.out.println("Current color is: " + colorBefore);
driver.findElement(By.xpath("//label[contains(#for,'edit-purchased-entity-0-attributes-attribute-color-17')]")).click();
new WebDriverWait(driver, 30).until(ExpectedConditions.numberOfElementsToBe(By.xpath("div[#class='message']"), 0));
String colorAfter = driver.findElement(By.tagName("legend")).getText().substring(5);
System.out.println("Current color is: " + colorAfter);
System.out.println("Color has been changed: " + !colorBefore.equals(colorAfter));
driver.quit();
}
}
Output:
Starting ChromeDriver 97.0.4692.71 (adefa7837d02a07a604c1e6eff0b3a09422ab88d-refs/branch-heads/4692#{#1247}) on port 38237
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
Úno 02, 2022 8:02:15 DOP. org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Current color is: Lime
Current color is: Blue
Color has been changed: true
Related
I can't get the Java / Javascript bridge to work on Java11 and Java13. The bridge seems to work fine i Java8 and Java10.
Here is essentially the same code as https://stackoverflow.com/a/34840552/11329518, which again works for me on Java8 and Java10:
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import javafx.application.Application;
import javafx.concurrent.Worker.State;
import javafx.scene.control.ButtonType;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebEvent;
import javafx.scene.web.WebView;
import javafx.stage.Stage;
import netscape.javascript.JSObject;
public class Main extends Application {
public static void main(String[] args) {
launch(args);
}
JavaBridge bridge;
WebEngine webEngine;
#Override
public void start(Stage primaryStage) throws MalformedURLException {
final URL url = new File("C:/test.html").toURI().toURL();
WebView webView = new javafx.scene.web.WebView();
webEngine = webView.getEngine();
webEngine.load(url.toExternalForm());
webEngine.setJavaScriptEnabled(true);
webEngine.setOnAlert(Main::showAlert);
webEngine.getLoadWorker().stateProperty().addListener((ov, oldState, newState) -> {
if (newState == State.SUCCEEDED) {
System.out.println("READY");
JSObject jsobj = (JSObject) webEngine.executeScript("window");
bridge = new JavaBridge();
jsobj.setMember("bridge", bridge);
}
});
primaryStage.setScene(new javafx.scene.Scene(webView, 300, 300));
primaryStage.show();
}
// Shows the alert, used in JS catch statement
private static void showAlert(WebEvent<String> event) {
javafx.scene.control.Dialog<ButtonType> alert = new javafx.scene.control.Dialog<>();
alert.getDialogPane().setContentText(event.getData());
alert.getDialogPane().getButtonTypes().add(ButtonType.OK);
alert.showAndWait();
}
public class JavaBridge {
public void hello() {
System.out.println("hello");
}
}
}
with test.html containing the Javascript:
<button onclick="try{bridge.hello();}catch(err){alert(err.message);}">call java</button>
What's going on?
I get the following error when clicking the button:
bridge.hello is not a function. (In 'bridge.hello()', 'bridge.hello' is undefined)
I can't reproduce your issue with JavaFX 13 or 14-ea+6, using Java 11 (OpenJDK 11.0.2) or Java 13 (OpenJDK 13).
However I can reproduce the issue, if I remove the strong reference to JavaBridge and I use Java 11.
This:
jsobj.setMember("bridge", new JavaBridge());
fails with the same error you have posted, with Java 11. But when using Java 13 (OpenJDK 13), that works fine (and also with Java 12).
Are you using other Java vendors? Can you try with OpenJDK https://jdk.java.net/13/?
There are two things that resolved this issue with OpenJDK 14 and JavaFx 14
Hard Reference to Bridge Object
bridge = new JavaBridge(); // create Bridge before hand
webEngine.getLoadWorker().stateProperty().addListener((ov, oldState,
newState) -> {
if (newState == State.SUCCEEDED) {
System.out.println("READY");
JSObject jsobj = (JSObject) webEngine.executeScript("window");
jsobj.setMember("bridge", bridge);
}
});
Use Bridge in Javascript/HTML page after the page is loaded
window.onload = function() {
bridge.hello();
}
I am a beginner with Selenium webdrivers. I'm getting a compilation error with the below code..Can anyone assist?
I am trying to copy a message into a file instead of showing it on the console.
testResultFile="C:\\CopyMessageTest.txt";
File file = new file(testResultFile).canWrite();
FileOutputStream fis = new FileOutputStream(file);
PrintStream out = new PrintStream(fis);
System.setOut(out);
System.out.println("----------Sucessfully Logged In ----------------");
The error is on the line
File file = new file(testResultFile).canWrite();
There are few mistakes that you have done in the code. In the first line you did not mention what type of data it is, it should be a String. In the second line, the object that you are creating should be of type File, you mentioned file, also, the return type of canWrite() method is a boolean, if you still want to use that mention, define it explicitly in the next line. see the corrected code below.
String testResultFile="C:\\CopyMessageTest.txt";
File file = new File(testResultFile);
file.canWrite();
FileOutputStream fis = new FileOutputStream(file);
PrintStream out = new PrintStream(fis);
System.setOut(out);
System.out.println("----------Sucessfully Logged In ----------------");
new file(testResultFile).canWrite(); will return the Boolean value to check can we writable or not. You have to use in this way to check that
boolean bool = new file(testResultFile).canWrite();
To write into file simply you can use
FileOutputStream fop = null;
File file;
String content = "This is the text content";
try {
file = new File("c:/newfile.txt");
fop = new FileOutputStream(file);
// if file doesnt exists, then create it
if (!file.exists()) {
file.createNewFile();
}
// get the content in bytes
byte[] contentInBytes = content.getBytes();
fop.write(contentInBytes);
fop.flush();
fop.close();
System.out.println("Done");
} catch (Exception e) {
e.printStackTrace();
}
Updating with another way as per comment
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintStream;
public class PrintTest {
public static void main(String[] args) throws FileNotFoundException {
// TODO Auto-generated method stub
File file=new File("C:\\CopyMessageTest.txt"); //my file where i want to write
boolean bool = file.canWrite(); //cross checking file can be writable or not
if(bool==false){
file.setWritable(true); //if not writable then set it to writable
}
FileOutputStream fis = new FileOutputStream(file);
PrintStream out = new PrintStream(fis);
System.setOut(out);
System.out.println("----------Sucessfully Logged In ----------------");
}
}
Thank You,
Murali
URL url = new URI("http://tamilblog.ishafoundation.org/").toURL();
I want to give 100 url in this connector?
I have the code to run single home page to extract its data
I need to get more contents From several links !
Is it any way possible ?
Here is my code
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class NewClass {
public static void main(String[] args) throws IOException {
Document doc = Jsoup.connect("http://tamilblog.ishafoundation.org").get();
Elements section = doc.select("section#content");
Elements article = section.select("article");
for (Element a : article) {
System.out.println("Title : \n" + a.select("a").text());
System.out.println("Article summary: \n" + a.select("div.entry-summary").text());
}
}
}
I got output from this single page Now i need to get it from several pages (100+links) at once.
And to save it in a document.
I have a problem when I run this code. I get always this message : "Document has no pages" with a blank page in the pdf file .
Note: I'm using NetBeans IDE 7.2 Beta.
Jar files imported are:
com.lowagie.text-2.1.7.jar
commons-beanutils-1.5.jar
commons-collections-2.1.jar
commons-digester-2.1.jar
commons-javaflow.jar
commons-logging-1.3.jar
jasperreports-5.5.1.jar
javax.servlet.jar
I need your help,please.
this is my code :
import java.math.BigDecimal;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.util.Currency;
import java.util.HashMap;
import javax.swing.JFrame;
import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JasperCompileManager;
import net.sf.jasperreports.engine.JasperExportManager;
import net.sf.jasperreports.engine.JasperFillManager;
import net.sf.jasperreports.engine.JasperPrint;
import net.sf.jasperreports.engine.JasperReport;
import net.sf.jasperreports.swing.JRViewer;
public class Report extends JFrame{
public Report(String month,int year){
try{
//load the driver
Class.forName("com.mysql.jdbc.Driver");
String url="jdbc:mysql://localhost:3306/motor";
String user="root";
String pass="";
Connection cn=DriverManager.getConnection(url,user,pass);
System.out.println("connected in report");
PreparedStatement ps = cn.prepareStatement("select number,title,name,ikar,echtirak,price,tarakom from users order by number asc");
ResultSet rs;
rs=ps.executeQuery();
JasperPrint jasperPrint=null;
HashMap<String, Object> mapParameters = new HashMap <String, Object>();
// jrxml compiling process
JasperReport jasperReport = JasperCompileManager.compileReport("C:/Users/user/Documents/NetBeansProjects/MotorApp/MyReports/motorReport.jrxml");
int i=1;
while(rs.next()){
mapParameters.put("Name", rs.getString("title")+" "+rs.getString("name"));
mapParameters.put("Number",rs.getString("number"));
mapParameters.put("Ikar",rs.getString("ikar"));
mapParameters.put("Amperage",rs.getString("echtirak")+" A");
mapParameters.put("tarakom",rs.getString("tarakom")+" $");
mapParameters.put("monthPrice",rs.getString("price")+" $");
Integer total=Integer.parseInt(rs.getString("price"))+Integer.parseInt(rs.getString("tarakom"));
mapParameters.put("totalPrice",total+" $");
mapParameters.put("monthYear",year+" "+month);
try{
System.out.println("Filling report...");
jasperPrint = JasperFillManager.fillReport(jasperReport, mapParameters,cn);
JRViewer viewer = new JRViewer(jasperPrint);
System.out.println("Done!");
JasperExportManager.exportReportToPdfFile(jasperPrint,"C:/Users/user/Desktop/"+month+" "+year+".pdf");
}catch(JRException elle){
System.out.println(elle.getMessage());
}
}
}catch(Exception ev){
System.out.println(ev.getMessage());
}
}
}
I truly appreciate your help.
My first thought is on the motorReport.jrxml file. It may have some issues on its content. Try reducing the contents of the report to a point where the code works and the reports can display properly. It may give you a clue to the cause of the problem. Then get back here with more specifics if you still can't solve this problem.
mapParameters in your code is not records of DataSource. It is parameters of report, therefore report haven't records.
One of the possible solutions: convert ResultSet into BeanCollection, make JRBeanCollectionDataSource and pass it into fillReport.
Make bean for saving result of query.
public class UserBean {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
....
}
For example result is
List myList = new ArrayList<UserBean>();
Convert into collection example here stackoverflow.com/questions/17206523/put-resultset-values-into-collection-object-then-add-to-arraylist
After convert you must call fillReport like this:
jasperPrint = JasperFillManager.fillReport(jasperReport, new HashMap<String,Object>, new JRBeanCollectionDataSource(myList));
Ok I am trying to implement an phone gap plugin, that consists of two parts. I am using cordova 2.0.0 and eclipse.
Here is the java part:
package org.apache.cordova;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.cordova.api.Plugin;
import org.apache.cordova.api.PluginResult;
import org.json.JSONArray;
import android.graphics.Bitmap;
import android.os.Environment;
import android.view.View;
public class Screenshot extends Plugin {
#Override
public PluginResult execute(String action, JSONArray args, String callbackId) {
// starting on ICS, some WebView methods
// can only be called on UI threads
final Plugin that = this;
final String id = callbackId;
super.cordova.getActivity().runOnUiThread(new Runnable() {
//#Override
public void run() {
View view = webView.getRootView();
view.setDrawingCacheEnabled(true);
Bitmap bitmap = Bitmap.createBitmap(view.getDrawingCache());
view.setDrawingCacheEnabled(false);
try {
File folder = new File(Environment.getExternalStorageDirectory(), "Pictures");
if (!folder.exists()) {
folder.mkdirs();
}
File f = new File(folder, "screenshot_" + System.currentTimeMillis() + ".png");
FileOutputStream fos = new FileOutputStream(f);
bitmap.compress(Bitmap.CompressFormat.PNG, 100, fos);
that.success(new PluginResult(PluginResult.Status.OK), id);
} catch (IOException e) {
that.success(new PluginResult(PluginResult.Status.IO_EXCEPTION, e.getMessage()), id);
}
}
});
PluginResult result = new PluginResult(PluginResult.Status.NO_RESULT);
result.setKeepCallback(true);
return result;
}
}
Here is the javascript part:
cordova.define("cordova/plugin/screenshot", function(require, exports, module) {
var exec = require('cordova/exec');
/**
* This class exposes the ability to take a Screenshot to JavaScript
*/
var Screenshot = function() {};
/**
* Save the screenshot to the user's Photo Library
*/
Screenshot.prototype.saveScreenshot = function() {
exec(null, null, "Screenshot", "saveScreenshot", []);
};
var screenshot = new Screenshot();
module.exports = screenshot;
});
if (!window.plugins) {
window.plugins = {};
}
if (!window.plugins.screenshot) {
window.plugins.screenshot = cordova.require("cordova/plugin/screenshot");
}
I am trying to call this with another javascript function on another page, but without success. I hide the anchors of an image on a canvas, then this line:
setTimeout(takeScreenShot,500);
EDIT -- made after Simon MacDonald's answer
this then relates to a javascript function:
function takeScreenShot() {
window.plugins.screenshot.saveScreenshot();
}
The following java prints:
System.out.println(folder);
System.out.println("screenshot_" + System.currentTimeMillis() + ".png");
Produce the following results:
/mdt/sdcard/Pictures
screenshot_1347893081276.png
EDIT After turning the device off and on again, the screenshots I took appeared, the phone seems to cache them, and not actually store them to the selected folder.
I have ensured that my config.xml and my android manifest have the right permissions and lines of code. Anyone see where im going wrong?
nowhere in your code do you ever call the saveScreenshot method. Your takeScreenShot method should look like this:
function takeScreenShot() {
window.plugins.screenshot.saveScreenshot();
}
Then the screen shot should be saved in "/sdcard/Pictures". This will work if:
You remembered to put a plugin line into your config.xml for the screen shot plugin
You mentioned this is on another page so make sure that page has a script tag for screenshot.js.
Did you try with the Obsolute Path?
File sdCard = Environment.getExternalStorageDirectory();
File folder = new File (sdcard.getAbsolutePath() + "Pictures");