I'm building an App (Electron based) where I need to get an information from a third party website before the main window is created, but I'm a little bit confused about security measures. I'm using axios to do the HTTP request inside the main process because it is promise based and I can create the window after the website is fetched. My concerns are:
Enabling nodeIntegration is not good when messing with the renderer process because of cross-site-scripting attack. Should I include all nodejs modules in a preload.js like the following, for example.
index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="Content-Security-Policy" content="script-src 'self';">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Viewer</title>
</head>
<body>
<div id="box">
<form id='fo'>
<input type="text" id="num">
<button type="button" id="bttn">Random</button>
</form>
</div>
<script src="renderer.js"></script>
</body>
</html>
main.js
const electron = require('electron');
const cheerio = require('cheerio');
const axios = require('axios').default;
const path = require('path');
const {app, BrowserWindow, ipcMain, Menu, MenuItem,session} = electron;
let win;
let url = 'sampletext';
function createWindow() {
win = new BrowserWindow({
width: 400,
height: 250,
webPreferences:{
nodeIntegration: false,
contextIsolation: true,
preload: path.join(app.getAppPath(), 'preload.js')
},
show: false,
});
win.loadFile('index.html');
win.once('ready-to-show', () =>{
win.show();
});
win.on('closed', () =>{
win = null;
});
}
app.whenReady().then(getRequest().then(res => {
const $ = cheerio.load(res);
if($('infoNeeded')){
random = get_numbers($('infoNeeded').attr('href'));
}
createWindow();
}));
app.on('window-all-closed', () =>{
app.quit();
});
function getRequest() {
return axios.get(url).then(res => res.data).catch(err => console.log(err));
}
preload.js
//Instead of using getRequest() on main.js use this file
const electron = require('electron');
const remote = require('electron').remote;
const cheerio = require('cheerio');
const axios = require('axios').default;
let url = 'sampletext';
//So I can use it in renderer.js
window.getReq = function () {
return axios.get(url).then(res => res.data).catch(err => console.log(err));
}
window.parseInfo = function (data) {
const $ = cheerio.load(data);
if($('infoNeeded')){
return random = get_numbers($('infoNeeded').attr('href'));
}
return;
}
//Preload first request
window.getReq().then(doStuffHere);
renderer.js
let info;
//Keep updating the info
setInterval( () =>{
window.getReq().then(data => {
info = window.parseInfo(data);
});
}, 10000);
1) Is it ok to do nodejs require inside main process? If not, what's the secure way of doing it?
2) May I make HTTP requests inside main process? If yes, should I send a CSP header when doing so?
3) Instead of doing the request inside the main.js, should I use "webPreferences: preload" property and make the first HTTP request inside preload.js (Just like the above example) ? (I need to get the info before sending it to renderer.js)
I've already read https://www.electronjs.org/docs/tutorial/security, but I couldn't grasp their teaching. If you could provide an answer for how and when to use preload.js and CSP header I'll be very grateful.
Yes it is ok to use node.js require in the main process(use any library with error handling, cause it may crash the app)
You can make an HTTP request from the main process
You can use Preload.js if you need the code execution result in the renderer process.(You can also use the ipc)
Related
I am building my first electron application and I am running into a JS error I am not sure how to fix. I have managed to setup the IPC for a basic password manager application. The use-case is simple:
I click an HTML button and the frontend connects to the backend and delivers a keyword from which to build a password.
Password is generated.
Upon completion, the password is returned to the frontend to be displayed via HTML in a header tag.
Here is an example of the expected behavior with the input string dog:
keyword --> dog
generated password --> dog-jdls4ksls
The issue I am seeing is that instead of printing the generated password, I am seeing:
[object Object]-jdls4ksls
My best guess is that, since I am using async/await, I am printing the promise memory object instead of the returned value. I do not know, however, how I would block to wait for completion. The code in question providing this output is the last line of render.js, which was called from the HTML body.
Any help would be appreciated!
For context, I am primarily a backend developer, with plenty of python and C/C++/C# experience. My goal is to rebuild a C#/.NET GUI application into an electron app.
Here is all my code.
main.js
const {app, BrowserWindow, ipcMain} = require("electron")
const path = require("path")
function generatePassword(keyword) {
console.log(keyword)
return keyword + '-' + Math.random().toString(36).substring(2,12)
}
function createWindow () {
const win = new BrowserWindow({
width: 800,
height: 600,
resizable: false,
webPreferences: {
preload: path.join(__dirname, 'preload.js')
}
})
win.loadFile('html/passwordGen.html')
}
app.whenReady().then(() => {
ipcMain.handle("generatePassword", generatePassword)
// console.log(generatePassword('test string')) // works
createWindow()
}).catch(error => {
console.log(error) // log error to console
app.quit() // quit the app
})
preload.js
const { contextBridge, ipcRenderer } = require('electron')
contextBridge.exposeInMainWorld('main', {
genPW: (keyword) => ipcRenderer.invoke("geåneratePassword", keyword)
})
render.js
async function testClick () {
const pw_root = document.getElementById("keyword")
const pw_label = document.querySelector("#password")
pw_label.innerText = await window.main.genPW(pw_root.value)
}
passwordGen.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Password Generator</title>
<link rel="stylesheet" href="../css/style.css">
<script src="../render.js"></script>
</head>
<body>
<h1>Password Generator</h1>
<input type="text" id="keyword" placeholder="Please enter a keyword...">
<button id="btn" onclick="testClick()">Generate Password</button>
<h1 id="password"></h1>
</body>
</html>
Edit:
Here is the code that worked. The accepted solution worked with the exception that I needed to either 1) keep the async/await on the generatePassword() or 2) convert it to .then() format as recommended in another solution.
main.js
const {app, BrowserWindow, ipcMain} = require("electron")
const path = require("path")
function generatePassword(keyword) {
console.log(keyword)
return keyword + '-' + Math.random().toString(36).substring(2,12)
}
function createWindow () {
const win = new BrowserWindow({
width: 800,
height: 600,
resizable: false,
webPreferences: {
preload: path.join(__dirname, 'preload.js')
}
})
win.loadFile('html/passwordGen.html')
}
app.whenReady().then(() => {
// ipcMain.handle("generatePassword", generatePassword)
// console.log(generatePassword('stink')) // works
ipcMain.handle('generatePassword', (_event, keyword) => {
console.log(keyword); // Testing
return generatePassword(keyword);
});
createWindow()
}).catch(error => {
console.log(error) // log error to console
app.quit() // quit the app
})
preload.js
const { contextBridge, ipcRenderer } = require('electron')
contextBridge.exposeInMainWorld('main', {
genPW: (keyword) => {
return ipcRenderer.invoke("generatePassword", keyword)
}
})
render.js
async function testClick () {
const pw_root = document.getElementById("keyword")
const pw_label = document.querySelector("#password")
pw_label.innerText = await window.main.genPW(pw_root.value)
// window.main.genPW(pw_root.value).then(res => {pw_label.innerText = res})
// ^^^ works as well if async/await removed
}
You were really close. Using Elctron's invoke method is the correct approach.
Within your main.js file, Electron's IPC handle signature contains the channel and listener arguments. In
your code you are calling your generatePassword() function in place of the listener argument. Instead, it should
be (event, ...args). In your specific case (event, keyword).
See ipcMain.handle(channel, listener)
for more information.
Additionally, within your preload.js script, all you need to do is add a return statement in front of your ipcRenderer.invoke method.
Finally, there is no need to use async on your testClick() function. Electron's invoke handles all of this.
main.js (main process)
const electronApp = require('electron').app;
const electronBrowserWindow = require('electron').BrowserWindow;
const electronIpcMain = require('electron').ipcMain;
const nodePath = require('path');
// Prevent garbage collection
let window;
function createWindow() {
window = new electronBrowserWindow({
x: 0,
y: 0,
width: 800,
height: 600,
resizable: false,
show: false,
webPreferences: {
nodeIntegration: false,
contextIsolation: true,
preload: nodePath.join(__dirname, 'preload.js')
}
});
window.loadFile('index.html')
.then(() => { window.show(); });
return window;
}
electronApp.on('ready', () => {
window = createWindow();
});
electronApp.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
electronApp.quit();
}
});
electronApp.on('activate', () => {
if (electronBrowserWindow.getAllWindows().length === 0) {
createWindow();
}
});
// ---
function generatePassword(keyword) {
console.log(keyword)
return keyword + '-' + Math.random().toString(36).substring(2,12)
}
electronIpcMain.handle('generatePassword', (event, keyword) => {
console.log(keyword); // Testing
return generatePassword(keyword);
});
preload.js (main process)
const contextBridge = require('electron').contextBridge;
const ipcRenderer = require('electron').ipcRenderer;
contextBridge.exposeInMainWorld(
'main', {
genPW: (keyword) => {
return ipcRenderer.invoke('generatePassword', keyword);
}
});
For sake of simplicity, I have included your render.js script within <script> tags below the closing </body> tag.
index.html (render process)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Emergency</title>
<meta http-equiv="Content-Security-Policy" content="script-src 'self' 'unsafe-inline';"/>
</head>
<body>
<div>
<label for="password">Password:</label>
<input type="text" id="password">
<input type="button" id="submit" value="Submit">
</div>
<div>
<label for="generated-password">Generated Password:</label>
<input type="text" id="generated-password" disabled>
</div>
</body>
<script>
document.getElementById('submit').addEventListener('click', () => {
window.main.genPW(document.getElementById('password').value)
.then((generatedPassword) => {
document.getElementById('generated-password').value = generatedPassword;
})
})
</script>
</html>
I dont think you can use async function as event listener, you need to use regular (not async) function here.
function testClick() {
const pw_root = document.getElementById("keyword")
const pw_label = document.querySelector("#password")
window.main.genPW(pw_root.value).then(res => {pw_label.innerText = res})
}
also, you have typo here: invoke("geåneratePassword")
Try using double await as in
await (await callToBackend)
I have an Electron app that is displayed using BrowserWindow. I want to open an external URL in the same window so that the user can log in (to an external website) and after the user logs in it should display the Electron application again instead of the external website that the user used to log in.
I've been able to open the external url in the same window by using:
<a href="https://loginsite-example.com" target="_blank" rel="noreferrer">
site where you have to log in
</a>
However, I don't know how to show the Electron application again after the user successfully logs in to the external website. Also, I would like to keep the session from the external website so that I could consume its API inside the electron application.
Moving between window sources, whether it be local (file) or remote (URL) can be accomplished by just calling window.loadFile(...) or window.loadURL(...), but only after the instance of the window has been created.
main.js (main thread)
const electronApp = require('electron').app;
const electronBrowserWindow = require('electron').BrowserWindow;
const electronIpcMain = require('electron').ipcMain;
const nodePath = require("path");
// Prevent garbage collection
let window;
function createWindow() {
return new electronBrowserWindow({
x: 0,
y: 0,
width: 800,
height: 600,
show: false,
webPreferences: {
nodeIntegration: false,
contextIsolation: true,
preload: nodePath.join(__dirname, 'preload.js')
}
});
}
function showMainWindow() {
window.loadFile('index.html')
.then(() => { window.show(); })
}
function showLoginWindow() {
// window.loadURL('https://www.your-site.com/login')
window.loadFile('login.html') // For testing purposes only
.then(() => { window.show(); })
}
electronApp.on('ready', () => {
window = createWindow();
showMainWindow();
});
electronApp.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
electronApp.quit();
}
});
electronApp.on('activate', () => {
if (electronBrowserWindow.getAllWindows().length === 0) {
createWindow();
}
});
// ----- IPC -----
electronIpcMain.on('message:loginShow', (event) => {
showLoginWindow();
})
electronIpcMain.on('message:loginSuccessful', (event, session) => {
showMainWindow();
})
index.html (render thread)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Main Window</title>
</head>
<body>
<div>Main Window</div><hr>
<button type="button" id="show-login">Login</button>
</body>
<script>
document.getElementById('show-login').addEventListener('click', () => {
window.ipcRender.send('message:loginShow');
});
</script>
</html>
login.html (render thread)
Used for testing purposes only as we do not have access to a real login page.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Login Window</title>
</head>
<body>
<div>Login Window</div><hr>
<label for="username">Username: </label>
<input type="text" id="username"><br>
<label for="password">Password: </label>
<input type="password" id="password"><br>
<button type="button" id="login">Login</button>
</body>
<script>
// For testing purposes only.
document.getElementById('login').addEventListener('click', () => {
window.ipcRender.send('message:loginSuccessful');
});
</script>
</html>
And finally, a preload.js script to communicate safely between the main thread and render thread(s).
preload.js (main thread)
// Import the necessary Electron components.
const contextBridge = require('electron').contextBridge;
const ipcRenderer = require('electron').ipcRenderer;
// White-listed channels.
const ipc = {
'render': {
// From render to main.
'send': [
'message:loginShow',
'message:loginSuccessful'
],
// From main to render.
'receive': [],
// From render to main and back again.
'sendReceive': []
}
};
// Exposed protected methods in the render process.
contextBridge.exposeInMainWorld(
// Allowed 'ipcRenderer' methods.
'ipcRender', {
// From render to main.
send: (channel, args) => {
let validChannels = ipc.render.send;
if (validChannels.includes(channel)) {
ipcRenderer.send(channel, args);
}
},
// From main to render.
receive: (channel, listener) => {
let validChannels = ipc.render.receive;
if (validChannels.includes(channel)) {
// Deliberately strip event as it includes `sender`.
ipcRenderer.on(channel, (event, ...args) => listener(...args));
}
},
// From render to main and back again.
invoke: (channel, args) => {
let validChannels = ipc.render.sendReceive;
if (validChannels.includes(channel)) {
return ipcRenderer.invoke(channel, args);
}
}
}
);
A problem that will need to be overcome is once you submit your login credentials to the server, how are you going to get the session data? Your html Javascript will need to detect if and when the necessary session data is available. Once the session data is available, it is an easy process of transferring the session via IPC from the render thread to the main thread. To figure this out, one will require some additional information and understanding of your login system (a separate StackOverflow question).
As a pre-cursor, I suspect one would need to detect in the main thread when the login page has been submitted to the server via something like window.webContents.on('did-navigate', ...). Once detected, check the next loaded page quickly to see if a session exists. If so, get it, send it to the main thread and then redirect back to the index.html page.
I think there should be an easier way if you can login via an API. Then the whole process can be self-contained in your Electron application. IE: Show local (file) login.html, submit data to server and await a "success" or "fail" response. If successful, pass the session data in the response. If unsuccessful, display an appropriate error message.
I have a very basic html file (using electron);
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title> File Uploader </title>
<link rel="stylesheet" href="style.css">
<script defer src="render.js"></script>
</head>
<body>
<h1>Drive File Uploader</h1>
<input type="file" id="myFile" name="myFile">
<button onclick="FileUploadPing()">Upload your file</button>
</body>
</html>
and an event listener named render.js;
const ipcRenderer = require("electron").ipcRenderer;
const FileUploadPing = () => {
var input = document.getElementById("myFile").value
if (input) {
ipcRenderer.send("FileUploadPing",inputVal);
}else{console.log("no path value")}
};
ipcRenderer.on("FileRecievePing", (event, data) => {
alert(data)
});
But when I click submit, document.getElementById("myFile").value returns undefined
how can I pull that value?
This is an interesting issue that confronts many people using Electron. One could either use (via Electron) the native OS dialog.showOpenDialog([browserWindow, ]options) dialog or the html <input type="file"> tag.
To circumvent the need to manage the is prefixed with C:\fakepath\ issue, it is often better to use the native approach. After all, that is what Electron is best at.
Let me show you how to quickly set up a html button that when clicked, will open the native file selector dialog, and when a path is selected, return the path to the render thread for display.
In the below code, we will be using a preload.js script configured to communicate (using IPC) between the main thread and render thread(s). Context Isolation will describe this in more detail.
First off, let's code the main.js file which will include the creation of the native file dialog.
main.js (main thread)
const electronApp = require('electron').app;
const electronBrowserWindow = require('electron').BrowserWindow;
const electronDialog = require('electron').dialog;
const electronIpcMain = require('electron').ipcMain;
const nodePath = require("path");
// Prevent garbage collection
let window;
function createWindow() {
const window = new electronBrowserWindow({
x: 0,
y: 0,
width: 1000,
height: 700,
show: false,
webPreferences: {
nodeIntegration: false,
contextIsolation: true,
preload: nodePath.join(__dirname, 'preload.js')
}
});
window.loadFile('index.html')
.then(() => { window.show(); })
return window;
}
electronApp.on('ready', () => {
window = createWindow();
});
electronApp.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
electronApp.quit();
}
});
electronApp.on('activate', () => {
if (electronBrowserWindow.getAllWindows().length === 0) {
createWindow();
}
});
// Open the file dialog
electronIpcMain.on('message:openDialog', (event) => {
let options = {
title: 'Select File',
properties: ['openFile']
};
electronDialog.showOpenDialog(window, options)
.then((result) => {
if (result.canceled) {
console.log('User cancelled file dialog.');
return;
}
event.reply('message:dialogPath', result.filePaths[0]);
})
.catch((error) => { console.error(error); });
})
Now, let's create the index.html file, which for the sake of simplicity, also includes the Javascript within the <script> tags.
NB: Instead of deferring your script in the <head>, you can include it just before the closing <html> tag. Placing it here effectively performs the same thing as defer in the <head> <script> tag.
index.html (render thread)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Drive File Uploader</title>
</head>
<body>
<div>Drive File Uploader</div>
<hr>
<label for="path">Path: </label>
<input type="text" id="path" name="path">
<input type="button" id="open-dialog" name="open-dialog" value="...">
<input type="button" id="upload" value="Upload">
</body>
<script>
// Let's declare it as it is used more than once
let filePath = document.getElementById('path');
// Event listeners
document.getElementById('open-dialog').addEventListener('click', () => { window.ipcRender.send('message:openDialog'); });
document.getElementById('upload').addEventListener('click', () => { console.log(filePath.value); });
// IPC message from the main thread
window.ipcRender.receive('message:dialogPath', (path) => { filePath.value = path; })
</script>
</html>
Finally, let's add the preload.js script to allow the main thread and render thread(s) to safely communicate with each other.
Note: This is where we define our whitelisted channel names.
preload.js (main thread)
const contextBridge = require('electron').contextBridge;
const ipcRenderer = require('electron').ipcRenderer;
// White-listed channels.
const ipc = {
'render': {
// From render to main.
'send': [
'message:openDialog'
],
// From main to render.
'receive': [
'message:dialogPath'
],
// From render to main and back again.
'sendReceive': []
}
};
// Exposed protected methods in the render process.
contextBridge.exposeInMainWorld(
// Allowed 'ipcRenderer' methods.
'ipcRender', {
// From render to main.
send: (channel, args) => {
let validChannels = ipc.render.send;
if (validChannels.includes(channel)) {
ipcRenderer.send(channel, args);
}
},
// From main to render.
receive: (channel, listener) => {
let validChannels = ipc.render.receive;
if (validChannels.includes(channel)) {
// Deliberately strip event as it includes `sender`.
ipcRenderer.on(channel, (event, ...args) => listener(...args));
}
},
// From render to main and back again.
invoke: (channel, args) => {
let validChannels = ipc.render.sendReceive;
if (validChannels.includes(channel)) {
return ipcRenderer.invoke(channel, args);
}
}
}
);
Hopefully, the above outlines how simple it can be to use (via Electron) the native dialog boxes. The benefit being they have OS specific functionality and feel.
I dont know what I did different, but when I tried this it suddenly worked.
main.js;
const { app, BrowserWindow, ipcMain } = require("electron");
let win = null;
const createWindow = () => {
win = new BrowserWindow({
width: 800,
height: 600,
resizable: true,
webPreferences: {
nodeIntegration: true,
contextIsolation: false,
enableRemoteModule: true,
nativeWindowOpen: true,
},
});
win.loadFile("index.html");
};
app.whenReady().then(createWindow);
render.js;
const ipcRenderer = require("electron").ipcRenderer;
const {uploadFileToCloud} = require("C:/Users/efeba/desktop/pythonAndJS/ITGSProject/app.js")
const FileUploadPing = () => {
var name = document.getElementById("myFile").value
name = name.replace("C:\\fakepath\\","")
console.log(name)
var path = document.getElementById("myFile").files[0].path
path = path.replace(name,"")
path = path.replaceAll("\\", "/") // beautify the string
console.log(path)
if (path) {
uploadFileToCloud(name, path)
alert("File Uploaded")
}else{console.log("no path value")}
};
//file uploads but content is empty
//I suspect pulling the libraries in renderer has some problems, will
//try to make it by sending a ping to main.js
index.html;
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title> File Uploader </title>
<link rel="stylesheet" href="style.css">
<script defer src="render.js"></script>
</head>
<body>
<h1>Drive File Uploader</h1>
<input type="file" id="myFile" name="myFile">
<button onclick="FileUploadPing()">Upload your file</button>
</body>
</html>
Hope I could help
I need to download some 300 files from a direct download link. When the link is opened directly in the browser, an automatic pdf download gets triggered. The file gets downloaded and the browser doesn't go anywhere. The links are as follows:
www.link.com/store/item/123
In the link, the 123 part would be changed on every loop.
I was thinking of using puppeteer (with goto), but I guess since visiting the link automatically triggers the download of the pdf and doesnt actually go to the page, it fails.
This is what I tried, but its not working at all:
const links = ['123', '456'];
(async () => {
const browser = await puppeteer.launch({
headless: false //preferably would run with true
});
links.forEach( async link => {
const page = await browser.newPage();
await page.goto(
linkBeginning + link
);
await browser.close();
})
})();
I searched around, but I could not really find this specific case, all other cases are more focused on the user side or have the target file in the actual link (like xx/store/doc.pdf). Not quite sure if that makes a difference though. I would just need a script that will get me the pdf files for a one time run.
If anyone has a solution in php/python that would work as well, as this is just a one off thing.
edit:
ended up doing it in html
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
<script src="sku.js"></script>
<script>
const linkStart = 'https://www.sols-europe.com/gb/pdfpublication/pdf/product/sku/';
sku.forEach(element => {
document.write('<a target = "_blank" class="click" href="' + linkStart + element.id +'">'+ element.id +'</a></br>')
});
</script>
</head>
<body></body>
</html>
<script>
const clickInterval = setInterval(function () {
const el = document.querySelector('.click:not(.clicked)');
if(el){
el.classList.add('clicked');
el.click()
} else {
clearInterval(clickInterval);
}
}, 2000);
</script>
You don't need puppeteer to do this, and you can achieve it fairly easily in NodeJS:
import http from "https";
import fs from "fs";
(async () => {
const skus = ["00548", "03575"];
const filesPromiseArray = skus.map(
(sku) =>
new Promise((resolve, reject) => {
const file = fs.createWriteStream(`${sku}.pdf`);
const request = http.get(`https://www.sols-europe.com/gb/pdfpublication/pdf/product/sku/${sku}`, (response) =>
response.pipe(file)
);
file.on("finish", resolve);
file.on("error", reject);
})
);
try {
await Promise.all(filesPromiseArray);
} catch {
console.log("There was an error downloading one of the files");
}
})();
What is this code doing?
Taking your array of skus, we are using .map() to transform them into an array of requests.
Inside the .map() we're creating a Promise which will be successful (resolve) when the file finishes downloading, or unsuccessful (reject) if the download errors.
We then await all of the requests that we have just created. If one of them fails it will log.
Note:
If you are using CommonJS ("type":"commonjs", in your package.json), replace the two imports with:
const http = require('https');
const fs = require('fs');
Your placement for browser.close() inside loop isn't a good thing.
So i moved it outside forEach and change it to page.close() instead.
const links = ['123', '456']
const linkBeginning = 'https://www.link.com/store/item/'
;(async () => {
const browser = await puppeteer.launch({
headless: false //preferably would run with true
})
links.forEach( async link => {
const page = await browser.newPage()
const session = await page.target().createCDPSession()
await session.send('Page.setDownloadBehavior', {
behavior: 'allow',
downloadPath: './pdf/'
})
await page.goto(linkBeginning + link)
await page.close() // Don't use browser.close() inside loop
})
await browser.close() // Use here instead
})()
Months ago I built an electron app that scrapes data from a web page using selenium and then visualizes it inside the electron webpage and everything worked fine.
After a few months of not using it, I'm having trouble because of many breaking changes inside electron and selenium themselves.
The major breaking change is that is not possible to start selenium webdrivers from the renderer process anymore, but I can start it only in the main process.
This below is a minimal non-working example of what I'm trying to do:
// index.js - entry point of the program
const electron = require("electron");
let app = electron.app
let mainWin;
app.on('ready', () => {
mainWin = new electron.BrowserWindow({
width: 100,
height: 100,
frame: true,
backgroundColor: '#222222',
webPreferences: {
nodeIntegration: true,
contextIsolation: false
},
});
mainWin.loadFile('./home.html');
mainWin.on('closed', function () {
console.log('byebye')
});
});
// home.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
hello
</body>
<script type="text/javascript">require('./home.js')</script>
</html>
//home.js - script file for the home.html launched in the main process
const { Builder, By } = require('selenium-webdriver');
process.env.PATH += `${__dirname};`;
(async () => {
let driver = await new Builder().forBrowser('chrome').build();
await driver.get('https://www.google.com');
let test = await driver.findElements(By.css('div'))
console.log(test)
})()
The program gets completely stuck on the build for chrome webdrivers.
I am 100% sure that I am using the right chromedriver version and I never get an error or something useful, just empty, endless-running nothing.
Am I missing something (like webpreferences flags for the window) or this is a bug of electron/selenium?
It appears that this happens only when I'm using this on Linux.
Rebuilding the program to launch the drivers from the main process would mean rebuilding the program from scratch as it uses different windows and so on, and I can't pass the driver or anything else from the main process to the renderer using IRC since it breaks the driver object itself.
Ok, I managed to make it work on Linux too. The tweak is to use a preload script that will initialize the driver instance and then pass it to the renderer process by polluting the window object (it is the recommended way as shown here https://www.electronjs.org/docs/latest/tutorial/security#how-1). In this way, it is possible to obtain a fully working driver instance in the renderer process in Linux with selenium and electron.
Below are the changes to make it work:
// index.js - entry point of the program
const electron = require("electron");
let app = electron.app
let mainWin;
app.on('ready', () => {
mainWin = new electron.BrowserWindow({
width: 100,
height: 100,
frame: true,
backgroundColor: '#222222',
webPreferences: {
nodeIntegration: true,
contextIsolation: false
preload: path.join(__dirname,'preload.js')
},
});
mainWin.loadFile('./home.html');
mainWin.on('closed', function () {
console.log('byebye')
});
});
//preload.js
const { Builder, By } = require('selenium-webdriver');
(async () => {
let driver = await new Builder().forBrowser('chrome').build();
window.pollutingDriver = driver
})()
//home.js
//retrieve the driver in this way
const sleep = (ms) => {
return new Promise(resolve => setTimeout(resolve, ms))
}
var driver = null ;
(async () => {
while(!driver) {
driver = window.pollutingDriver
await sleep(500)
}
//do the main work on the page
})()