Connecting Browsers in Puppeteer - javascript

Is it possible to connect a browser to puppeteer without instantiating it in puppeteer? For example, running an instance of chromium like a regular user and then connecting that to an instance of puppeteer in code?

The answer is Yes and No.
You can connect to an existing using the connect function:
const browserURL = 'http://127.0.0.1:21222';
const browser = await puppeteer.connect({browserURL});
But, if you want to use those 2 lines you need to launch Chrome with the "--remote-debugging-port=21222 argument.

I believe you need to connect to an address ended with an id:
ws://127.0.0.1:9222/devtools/browser/{id}
When you launch Chrome with --remote-debugging-port, you'll see something like
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 [17:57:55]
...
DevTools listening on ws://127.0.0.1:9222/devtools/browser/44b3c476-5524-497e-9918-d73fa39e40cf
The address on the last line is what you need, i.e.
const browser = await puppeteer.connect({
browserWSEndpoint: "ws://127.0.0.1:9222/devtools/browser/44b3c476-5524-497e-9918-d73fa39e40cf"
});

Related

connect puppeteer to already open browser page

I'm creating an chrome extension that helps people to login in one webesite, and I need to puppeteer connect in this url that the user is, can I connect in one already open website page to manipulate it?
I've tried to
const browserURL = "http://127.0.0.1:21222";
const browser = await puppeteer.connect({ browserURL });
and I tried start the chrome with:
chrome.exe --remote-debugging-port=21222
I need to connect one specific url for example fecebook.com, I tied:
example:
const browserURL = "http://facebook.com:21222";
and without the ":21222"...
I'm using window 10
node v16.16.0
thanks for helping!

how to connect puppeteer with an open chrome browser using authenticated proxies

so am connection puppeteer to an already opened chrome browser Using This Peace of code
const browserURL = 'http://127.0.0.1:9222';
const browser = await puppeteer.connect({browserURL , defaultViewport : null });
const page = await browser.newPage();
but most of the times i want to connect to the browser using an authenticated proxies , maybe launching the chrome with specific flags ?
i tried proxy-login-Automator and it would launch chrome with authenticated proxy that i would connect to it but this so complicated if i want to keep changing proxies + if am using many instances of the same code
am launching Chrome with this.
chrome --remote-debugging-port=9222 --user-data-dir="C:\Users\USER\AppData\Local\Google\Chrome\User Data

puppeteer chromium url redirects to malware

I have a node.js (v12.17.0) npm (6.14.4) project I am running on my Windows 10 command prompt. It is using puppeteer ("puppeteer": "^5.3.1",) to go to a url and get info.
My puppeteer code runs this to go to the page:
let targetURL = "https://www.walmart.com/ip/Small-Large-Dogs-Muzzle-Anti-Stop-Bite-Barking-Chewing-Mesh-Mask-Training-S-XXL/449423974";
await page.goto(targetURL, { waitUntil: 'networkidle2' });
And my puppeteer is running a chromium exe I just downloaded, I specify it's locating when I start puppeteer:
var options = {
executablePath: "C:\\Users\\marti\\Downloads\\chrome-win-new\\chrome-win\\chrome.exe",
So when puppeteer stars and goes to that URL, it loads the walmart page at first and seems fine, then a couple of seconds later the page looks like this:
I ran windows defender and malware antibytes to double check if i had a virus, I checked and if i open chromium without puppeteer it doesnt redirect, only seems to happen in my node js program when I run it.
I tried using a different url (https://www.petcarerx.com/chicken-soup-for-the-soul-bacon-and-cheese-crunchy-bites-dog-treats/32928?sku=50857#50857) and it did not redirect, does anyone know how i can fix this for my Walmart url?

Connect to already started web driver with selenium js

I'm not an expert of Selenium, so I may miss something here.
One of the software in the corp starts a google chrome with ChromeDriver.
I would like to attach to this browser from my JavaScript code.
I know the port where the ChromeDriver starts:
Starting ChromeDriver 77.0.3865.10 (bc3579f611bbc73331171afe020ec7a45e6ccc55-refs/branch-heads/3865#{#93}) on port 55848
I try to connect from JS:
const webdriver = require('selenium-webdriver')
void async function() {
let driver = await new webdriver.Builder().forBrowser('chrome').usingServer('http://localhost:55848/').build();
await driver.get('http://www.google.com/ncr');
await driver.findElement(By.name('q')).sendKeys('webdriver');
await driver.findElement(By.name('btnG')).click();
await driver.wait(until.titleIs('webdriver - Google Search'), 1000);
driver.quit();
}();
The connection is not successful. What I can think is that this code tries to start a new instance.
There is an error message:
SessionNotCreatedError: session not created: This version of ChromeDriver only supports Chrome version 77
I checked that the running chrome version is 77 and the ChromeDriver is also 77. This Chrome that is started by the corp software is actually a portable version of Chrome. I have a Chrome 76 installed on my computer. What I can think is that the code I wrote tries to start a new instance of Chrome. And there the version does not match.
Any idea how I can connect to the existing one? And control it?
UPDATE:
I managed to do the same with Firefox. The Firefox is started with geckodriver. I still cannot connect it. The error message is:
SessionNotCreatedError: Session is already started
So I'm pretty sure that is not in connection with the chrome versions but it tries to create a new session instead of connecting the existing one.
Yes, the problem is with the version of chrome driver. Get chromedriver.exe version 77... or else you will keep getting this error as latest version of selenium 3.141.59 doesnt support other version of chrome. Go to this link (https://www.seleniumhq.org/download/) and get the chromedriver.exe latest version and I think it will solve your problem.
Error - SessionNotCreatedError: session not created: This version of ChromeDriver only supports Chrome version 77
The error is related to mismatch between chromedriver.exe and chromebrowser version.
So based on browser version installed in operating system we have, we can download chromedriver.exe from below link https://chromedriver.chromium.org/downloads
Actual question answer -
Yes, we can connect to an existing running selenium server by setting desired capabilities provided by chrome. Then passing this capability while creating selenium session
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("debuggerAddress", "77.0.3865.10:55848");
WebDriver driver = new ChromeDriver(options);
The only thing to be searched is equivalent javascript code for doing same as above code is using java binding with selenium. I will surely update this answer with js binding.
#Edit=>
This is something I was able to find the way to pass chrome options in javascript. But sadly I didn't found setExperimentalOptions method here inside ChromeOptions. So I have used addArguments method as shown below.
const { Options } = require('selenium-webdriver/chrome');
const options = new Options();
options.addArguments('debuggerAddress=77.0.3865.10:55848');
builder.setChromeOptions(options);
const driver = builder.build();
driver.get('url');

How can I add a custom chrome extension to my Electron app?

I am facing some trouble adding chrome addons into my Electron BrowserWindow.
Before creating my window (and after the ready event has fired), I try to add a devtools extension that my browser needs to do screen sharing.
BrowserWindow.addDevToolsExtension('/home/USER/.config/chromium/Default/Extensions/dkjdkjlcilokfaigbckcipicchgoazeg/1.5_0');
I followed this Electron guide, and it worked for their example (adding the react develop tool). When I do the exact same thing with my own chrome extension I have this error:
[4735:1116/163422.268391:ERROR:CONSOLE(7701)] "Skipping extension with invalid URL: chrome-extension://extension-name", source: chrome-devtools://devtools/bundled/shell.js (7701)
I don't really get why the error specified is "invalid URL" since I'm doing the exact same thing / process with the react addon without a problem. I also have no idea what to do. Is it possible that my chrome addon is not Electron-compatible?
It looks like you're trying to add a regular Chrome extension instead of a Dev Tools extension.
The BrowserWindow.addExtension(path) method is for regular Chrome extensions:
BrowserWindow.addExtension(path)
path String
Adds Chrome extension located at path, and returns extension's name.
The method will also not return if the extension's manifest is missing or incomplete.
Note: This API cannot be called before the ready event of the app module is emitted.
- https://electronjs.org/docs/api/browser-window#browserwindowaddextensionpath
Conversely, the BrowserWindow.addDevToolsExtension(path) method is for Dev Tools extensions:
BrowserWindow.addDevToolsExtension(path)
path String
Adds DevTools extension located at path, and returns extension's name.
The extension will be remembered so you only need to call this API once, this API is not for programming use. If you try to add an extension that has already been loaded, this method will not return and instead log a warning to the console.
The method will also not return if the extension's manifest is missing or incomplete.
Note: This API cannot be called before the ready event of the app module is emitted.
- https://electronjs.org/docs/api/browser-window#browserwindowadddevtoolsextensionpath
Note that in both cases you need to wait for the ready event from the app module to be emitted:
const { BrowserWindow, app } = require('electron')
let mainWindow = null
function main() {
BrowserWindow.addExtension('/path/to/extension')
mainWindow = new BrowserWindow()
mainWindow.loadURL('https://google.com')
mainWindow.on('close', event => {
mainWindow = null
})
}
app.on('ready', main)
Support for Chromium extensions in Electron is actively being worked on at the moment. The support isn't complete yet, but the GitHub issue seems to have regular updates being posted.
Fingers crossed!
A current pull request is open for 'just enough extensions [api] to load a simple ... extension'
Electron 9 has much more support for extensions!
To load them, use session.loadExtension: https://github.com/electron/electron/blob/master/docs/api/extensions.md
const { app, BrowserWindow, session } = require('electron')
// ... in your createWindow function, which is called after app.whenReady
const mainWindow = new BrowserWindow({...})
const ext = await session.defaultSession.loadExtension('/path/to/unpacked/chrome-ext')
console.log('ext', ext)
// outputs config file
// {
// id: 'dcpdbjjnmhhlnlbibpeeiambicbbndim',
// name: 'Up! – Free Social Bot',
// path: '/Users/caffeinum/Development/GramUp/chrome-ext',
// url: 'chrome-extension://dcpdbjjnmhhlnlbibpeeiambicbbndim/',
// version: '1.7.0',
// manifest: { ... }
// }
Read more: https://github.com/electron/electron/blob/master/docs/api/extensions.md
Also, there's another project that helps you do that, also adds additional functionality: https://github.com/sentialx/electron-extensions
While there is a documented method to register a normal extension, in majority of cases it won't do much, as Electron supports only an accessibility subset of the chrome.* APIs (apparently only the stuff required by Spectron and Devtron) and as they've stated a while ago, they don't have any plans to support Chrome extension APIs at a full scale.

Categories

Resources