Zombiejs empty HTML - javascript

I'm deploying a Zombiejs application to Openshift, but Zombie seems to be unable to fetch the HTML.
I have an object (called Poker) that maintains the headless browser and does things with it. One of the methods, called init, logs in to a website and returns the "initialized" browser.
Poker.prototype.init = function (email, password, ip) {
var self = this;
var browser;
// Have to use the ip that Openshift provides
// See SO question http://goo.gl/n2TfMC
if (ip) {
browser = Zombie.create({
'localAddress': ip
});
} else {
browser = Zombie.create();
}
return new Promise(function (resolve, reject) {
browser
.visit('http://some.login/page')
.then(function () {
// Some debugging stuff :p
console.log('body: ');
console.log(browser.html('body'));
// Fill in the credentials
browser.fill('email', email);
browser.fill('password', password);
return browser.pressButton('Log In');
})
.done(function() {
// Logged in, new page loaded
// Check if login was successful
var title = browser.text('title');
console.log(title);
});
});
}
The console remains empty after printing body:, anden Zombie attempts to fill in the email address, I receive this error:
Possibly unhandled TypeError: Cannot use 'in' operator to search for 'compareDocumentPosition' in null
at /var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/node_modules/jsdom/node_modules/nwmatcher/src/nwmatcher-noqsa.js:267:43
at module.exports (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/node_modules/jsdom/node_modules/nwmatcher/src/nwmatcher-noqsa.js:37:7)
at addNwmatcher (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/node_modules/jsdom/lib/jsdom/selectors/index.js:6:27)
at HTMLDocument.<anonymous> (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/node_modules/jsdom/lib/jsdom/selectors/index.js:18:29)
at HTMLDocument.querySelectorAll (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/node_modules/jsdom/lib/jsdom/level1/core.js:63:53)
at Browser.queryAll (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/lib/zombie/browser.js:348:26)
at Browser.field (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/lib/zombie/browser.js:591:17)
at Browser._findOption (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/lib/zombie/browser.js:662:18)
at Browser.select (/var/lib/openshift/[app-id]/app-root/runtime/repo/node_modules/zombie/lib/zombie/browser.js:692:19)
at /var/lib/openshift/[app-id]/app-root/runtime/repo/poker.js:61:17
After I saw this, I tried to visit a different page (Google) through Zombie, but it returned empty HTML as well.
I took a look at some other StackOverflow questions about the compareDocumentPosition error, but I think the one I'm having is related to deployment on Openshift rather than an issue with the HTML of the page I'm visiting.
I'm using Node.js v0.10.25 and Zombie v2.2.1.

Can it be a CORS problem? .login is not a valid TLD and rules vary per TLD.

Are you sure the variable "browser" you are trying to work with is an Object, that error can occur if the variable is a string.

Related

Outlook AddIn GetAsync successful but returns nothing

I've got an Outlook Add In that was developed using the Office Javascript API.
It looks at the new email being composed & does things based on who it's going to: https://learn.microsoft.com/en-us/office/dev/add-ins/reference/objectmodel/requirement-set-1.3/office.context.mailbox.item
The code correctly returns the TO email when you 'select' the email from the suggested email list... screenshots shown # bottom of this thread
To debug the Javascript, I use C:\Windows\SysWOW64\F12\IEChooser.exe
It was working fine until last week. Is it possible a Windows update broke functionality?
I'm the only person with access to the code. It hadn't been modified for months.
When debugger is running, getAsync correctly returns the 'TO' value. I needed to write the response to a global variable to prove the values were 'undefined' while not in debug.
var resultObjects;
var resultObjects2;
var strMessages = '';
var strTo = '';
var mailbox;
var mailitem;
(function () {
"use strict";
// The Office initialize function must be run each time a new page is loaded.
Office.initialize = function (reason) {
$(document).ready(function () {
mailbox = Office.context.mailbox;
mailitem = mailbox.item;
mailitem.to.getAsync(function (result) {
if (result.status === 'failed') {
strMessages = 'FAILED';
} else {
strMessages = 'SUCCESS';
strTo = result.value[0];
resultObjects = result;
resultObjects2 = result.value;
}
});
loadApp();
});
};
})();
Here are the values of the variables, when the app is loaded & debugger is not running
EDIT
If you 'select' the TO email so that it is bolded... the code works correctly. If you leave the typed-in-text field without selecting the suggested email, it does not work. The same behavior is true for both the Outlook Web Application (# https://outlook.office.com) and the desktop outlook application.
Does not work
Does Work
The Office.context.mailbox.item.to.getAsync API will only return resolved recipients. If the TO email address is not resolved (as in the first screenshot titled "Does not Work"), then API will not return the email address until it is resolved (in both desktop and OWA).
You can use the RecipientsChanged Event, to get newly resolved recipients after you have queried for to.getAsync. This event would fire when a recipient is newly resolved.

Re-using same instance again webdriverJS

I am really new to Selenium. I managed to open a website using the below nodejs code
var webdriver = require('selenium-webdriver');
var driver = new webdriver.Builder()
.forBrowser('chrome')
.build();
console.log(driver);
driver.get('https://web.whatsapp.com');
//perform all other operations here.
https://web.whatsapp.com is opened and I manually scan a QR code and log in. Now I have different javascript files to perform actions like delete, clear chat inside web.whatsapp.com etc...
Now If I get some error, I debug and when I run the script again using node test.js, it takes another 2 minutes to load page and do the steps I needed. I just wanted to reopen the already opened tab and continue my script instead new window opens.
Edit day 2 : Still searching for solution. I tried below code to save object and reuse it.. Is this the correct approach ? I get a JSON parse error though.
var o = new chrome.Options();
o.addArguments("user-data-dir=/Users/vishnu/Library/Application Support/Google/Chrome/Profile 2");
o.addArguments("disable-infobars");
o.addArguments("--no-first-run");
var driver = new webdriver.Builder().withCapabilities(webdriver.Capabilities.chrome()).setChromeOptions(o).build();
var savefile = fs.writeFile('data.json', JSON.stringify(util.inspect(driver)) , 'utf-8');
var parsedJSON = require('./data.json');
console.log(parsedJSON);
It took me some time and a couple of different approaches, but I managed to work up something I think solves your problem and allows to develop tests in a rather nice way.
Because it does not directly answer the question of how to re-use a browser session in Selenium (using their JavaScript API), I will first present my proposed solution and then briefly discuss the other approaches I tried. It may give someone else an idea and help them to solve this problem in a nicer/better way. Who knows. At least my attempts will be documented.
Proposed solution (tested and works)
Because I did not manage to actually reuse a browser session (see below), I figured I could try something else. The approach will be the following.
Idea
Have a main loop in one file (say init.js) and tests in a separate file (test.js).
The main loop opens a browser instance and keeps it open. It also exposes some sort of CLI that allows one to run tests (from test.js), inspect errors as they occur and to close the browser instance and stop the main loop.
The test in test.js exports a test function that is being executed by the main loop. It is passed a driver instance to work with. Any errors that occur here are being caught by the main loop.
Because the browser instance is opened only once, we have to do the manual process of authenticating with WhatsApp (scanning a QR code) only once. After that, running a test will reload web.whatsapp.com, but it will have remembered that we authenticated and thus immediately be able to run whatever tests we define in test.js.
In order to keep the main loop alive, it is vital that we catch each and every error that might occur in our tests. I unfortunately had to resort to uncaughtException for that.
Implementation
This is the implementation of the above idea I came up with. It is possible to make this much fancier if you would want to do so. I went for simplicity here (hope I managed).
init.js
This is the main loop from the above idea.
var webdriver = require('selenium-webdriver'),
by = webdriver.By,
until = webdriver.until,
driver = null,
prompt = '> ',
testPath = 'test.js',
lastError = null;
function initDriver() {
return new Promise((resolve, reject) => {
// already opened a browser? done
if (driver !== null) {
resolve();
return;
}
// open a new browser, let user scan QR code
driver = new webdriver.Builder().forBrowser('chrome').build();
driver.get('https://web.whatsapp.com');
process.stdout.write("Please scan the QR code within 30 seconds...\n");
driver.wait(until.elementLocated(by.className('chat')), 30000)
.then(() => resolve())
.catch((timeout) => {
process.stdout.write("\b\bTimed out waiting for code to" +
" be scanned.\n");
driver.quit();
reject();
});
});
}
function recordError(err) {
process.stderr.write(err.name + ': ' + err.message + "\n");
lastError = err;
// let user know that test failed
process.stdout.write("Test failed!\n");
// indicate we are ready to read the next command
process.stdout.write(prompt);
}
process.stdout.write(prompt);
process.stdin.setEncoding('utf8');
process.stdin.on('readable', () => {
var chunk = process.stdin.read();
if (chunk === null) {
// happens on initialization, ignore
return;
}
// do various different things for different commands
var line = chunk.trim(),
cmds = line.split(/\s+/);
switch (cmds[0]) {
case 'error':
// print last error, when applicable
if (lastError !== null) {
console.log(lastError);
}
// indicate we are ready to read the next command
process.stdout.write(prompt);
break;
case 'run':
// open a browser if we didn't yet, execute tests
initDriver().then(() => {
// carefully load test code, report SyntaxError when applicable
var file = (cmds.length === 1 ? testPath : cmds[1] + '.js');
try {
var test = require('./' + file);
} catch (err) {
recordError(err);
return;
} finally {
// force node to read the test code again when we
// require it in the future
delete require.cache[__dirname + '/' + file];
}
// carefully execute tests, report errors when applicable
test.execute(driver, by, until)
.then(() => {
// indicate we are ready to read the next command
process.stdout.write(prompt);
})
.catch(recordError);
}).catch(() => process.stdin.destroy());
break;
case 'quit':
// close browser if it was opened and stop this process
if (driver !== null) {
driver.quit();
}
process.stdin.destroy();
return;
}
});
// some errors somehow still escape all catches we have...
process.on('uncaughtException', recordError);
test.js
This is the test from the above idea. I wrote some things just to test the main loop and some WebDriver functionality. Pretty much anything is possible here. I have used promises to make test execution work nicely with the main loop.
var driver, by, until,
timeout = 5000;
function waitAndClickElement(selector, index = 0) {
driver.wait(until.elementLocated(by.css(selector)), timeout)
.then(() => {
driver.findElements(by.css(selector)).then((els) => {
var element = els[index];
driver.wait(until.elementIsVisible(element), timeout);
element.click();
});
});
}
exports.execute = function(d, b, u) {
// make globally accessible for ease of use
driver = d;
by = b;
until = u;
// actual test as a promise
return new Promise((resolve, reject) => {
// open site
driver.get('https://web.whatsapp.com');
// make sure it loads fine
driver.wait(until.elementLocated(by.className('chat')), timeout);
driver.wait(until.elementIsVisible(
driver.findElement(by.className('chat'))), timeout);
// open menu
waitAndClickElement('.icon.icon-menu');
// click profile link
waitAndClickElement('.menu-shortcut', 1);
// give profile time to animate
// this prevents an error from occurring when we try to click the close
// button while it is still being animated (workaround/hack!)
driver.sleep(500);
// close profile
waitAndClickElement('.btn-close-drawer');
driver.sleep(500); // same for hiding profile
// click some chat
waitAndClickElement('.chat', 3);
// let main script know we are done successfully
// we do so after all other webdriver promise have resolved by creating
// another webdriver promise and hooking into its resolve
driver.wait(until.elementLocated(by.className('chat')), timeout)
.then(() => resolve());
});
};
Example output
Here is some example output. The first invocation of run test will open up an instance of Chrome. Other invocations will use that same instance. When an error occurs, it can be inspected as shown. Executing quit will close the browser instance and quit the main loop.
$ node init.js
> run test
> run test
WebDriverError: unknown error: Element <div class="chat">...</div> is not clickable at point (163, 432). Other element would receive the click: <div dir="auto" contenteditable="false" class="input input-text">...</div>
(Session info: chrome=57.0.2987.133)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.9.0-2-amd64 x86_64)
Test failed!
> error
<prints complete stacktrace>
> run test
> quit
You can run tests in other files by simply calling them. Say you have a file test-foo.js, then execute run test-foo in the above prompt to run it. All tests will share the same Chrome instance.
Failed attempt #1: saving and restoring storage
When inspecting the page using my development tools, I noticed that it appears to use the localStorage. It is possible to export this as JSON and write it to a file. On a next invocation, this file can be read, parsed and written to the new browser instance storage before reloading the page.
Unfortunately, WhatsApp still required me to scan the QR code. I have tried to figure out what I missed (cookies, sessionStorage, ...), but did not manage. It is possible that WhatsApp registers the browser as being disconnected after some time has passed. Or that it uses other browser properties (session ID?) to recognize the browser. This is pure speculating from my side though.
Failed attempt #2: switching session/window
Every browser instance started via WebDriver has a session ID. This ID can be retrieved, so I figured it may be possible to start a session and then connect to it from the test cases, which would then be run from a separate file (you can see this is the predecessor of the final solution). Unfortunately, I have not been able to figure out a way to set the session ID. This may actually be a security concern, I am not sure. People more expert in the usage of WebDriver might be able to clarify here.
I did find out that it is possible to retrieve a list of window handles and switch between them. Unfortunately, windows are only shared within a single session and not across sessions.

best way to retrieve URLs from a mail body

The following line was making add-in non-responsive on Mac clients. The add-in works absolutely fine on mac client if we remove this line and execute the function someFunction directly.
Office.context.mailbox.item.body.getAsync("html", someFunction);
We used body.getAsync() because we need to extract all the URLs in the mail body by processing the html and those urls containing certain IDs.
tried using the below but didn't give the expected URLS.
var links = Office.context.mailbox.item.getEntities().urls;
I am also trying the following
Office.initialize = function (reason) {
$(document).ready(function () {
app.initialize();
Office.context.mailbox.item.body.getAsync("html", processHtmlBody);
});
};
function processHtmlBody(asyncResult) {
var htmlParser = new DOMParser().parseFromString(asyncResult.value, "text/html");
var links = htmlParser.getElementsByTagName("a");
}
Is there a better alternative to fetch the URLs from the mail body.
Note that getAsync is part of the 1.3 Mailbox Requirement set, which Outlook for Mac doesn't currently support:
https://dev.outlook.com/reference/add-ins/tutorial-api-requirement-sets.html
Otherwise using Entities is your only option, but getElementsByTagName would probably work best (if you had access to the email body).
We tried both alternatives/workarounds with makeEwsRequestAsync() and getEntities().urls to retrieve the URLs from the mailbody on a Mac client. But both were not successful:
makeEwsRequestAsync(): "no data error".
getEntities().urls : did not give the intended result/URLs
So for Mac client, we ended up changing the logic and added conditional logic which did not include any code to retrieve the URLs.
if (Office.context.requirements.isSetSupported("mailbox", 1.3))
{
//conditional code
}
I had the same problem and I resolved it in a similar way:
return new Promise((resolve, reject) => {
mailbox.item.body.getAsync(window.Office.CoercionType.Html, asyncResult => {
if (asyncResult.status == window.Office.AsyncResultStatus.Succeeded) {
var $dom = $(asyncResult.value);
var url = $dom.find('#x_hiddenURL').text();
resolve(url);
} else {
reject(new Error('Failed to load email body'));
}
});
});

Getting unique ClientID from chrome extension?

I'm developing chrome extension. I need the ability to identify each client as a unique client.
I can't store guid in a cookie since cookie can be deleted. I need something to be read from the system itself which is unique.
Now - I know that JS doesn't has access to client resources ( local resources) but - and here is my question :
Question
Does chrome extensions Js's provide API for getting unique client information ( I dont care what data - as long as it is unique).
Edit :
Just to clarify :
The user will be shown a unique key ( which is a hash data of his computer). this code will be sent to me , and I will provide matching result which the user will be sent (via email) and only then - he will be able to use the extension.
(no , not all countries support extension payment via wallet , im at one of those countries)
To uniquely identify a user, I would suggest to generate a random token and store it in your extension's storage (chrome.storage). The userid has to be generated only once, when the token does not exist in storage.
For example:
function getRandomToken() {
// E.g. 8 * 32 = 256 bits token
var randomPool = new Uint8Array(32);
crypto.getRandomValues(randomPool);
var hex = '';
for (var i = 0; i < randomPool.length; ++i) {
hex += randomPool[i].toString(16);
}
// E.g. db18458e2782b2b77e36769c569e263a53885a9944dd0a861e5064eac16f1a
return hex;
}
chrome.storage.sync.get('userid', function(items) {
var userid = items.userid;
if (userid) {
useToken(userid);
} else {
userid = getRandomToken();
chrome.storage.sync.set({userid: userid}, function() {
useToken(userid);
});
}
function useToken(userid) {
// TODO: Use user id for authentication or whatever you want.
}
});
This mechanism relies on chrome.storage.sync, which is quite reliable. This stored ID will only be lost in the following scenarios:
The user re-installs the extension. Local storage will be cleared when uninstalling the extension.
One of the storage quotas has been exceeded (read the documentation).
This is not going to happen because the only write operation occurs at the first run of your extension.
Chrome's storage gets corrupted and fails to save the data.
Even if the user does not have Chrome Sync enabled, data will still be saved locally. There have been bugs with Chrome's internals that resulted in data loss, but these are incidents.
The user has opened the developer tools for your extension page and ran chrome.storage.sync.clear() or something similar.
You cannot protect against users who possess the knowledge to mess with the internals of Chrome extensions.
The previous method is sufficient if you want to uniquely identify a user. If you really want to get a hardware-based ID, use chrome.storage.cpu and chrome.storage.memory as well. I don't see any benefits in using these additional sources though, because they can change if the user replaces hardware, and they are not unique either (two identical laptops would report the same values, for instance).
As Xan suggested, the chrome.identity API is probably your best choice. You can get the users e-mail address and use that as a random seed to generate a code of your choosing. The user info also includes an "id" field which I believe is unique but I haven't ever seen any documentation that substantiates that. You can then use the chrome.storage.sync API to store the generated key in the users online data storage for your app. This way the user will be able to access their private key whenever and where ever they log in on any device.
Please note that you will have to enable the oAuth2 api's in the developers console for your application and include the application key and proper scopes in your app manifest.
Here is a crude example:
function getUserInfo (interactive, callback )
{
var xmlhttp = new XMLHttpRequest();
var retry = true;
var access_token;
getToken();
/**
* Request the Auth Token
*/
function getToken()
{
chrome.identity.getAuthToken( { 'interactive': interactive }, function (token) {
if ( chrome.runtime.lastError )
{
console.log( "ERROR! " + chrome.runtime.lastError.message );
return;
}
if ( typeof token != 'undefined ')
{
access_token = token;
sendRequest( );
}
else
callback( );
});
}
function sendRequest()
{
xmlhttp.open('GET', 'https://www.googleapis.com/userinfo/v2/me' );
xmlhttp.setRequestHeader('Authorization','Bearer ' + access_token );
xmlhttp.onload = requestComplete;
xmlhttp.send();
}
function requestComplete()
{
if ( this.status == 401 && retry )
{
retry = false; // only retry once
console.log( "Request failed, retrying... " + this.response );
}
else
{
console.log( "Request completed. User Info: " + this.response );
callback(null, this.status, this.response );
var userInfo = JSON.parse( this.response );
storeUniqueKey( userInfo );
}
}
}
function storeUniqueKey( info )
{
var key;
// TODO: Generate some key using the user info: info.loginName
// user info here contains several fields you might find useful.
// There is a user "id" field here which is numeric and I believe that
// is a unique identifier that could come in handy rather than generating your
// own key.
...
chrome.storage.sync.set ( { user_key: key } );
}
To add to Rob W's answer. In his method, the saved string would propagate to every Chrome instance signed in with the same Google user account - with a lot of big and small if's.
If you need to uniquely identify a local user profile, and not all Chrome profiles with the same Google user, you want to employ chrome.storage.local in the same manner. This will NOT be a unique Chrome install identifier though - only a profile within that install.
What also needs to be noted is that all this data is not in any way or form tied to anything - it just has a good probability of being unique. But absolutely nothing stops user from reading and cloning this data as he sees fit. You cannot, in this scenario, secure the client side.
I'm thinking that a more secure way would be to use chrome.identity API to request and maintain an offline (therefore, not expiring) token as proof of license. The user cannot easily clone this token storage.
I'm not versed in OAuth yet, so if anyone can point out what's wrong with this idea - they are welcome to.
We can also use Crypto.randomUUID() for generating a UUID and then save it to web storage. Refer to MSDN for details this API.
let uuid = self.crypto.randomUUID();
console.log(uuid); // for example "36b8f84d-df4e-4d49-b662-bcde71a8764f"

Get notified when JS breaks?

I have configured PHP to send me mails whenever there is an error. I would like to do the same with Javascript.
Also given the fact that this will be client side it is open to abuse.
What are good ways to get notified by mail when JS breaks in a web application?
Update:
Just to give some perspective, i usually load several js files including libraries (most of the time jQuery).
You can listen to the global onError event.
Note that you need to make sure it doesn't loop infinitely when it raises an error.
<script type="text/javascript">
var handlingError = false;
window.onerror = function() {
if(handlingError) return;
handlingError = true;
// process error
handlingError = false;
};
</script>
The code below relies on the global onError event, it does not require any external library and will work in any browser.
You should load it before any other script and make sure you have a server-side jserrorlogger.php script that picks up the error.
The code includes a very simple self-limiting mechanism: it will stop sending errors to the server after the 10th error. This comes in handy if your code gets stuck in a loop generating zillions of errors.
To avoid abuse you should include a similar self-limiting mechanism in your PHP code, for example by:
saving and updating a session variable with the error count and stop sending emails after X errors per session (while still writing them all down in your logs)
saving and updating a global variable with the errors-per-minute and stop sending emails when the threshold is exceeded
allowing only requests coming from authenticated users (applies only if your
application requires authentication)
you name it :)
Note that to better trace javascript errors you should wrap your relevant code in try/catch blocks and possibly use the printstacktrace function found here:
https://github.com/eriwen/javascript-stacktrace
<script type="text/javascript">
var globalOnError = (function() {
var logErrorCount = 0;
return function(err, url, line) {
logErrorCount++;
if (logErrorCount < 10) {
var msg = "";
if (typeof(err) === "object") {
if (err.message) {
// Extract data from webkit ErrorEvent object
url = err.filename;
line = err.lineno;
err = err.message;
} else {
// Handle strange cases where err is an object but not an ErrorEvent
buf = "";
for (var name in err) {
if (err.hasOwnProperty(name)) {
buf += name + "=" + err[name] + "&";
}
}
err = "(url encoded object): " + buf;
}
}
msg = "Unhandled exception ["+err+"] at line ["+line+"] url ["+url+"]";
var sc = document.createElement('script'); sc.type = 'text/javascript';
sc.src = 'jserrorlogger.php?msg='+encodeURIComponent(msg.substring(0, Math.min(800, msg.length)));
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(sc, s);
}
return false;
}
})();
window.onerror = globalOnError;
</script>
You would wrap your entire program in a try/catch and send caught exceptions over AJAX to the server where an email could be generated. Short of that (and I wouldn't do that) the answer is "not really."
JA Auide has the basic idea. You could also go somewhat in between, ie.:
Write an AJAX "errorNotify" function that sends error details to the server so that they can be emailed to you.
Wrap certain parts of your code (the chunks you expect might someday have issues) with a try/catch which invokes errorNotify in the catch block.
If you were truly concerned about having 0 errors whatsoever, you'd then be stuff with try/catching your whole app, but I think just try/catching the key blocks will give you 80% of the value for 20% of the effort.
Just a note from a person that logs JavaScript errors.
The info that comes from window.onerror is very generic. Makes debugging hard and you have no idea what caused it.
User's plugins can also cause the issue. A very common one in certain Firebug versions was toString().
You want to make sure that you do not flood your server with calls, limit the amount of errors that can be sent page per page load.
Make sure to log page url with the error call, grab any other information you can too to make your life easier to debug.

Categories

Resources