Develop Reference

How do I navigate to bing.com and enter a search text using the chrome console?

Below is my code.
It is resulting in unexpected behaviour.
It navigates to bing.com but it does not fill in the text field. Also, I have noticed that the console get cleared after navigating to a new webpage.
window.location = "https://www.bing.com";
window.onload = function(){
var editSearch = document.getElementById("sb_form_q");
editSearch.value = "Quux";
}

You are binding the onload function to the existing window object.
When the browser loads the new page, it will make a new window object which won't have your property set on it.
JavaScript run in one page (even when you are running it through developer tools) can't persist variables onto a different page.
(Storage mechanisms like localStorage and cookies are available, but you would need code in the subsequent page to look for them).

JavaScript is only valid for the current page you are on. When you are executing code from DevTools console, you are executing code on that page itself. So, when you navigate to another page using window.location you loose the onload handler you have defined.
To add handlers to a different page, it must be connected to your page (the parent) in some way, like an iframe or a popup.
ifrm = document.getElementById('frame');
ifrm.src = 'http://example.com';
ifrm.contentWindow.onload = function () {
// do something here with
// ifrm.contentWindow.document.getElementById('form')
}

As #Quentin said.
But you can do another way like ..
var keyword = "Quux";
window.location = "https://www.bing.com/search?q="+keyword;

How to Handle redirects in Node.JS with HorsemanJs and PhantomJS

I´ve recently started using horseman.js to scrap a page with node. I can´t figure out how exactly it works and I can´t find good examples on the internet.
My main goal is to log on a platform and extract some data. I´ve managed to do this with PhantomJS, but know I want to learn how to do it with horseman.JS.
My code should open the login page, fill the login and password inputs and click on the "login" button. Pretty easy so far. However, after clicking on the "login" button the site makes 2 redirects before loading the actual page where I want to work.
My problem is that I don´t know how to make my code wait for that page.
With phantomJS I had a workaround with the page URL. The following code shows how I´ve managed to do it with phantomJS and it works just fine:
var page = require('webpage').create();
var urlHome = 'http://akna.com.br/site/montatela.php?t=acesse&header=n&footer=n';
var fillLoginInfo = function(){
$('#cmpLogin').val('mylogin');
$('#cmpSenha').val('mypassword');
$('.btn.btn-default').click();
};
page.onLoadFinished = function(){
var url = page.url;
console.log("Page Loaded: " + url);
if(url == urlHome){
page.evaluate(fillLoginInfo);
return;
}
// After the redirects the url has a "sid" parameter, I wait for that to apear when the page loads.
else if(url.indexOf("sid=") >0){
//Keep struggling with more codes!
return;
}
}
page.open(urlHome);
However, I can´t find a way to handle the redirects with horseman.JS.
Here is what I´ve been trying with horseman.JS without any success:
var Horseman = require("node-horseman");
var horseman = new Horseman();
var urlHome = 'http://akna.com.br/site/montatela.php?t=acesse&header=n&footer=n';
var fillLoginInfo = function(){
$('#cmpLogin').val('myemail');
$('#cmpSenha').val('mypassword');
$('.btn.btn-default').click();
}
var okStatus = function(){
return horseman.status();
}
horseman
.open(urlHome)
.type('input[name="cmpLogin"]','myemail')
.type('input[name="cmpSenha"]','mypassword')
.click('.btn-success')
.waitFor(okStatus, 200)
.screenshot('image.png')
.close();
How do I handle the redirects?

I'm currently solving the same problem, and my best solution so far is to use the waitForSelector method to target something on the final page.
E.g.
horseman
.open(urlHome)
.type('input[name="cmpLogin"]','myemail')
.type('input[name="cmpSenha"]','mypassword')
.click('.btn-success')
.waitForSelector("#loginComplete")
.screenshot('image.png')
.close();
Of course you have to know the page you're waiting for to do this.
If you know there are two redirects, you can use the approach of .waitForNextPage() twice. A naive approach if you didn't know how many redirects to expect would be to chain these until a timeout is reached (I don't recommend this as it will be slow!),
Perhaps a cleverer way, you can also use on events to capture redirects, like .on('navigationRequested') or .on('urlChanged').
Although it doesn't answer your question directly, this link may help: https://github.com/ariya/phantomjs/issues/11507

Need to navigate users to landing page when browser back button is pressed

I have a ASP.net MVC web application which consists of several pages. The requirement is like this:
when users are using the application, suppose user is in page 7, suddenly user navigates away from the application by typing a external internet URL say Google.com.
Now when user presses the back button of the browser, Instead of bringing him back to page 7, we need to redirect him to Page 0 which is the landing page of the application.
Is there any way to achieve this? we have a base controller which gets executed every time a page loads as well as a master page (aspx). Can we do something there so that this behavior can be implemented in all the pages?

I think the best solution is to use iframe and switch between your steps inside of iframe. It would be quite easy to do, because you don't need to redesign your application. Anytime when user tries to switch to other url and come back, the iframe will be loaded again from the first step.
Be sure to disable caching on every step of your application. You can do this by applying NoCache attribute to your controller's actions:
public class NoCache : ActionFilterAttribute
{
public override void OnResultExecuting(ResultExecutingContext filterContext)
{
filterContext.HttpContext.Response.Cache.SetExpires(DateTime.UtcNow.AddDays(-1));
filterContext.HttpContext.Response.Cache.SetValidUntilExpires(false);
filterContext.HttpContext.Response.Cache.SetRevalidation(HttpCacheRevalidation.AllCaches);
filterContext.HttpContext.Response.Cache.SetCacheability(HttpCacheability.NoCache);
filterContext.HttpContext.Response.Cache.SetNoStore();
base.OnResultExecuting(filterContext);
}
}

There is 2 case over here
First is browser in online mode, in this case you have to store your last page get request in session, if user hit back button it will re initiate get request for that page again you can trap it and send them to landing page, You have to take care that get request for page happen only once other action must be post.
Second is browser in offline mode, in this case you have to take care that your response should not put any cache foot print in browser, there are many code example you can found on net for this purpose.

I can offer the following idea:
When user press <a href='external url' onclick='clearHistory'>link</a>
You can save in browser history of the desired url:
<script>
function clearHistory()
{
var reternUrl = getReternUrl();
History.pushState({}, null, reternUrl);
}
</script>
more about history.js
Edit: ok, then handle beforeunload event:
$(window).on('beforeunload', function () {
var reternUrl = getReternUrl();
History.pushState({}, null, reternUrl);
});

EDIT: Shortened and slightly changed code to better answer exact question (based on first comment to this answer)
Addition to answer above about editing the browser history for the case where the user types the external URL in the browser address bar.
You could try to detect url change as posted in How to detect URL change in JavaScript.
Example of this using jquery (taken and edited slightlyfrom post linked to above):
For newer browsers:
$(window).bind('hashchange', function() {
/* edit browser history */
});
For older browsers:
function callback(){
/* edit browser history */
}
function hashHandler(callback){
this.oldHash = window.location.hash;
this.Check;
var that = this;
var detect = function(){
if(that.oldHash!=window.location.hash){
callback("HASH CHANGED - new hash" + window.location.hash);
that.oldHash = window.location.hash;
}
};
this.Check = setInterval(function(){ detect() }, 100);
}
hashHandler(callback); //start detecting (callback will be called when a change is detected)
I'll get back to you on bookmarks (still need to check that out).

why is my window.dialogArguments undefined

I need to open up a .aspx page in a modal dialog. Here is the JS code I use to open the dialog:
if (url) {
var fullPath = url + "/Validation.aspx";
}
else {
alert("Could not find the location of the merge dialog. Please contact your System admin and have them update the configuration entity.");
return;
}
var features = "unadorned:yes;scroll:yes;resizable:yes;status:yes;dialogHeight:480px;dialogWidth:480px;";
var args = {
selected: selectedIds,
page: pageIds,
fetchXml: xml,
entity: "xyz"
};
window.showModalDialog(fullPath, args, features);
In my validation.aspx page I need to be able to grab the JS arguments, assign them to hidden fields, then repost, so I can use those arg values server side.
here is my JS code in my .aspx page:
window.onload = function(){
if (!window.dialogArguments)
return;
var args = window.dialogArguments;
...
}
I have seen tons of examples of this working throughout the web. But...My window.dialogArguments is always undefined in my .aspx page. What gives? anyone have any thoughts or solutions?

My assumption here is that the ASPX dialog page is being opened cross-domain.
This would mean that your parent page is in one domain aka: http://abc/page.html, and that your child dialog page is in another domain like: http://def/dialog.html.
If this is the case, it seems as though there are restrictions against accessing dialogArguments and returnValue. Check out the comments on this previous answer for example.

Navigating / scraping hashbang links with javascript (phantomjs)

I'm trying to download the HTML of a website that is almost entirely generated by JavaScript. So, I need to simulate browser access and have been playing around with PhantomJS. Problem is, the site uses hashbang URLs and I can't seem to get PhantomJS to process the hashbang -- it just keeps calling up the homepage.
The site is http://www.regulations.gov. The default takes you to #!home. I've tried using the following code (from here) to try and process different hashbangs.
if (phantom.state.length === 0) {
if (phantom.args.length === 0) {
console.log('Usage: loadreg_1.js <some hash>');
phantom.exit();
}
var address = 'http://www.regulations.gov/';
console.log(address);
phantom.state = Date.now().toString();
phantom.open(address);
} else {
var hash = phantom.args[0];
document.location = hash;
console.log(document.location.hash);
var elapsed = Date.now() - new Date().setTime(phantom.state);
if (phantom.loadStatus === 'success') {
if (!first_time) {
var first_time = true;
if (!document.addEventListener) {
console.log('Not SUPPORTED!');
}
phantom.render('result.png');
var markup = document.documentElement.innerHTML;
console.log(markup);
phantom.exit();
}
} else {
console.log('FAIL to load the address');
phantom.exit();
}
}
This code produces the correct hashbang (for instance, I can set the hash to '#!contactus') but it doesn't dynamically generate any different HTML--just the default page. It does, however, correctly output that has when I call document.location.hash.
I've also tried to set the initial address to the hashbang, but then the script just hangs and doesn't do anything. For example, if I set the url to http://www.regulations.gov/#!searchResults;rpp=10;po=0 the script just hangs after printing the address to the terminal and nothing ever happens.

The issue here is that the content of the page loads asynchronously, but you're expecting it to be available as soon as the page is loaded.
In order to scrape a page that loads content asynchronously, you need to wait to scrape until the content you're interested in has been loaded. Depending on the page, there might be different ways of checking, but the easiest is just to check at regular intervals for something you expect to see, until you find it.
The trick here is figuring out what to look for - you need something that won't be present on the page until your desired content has been loaded. In this case, the easiest option I found for top-level pages is to manually input the H1 tags you expect to see on each page, keying them to the hash:
var titleMap = {
'#!contactUs': 'Contact Us',
'#!aboutUs': 'About Us'
// etc for the other pages
};
Then in your success block, you can set a recurring timeout to look for the title you want in an h1 tag. When it shows up, you know you can render the page:
if (phantom.loadStatus === 'success') {
// set a recurring timeout for 300 milliseconds
var timeoutId = window.setInterval(function () {
// check for title element you expect to see
var h1s = document.querySelectorAll('h1');
if (h1s) {
// h1s is a node list, not an array, hence the
// weird syntax here
Array.prototype.forEach.call(h1s, function(h1) {
if (h1.textContent.trim() === titleMap[hash]) {
// we found it!
console.log('Found H1: ' + h1.textContent.trim());
phantom.render('result.png');
console.log("Rendered image.");
// stop the cycle
window.clearInterval(timeoutId);
phantom.exit();
}
});
console.log('Found H1 tags, but not ' + titleMap[hash]);
}
console.log('No H1 tags found.');
}, 300);
}
The above code works for me. But it won't work if you need to scrape search results - you'll need to figure out an identifying element or bit of text that you can look for without having to know the title ahead of time.
Edit: Also, it looks like the newest version of PhantomJS now triggers an onResourceReceived event when it gets new data. I haven't looked into this, but you might be able to bind a listener to this event to achieve the same effect.