I have a php file that calls phantomjs via the exec command. The phantom.js file calls a url that needs to include variables. I need to send these variables from php along with the exec command and have the variables show up in the phantom.js url.
Here is my current hardcoded php:
$response = [];
exec('/usr/bin/phantomjs phantomjstest.js', $response);
$data = $response[19];
and phantom.js
var page = require('webpage').create();
console.log('The default user agent is ' + page.settings.userAgent);
page.settings.userAgent = 'SpecialAgent111';
page.open('https://www.aa.com/travelInformation/flights/status/detail?search=AA|1698|2019,1,23&ref=search', function(status) {
if (status !== 'success') {
console.log('Unable to access network');
} else {
var ua = page.evaluate(function() {
return document.getElementById('aa-content-frame').innerHTML;
});
console.log(ua);
}
phantom.exit();
});
What I would like to do is change the php so that it passes 2 variables (from a form submit) to the javascript. Something like:
PHP:
exec('/usr/bin/phantomjs phantomjstest.js?variable1=$forminput1&variable2=$forminput2', $response);
JS
page.open('https://www.aa.com/travelInformation/flights/status/detail?search=AA|variable1|variable2&ref=search', function(status) {
Or I can construct the entire URL in the php and send it along with the exec command.
Any ideas on either method, or some other way to get from here to there, are most appreciated. Thanks.
Based on suggestions in comments, I have updated my PHP to contain:
exec("/usr/bin/phantomjs phantomjstest.js https://www.aa.com/travelInformation/flights/status/detail?search=AA|1698|2019,1,23&ref=search", $response);
and my JS to:
var page = require('webpage').create();
console.log('The default user agent is ' + page.settings.userAgent);
page.settings.userAgent = 'SpecialAgent111';
var system = require('system');
var args = require('system').args;
var address = system.args[0];
page.open(address, function (status) {
if (status !== 'success') {
console.log('Unable to access network');
} else {
var ua = page.evaluate(function() {
return document.getElementById('aa-content-frame').innerHTML;
});
console.log(ua);
}
phantom.exit();
});
But all I get back is a null response. Any idea why the address is either not getting passed with PHP exec or not getting picked up and run by JS?
Thanks.
****** SOLUTION ********
There were a couple of things that needed fixing, and I want to thank those that commented below, as well as others on SO that offered solutions to similar issues.
First, the JS did not like the & symbol in the URL. I had to use # in the passed argument instead and then replace it on the JS side.
Second, the JS did not like the pipe symbol in the URL so I had to escape them.
The finished PHP looks like:
exec("/usr/bin/phantomjs phantomjstest.js https://www.aa.com/travelInformation/flights/status/detail?search=AA\|$flight\|$date#ref=search", $response);
and the JS like:
var page = require('webpage').create();
console.log('The default user agent is ' + page.settings.userAgent);
page.settings.userAgent = 'SpecialAgent111';
var system = require('system');
var args = require('system').args;
var address = system.args[1];
var address = address.replace(/#/gi,"&");
page.open(address, function (status) {
if (status !== 'success') {
console.log('Unable to access network');
} else {
var ua = page.evaluate(function() {
return document.getElementById('aa-content-frame').innerHTML;
});
console.log(ua);
}
phantom.exit();
});
Again, my thanks to all!!!!
****** SOLUTION ********
There were a couple of things that needed fixing, and I want to thank those that commented below, as well as others on SO that offered solutions to similar issues.
First, the JS did not like the & symbol in the URL. I had to use # in the passed argument instead and then replace it on the JS side.
Second, the JS did not like the pipe symbol in the URL so I had to escape them.
The finished PHP looks like:
exec("/usr/bin/phantomjs phantomjstest.js
https://www.aa.com/travelInformation/flights/status/detail?
search=AA\|$flight\|$date#ref=search", $response);
and the JS like:
var page = require('webpage').create();
console.log('The default user agent is ' + page.settings.userAgent);
page.settings.userAgent = 'SpecialAgent111';
var system = require('system');
var args = require('system').args;
var address = system.args[1];
var address = address.replace(/#/gi,"&");
page.open(address, function (status) {
if (status !== 'success') {
console.log('Unable to access network');
} else {
var ua = page.evaluate(function() {
return document.getElementById('aa-content-frame').innerHTML;
});
console.log(ua);
}
phantom.exit();
});
Related
I got data from MySQL to check if my login is correct, so I need to save some data as cookies for 30 minutes and some redirect to my homepage. I tried to do so in a final code but, didn't work.
And to sane another curiosity: may I call a node.js code file (file that links to database e execute queries) in html file as script then use the ResultSet to fill the html form with JQuery?
// This one is what is in the top of modelAutonomo.js
var dadosJSON = JSON.stringify("{}");
const sql = require('../config/sql');
const requisicao = require('request');
var cookieNome = '';
var cookieEmail = '';
var cookieCPF = '';
var autonomo = {
...
...
// This is the function to login
fazerLogin: async function (entradaJSON) {
var aux;
console.log("Encontrado " + entradaJSON);
dadosJSON = converterParaObjetoJSON(entradaJSON);
aux = await sql.consultarAutonomoCPFeEmail(dadosJSON);
console.log("Aux: " + aux);
aux = JSON.parse(aux);
console.log(dadosJSON.senha + " e " + aux.senha);
if(dadosJSON.senha != aux.senha) {
console.log("Senha incorreta!");
} else if (dadosJSON.cpf != aux.cpf) {
console.log("CPF incorreto!");
} else {
preencher(aux);
console.log("Logado: " + autonomo.nome);
cookieEmail = autonomo.email;
cookieNome = autonomo.nome;
cookieCPF = autonomo.cpf;
}
// I tried this to make the redirect, but don't even write the page html code in the console
requisicao(__dirname + '/../html/index.html', function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body) // Print the my web homepage.
}
});
}
}
The structure of app is:
index.js
|--model
|--modelAutonomo.js
|--config
|--routes.js (who redirects to another pages/files of code)
|--html
|--index.html (can be redirected by sending a request to "/")
|--controller
And this is the git repository of him:
https://github.com/diegossilva-1995-01-25/ReformaAqui
I see in your repo that you are using express.js. The express documentation is fairly straightforward:
set a cookie
redirect
res.cookie(cookieName, cookieValue);
res.redirect(redirectPath);
That res variable is available in the request handler.
I have a phantom js script that checks every redirection and shows it in the console by the page.onNavigationRequested callback method.
but when i want to catch all the URLs that returned from the page.onNavigationRequested callback method and pushed them to an array and finally show all the URLs at the end of the script, it only shows the first redirection URL.
can you please check the script and advice.
var page = require('webpage').create();
var sys = require('system');
var fs = require('fs');
var response = {};
var arrayOfResponses = [];
var pageUrl = 'http://example.com/r1.php';
phantom.onError = function (msg, trace) {
phantom.exit(1);
};
function forceExit(){
phantom.exit(0);
}
page.onNavigationRequested = function(url, type, willNavigate, main) {
arrayOfResponses.push(url) ;
}
response.content = arrayOfResponses;
page.open(pageUrl, function(status) {
if ( status !== 'success' ) {
phantom.exit( 1 );
} else {
phantom.exit( 0 );
}
}, 100);
setTimeout(forceExit,2000);
console.log(JSON.stringify(response));
and thank you in advance.
There are two issues with your script:
You make PhantomJS exit too early, after the first url is opened. It doesn't have time to follow redirects.
You write the script from top to bottom as if the program flow is linear/synchronous, whereas in javascript it's not — onNavigationRequested can be called many times.
So with that in mind let's rewrite the script to collect all the redirects and exit if no new redirect is made for 2 seconds.
var page = require('webpage').create();
var response = {};
var arrayOfResponses = [];
var pageUrl = 'http://admin.weeqo.com/redirect/r1.php';
var exitTimeout;
// This will be called if no redirects are requested in 2 seconds
function forceExit(){
// Just for fun we'll note the final URL
var curURL = page.evaluate(function(){
return document.location.href
});
console.log("Final URL is " + curURL);
// Prepare and output the report:
response.content = arrayOfResponses;
console.log("List of all requested URLs: " + JSON.stringify(response));
// Now we can exit safely
phantom.exit(0);
}
// This is called before each redirect
page.onNavigationRequested = function(url, type, willNavigate, main) {
// Clear timeout so that script is not shut down
// because we have a new redirect
if(exitTimeout) {
clearTimeout(exitTimeout);
}
arrayOfResponses.push(url);
console.log("Navigation requested: " + url);
// Create timeout that will shut down the script
// in two seconds unless cancelled
exitTimeout = setTimeout(forceExit, 2000);
}
// open the first page
page.open(pageUrl, function(status) {
// We only care for errors because
// who knows how many time will pass before
// we hit the last redirect
if ( status !== 'success' ) {
phantom.exit( 1 );
}
});
Can you store an array using browser.storage.local.set or achieve the same result with a different method?
Details:
My extension currently will redirect a website specified through the options.html form. Currently when you specify a new website the old one will be replaced. Is there a way I can append to an array of websites that will be redirected instead of replacing the website?
options.js: (will process information from form in options.html)
function saveOptions(e) {
e.preventDefault();
browser.storage.local.set({
url: document.querySelector("#url").value
});
}
function restoreOptions() {
function setCurrentChoice(result) {
document.querySelector("#url").value = result.url || "reddit.com";
}
function onError(error) {
console.log(`Error: ${error}`);
}
var getting = browser.storage.local.get("url");
getting.then(setCurrentChoice, onError);
}
document.addEventListener("DOMContentLoaded", restoreOptions);
document.querySelector("form").addEventListener("submit", saveOptions);
redirect.js:
function onError(error) {
console.log(`Error: ${error}`);
}
function onGot(item) {
var url = "reddit.com";
if (item.url) {
url = item.url;
}
var host = window.location.hostname;
if ((host == url) || (host == ("www." + url))) {
window.location = chrome.runtime.getURL("redirect/redirect.html");
}
}
var getting = browser.storage.local.get("url");
getting.then(onGot, onError);
One thought I had was to add a storage location per url, however i would have to also be stored to prevent it getting reset each time options.js is loaded. (Something similar to the below code)
var i = 0;
browser.storage.local.set({
url[i]: document.querySelector("#url").value
});
i++;
A more logical solution would be for the url storage location to be an array.
If there is a way for url to be an array then redirect.html could contain the following:
if ( (url.includes (host) ) || (url.includes ("www." + host) ) ){
window.location = chrome.runtime.getURL("redirect.html");
}
Fresh eyes has solved my problem.
In options.js:
function saveOptions(e) {
e.preventDefault();
var array = (document.querySelector("#url").value).split(",");
browser.storage.local.set({
url: array
});
In redirect.js:
function onGot(item) {
var url = "";
if (item.url) {
url = item.url;
}
var host = window.location.hostname;
if ( (url.includes(host)) || (url.includes("www." + host)) ) {
window.location = chrome.runtime.getURL("redirect/redirect.html");
}
}
I have a PhantomJS script that works when I run it locally (Mac), but when I run it on my Linux server, it returns the following error:
ReferenceError: Can't find variable: $
https://fantasy.premierleague.com/a/statistics/value_form:5712 in global code
The code is:
var page = require('webpage').create();
var fs = require('fs');
var args = require('system').args;
page.settings.userAgent = 'SpecialAgent';
page.open('https://fantasy.premierleague.com/a/statistics/value_form', function (status) {
if (status !== 'success') {
console.log('Unable to access network');
} else {
var ua = page.evaluate(function () {
var result ="";
// ...
return result;
});
}
phantom.exit();
});
There may be a race condition between your code and jQuery being loaded on the page. Wrap the statements in your page.evaluate callback with a $(document).ready(function() { /* your statements here */ }); to ensure scripts on the page have loaded fully.
For anyone who is still using PhantomJS and encounters this problem, I solved it with
phantomjs --ignore-ssl-errors=yes
I don't intend for upvote.
I'm providing solution to solve some situations without simulating browser behavior with phantomjs just to retrieve data that can be handled directly by requesting to url.
You need the data from the page, so why not just do request to this url: https://fantasy.premierleague.com/drf/bootstrap-static
var request = require('request'); // install: npm i request
var fs = require('fs');
var args = require('system').args;
request.get({url: 'https://fantasy.premierleague.com/drf/bootstrap-static'}, function(err, response, body) {
console.log(body);
});
How I found this url?
Simple:
So I keep Receiving an error when I'm trying to use OAuth with Asana's API. The error I'm receiving is "Error no route found". What am I doing wrong? I know that the request is most likely correct, but I believe it returns a hashed URL and i'm supposed to unhash it. This is sample code I am using from a Facebook OAuth though, so perhaps the code is incorrect and facebook api specific.
Here is my code:
$(function () {
checkHashLogin();
$('#signon').click(function () {
asanaLogin();
});
})
});
var appID = ****************;
function asanaLogin() {
var path = 'https://app.asana.com/-/oauth_authorize';
var queryParams = ['client_id=' + appID,
'redirect_uri=' + window.location,
'response_type=token'];
var query = queryParams.join('&');
var url = path + query;
window.location.replace(url);
}
function checkHashLogin() {
if (window.location.hash.length > 3) {
var hash = window.location.hash.substring(1);
if(hash.split('=')[0] == 'access_token')
{
var path = "https://app.asana.com/-/oauth_authorize";
var queryParams = [hash, 'callback=displayUser'];
var query = queryParams.join('&');
var url = path + query;
//use jsonp to call the graph
var script = document.createElement('script');
script.src = url;
document.body.appendChild(script);
}
}
}
function displayUser(user) {
setTimeout(function () { }, 1000);
if (user.id != null && user.id != "undefined") {
//Do Stuff
}
else {
alert('user error');
}
}
Here is a photo of my app credentials. My redirect location is just local because I am not hosting it on a server yet.
Looks like you're doing url = path + query when you might need to do url = path + "?" + query - the query string isn't separated, which means you end up requesting a path like https://app.asana.com/-/oauth_authorizeclientId=... which isn't recognized: hence, "no route found".
Hope that helps!