I am working on a webextension in Firefox for use internally at work. The purpose of the extension is to insert relevant information from our ServiceNow instance into Nagios host/service page.
I am currently trying to insert the state of tickets into the history tab of Nagios. My script looks like this:
var table = document.getElementById('id_historytab_table');
var table = table.getElementsByTagName('tbody');
var table = table[1];
var len = table.children.length
const url = "https://[domain].service-now.com/api/now/table/task?sysparm_limit=1&number="
for (i = 1; i <= len; i++) {
var col = table.rows[i].cells[2];
if (col.textContent.startsWith("TKT")) {
var tkt = col.textContent;
//console.log(tkt);
//console.log(url+tkt);
var invocation = new XMLHttpRequest();
invocation.open("get",url+tkt, true);
invocation.withCredentials = true;
invocation.onreadystatechange = function() {
if(this.readyState == this.DONE) {
//console.log('recieved');
console.log(invocation.responseText);
//console.log(JSON.parse(invocation.responseText).result[0].state);
}
};
invocation.send();
};
};
This successfully gets the ticket number from each row of the history tab and makes a GET request. I can see the requests on my ServiceNow REST log and it looks good there. However, the response is never received.
If I copy and paste the above from my content-script.js and put it directly into my console I am able to iterate through the rows, get the ticket numbers, and successfully receive responses from ServiceNow. So this works, but not in WebExtension for some reason. I am about at the end of my knowledge of extensions and javascript though and am not sure what else to do.
I figured out the problem. In order for the WebExtension to receive the response the URL needs to be under permissions in the manifest.json. Adding:
"permissions": [ "url" ],
resolved the issues and I immediately began seeing the response bodies I was expecting.
Related
I want to view and manipulate Javascript localStorage information on a PHP page using PHP. I have gotten pretty far with this, but I'm not where I need to be. I'm using PHP 7.3 and vanilla JS.
I have AJAX POSTing the data to a PHP processing page (not the one that calls the javascript function). However, I need to access the POST variable on the page that called the javascript. How can I pass the information back without a click?
The page wishlist.php contains a link to the js file and <div id="temporary_wishlist"></div>.
Javascript called from wishlist.php:
window.addEventListener("load", function() {
loadWishlist();
});
// get wishlist contents
function loadWishlist() {
var items = wishlistStorage.data.items.join("%20");
var item_notes = wishlistStorage.data.item_notes.join("%20");
var comments = wishlistStorage.data.comments;
var wishlistRequest;
var response = null;
if(window.XMLHttpRequest) {
wishlistRequest = new XMLHttpRequest();
} else {
wishlistRequest = new ActiveXObject("Microsoft.XMLHTTP");
}
wishlistRequest.onreadystatechange = function() {
if(wishlistRequest.readyState == 4 && wishlistRequest.status == 200) {
response = wishlistRequest.responseText;
document.getElementById("temporary_wishlist").innerHTML = response;
}
}
wishlistRequest.open("POST", "/wishlist-processor.php", true);
wishlistRequest.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
wishlistRequest.send("wishlist_items=" + encodeURIComponent(items)
+ "&wishlist_itemnotes=" + encodeURIComponent(item_notes)
+ "&wishlist_comments=" + encodeURIComponent(comments));
}
Given the above, I have access to $_POST['wishlist_items'] inside wishlist-processor.php. Whatever I output inside of wishlist-processor.php is visible to the visitor on wishlist.php inside <div id="temporary_wishlist"></div>. It is not, however, visible in the Console.
On wishlist.php, I want to load the information into a PHP variable so I can send it to a MySQL query. If I could just POST to self like I do with form validation I would have access to $_POST['wishlist_items'] from wishlist.php. I already tried setting the URL in the open("POST", URL, true) function to _self or wishlist.php, and that didn't work for me.
I know it's suggested that I use Fetch, but I'm already overwhelmed learning new things I want to understand AJAX better. Also, Fetch doesn't work on Firefox Android.
I'm scraping dynamic data from a website. For some reason the PageSource that I get() is partial. However, it is not partial when I view the page source directly from Chrome or Firefox browsers. I would like to know an answer that will enable me to completely scrape the data from the page.
For my application, I want to scrape programmatically using a .Net web browser or similar. I've tried using Selenium WebDriver 2.48.2 with ChromeDriver; I've also tried PhantomJSDriver; I've also tried WebClient; and also HttpWebRequest. All with .Net 4.6.1.
The url: http://contests.covers.com/KingOfCovers/Contestant/PendingPicks/ARTDB
None of the following are working...
Attempt #1: HttpWebRequest
var urlContent = "";
try
{
var request = (HttpWebRequest) WebRequest.Create(url);
request.CookieContainer = new CookieContainer();
if (cookies != null)
{
foreach (Cookie cookie in cookies)
{
request.CookieContainer.Add(cookie);
}
}
var responseTask = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse,request.EndGetResponse,null);
using (var response = (HttpWebResponse)await responseTask)
{
if (response.Cookies != null)
{
foreach (Cookie cookie in response.Cookies)
{
cookies.Add(cookie);
}
}
using (var sr = new StreamReader(response.GetResponseStream()))
{
urlContent = sr.ReadToEnd();
}
}
Attempt #2: WebClient
// requires async method signature
using (WebClient client = new WebClient())
{
var task = await client.DownloadStringTaskAsync(url);
return task;
}
Attempt #3: PhantomJSDriver
var driverService = PhantomJSDriverService.CreateDefaultService();
driverService.HideCommandPromptWindow = true;
using (var driver = new PhantomJSDriver(driverService))
{
driver.Navigate().GoToUrl(url);
WaitForAjax(driver);
string source = driver.PageSource;
return source;
}
public static void WaitForAjax(PhantomJSDriver driver)
{
while (true) // Handle timeout somewhere
{
var ajaxIsComplete = (bool)(driver as IJavaScriptExecutor).ExecuteScript("return jQuery.active == 0");
if (ajaxIsComplete)
break;
Thread.Sleep(100);
}
}
I also tried ChromeDriver using page object model. That code is too long to paste here; nonetheless: it has the exact same result as the other 3 attempts.
Expected Results
The data table from the url is complete, without any missing data. For example, here is a screenshot to compare to the screen shot below. The thing to observe is that there is NOT an "...". Instead there is the data. This can reproduced by opening the url in Firefox or Chrome, right click, and View Page Source.
Actual Results
Observe that where the "..." is a big gap, as the arrow indicates in the screen shot. There should be many rows of content in place of that "...". This can be reproduced using any of the above attempts above.
Please note that the url is dynamic data. You will likely not see the exact same results as the screen shots. Nonetheless, the exercise can be repeated it will simply look different than the screen shots. A quick test to understand that there is missing data is to compare the Page Source line count: the "complete" data set will have nearly twice as many rows in the html.
Ok, as requested. glad to have helped. :)
But in your C# were are you copying from?, in your code you have -> urlContent = sr.ReadToEnd(); How are you seeing, copying the result from this?. Are you copying from the debugger?, if so it's maybe the object inspector of the debugger that's trimming. Have you tried getting the result from urlContent and saving to file?. eg. System.IO.File.WriteAllText(#"temp.txt",urlContent);
I'm trying to create a desktop app with node-webkit. A part of this app require the use of facebook to get/post some content to any profile (facebook user) that use my desktop app on my personal computer.
Regarding facebook api documentation (https://developers.facebook.com/docs/facebook-login/manually-build-a-login-flow/v2.0) , I have to manually implement the login flow and use the following URI as the redirect URI: https://www.facebook.com/connect/login_success.html
Currently, in my node-webkit app, via a child window (a popup), a user can login to facebook and authorize my desktop app to interact with it's profile.
Here is a part of the code:
var url = "https://www.facebook.com/dialog/oauth?client_id=myclientID&redirect_uri=https://www.facebook.com/connect/login_success.html&response_type=token&scope=publish_actions";
loginWindow = window.open(url, 'Login facebook', 'location=yes,toolbar=yes,menubar=yes,directories=yes,status=yes,resizable=yes,scrollbars=yes,height=480,width=640', false);
After that, the user is redirected to the following URI and as mentioned in the doc, the access token appear correctly:
https://www.facebook.com/connect/login_success.html#access_token=theBigTokenString&expires_in=3864.
But it appears only for few seconds and after that the url is replaced by https://www.facebook.com/connect/blank.html#_=_ (with a security warning message).
I've read some post that propose to add eventListener like hashchange to the opened window in order to capture the access token. But after some redirect within the child window, I'm no longer available to interact with it via javascript.
So finally, I can't get the access token that the child window has retrieved and make visible for few seconds.
Is anyone can help me to get this user access token with node-webkit?
I really don't want to use a server (for hosting a web page) between my desktop app and facebook.
Thanks in advance for help.
With the help of an other question (here) I found a solution to my problem.
I post the code that I use now and I Hope it will help someone else:
var url = "https://www.facebook.com/dialog/oauth?&client_id=myBigClientID&redirect_uri=https://www.facebook.com/connect/login_success.html&response_type=token&scope=publish_actions";
function Response() {
this.access_token = null;
this.expires_in = null;
};
var response = new Response();
//function called every 500ms to check if token is in url
window.hashUpdate = function() {
if(window.loginWindow.closed){
window.clearInterval(intervalId);
start(); //just a callback that I'm using to start another part of my application (after I caught the token)
}
else {
var url = window.loginWindow.document.URL;
var tabUrl = url.split("#"); //first split to separate the domain part and the parameter part
var paramString = tabUrl[1]; //I concerned only by the second part after the '#'
if(paramString != undefined){
var allParam = paramString.split("&");
for (var i = 0; i < allParam.length; i++) {
var oneParamKeyValue = allParam[i].split("=");
response[oneParamKeyValue[0]] = oneParamKeyValue[1]; //store my token in form of key => value
};
//close the window after 1500ms
setTimeout(function(){
window.loginWindow.close();
}, 1500);
}
}
}
//open the url and start the watching process
window.loginWindow = window.open(this.url, 'Login facebook', false);
this.intervalId = window.setInterval("window.hashUpdate()", 500);
I'm developing an add-on for the first time. It puts a little widget in the status bar that displays the number of unread Google Reader items. To accommodate this, the add-on process queries the Google Reader API every minute and passes the response to the widget. When I run cfx test I get this error:
Error: The page has been destroyed and can no longer be used.
I made sure to catch the widget's detach event and stop the refresh timer in response, but I'm still seeing the error. What am I doing wrong? Here's the relevant code:
// main.js - Main entry point
const tabs = require('tabs');
const widgets = require('widget');
const data = require('self').data;
const timers = require("timers");
const Request = require("request").Request;
function refreshUnreadCount() {
// Put in Google Reader API request
Request({
url: "https://www.google.com/reader/api/0/unread-count?output=json",
onComplete: function(response) {
// Ignore response if we encountered a 404 (e.g. user isn't logged in)
// or a different HTTP error.
// TODO: Can I make this work when third-party cookies are disabled?
if (response.status == 200) {
monitorWidget.postMessage(response.json);
} else {
monitorWidget.postMessage(null);
}
}
}).get();
}
var monitorWidget = widgets.Widget({
// Mandatory widget ID string
id: "greader-monitor",
// A required string description of the widget used for
// accessibility, title bars, and error reporting.
label: "GReader Monitor",
contentURL: data.url("widget.html"),
contentScriptFile: [data.url("jquery-1.7.2.min.js"), data.url("widget.js")],
onClick: function() {
// Open Google Reader when the widget is clicked.
tabs.open("https://www.google.com/reader/view/");
},
onAttach: function(worker) {
// If the widget's inner width changes, reflect that in the GUI
worker.port.on("widthReported", function(newWidth) {
worker.width = newWidth;
});
var refreshTimer = timers.setInterval(refreshUnreadCount, 60000);
// If the monitor widget is destroyed, make sure the timer gets cancelled.
worker.on("detach", function() {
timers.clearInterval(refreshTimer);
});
refreshUnreadCount();
}
});
// widget.js - Status bar widget script
// Every so often, we'll receive the updated item feed. It's our job
// to parse it.
self.on("message", function(json) {
if (json == null) {
$("span#counter").attr("class", "");
$("span#counter").text("N/A");
} else {
var newTotal = 0;
for (var item in json.unreadcounts) {
newTotal += json.unreadcounts[item].count;
}
// Since the cumulative reading list count is a separate part of the
// unread count info, we have to divide the total by 2.
newTotal /= 2;
$("span#counter").text(newTotal);
// Update style
if (newTotal > 0)
$("span#counter").attr("class", "newitems");
else
$("span#counter").attr("class", "");
}
// Reports the current width of the widget
self.port.emit("widthReported", $("div#widget").width());
});
Edit: I've uploaded the project in its entirety to this GitHub repository.
I think if you use the method monitorWidget.port.emit("widthReported", response.json); you can fire the event. It the second way to communicate with the content script and the add-on script.
Reference for the port communication
Reference for the communication with postMessage
I guess that this message comes up when you call monitorWidget.postMessage() in refreshUnreadCount(). The obvious cause for it would be: while you make sure to call refreshUnreadCount() only when the worker is still active, this function will do an asynchronous request which might take a while. So by the time this request completes the worker might be destroyed already.
One solution would be to pass the worker as a parameter to refreshUnreadCount(). It could then add its own detach listener (remove it when the request is done) and ignore the response if the worker was detached while the request was performed.
function refreshUnreadCount(worker) {
var detached = false;
function onDetach()
{
detached = true;
}
worker.on("detach", onDetach);
Request({
...
onComplete: function(response) {
worker.removeListener("detach", onDetach);
if (detached)
return; // Nothing to update with out data
...
}
}).get();
}
Then again, using try..catch to detect this situation and suppress the error would probably be simpler - but not exactly a clean solution.
I've just seen your message on irc, thanks for reporting your issues.
You are facing some internal bug in the SDK. I've opened a bug about that here.
You should definitely keep the first version of your code, where you send messages to the widget, i.e. widget.postMessage (instead of worker.postMessage). Then we will have to fix the bug I linked to in order to just make your code work!!
Then I suggest you to move the setInterval to the toplevel, otherwise you will fire multiple interval and request, one per window. This attach event is fired for each new firefox window.
I am having some issues with an extension that i am trying to make. The extension's job is to check URLs before making the request and block the ones that are available in the list provided. The manifest.json file loads flagged.js which declares and initializes the array. Then, it loads the file that has the function to check URLs:
// To detected and filter globally flagged websites
chrome.webRequest.onBeforeRequest.addListener(
// What to do if detected a website
function(info) {
return {redirectUrl: 'chrome-extension://' + chrome.i18n.getMessage('##extension_id') +
'/blocked.html?flag_type=globally&url=' + escape(info.url)};
},
// Website to watchout for
{
urls: flagged,
types: []
},
// Action to be performed?
["blocking"]);
My issue starts with the part urls: flagged, it seems the list must be hardwired in order to be accepted. What i mean to say, is that i must specify the list items as such:
var flagged = [
"*://*.domain_1.com/*",
"*://*.domain_2.com/*",
"*://*.domain_3.com/*",
"*://*.domain_4.com/*",
];
If i attempt to go to any of the above specified domains, the extension will redirect them to the block.html page no problems. However, i want that list to be automatically or periodically updated, so i introduced this part:
// Get list from the text file we have stored
function getFlagged(url) {
var list = [];
var file = new XMLHttpRequest();
file.open("GET", url, true);
file.onreadystatechange = function() {
if (file.readyState === 4) {
if (file.status === 200) {
allText = file.responseText;
list = file.responseText.split('|');
for( var i=0; i<list.length; i++ )
flagged.push(list[i]);
}
}
}
file.send(null);
}
I have tried to change and add things that may not make sense or could be done differently, i was just desperate. The list will be populated with the new items, say *domain_5* and *domain_6* for example. This operation will occur first thing when the extension is being loaded, or so i think. However, when i try to access *domain_5* and *domain_6*, the extension does NOT block them despite the fact that they are in the list flagged.
Any help with this issue would be very appreciated!
Thank you
EDIT: I am not an expert on JS or Chrome.APIs, this is my first attempt at chrome extensions.
Yeah, Your Listener is registering before the XMLHttpRequest is finished.
Call the listener in your onreadystatechange function and you'll get it to work.