Basically i want to have an update content of the web,
https://www.investing.com/indices/indices-futures
In Dow 30, the last value is updating itself(real-time update from Investing.com server) and i would like to know is there any method to capture the change of values without requesting the website again, so that i can update on my code asynchronously. Since all i found online about being notified on change is based on their own html, but in my case it is external url, so i am asking here to gain some insight
You can add some code into the chrome console and track this value every second to notify you.
let last_value = -1
let class_selector = 'pid-8873-last'
setInterval(function() {
let v = document.getElementsByClassName(class_selector)[0].innerText
if (v != last_value) {
console.log("Value as been updated to " + v)
last_value = v
}
}, 1000)
> Value as been updated to 25,799.5
> Value as been updated to 25,798.5
But you must have a browser open, and create an ajax request when value is updated.
If you don't want any Browser, but be run into a server, you can check PhantomJS
EDIT WITH PHANTOMJS
They're some update to do to work with PhantomJS.
You need to replace let by var
document isn't accessible, so you need to use evaluate
https may require to add --ssl-protocol=any
./test.js
var page = require('webpage').create();
page.open('https://www.investing.com/indices/indices-futures', function(status) {
var last_value = -1
setInterval(function() {
var value = page.evaluate(function() {
return document.getElementsByClassName('pid-8873-last')[0].innerText
})
if (value != last_value) {
console.log("Value as been updated to " + value)
last_value = value
}
}, 1000)
// phantom.exit()
})
Then run it from the same directory:
# phantomjs test.js
Value as been updated to 25,799.0
I think you need to check what is websocket. this would be cool start; https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
Related
How can I set the badge number on a specific tab only? So far I have a code that sets the badge number on all the tabs.. I've been reading around A LOT, but there doesn't seem to be a whole lot of information about this, so perhaps I will find a solution for this here.
I would like something like Adblock Plus, which sets the badge number for a specific tab. This is pretty easy in Chrome etc, but doesn't seem to be the case in Safari.
Does anyone know how extensions like Adblock plus shows the badge number on a specific tab?
So far I only have this code, but as mentioned, it sets the badge on all the tabs, which is not the result I want.
safari.extension.toolbarItems[0].badge = 2;
Edit:
I have been looking at the source code of Adblock plus, and a few other extensions that had this function. And it seems it is using some prototype.
Adblock plus background snippet:
BrowserAction.prototype = {
_set: function(name, value)
{
var toolbarItem = getToolbarItemForWindow(this._page._tab.browserWindow);
if (!toolbarItem)
{
return;
}
var property = toolbarItemProperties[name];
if (!property)
{
property = toolbarItemProperties[name] = {
pages: new ext.PageMap(),
global: toolbarItem[name]
};
}
property.pages.set(this._page, value);
if (isPageActive(this._page))
{
toolbarItem[name] = value;
}
},
setIcon: function(path)
{
this._set("image", safari.extension.baseURI + path.replace("$size", "16"));
},
setBadge: function(badge)
{
if (!badge)
{
this._set("badge", 0);
}
else if ("number" in badge)
{
this._set("badge", badge.number);
}
}
};
Content script (adblockplus.js)
FilterNotifier.on("filter.hitCount", function(filter, newValue, oldValue, page)
{
if (!(filter instanceof BlockingFilter) || !page)
{
return;
}
Prefs.blocked_total++;
var blocked = blockedPerPage.get(page) || 0;
blockedPerPage.set(page, ++blocked);
if (Prefs.show_statsinicon)
{
page.browserAction.setBadge(
{
color: badgeColor,
number: blocked
});
}
});
It seems this is how Adblock plus does it, but so far I haven't been able to replicate it. Still trying though..
Okay, so I finally found a solution for this, and thought I would share what I did, in case somebody else is in the same situation.
This morning I got the idea of storing the data in an array, when the user visits one of the websites I want to display the badge number on (doesn't store all websites the user visits), only if it matched one of the websites I wanted to target. I stored the following data in the array: root domain (example.com) and the badgeNumber.
For this to work, you need to make an array of the root domain of the websites you want to target, and then only execute the following when it matches, otherwise the array would fill up very quickly, and we don't want too much data in it.
In the global page, start by making an empty array to store the data
var badgeUpdateArray = [];
You then need to set up message handling in your global page as well.
safari.application.addEventListener('message', handleMessage, false);
function handleMessage(event) {
if(event.name === "setBadgeText"){
var id = badgeUpdateArray.length + 1;
var isFound = 0;
var found = badgeUpdateArray.some(function (el) {
if(el.identifier === event.message.identifier){
// Was found
isFound = 1;
}
});
if (isFound == 0) {
// Not found, add to the array
badgeUpdateArray.push({identifier:event.message.identifier,badgeNumber:event.message.badgeNumber});
}
// Set the badge number
safari.extension.toolbarItems[0].badge = event.message.badgeNumber;
}
}
Now we need to send the message from the content script to the global page. You need to get the root domain (example.com), I'm not getting into that here, as it's pretty easy. You will also need the badgeNumber value, this can be gathered from wherever (GET request, or elsewhere..)
Remember, only execute this code if the website matches your target domains.
var message = {
identifier: domain,
badgeNumber: rows.length
}
safari.self.tab.dispatchMessage("setBadgeText", message);
This will send the message, and store the data in the array, it will also set the badge number.
Now, for this to be working on different tabs, you will need to make an event handler for "activate" on the global page, this will run whenever a tab is active.
safari.application.addEventListener("activate", updateBadge, true);
function updateBadge(){
var cDomain = safari.application.activeBrowserWindow.activeTab.url;
cDomain = cDomain.replace("www3.","");
cDomain = cDomain.replace("www2.","");
cDomain = cDomain.replace("www1.","");
cDomain = cDomain.replace("www.","");
cDomain = new URL(cDomain);
cDomain = cDomain.hostname;
var id = badgeUpdateArray.length + 1;
var isFound = 0;
var badgeNumber = 0;
var found = badgeUpdateArray.some(function (el) {
badgeNumber = el.badgeNumber;
if(el.identifier === cDomain){
// Was found, set the badge number
isFound = 1;
safari.extension.toolbarItems[0].badge = el.badgeNumber;
}
});
if (isFound == 0) {
// Was not found
safari.extension.toolbarItems[0].badge = 0;
}
}
Hopefully I've got it all in here, and at least something that works, though I have to say that I would prefer an easier way of storing it.. like Chrome etc does it, with the tab API.
Is it possible to detect situation when page is entered and when the same page is refreshed
if (entered) alert("hi");
if (refreshed) alert("you've refreshed");
Somehow there are some little differences between page rendering when entered and when refreshed and it would be much easier to detect the case than to debug it for me (if its even possible - maybe some browser optimization stuff is causing it).
This isn't an ideal solution, but if your page can load in under 5 seconds than this will work, and assuming you are not navigation to another page, then returning within 5 seconds.
window.onbeforeunload = function(){
window.sessionStorage.setItem('lastvisit', new Date().getTime());
}
var lastVisit = +window.sessionStorage.getItem('lastvisit');
var isRefresh = (new Date().getTime() - lastVisit) < 5000;
console.log(isRefresh);
There is no perfect way of tracking reloads verses new page loads but this solution works in most situations. Use sessionStorage in combination with an unload event:
(function (win) {
'use strict';
var reloaded = false,
ss = win.sessionStorage,
offset = 1000, // 1 second, may need tweaking if
// your page takes a long time to load/where
// this code is located in your page
now = function () {
return (new Date()).getTime();
},
lastUnload = ss.getItem('lastunload'),
loadStatus = document.getElementById('status');
// sessionStorage returns null if nothing was stored
if (lastUnload !== null) {
// sessionStorage returns a string, +lastUnload
// coerces the string held in lastUnload into an integer
// see https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Operators/Arithmetic_Operators#-_.28Unary_Negation.29
if (+lastUnload + offset > now()) {
reloaded = true;
}
}
win.addEventListener('unload', function () {
ss.setItem('lastunload', now());
}, false);
if (lastUnload === null) {
loadStatus.innerHTML = 'First visit of session.';
} else if (reloaded) {
loadStatus.innerHTML = 'Page was reloaded.';
} else {
loadStatus.innerHTML = 'Navigated back to page after leaving';
}
}(window));
This code defines a page reload as returning to the page within 1 second of leaving it, so there could be false positives if someone leaves the page and immediately hits the back button but with normal browsing behavior that really shouldn't happen. You can modify the offset variable if you want to give more or less leeway, but 1 second seems to be a good default.
After developing this code I also found this similar answer.
If sessionStorage is available, you can use that.
if (!window.sessionStorage.getItem('visited')) {
//entered
window.sessionStorage.setItem('visited', true);
}
else {
//refreshed
}
More on sessionStorage
I want to fetch a list online from a certain URL that is in JSON format and then use the DATA_ID from each item in that list to call a new URL. I'm just new with PhantomJS and I can't figure out why nest loops inside the page.open() acts all weird. Also the way to use phantom.exit() seems to be really weird doing what I want to achieve.
Here's my code:
console.log('Loading recipes');
console.log('===============================================================');
var page = require('webpage').create();
var url = 'http://www.hiddenurl.com/recipes/all';
page.open(url, function (status) {
//Page is loaded!
var js = page.evaluate(function () {
return document.getElementsByTagName('pre')[0];
});
var recipes = JSON.parse(js.innerHTML).results;
//console.log(recipes[0].name.replace('[s]', ''));
for (i = 0; i < recipes.length; i++) {
console.log(recipes[i].name.replace('[s]', ''));
var craft_page = require('webpage').create();
var craft_url = 'http://www.hiddenurl.com/recipe/' + recipes[i].data_id;
craft_page.open(craft_url, function (craft_status) {
//Page is loaded!
var craft_js = craft_page.evaluate(function () {
return document.getElementsByTagName('body')[0];
});
var craftp = craft_js.innerHTML;
console.log('test');
});
if (i == 5) {
console.log('===============================================================');
phantom.exit();
//break;
}
}
});
The thing that happens here is that this line:
console.log(recipes[i].name.replace('[s]', ''));
..prints the following:
===============================================================
Item from DATA_ID 1
Item from DATA_ID 2
Item from DATA_ID 3
Item from DATA_ID 4
Item from DATA_ID 5
..then it just prints the next:
===============================================================
..followed by:
'test'
'test'
'test'
'test'
'test'
Why is this not happening serial? The data from the innerly called page() request gets heaped up and dumped at the end, even after phantom.exit() should actually already be called.
Also when I free-loop a normal data-set I get this error:
QEventDispatcherUNIXPrivate(): Unable to create thread pipe: Too many open files
2013-01-31T15:35:18 [FATAL] QEventDispatcherUNIXPrivate(): Can not continue without a thread pipe
Abort trap: 6
Is there any way I can set GLOBAL_PARAMETERS or direct the process in some way so I can just handle 100's of page requests?
Thanks in advance!
I've made a workaround with Python by calling PhantomJS separately through the shell, like this:
import os
import json
cmd = "./phantomjs fetch.js"
fin,fout = os.popen4(cmd)
result = fout.read()
recipes = json.loads(result)
print recipes['count']
Not the actual solution for the PhantomJS issue, but it's a working solution and has less problems with memory and code-structure.
Thanks to everyone in advance -
I need to load a preference before any windows are loaded at startup. Below is some /component code I have been working with. The SetPreference method seems to fail when it is called (nothing executes afterwords either) - I am assuming because the resources that it needs are not available at the time of execution...or I am doing something wrong. Any suggestions with this code or another approach to setting a preference at startup?
Thanks again,
Sam
For some reason the code formatting for SO is not working properly - here is a link to the code as well - http://samingrassia.com/_FILES/startup.js
Components.utils.import('resource://gre/modules/XPCOMUtils.jsm');
const Cc = Components.classes;
const Ci = Components.interfaces;
const ObserverService = Cc['#mozilla.org/observer-service;1'].getService(Ci.nsIObserverService);
function MyStartupService() {};
MyStartupService.prototype = {
observe : function(aSubject, aTopic, aData) {
switch (aTopic) {
case 'xpcom-startup':
this.SetPreference("my.extension.is_running", "false");
break;
case 'app-startup':
this.SetPreference("my.extension.is_running", "false");
ObserverService.addObserver(this, 'final-ui-startup', false);
break;
case 'final-ui-startup':
//make sure is_running is set to false
this.SetPreference("my.extension.is_running", "false");
ObserverService.removeObserver(this, 'final-ui-startup');
const WindowWatcher = Cc['#mozilla.org/embedcomp/window-watcher;1'].getService(Ci.nsIWindowWatcher);
WindowWatcher.registerNotification(this);
break;
case 'domwindowopened':
this.initWindow(aSubject);
break;
}
},
SetPreference : function(Token, Value) {
var prefs = Components.classes["#mozilla.org/preferences-service;1"].getService(Components.interfaces.nsIPrefService);
var str = Components.classes["#mozilla.org/supports-string;1"].createInstance(Components.interfaces.nsISupportsString);
str.data = Value;
prefs.setComplexValue(Token, Components.interfaces.nsISupportsString, str);
//save preferences
var prefService = Components.classes["#mozilla.org/preferences-service;1"].getService(Components.interfaces.nsIPrefService);
prefService.savePrefFile(null);
},
initWindow : function(aWindow) {
if (aWindow != '[object ChromeWindow]') return;
aWindow.addEventListener('load', function() {
aWindow.removeEventListener('load', arguments.callee, false);
aWindow.document.title = 'domwindowopened!';
// for browser windows
var root = aWindow.document.documentElement;
root.setAttribute('title', aWindow.document.title);
root.setAttribute('titlemodifier', aWindow.document.title);
}, false);
},
classDescription : 'My Startup Service',
contractID : '#mystartupservice.com/startup;1',
classID : Components.ID('{770825e7-b39c-4654-94bc-008e5d6d57b7}'),
QueryInterface : XPCOMUtils.generateQI([Ci.nsIObserver]),
_xpcom_categories : [{ category : 'app-startup', service : true }]
};
function NSGetModule(aCompMgr, aFileSpec) {
return XPCOMUtils.generateModule([MyStartupService]);
}
To answer your real question, which is
I have code that loads on every window load and I need to make sure that only gets executed once every time firefox starts up.
..you should just use a module, in the load handler that you wish to execute once, check a flag on the object exported from (i.e. "living in") the module, then after running the code you need, set the flag.
Since the module is shared across all windows, the flag will remain set until you close Firefox.
As for your intermediate problem, I'd suggest wrapping the code inside observe() in a try { ... } catch(e) {dump(e)} (you'll need to set a pref and run Firefox in a special way in order to see the output) and check the error returned.
I guess xpcom-startup and app-startup is too early to mess with preferences (I think you need a profile for that), note that you don't register to get xpcom-startup notification anyway. You probably want to register for profile-after-change instead.
Example:
In the main page cliked on a button (NEW), the
page then will using Javascript to
open a new page in a new window by
calling redirectPage().
In the main page clicked on a button (EXIT),
then page then will call
confirmExit(), then
closeChildWindows() to closed all
popup new window before redirect to
another new page.
However, the JS variable
(childWindowHandles) will be always
reset if I refresh the main page,
and this cause the page unable to
close all other popup window before
relocated while EXIT button being
clicked
Does anyone know how can I solve this problem? By able to keep the JS variable (childWindowHandles) even the main page being refresh?
var childWindowHandles = new Array();
function redirectPage(url)
{
childWindowHandles[childWindowHandles.length] = window.open(url)
}
function confirmExit(url)
{
closeChildWindows()
window.location=url
}
function closeChildWindows()
{
for (var loop=0; loop<childWindowHandles.length; loop++)
{
if (!childWindowHandles[loop].closed)
{
childWindowHandles[loop].close();
}
}
}
You can use cookies to persist values...
Edit: You might find useful a simple object that I use:
Usage:
// Store a key/value for 1 day:
cookieManager.set('name', 'a value', 1);
// Retrieve a value associated to a key:
var value = cookieManager.get('name');
// Remove a key/value:
cookieManager.remove('name');
Implementation:
var cookieManager = {
set: function (name, value, expireDays) {
var expireDate = new Date();
expireDate.setDate(expireDate.getDate() + expireDays);
document.cookie = name + "=" + escape(value) +
((!expireDays) ? "" : ";expires="+expireDate.toGMTString());
},
get: function (key) {
var start,end;
if (document.cookie.length > 0) {
start = document.cookie.indexOf(key + "=");
if (start != -1) {
start = start + key.length + 1;
end = document.cookie.indexOf(";",start);
if (end == -1) {
end = document.cookie.length;
}
return unescape(document.cookie.substring(start,end));
}
}
return "";
},
remove: function (key) {
this.set(key, '', -1);
}
}
You can use cookies or window.name:) window.name to store session variables
Per this post here on SO, Firefox 3.5, Safari 4, and IE8 support HTML5 Storage.
Or use PersistJS which simplifies your access to whichever back-end storage mechanisms are available. (But cookie-less)
Use window.name
Positives:
it will live for the time of browser session - user closes window and it's gone
it won't put additional traffic on the wire like cookies do
it works even when cookies are disabled
at least 2MB space (Opera's limit is this low, other's have 32/64MB)
I also suggest you use javascript object for storing various values and serialize it using JSON and put that string into window.name.
Just make sure you don't persist any vulnerable data inside... For security reasons.
You can use sessionStorage.
Check this out:
html5_webstorage