I would like to know how to handle both local and sync storage in the right way in Chrome extension please.
This is my case:
I'm working on an extension for only a specific site (for now),
which contains a content-script and a popup.
The popup contains options where the user can make changes, then the values are sent to the content-script to show the changes on the page.
I'm looking to make as less saving and retrieving storage tasks as possible, and that in the end it will get saved in the sync storage and not just in local.
The sync storage got a per-minute limit, where the local one doesn't.
I know how to listen to the popup closed call from the content-script using a long-lived connection and listen to the onConnect and onDisconnect, and then I can do a save task, but is there a better way to save reading and writing to the storage?
All I can think of was having a background script where I can store the changes in variables and then just send them back and forward to and from the content-script and popup, so it's like having a storage without actually using the storage, but then how can I detect when the user leaves the specific domain and then do the single saving task, and also close/stop the background/event script?
The current limit on chrome.storage.sync sustained operations is 1 every 2 seconds (more accurately 1800 per hour), and a burst rate limit of 120 per minute.
So, your job is to ensure sync happens no more often than once per 2 seconds.
I would make an event page that deals with chrome.storage.onChanged event and syncs the two areas. Which is a surprisingly hard task due to local echo!
// event.js, goes into background.scripts in manifest
// Those will not persist if event page is unloaded
var timeout;
var queuedChanges = {};
var syncStamp = 1;
chrome.storage.onChanged.addListener(function(changes, area) {
// Check if it's an echo of our changes
if(changes._syncStamp && changes._syncStamp.newValue == syncStamp) {
return;
}
if(area == "local") {
// Change in local storage: queue a flush to sync
// Reset timeout
if(timeout) { clearTimeout(timeout); }
// Merge changes with already queued ones
for(var key in changes) {
// Just overwrite old change; we don't care about last newValue
queuedChanges[key] = changes[key];
}
// Schedule flush
timeout = setTimeout(flushToSync, 3000);
} else {
// Change in sync storage: copy to local
if(changes._syncStamp && changes._syncStamp.newValue) {
// Ignore those changes when they echo as local
syncStamp = changes._syncStamp.newValue;
}
commitChanges(changes, chrome.storage.local);
}
});
function flushToSync() {
// Be mindful of what gets synced: there are also size quotas
// If needed, filter queuedChanges here
// Generate a new sync stamp
// With random instead of sequential, there's a really tiny chance
// changes will be ignored, but no chance of stamp overflow
syncStamp = Math.random();
queuedChanges._syncStamp = {newValue: syncStamp};
// Process queue for committing
commitChanges(queuedChanges, chrome.storage.sync);
// Reset queue
queuedChanges = {};
timeout = undefined;
}
function commitChanges(changes, storage) {
var setData = {};
for(var key in changes) {
setData[key] = changes[key].newValue;
}
storage.set(setData, function() {
if(chrome.runtime.lastError) {
console.error(chrome.runtime.lastError.message);
}
});
}
The idea here is to sync 3 seconds after the last change to local. Each new change is added to the queue and resets the countdown. And while Chrome normally does not honor DOM timers in event pages, 3 seconds is short enough to complete before the page is shut down.
Also, note that updating an area from this code will fire the event again. This is considered a bug (compare with window.onstorage not firing for changes within current document), but meanwhile I added the _syncStamp property. It is used to distinguish the local echo, though there is a tiny chance that the stamp will result in a collision
Your other code (content script) should probably also rely on onChanged event instead of a custom "okay, I changed a value!" message.
Related
I have a POST request that pulls data from a server, according to parameters that are adjustable by the user through number inputs. Simply listening to the change event is not enough, because I want the data to refresh while using the mousewheel to change the input value.
Calling my refresh function with something like that is clearly not optimal:
$('input').on('mousewheel', function(){
refresh_data();
});
There would be a lot of requests, and because they are asynchronous, I can never know for sure if the last request to be completed will be the last one I send.
What I currently use is a setInterval to watch an object containing intervals (rounded timestamps) where a refresh has been requested. Inside the mousewheel listener, I add the next interval to the object, as a property/key, to avoid duplicates.
Something like that:
var i = 100;
var obj = [];
$('input').on('mousewheel', function(){
// add next interval to obj;
obj[Math.ceil(Date.now()/i)*i] = true;
});
var check = setInterval(function(){
// refresh if current interval is in obj
var t = Math.floor(Date.now()/i)*i;
if(obj[t]){
delete obj[t]; // remove from obj
refresh_data();
}
}, i);
I tested this code, on my machine, with i=50, and the object is always empty, which is what we want, but with i=30, as soon as I go too fast with the wheel, I see some old intervals accumulating in the object. This is caused by the setInterval is skipping beats, so whatever value I choose for i, some user with less CPU could "skip the wrong beat", and end up finishing his mouse wheel movement seeing a result that does not match the parameters value.
I feel like maybe i'm making this more complicated than it has to be, so i'm asking:
What would be the best way to pull data from server onmousewheel, considering:
1: I want to constantly refresh the data as the user rolls the wheel over an input field.
2: I want to be 100% sure the presented data after the wheel movement is always accurate.
Try using the document offsetheight like the below code. This will work when user scrolls down and reaches the end of scroll. The behavior is kind of like a recycler view of android.
window.onscroll = function(ev) {
if ((window.innerHeight + window.pageYOffset ) >=
document.body.offsetHeight) {
alert("you're at the bottom of the page");
}
};
I have a website, and I only want the client to be able to have 1 WebSocket connection at a time (when they open another tab while there is already another connection display, I display an error to them).
I'm working on a client-side solution where I update a flag in local storage to true when the connection is requested (It won't request if the flag is already true) then I listen for the beforeunload event and set the local storage flag to false if that tab had an open connection.
This seems to be working great except for the edge case of when a user shuts down their computer abruptly and thus beforeunload never fires, so when they turn their computer back on the local storage flag is stuck at true and they are stuck not being able to connect in any tabs.
Is there an event that will be called before the shutdown where I can set my local storage flag to false?
If not is there another solution for the client to keep track that it has only 1 WebSocket connection across all tabs so it can block a connection if there is already one?
window.addEventListener('beforeunload', this.setFlagToFalse);
As correctly stated in Jaromanda's comment, a computer without power can not emit an Event to the browser (which doesn't even exist anymore...).
However, one solution to your root problem is to listen to the storage event.
This event will fire across all the Windows that do share the same Storage area, when an other Window will make any modification to this Storage.
So we can use it as a mean to communicate between Windows from the same domain, in almost real time. This means that you don't have to keep your flag up to date, you can now know directly if an other Window is already active.
Here is a basic implementation. I'll let you the joy of making it more suited to your needs.
let alone = true; // a flag to know if we are alone
onstorage = e => { // listen to the storage event
if(e.key === 'am_I_alone') {
if(e.newValue === 'just checking') { // someone else is asking for permission
localStorage.am_I_alone = 'false'; // refuse
}
else if(e.newValue === 'false') { // we've been refused access
alone = false;
}
}
};
localStorage.am_I_alone = 'just checking'; // trigger the event on the other Windows
setTimeout(()=>{ // let them a little time to answer
if(alone) { // no response, we're good to go
// so the next one can trigger the event
localStorage.am_I_alone = "true";
startWebSocket();
}
else { // we've been rejected...
error();
}
}, 500);
Live Plnkr
I am working on a Chrome extension that processes items on a page via the content script and processes the items in the background script. It's my understanding that the background script is useful for long processing (e.g. if I have a queue of items I need to process over time).
// contentScript.js
function process(matches) {
if(matches && matches.length) {
chrome.storage.local.get({ queue: [] }, function(result) {
var updated_queue = result.queue;
var item = matches.pop();
updated_queue.push(item);
chrome.storage.local.set({ queue: updated_queue }, function() {
process(matches);
});
});
}
}
This works and stores each item in the local storage, which I should be able to process later. I'm not if this is the correct way to do this, but it addresses what appears to be a race condition otherwise.
I would like to process each of these items in the event page and have tried using an event listener to track changes to my local storage:
// eventPage.js
chrome.storage.onChanged.addListener(function(changes, namespace) {
// Pop the item from the local storage, process it and save the
// updated queue back to the local storage for additional processing.
});
The problem is, this introduces a similar race condition I was experiencing before. As the content script adds new items to the local storage key, popping them off in the event page causes issues. There doesn't appear to be any sort of mutex or locking mechanism that I can use.
What's the correct way to do this?
My data is object
I save it use local storage javascript like this :
localStorage.setItem('storedData', JSON.stringify(data))
I just want to keep that data for 1 hour. So if it's been more than 1 hour then the data will be removed
I know to remove data like this :
localStorage.removeItem("storedData")
But how can I set it to auto delete after 1 hour?
You can't.
When do items in HTML5 local storage expire?
The only thing you can do is set the delete statement in a timeout of 1 hour. This requires the user to stay on your page or the timeout won't be executed.
You can also set an expiration field. When the user revisits your site, check the expiration and delete the storage on next visit as the first thing you do.
Add the following code in your App.js file, then the localStorage will be cleared for every 1 hour
var hours = 1; // to clear the localStorage after 1 hour
// (if someone want to clear after 8hrs simply change hours=8)
var now = new Date().getTime();
var setupTime = localStorage.getItem('setupTime');
if (setupTime == null) {
localStorage.setItem('setupTime', now)
} else {
if(now-setupTime > hours*60*60*1000) {
localStorage.clear()
localStorage.setItem('setupTime', now);
}
}
use setInterval, with setting/pushing expiration key in your local data,
check code below.
var myHour = new Date();
myHour.setHours(myDate.getHours() + 1); //one hour from now
data.push(myHour);
localStorage.setItem('storedData', JSON.stringify(data))
function checkExpiration (){
//check if past expiration date
var values = JSON.parse(localStorage.getItem('storedData'));
//check "my hour" index here
if (values[1] < new Date()) {
localStorage.removeItem("storedData")
}
}
function myFunction() {
var myinterval = 15*60*1000; // 15 min interval
setInterval(function(){ checkExpiration(); }, myinterval );
}
myFunction();
You can't use setTimeout().Since you can't guarantee your code is going to be running on the browser in 1 hours. But you can set a timestamp and while returning back just check with Timestamp and clear the storage out based on expiry condition.
Use cookies instead
Cookies.set('foo_bar', 1, { expires: 1/24 })
Alternatively you can use IndexedDB in WebWorkers. What it basically means is you can create a parallel to your main thread JavaScript program, like this
var worker = new Worker('worker.js');
The advantage is when it finishes its work, and you don't have a reference to it, it gets cleaned up automatically.
However, since I'm not fully sure about the above, here is The worker's lifetime from Living Standard
Workers communicate with other workers and with browsing contexts
through message channels and their MessagePort objects.
Each WorkerGlobalScope object worker global scope has a list of the
worker's ports, which consists of all the MessagePort objects that are
entangled with another port and that have one (but only one) port
owned by worker global scope. This list includes the implicit
MessagePort in the case of dedicated workers.
Given an environment settings object o when creating or obtaining a
worker, the relevant owner to add depends on the type of global object
specified by o. If o specifies a global object that is a
WorkerGlobalScope object (i.e., if we are creating a nested dedicated
worker), then the relevant owner is that global object. Otherwise, o
specifies a global object that is a Window object, and the relevant
owner is the responsible document specified by o.
A worker is said to be a permissible worker if its WorkerGlobalScope's
owner set is not empty or:
its owner set has been empty for no more than a short implementation-defined timeout value,
its WorkerGlobalScope object is a SharedWorkerGlobalScope object (i.e., the worker is a shared worker), and
the user agent has a browsing context whose Document object is not completely loaded.
Another advantage is you can terminate a worker, like this
worker.terminate();
Also, it is worth mentioning
IndexedDB has built-in support for schema versions and upgrading via
its IDBOpenDBRequest.onupgradeneeded() method; however, you still
need to write your upgrade code in such a way that it can handle the
user coming from a previous version (including a version with a bug).
This works for a SPA where the browser window doesn't reload on page renders:
Set an expiresIn field in your local storage.
On window reload check to see if current time >= expiresIn
if true clear localStorage items else carry on
You can also do this check whenever your business logic requires it.
Just add the check in your app to check if the variable exists and then do a check on it. Put a unix timestamp on the variable, and compare that the next time the user visits your site.
I think setTimeout() method will do the trick.
UPDATED:
While setting the value to localStorage also set the time of recording the data.
// Store the data with time
const EXPIRE_TIME = 1000*60*60;
localStorage.setItem('storedData', JSON.stringify({
time: new Date(),
data: "your some data"
}));
// start the time out
setTimeout(function() {
localStorage.removeItem('storedData');
}, EXPIRE_TIME); // after an hour it will delete the data
Now the problem is if user leave the site. And setTimeout will not work. So when user next time visit the site you have to bootstrap the setTimeout again.
// On page load
$(document).ready(function() {
let userData = JSON.parse(localStorage.getItem('storedData')) || {};
let diff = new Date() - new Date(userData.time || new Date());
let timeout = Math.max(EXPIRE_TIME - diff, 0);
setTimeout(function() {
localStorage.removeItem('storedData');
}, timeout);
})
I am coding a Greasemonkey script to click the play button for a script in Google Apps Script every five 5 minutes to avoid the execution time limit set by Google.
I was able to identify with the script when the time is over but am unable to click the "run" button by using JavaScript. I have been inspecting the button with Google Chrome and tried a few things but I couldn't make it work.
Can anyone please help me?
I guess clicking any button in the toolbar of Google Sheets would be exactly the same..
Thanks!
You are approaching this in a completely wrong way. You should be including the possibility to restart the execution with a trigger internally in your script. I will show you how I did it. Keep in mind that my script is quite large and it performs a loop and I had to make it remember where in the loop it stopped, so it could continue with the same data.
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Purpose: Check if there is enough time left for another data output run
// Input: Start time of script execution
// Output: Boolean value if time is up
function isTimeUp(start, need) {
var cutoff = 500000 // in miliseconds (5 minutes)
var now = new Date();
return cutoff - (now.getTime() - start.getTime()) < need;
}
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here the start is simply a new Date() that you create when you start the script. The need is simply an average time it takes for my script to perform 1 loop. If there is not enough time for a loop, we will cut off the script with another function.
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Purpose: Store current properties and create a trigger to start the script after 1 min
// Input: propertyCarrier object (current script execution properties)
// Output: Created trigger ID
function autoTrigger(passProperties, sysKeys) {
var sysProperties = new systemProperties();
if (typeof sysKeys === 'undefined' || sysKeys === null) {
sysKeys = new systemKeys();
}
var triggerID = ScriptApp.newTrigger('stateRebuild')
.timeBased()
.after(60000)
.create()
.getUniqueId();
Logger.log('~~~ RESTART TRIGGER CREATED ~~~');
//--------------------------------------------------------------------------
// In order to properly retrieve the time later, it is stored in milliseconds
passProperties.timeframe.start = passProperties.timeframe.start.getTime();
passProperties.timeframe.end = passProperties.timeframe.end.getTime();
//--------------------------------------------------------------------------
//--------------------------------------------------------------------------
// Properties are stored in User Properties using JSON
PropertiesService.getUserProperties()
.setProperty(sysKeys.startup.rebuildCache, JSON.stringify(passProperties));
//--------------------------------------------------------------------------
Logger.log('~~~ CURRENT PROPERTIES STORED ~~~');
return triggerID;
}
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yours can be more simplistic if you do not need to remember where you stopped (judging by your current implementation you do not care whether you start from the beginning or not).
The trigger you create should either aim at the main function or if you need to pass on the data like I do, you will need to have a separate starter function to get the data back from the user properties and pass it on.