Develop Reference

How to set cookies client-side taking into account Intelligent Tracking Protection?

According to this blog post on webkit.org, cookies set client-side using document.cookie are capped to a 7 day expiry.
I understand the rationale behind using httpOnly for sensitive cookies such as auth tokens, but if I need to store something for a long duration and have it available across subdomains of a site, then cookies are the only option, right?
With these new ITP restrictions, setting cookies client-side which should live for any longer duration of time is not going to work, so what's the best way to approach this? One idea was make a route which takes params and converts them into a Set-Cookie header and then make a request to that instead of using document.cookie. Is there a better way?

One attempt would be to use localStorage instead of cookies, technically they don't expire. The Problem however can be, that the user can decide to empty the localStorage.
Here's an example
//use this if you only need it for the current page and remove it after leaving the page
const exampleStorage = window.localStorage;
exampleStorage.setItem('currentUser', 'Manuel');
//use this if you need to keep it even after leaving the page
localStorage.setItem('glob_currentUser', 'Max');
//and finally this if you need to keep it only for the session
sessionStorage.setItem("session", "Morning")
If you need more Information about LocalStorage here are 2 helpful websites:
MDN Window​.local​Storage
The W3C Specification for localStorage

Why is Javascript not url-decoding the cookie value, and should it? How can I normalise the cookie encoding policy between languages?

I just noticed that when I setcookie('foo','a#b.com',...) in PHP and read it out in document.cookie in Javascript, the read-out value is 'a%40b.com'
In the PHP documentation:
Note that the value portion of the cookie will automatically be urlencoded when you send the cookie, and when it is received, it is automatically decoded and assigned to a variable by the same name as the cookie name. If you don't want this, you can use setrawcookie() instead if you are using PHP 5.
Seems like PHP is doing this a matter of its own policy, but Javascript (in Firefox) isn't playing the same game.
The data in question is unsanitised user input, so it should be encoded one way or another, for security and sanity purposes. Thing is, if I make Javascript use decodeURIComponent but then if the cookie is set by some other code/language/circumstance which doesn't have a policy of urlencodeing the value, then the decoding stage might yield an incorrect value.
I'd rather not hunt down every cookie-setting bit of code now and forever and consider manually normalising the encoding policy on each of them.
Is Javascript doing The Right Thing here? To what extent is this behaviour guaranteed? Can I set a HTTP header or something which will inform browsers of the cookie encoding scheme? Is there a fix-once-an-for-all solution for this?

Possible to cache JSON to increase performance / load time?

I'm using a JSON file to autopopulate a drop down list. It's by no means massive (3000 lines and growing) but the time taken to refresh the page is becoming very noticeable.
The first time the page is loaded the JSON is read, depending on what option the user has selected dictates which part of the JSON is used to populate the drop down.
It's then loaded on every refresh or menu selection after. Is it possible to somehow cache the values to prevent the need for it to be reloaded time and time again?
Thanks.
EDIT: More Info:
It's essentially a unit converter. The JSON holds all the details. When a users selects 'Temp' for example a call is made and the lists are populated. Once a conversion is complete you can spend all day running temp conversions and they'll be fine but everytime a user changes conversion type so now length, the page refreshes and takes a noticeable amount of time.

Unfortunately, I don't know of a standardized global caching mechanism in PHP. This article says that Optimizer Plus, a third party accelerator, is being included in core PHP starting in version 5.5. Not sure what version you are using but you could try that.
On a different note, have you considered file storage as andrew pointed out? I think it combined with $_SESSION could really help you in this case. Let me give you an example that would work with your existing JSON data:
Server Side
Store your JSON data in a .json file on your PHP server:
{
"data": "some data",
"data2": "more data",
"data3": [
...
],
etc.
}
Note: Make sure to properly format your JSON data. Remember all strings must be enclosed in double quotes ".
In PHP, use an if statement to decide the appropriate action:
error_reporting(E_ALL);
ini_set("display_errors", "On");
session_start();
if(isset($_SESSION['dataCache'])) {
echo json_encode($_SESSION['dataCache']);
} else {
$file = 'data.json';
if (!is_file($file) || !is_readable($file)) {
die("File not accessible.");
}
$contents = file_get_contents($file);
$_SESSION['dataCache'] = json_decode($contents, true);
echo $contents;
}
So lets dig into the above coding a little more. So here's what we are doing in a nutshell:
Turn on error reporting and start session support.
Check to see if we've already read the file for this user.
If so, pull the value from storage and echo it out and exit. If not continue below.
Save off the file name and do a little error checking to ensure PHP can find, open and read the contents of the file.
Read the file contents.
Save the decoded json, which is not an array because of the `true` parameter passed to `json_decode`, into your `$_SESSION` variable.
Echo the contents to the screen.
This will save you the time and hazzle of parsing out JSON data and/or building it manually on the server. It will be cached for the users session so that they can use it through out.
Client Side
I assume you are using ajax to fetch the information? If not correct me, but I was assuming that's where some of your JavaScript comes into play. If so you may consider this:
Store the returned data in sessionStorage on the user's browser when it's returned from the server:
$.ajax({
...
success: function (res) {
localStorage.setItem("dataCache", JSON.stringify(res));
},
...
});
Or if you use promise objects:
$.ajax({
...
}).done(function (res) {
localStorage.setItem("dataCache", JSON.stringify(res));
});
When you need to read it you can do a simple test:
var data;
// This returns null if the item is not in local storage.
// Since JavaScript is truthy falsy, it will be evaluated as false.
if(localStorage.getItem("dataCache")) {
data = JSON.parse(localStorage.getItem("dataCache"));
} else {
// Make ajax call, fetch object and store in localStorage in the success or done callbacks as described above
}
Notes:
localStorage is a new feature in HTML5, so it's not fully supported on all browsers yet. Most of the major ones do however, even as far back as IE8 (I think). However, there is no standardized size limit on how much these browsers are required to hold per site.
It's important to take that into consideration. I can guarantee you probably will not be able to store the entire 30,000 line string in localStorage. However, you could use this as a start. Combined with the server side solution, you should see a performance increase.
Hope this helps.

I use the browser's cache to ensure that my large chunk of JSON is only downloaded once per session. I program in ASP.NET, but I'm sure PHP has the same mechanisms:
On session start, I generate a random string as session key for my dynamic JavaScripts. This key get stored in the ASP.NET session state under the key JsonSessionID. That way I can refer to it in my page markup.
I have a "generic http handler" (an ashx file) that when called by the browser, returns a .js file containing my JSON.
In my HTML I include the dynamic script:
<script type="text/javascript" src="/dynamicJSON.ashx?v=<%= JsonSessionID %>"></script>
The browser will automatically cache any URLs included as scripts. The next time the browser is asked to load a cached script from a URL, it will just load up the file from the local disk. This includes dynamic pages like this.
By adding the ?v= in there, I ensure that the JSON is updated once per session.
Edit
I just realized that your JSON is probably static. If that's the case, you can just put your JSON into a static .js file that you include in your HTML, and the browser will cache it.
// conversionData.js
var conversionData = { "a":1,"b":2,"c":3 };
When you include the conversionData.js, the conversionData variable will be in scope with the rest of your page's JavaScript that dynamically updates the drop-downs.
Edit 2
If you are serving static files, this blog post has a good pattern for cache-busting based on the file's date modified property. i.e. the file is only downloaded when it is changed on the server.
I have yet to find a good method for cache-busting JSON created via database lookup tables, other than per-session. Which isn't ideal because the database could change mid-session.

Once you've got your JSON data decoded into an object you can just keep the object around, it should persist until a page reload at least.
If you want to persist between reloads you might want to look at HTML5's localStorage etc.
You would need to come up with an age strategy, maybe just dump the current date in there with it as well so you can compare that and expire as needed.

I would suggest storing your json data to a session. On first page load you can write a script to get your json data, then store them into a session.
on each page load/refresh afterwards you can check our session to decide what to do - use the session data or fetch again your json data.
This approach suites me for small scale data (for example: an array of products - colors - sizes - prices).
Based on your data you should test you loading times.

Here is a simple hack:
Create a call to a php file as GET request with parameter "bla-bla.html"
or "bla-bla.css"... well you know, it makes browser think it is not a php, but rather "html" or "css". And browser will cache it.
To verify that the trick is working - go to the "network" tab of the browser dev panel and you will see column "type" there along with "transferred" - instead of having php there and actual size, you will find "html" and "(cached)"
This is also good to know when you passing parameters like "blah-blak.html" to the php file and expect it will not be cached. Well, it will be cached.
Tested on FireFox Quantum 57.0.1 (Mac 64bit)
P.S.
Chrome 63 on Mac is capable of recognising real file type in this situation. So it cannot be fooled.

Thinking out of the box here:
but if your list has 3000 lines and growing (as you said)
is it possible for you to establish its maximum size ?
let's say the answer is 10,000 (max) items; then do you really need an ajax call ?
you could transfer the data straight away with the page
(depending on your architecture of course, you could come out with different solution)

What purpose is of "&rnd=" parameter in http requests?

Why do some web-applications use the http-get parameter rnd? What is the purpose of it? What problems are solved by using this parameter?

This could be to make sure the page/image/whatever isn't taken from the user's cache. If the link is different every time then the browser will get it from the server rather than from the cache, ensuring it's the latest version.
It could also be to track people's progress through the site. Best explained with a little story:
A user visits example.com. All the links are given the same random number (let's say 4).
The user opens a link in a new window/tab, and the link is page2.php?rnd=4. All the links in this page are given the random number 7.
The user can click the link to page3.php from the original tab or the new one, and the analytics software on the server can tell which one by whether it has rnd=4 or rnd=7.
All we can do is suggest possibilities though. There's no one standard reason to put rnd= in a URL, and we can't know the website designer's motives without seeing the server software.

Internet Explorer and other browsers will read an image URL, download the image, and store it in a cache.
If your application is going to be updating the image regular, and so you want your users to not see a cached image, the URL needs to be unique each time.
Therefore, adding a random string ensures this will be unique and downloaded into the cache each time.

It's almost always for cache-busting.

As has been suggested by others. This kind of behaviour is usually used to avoid caching issues when you are calling a page that returns dynamic content data.
For example, say you have a page that gets some current user information such as "mysite.com/CurrentUserData". Now on the first call to this page, the user data will be returned as expected, but depending on the timing and caching settings, the second call may return the same data - even though the expected data may have been updated.
The main reason for caching is of course to optimise the speed of frequent request. But in the instance where this is not wanted, adding a random value as a query string parameter is known to be a widely used solution.
There are however other ways to get around this issue. For example if you were doing an Ajax request with javascript/JQuery. You could set the cache to false in your call...
$.ajax({url: 'page.html', cache: false});
you could also change it for all page calls on document load with...
$.ajaxSetup({cache: false}});
If you were to do an MVC application, you can even disable the caching on the control action methods with an attribute like so...
[OutputCache(NoStore = true, Duration = 0, VaryByParam = "*")]
public ActionResult NonCacheableData()
{
return View();
}
(thanks to a quick copy and paste from here)
I dare say there are also settings in IIS you could apply to get the same affect - though I have not been that far with this yet.

Cookie is set twice; how to remove the duplicate?

So I have a website that uses a cookie to remember the current layout state across visits. Everything was working great until I added a Facebook 'Like' button to the site which generates links that allow users to share a certain UI state (a little confusing but not really relevant to the problem).
The problem is that when I visit the site via one of these Facebook links a second copy of my layout cookie seems to be created (as in, I see two cookies with the same name and different values). This wouldn't be too terrible except that the value of the duplicate cookie appears to be stuck, coupled with the fact that when the user returns to the site the browser remembers the stuck value instead of the most recently set value (so it's kind of like there's a "good" cookie that I can still work with, and a "bad" one which I cannot, and the browser likes to remember the "bad" cookie instead of the "good" cookie). This breaks my layout tracking/remembering functionality.
So there are two questions here:
How do I stop this from happening/why is this happening in the first place?
How do I fix things for any users that already have a stuck cookie (I know I could just pick a new name for the cookie, but I'd rather do it by finding a way to properly unstick the stuck cookie)?
If I use Chrome's developer console after visiting the page in a stuck state, I can see that document.cookie is (formatting added for readability):
layoutState=[{'id':6,'x':8,'y':1525,'z':4,'url':'undefined'}, {'id':1,'x':625,'y':709,'z':2,'url':'undefined'}, {'id':2,'x':8,'y':37,'z':3,'url':'undefined'}, {'id':3,'x':625,'y':1179,'z':5,'url':'undefined'}, {'id':4,'x':626,'y':37,'z':1,'url':'undefined'}, {'id':5,'x':626,'y':357,'z':1000000,'url':'http://m.xkcd.com/303/'}];
WibiyaNotification1=1;
WibiyaNotification213286=213286;
WibiyaNotification213289=213289; wibiya756904_unique_user=1;
JSESSIONID=DONTHIJACKMEPLEASE;
WibiyaProfile={"toolbar":{"stat":"Max"},"apps":{"openApps":{}},"connectUserNetworks":[null,null,null,null,null,null]};
WibiyaLoads=59;
layoutState=[{'id':6,'x':8,'y':1525,'z':4,'url':'undefined'}, {'id':1,'x':625,'y':709,'z':2,'url':'undefined'}, {'id':2,'x':8,'y':37,'z':3,'url':'undefined'}, {'id':3,'x':625,'y':1179,'z':5,'url':'undefined'}, {'id':4,'x':626,'y':37,'z':1,'url':'undefined'}, {'id':5,'x':626,'y':357,'z':6,'url':'http://m.xkcd.com/303/'}]"
Ignore the Wibiya cookies and the JSESSIONID. The stuck cookie is the first 'layoutState' instance, and the one that I can still manipulate in JavaScript is the second 'layoutState' instance. Here is what I get if I change some things around:
layoutState=[{'id':6,'x':8,'y':1525,'z':4,'url':'undefined'}, {'id':1,'x':625,'y':709,'z':2,'url':'undefined'}, {'id':2,'x':8,'y':37,'z':3,'url':'undefined'}, {'id':3,'x':625,'y':1179,'z':5,'url':'undefined'}, {'id':4,'x':626,'y':37,'z':1,'url':'undefined'}, {'id':5,'x':626,'y':357,'z':1000000,'url':'http://m.xkcd.com/303/'}];
WibiyaNotification1=1;
WibiyaNotification213286=213286;
WibiyaNotification213289=213289;
wibiya756904_unique_user=1;
JSESSIONID=DONTHIJACKMEPLEASE;
WibiyaProfile={"toolbar":{"stat":"Max"},"apps":{"openApps":{}},"connectUserNetworks":[null,null,null,null,null,null]};
WibiyaLoads=59;
layoutState=[{'id':1,'x':8,'y':39,'z':1000000,'url':'undefined'}]
The second 'layoutState' has the correct information that I want the browser to remember. However what the browser actually remembers is the value of the first instance.
I've tried unsetting the cookie entirely, which causes the second instance to disappear, but nothing I do seems to get rid of the first instance. I get the same behavior in all major browsers (Chrome, Firefox, IE), which makes me suspect that I must be doing something fundamentally wrong here, but I'm not sure what it is.
You can view the site itself here. Or click here to access it via a Facebook link (should generate a stuck cookie). Any help is much appreciated.
Update:
So the steps to reliably reproduce the error are as follows:
Visit the site via the Facebook-style link
Make some changes to the layout, and then close the tab.
Visit the site via the normal URL.
Your layout from the initial visit should be correctly remembered, so change some things around and then refresh the page. When the page reloads, your changes will no longer be remembered.
I've also noticed that revisiting the site via the Facebook-style URL is able to clear/reset the stuck cookie. So it's like the browser is keeping a separate cookie for each URL path, or something, and not allowing the root page to access the cookie that was set on the other URL path. I thought I might be able to fix this by explicitly setting path=/ on the cookie, but no dice.
Update 2:
I've found that if I set both the path and the domain of the cookie I get different behavior in all browsers:
Firefox - Works correctly now, hooray! Worked correctly once, then broke, boo!
Chrome - No change
IE - Seems to be keeping separate cookies for each URL, so the Facebook-style URL remembers one state, and the standard URL remembers a different state. Both update correctly and independently of each other. This is kind of funky, but still way better than the stuck/broken state.

Dude(tte), there are inconsistencies, and a bug, in your cookie setter.
1. Make sure path and domain is correctly set
The path and domain should be the same for both clearing the cookie and setting it. See your code here:
document.cookie = c_name + "=; expires=Fri, 31 Dec 1999 23:59:59 GMT;";
and compare it to:
var c_value=escape(value) + "; expires=" + exdate.toUTCString(); + "; path=/spring; domain=aroth.no-ip.org";
you will see that the setter has both of them, but the deleter doesn't. You will bring about chaos.
2. Oh, and that nasty semicolon
That second line of code I quoted above, has a semicolon introduced in the middle of a string concatenation expression. Right after exdate.toUTCString(). Kill it. Kill it…now.
At least on my Google Chrome, I managed to get it run correctly, if I set a breakpoint at json = "[" + json + "]"; and modify setCookie before it is executed.
P/S: It was a bizzare debugging experience, where I managed to set 4 layoutState cookies, by fiddling with path & domain.

This may be too simple, but just in case, are the cookies recorded for two different paths? If the URL is different, you may be setting your cookies for a restricted path, so the system would take them differently.

Here is a solution, the / slash help to do not set duplicate cookie of same name
setcookie('YourCookieName','yes', time() + 400, '/');

check in Chrome console -> Resources if your page gets loaded twice. That would be the reason of double cookie.

There is again the problem left after identifying the problem and taking prevention by correctly setting the cookie.
You also need to delete previous incorrectly set cookies in your or in your client's browser.
So observe the cookie set from developer tools and search for path and subdomain and put those explicitly on your code to delete.
function eraseCookie(c_name) {
document.cookie = c_name + "=; expires=Fri, 31 Dec 1999 23:59:59 GMT;";
}
function eraseCookieWithPathDomain(c_name) {
document.cookie = c_name + "=; expires=Fri, 31 Dec 1999 23:59:59 GMT;path=/yourpath/to; domain=sub.domain.com";
//you can remove this function call on your second upload if you are confirm that the previous cookie setter expired
}
You may need to call function eraseCookieWithPathDomain right after eraseCookie or even every time after document load depending in your application.

You can add the following key in the AppSettings in the web config file it solves the issue of duplicate cookie.
<!-- Tell ASPNET to avoid duplicate Set-Cookies on the Response Headers-->
<appSettings>
<add key="aspnet:AvoidDuplicatedSetCookie" value="true" />
</appSettings>
This will help avoiding the duplicate Set-Cookie() in Response Headers.

It seems the issue is not a duplicate cookie (cookies overwrite themselves) but a duplication of the DATA in your cookie.
I think you'll have to modify the script that reads the cookie and clean out the duplicate value if it's detected.