Puppeteer: Αccept all cookies in multiple websites with cookie banner - javascript

I create a script with Node.js and Puppeteer that loads multiple sites like (site A, B, C etc). I want to find all the cookies that site uses.The problem is, that some sites have a Cookie Banner to accept or decline. If you accept the banner the website adds some extra cookies.
So to capture all that cookies is there a general solution to accept all the different banners or to set some initial parameter on header of the initial request to inform the site that I accept all that cookies?
Here is an example of a site with Cookie Bannner initial set 6 cookies and if you accept the banner the total cookies are 48.
https://siteimprove.com/en/gdpr/who-gdpr-affects-and-whose-data-is-protected/
I need a general solution. Because I have a list of websites.

You can use the identifiers of cookies banners : https://www.fanboy.co.nz/fanboy-cookiemonster.txt

Unfortunately, there is no standard way to do this. Because there is no standard for "cookie banners". If your list of websites isn't terribly large, your best bet is to figure out what the cookie is for each site, store them systematically, and use the appropriate cookies based on the domain being navigated.
UPDATE: see the puppeteer docs here on how to add cookies to your request: https://pptr.dev/#?product=Puppeteer&version=v5.4.0&show=api-pagecookiesurls

Related

Set cross domain cookie using javascript

Hello I want to set crossdomain cookie using javascript.
So on domain localhost I can easily set cookie. So I load the page on localhost:300 and in the Browser's console I type:
document.cookie = "my_cookie=works; Domain=localhost; path=/;"
This method works - I can see the cookie being set.
But now, I want to set cookie to a different domain
from the same place:
document.cookie = "my_cookie=doesntwork; Domain=.hubspot.com; path=/;"
And this time it doesnt work - I still have the old cookie value and domain:
I know that what I am trying to do is possible because I have seen it on many sites.
For example if check the cookies on this site you will see many cookies with domain google but the domain is stackoverflow.com .
For example:
So, how can I imitate/mimic this? I see some suggest using iframes - I tried without success - the cookies are still blocked. Please provide workable example (I believe 5 lines of code can do )
I want to open localhost page in my browser
and run JS code that would put cookie with domain like .example.com.
This is obviously possible. But I cant do it.
I'm not trying to complete some job here - I just want to understand how the cool kids do it.
You cannot set a "cross-domain" cookie for security reasons. If this was possible, you'd be able to remotely sign people out of their accounts on other websites if they visited yours by overwriting their cookie for their site. The cookies you are seeing on pages like StackOverflow are because that page includes resources from that server. For example, you can see imgur cookies on this page since the image you linked to is hosted there.
SORRY FOR MY BAD ENGLISH!
There is no such thing as a cross-domain cookie that said you can get the same tracking functionality with sharing a cookie.
for example we want to track a user in our ad network across affiliated sites.
our tracking endpoint is : https://tracking.a.com
here we set a deviceid in the cookies.
Our Affiliated Webmaster b.com adds a javascript which adds an iframe to the page. iframe loads somecontent from a.com hence the cookies are shared then you can inside the iframe send the users current location ( b.com/shop ) to a tracking endpoint and attach deviceid cookie which can only be accessed by webpage on a.com. now deviceid or user can be related to the affiliate website and their intrests can be pridicted - but it is illegal - and safari deletes these cookies

How to prevent cookies from being set or stored until the consent?

I have a website developed in PHP and JavaScript language and I am using cookies on my website. Also there are many third party scripts like google analytics, mouseflow, third party chat script etc on my website. These scripts are also storing cookies.
To get my website GDPR compliant, before storing marketing cookies (like analytics) I need to make sure that the visitor has given his/her consent for storing it.
We can show the visitors a pop-up stating the cookie policy and once they accept, we will start storing cookies.
So, How can we prevent any of the cookie to be stored before the consent of the user.
Well what I always do is check if the cookie exists, if not I won't allow access to the features of the website that allow that. If it exists, the user has given consent.
To make it cleanly, you need to check server side that their is a "cookie consent" cookie/session var present. If that's the case, serve the normal website, if not, render a barebone page with just the cookie consent form and 0 tracking scripts.
It's not so easy, but i guess getting rid of the tracking scripts is not an option for you

How would I allow third party cookies in safari or create a workaround?

I am trying to find a way to share a session cookie between my two sites. I am unsure if this is the best approach or if I should place these sites on the same domain or switch to a different approach.
I have two sites: adminCRM.com and customerSite.com. The admin site is based on a 1.5 Mean skeleton, using oauth and passport to generate session cookies that keep users logged in. Users are given admin and/or customer roles. customerSite.com has an iframe which calls a login api on adminCRM.com. customerSite then gets a session cookie from adminCRM's response header.
For the most part, this session cookie approach appears sufficient.
The problem is that this approach does not work with safari and other browsers that deny third party cookies by default.
I have researched work-around and have yet to find a current, viable solution. However, I am very new to all of this and I feel like I may be tackling this the wrong way.
Is there a workaround to set third party cookies on safari? Should I abandon third party cookies and switch to a different approach? Any help is much appreciated.
Update: Eliminated the 3rd party cookie Safari issue. Kept the domains separate.
I was not able to find a way to allow 3rd party cookies, but that would have been hackish anyway.
I ended up still keeping the domains separate and just switching from session cookies to JWT. I used mleanos meanstack feature:
https://github.com/mleanos/mean/tree/feature/JWT-Auth-Lib
By making one of the sites subdomain of the other, I think it is no longer considered third party by the browsers. E.g. adminCRM.customerSite.com

Visits without cookies

I am trying to add a feature to a Java/Tomcat web-application to follow the users during their visits on a website A through Javascript calls from the pages of A to a server B (B's applications run on Tomcat 7). The server has a different address from the website.
Each time a user opens a page on A a piece of Javascript is loaded from a url on B. The javascript is served on B by a controller method, which calls HttpServletRequest.getCookies() and looks for a cookie with the right name. If no such cookie is found, a new one is created and added to the response through HttpServletResponse.addCookie(). The cookie mentions the path and IP of B, so they are available there.
The Javascript on A will then regularly send Ajax POST calls to B, where cookies are checked.
While this scenario works fine for many users, I still get a large number of Ajax calls from A to B without any cookie. Not only the cookies I set manually are missing, but also the session cookies set by Tomcat on B. I understand some users may have just set their browser to prevent or block cookies, but there seem to be too many empty requests. Despite a thorough inspection of the logs, I could not find any common feature that would explain the lack of cookies.
Any idea of what I might be doing wrong ?

Facebook Connect for one application with multiple domains?

I'm implementing a plug-in that's embeddable in different sites (a la Meebo, Wibiya), and I want to use Facebook Connect. The plug-in is supposed to be embeddable in sites with different domain names. Problem is, Facebook connect allows only one domain per application you register.
The question is, how can I have multiple domains for a single Facebook application, assuming:
When users "Allow" the application on one site, they won't have to "Allow" it on other sites as well.
Preferably, after the initial log-in, users won't see a pop-up opening on every site they log-in to (i.e. - I'd rather not open a link to my domain and do the log-in process from there).
Is there anyway of doing that?
If not, is my only option is to manage all the log-ins from a single domain and pass the cookies back to the original domains?
And if I pass the cookies between domains, how can I be sure that Facebook won't block this kind of behavior in the future?
I'd appreciate any suggestions, though I'd prefer an official solution over hacks, if at all possible.
Im assuming you are using facebook.php by Naitik Shah? Your widget would need to be on every page of course and include the async script connect-js.
I am currently developing a facebook login based application myself.
I would say the best solution is too login through your own domain and pass the cookie. Your app/widget will be the only one they allow to share information with. Nothing should be different in operation from a single page solution. I envisage a PHP plugin which executes a login from an outside domain and passes through the cookie to the site via the widget. return the cookie securely how you wish (except for something dodgy like storing it in a div and retrieving it..or something a hacker could try to spoof). the site will then use the cookie for account and user id purposes and the widget will control all login actions and session finding using the async script (but routed through a different domain).
Sorry I can't be more help but this is the only solution I can think of, and it seems you have already anyway.
In terms of keeping session control across different domains you only need the 3rd party cookie to be active. Once your page is activated for your domain you will already have the cookie for that domain if you haven't logged out or it hasn't expired. A benefit of using an outside management domain.
It would seem this is also the most reliable way compared to any successful hack for multiple domains, because I would see fb and Oauth2.0 as being ok with an approved party sharing info (cookies) to another party approved by the approved party. But.. It could be problematic if they think the user will have privacy issues, because you could potentially share the cookie on any site without the users permission. So you have to be careful about notifying the user about all the sites they will be auto logged into and treating them with respect.
Good luck with it, hope you let us know how it goes.
There is easy and clean technique -> Single Sign On (SSO). You can search on about it.

Categories

Resources