I'm wondering how Google Analytics Real Time user interface works, what's the technique ? Do they use long-polling from the client to keep the UI statistics instantly up to date by delivering realtime information from the server to the client?
I just open Chrome dev tool on network tab and there is a infinite request on https://www.google.com/analytics/realtime/bind
Does anybody know the trick? It works flawless...
The below refers to how the real time data is collected, not how the UI updates. (It looks like the UI is just using AJAX polling on the client-side, though)
No special polling or client-side technique is used. Data collection is the same as it always has been.
Instead, Google Analytics will assume that someone who's triggered a pageview in the last 5 minutes is still an "active" visitor.
From e-nor:
These visitors have been active in the last 5 minutes, any one not active for over 5 minutes is dropped.
I was one of two people who built the first version of the Realtime Analytics UI. We used Closure's BrowserChannel.
Related
I want to display the number of active visitors on my website, I tried using django-tracking, django-active-users but none of them worked because they are outdated. I am using Django 3.0.8 and all of those modules aren't supported with django3.
I also tried doing this with Google analytics but their real-time reporting API is in limited beta and not open to everyone.
Is there any way I can get this to work using Django active sessions or something?
Create a DailyCount model which has date(DateField), active_users(PositiveIntegerField).
Then use celery periodic tasks with a function that is triggered at hour=0, minute=0, i.e., every day.
Create a DailyCount instance in the triggered function.
Whenever a user visits the site for the first time that day, increase DailyCount.active_users by 1.
Some background, the environment I am working in is PHP on the server side and Javascript on the client side. To track a user session, I make a JS PUT request to a server every 5 seconds. Through this I get the amount of time they were on the site, scroll depth, click events, etc.
With tools such as MixPanel or Google Anaylytis, they have the same metrics being measured, yet there isn't constant js calls being made. So my question is how are they tracking the user without javascript?
Your question seems to be... "how does google analytics know how long a user spends on a site without sending constant updates"? Well, short answer is that it doesn't (but it can, read on...!)
In the simplest setup, it will create a session when the first page is opened. Then it will track the time until the second page is opened, then the third. But it doesn't know when the third page was closed, so the total time for the session will only be "time page 3 was opened" minus "time page 1 was opened". And that's what you'll see on the analytics page. This is how most analytics systems work and yes, it's innaccurate. But that's how it is.
HOWEVER! There is a way around this, and it's called an "engagement hit" - essentially just tracking when a user does something on your page, e.g. playing a video or clicking a banner. In fact, you can track all sorts of events with Google Analytics and this will all help towards accurately reporting time-spent. This will involve more JS calls being made than just the typical onload event, but they certainly will not be constant (unless you set up some crazy events).
There's some good information about this on the GA docs site. It might be worth having a look and trying to implement something similar.
I have an html5/javascript application in which multiple users can be viewing the same set of data of any given time. For the sake of a real world example, lets say its a calendar type page.
So user1 is looking has the browser open and looking at the calendar page and user2 is also on the calendar page. User2 makes a change to the calendar and i'd like (as quickly as possible) for those changes the be recognized and refreshed on user1's screen. What is the best way to do this?
I'm thinking about have a mysql table for active users that stores the page they are currently on and a timestamp for its last update, then use ajax calls to ping the server every few seconds and check for an updated timestamp, if its newer than what they have client side, the new data gets sent and the page "reloaded." I am putting reloaded in quotes because the actual browser window will not be refreshed, but a function will be called via javascript that will reload the page. Sort of the way stack overflow performs its update checks, but instead of telling the user the page has changed and providing a button for reload, it should happen automatically. If user1 is working away on the calendar, it seems it might be quite annoying for user2's screen to constantly be refreshing...
Is this a horrible idea? Is pinging the server with an ajax request every few seconds going to cause major slow downs? Is there a better way to do this? I would like the views on either users side to be real time because its important that user1 not be able to update an element on the calendar page that user2 has already changed.
Update: based on some web sockets research it doesnt seem like a proper solution. First its not compatible with older browsers and i support ie8+ and second i dont need real time updstes for all users on the site. The site is an account based applicatiin and an account can have multiple users. The data needs to sync between those users only. Any other recommendations would be great.
You need realtime app for this. You should have a look at socketio. Everytime a user log in, you make him listen for changes on the server. Then when something changed on the server, every users listening are notified.
you can find examples on the official website : http://socket.io/
I want to know whether Google Analytics will track traffic on my website, if my browser has cookies disabled?
I tried to search on Google but couldn't find much information about this question.
I would appreciate if you can also provide me with a source link.
Thanks
No, if you disable cookies, Google Analytics will not track you.
Google Analytics tracks you by creating cookies (or using existing cookies it finds). Then it uses that cookie information in the request it sends to http://google-analytics.com/__utm.gif. That "request" to get the __utm.gif is how the data is transmitted to Google Analytics.
Googling this is a bit difficult, since it mostly just turned up information on the cookies themselves, not what happens when they're disabled.
So, I did an experiment to prove it. I loaded StackOverflow in a fresh FireFox install with cookies disabled.
Below is the list of HTTP requests. Note that it loads ga.js, the Google Analytics script that attempts to track you. But, no request for __utm.gif is made. ga.js merely runs, realizes its unable to create cookies, and as a result has no way to create "state" from pageview to pageview.
Without cookies, Google Analytics would view every single "hit" as its own visit, and each of those visits would be a bounce. This data would be useless. As a result, GA makes the logical choice to just not track those people.
The answer appears to be YES, below is what google has to say.
The "measurement protocol" can even work withOUT cookies or js, it seems to be in beta so you have to request access.
Universal Analytics supports data collection without browser cookies.
The Universal Analytics collection methods (analytics.js and the Measurement Protocol) can be implemented and used to collect visitor usage data without cookies. These methods also work if cookies are cleared or disabled. Website visitors that don’t want their visit data reported by Google Analytics can install the Google Analytics opt-out browser add-on to opt-out.
Source: http://support.google.com/analytics/bin/answer.py?hl=en&answer=2838718
Thanks for the explanation however I thougfht cookies or no cookies Analytics will still record a vist but just can't track it. So when you say "Google Analytics would view every single "hit" as its own visit, and each of those visits would be a bounce." are you implying this is still recorded as say a 0 second or 100% Bounce visit? Or Analytics records no visit at all?
It can, but only if the site owner sets it up that way, which is difficult. The problem is tracking a user across different URLs while maintaining a UUID.
The short answer is yes - but only if the site owner found his way around the pitfalls.
I did it on my website.
I am trying to prevent fraud in a webproject I am building.
The project is a game which includes multiple websites.
Each website does a ajax check for with each pageview to a webpage on my server for a status update of the game.
The response page, lets say www.domain.com/response.cfm (it is coldfusion) normally returns nothing, but at a certain point of time within the games timeframe, it will display a JSON string with information.
This information is then used by the script that is included on the websites.
So website A has been viewed 100 times (all of its pages), which will generate 100 ajax calls.
The problem I have is that a robot could check the ajax destination too, and much faster. Now I can detect a robot, or could make it difficult for him by using a session or checking for cookies, BUT...
the biggest issue is that I found out you can do a lot in the Firebug script console, or the Safari console. Probably Chrome too.
With this console, they can even evade the crossdomain restriction. I created a simple script that does a couple of calls to the Ajax page and when I go to the same domain first, and then use the console...there is no crossdomain limitation. And you execute all kind of javascript, so in essence someone like me could commit fraud in the game by using the javascript console which masks him as regular browser user.
My question now is: Does anyone know how to prevent this? I tried to disable the usage of the console but I don't think I can. It may be possible to detect if the console is active and then disable MY scripts so the game doesn't work. But I think they can load the script source in the console manually and then the game does work.
Looks like console is a beautiful thing, but a nightmare for me now to prevent people cheating in the game I am creating.
Hope anyone has suggestions.
ps: of course I am trying to implement som server side checks to detect cheating, but most of the time it is not realtime.
UPDATE 19/3/2012
The fraud that I am trying to prevent is cheating in the game by polling the page that generates logic for the next step of the game. This is a serverscript page which generates json code which will trigger a change on the website the game is played on. For your information, websites the are involved have a script in there header, like google analytics, so they will communicate with my server every pageview.
Polling that serverpage can reveal information which will gain the cheaters knowledge or progress.
So i have to prevent people from getting knowledge ahead of other earnest players by monitoring the serverpage which will reveal information at a certain time. I don't want them auto polling it and when info is revealed, the send themselves a notifcation and check the website.
So what I will do is make sure that if people have to many pageviews per second, they are blocked. Plus you need a cookie to be able to join in and you only get a cookie by logging in. Hopefully this will give me enough tools to make it as robust as possible.
Thanks for all your knowledge, people.
It would be very, very difficult to disable web consoles across the majority of browsers, and anyone who managed to do this would probably be exploiting a browser bug. But read on...
First rule of web programming: You can never trust anything you receive from the web client. Anything that gets sent to your data might have been forged or altered intentionally or unintentionally, and even if you did manage to block a web console, what's to stop me from opening it in a different browser which specifically disallows websites with the console? So that's out. As #DCoder mentions in the comments, there are other methods as well, including browser extensions, which would allow user-defined JavaScript to be executed.
So any checking you do has to be server side. I know you're trying to do some checking already, and it's hard to give advice without having more specifics. That said, one way to do this, as far as I can see right now, is to issue each client an ID and store that in a database somewhere. They can't be sequential IDs, and make sure that they're not trivially forgeable even if someone has a bunch of different IDs (for example, you might want to salt the username, and then hash it). Each time a request is made to the server, only issue a response if the last request was >500 ms ago, and update the database accordingly. Expire the ID after logoff or some time.
The first thing you should think about is securing your server, not the client. It's impossible to hide client code from the client. While it might arguably help prevent a few people who want to cheat from cheating, it's not your primary objective. You have to do this from the server side. This means validating the requests on the server to ensure that they conform to your expectations to some degree.
Game companies will
Require user authentication of some kind so they can identify users
Create some rules about possibilities. For example, the laws of physics should apply, so you know when someone has cheated. Something they can validate as human activity.
Ban people who cheat
If you are not sending data continuously over the network, then you have an issue which is unsolvable unless you are willing to make checks on the server securely and continuously over the course of the game. This will increase server load, but that's the unfortunate cost of preventing cheats.