In my current job I have to import an analytics framework into our clients pages (we're a 3rd party also, our services are provided by including our tag in the client's pages) for counting basic metrics such as: Visitors,Visits,Views,Avg. Visit duration,Bounce Rate,Pages/Visit .
I've looked at several opensource solution such as piwik etc.. but as we handle allot of traffic this will come at a pretty hefty price, so obviously GA seems like the most reasonable solution for us. the problem is that the bulk of my clients already have GA on their pages and everybody has a different version of it (analytics.js,ga.js,dc.js etc..) where each version has a different method of working with multiple trackers on page and some don't support that all together. so using a pixel reporting technique seems like the safest method to use here but .. I can't find any documentation on google or anywhere else for that matter explaining what functionality I will loose when using this approach.
The API documentation does a decent job of showing me how to do it myself and there's also nojsstats (which i prefer not to use) but has anyone had any experience with this kind of a scenario and can eloborate on what functionality will be lost as a result of this approach? or has found a different solution for this kind of problem?
Thanks in advance,
Amnon
You can look into the parameter reference for the measurement protocol - that's what you can get, provided you pass all the parameters (so everything that's not in there you lose, most notably all Geo information which will be set to the location of your server).
The biggest challenge would be that you'd have to maintain a session id in your respective clients applications, else you'll only register single page visits with 100% bounce rate (since Google will not know which visits belong to the same user). That seems a lot more cumbersome than additional javascript trackers.
However since they already have Google Analytics you do not need to implement any tracker at all, you can query the existing accounts via a serverside language of your choice (or via Google Apps Script). Which is actually what the documentation you've linked to describes (Core reporting API is not about collecting data, but about querying existing data).
Related
I'm building a website where users can create new pages with dynamic URLs. I would like to add traffic analytics for each of these pages individually and allow users to view a dashboard of the data collected for the page they have created (just view, they shouldn't be able to edit the dashboard they are given). This should also be automated so that when a new user registers and creates their own page, this is tracked automatically.
If possible, this should work in Vue.js, but anything that works with vanilla JS should be fine too.
Is this possible with a tool like Google Analytics or a similar alternative? Is this expensive?
Most of the Web applications now a days are single page apps or Client side web applications. This means most of the analytics platforms like GA etc will work with your web application.
However, You will need to implement the analytics in a specific way for you to capture the required metrics. I think using Google Tag manager and creating a DataLayer will make it easier in the long run. Once the datalayer is available you can use this data with any analytics tool.
In order to track a screenview change, you can use any custom trigger available with your application that notifies of the screen view change or you can use the History change as mentioned here: Google Tag Manager and Single Page apps.
You can also use the App Rote changes as mentioned here on simple routing to populate the datalayer and use the values from the datalayer in any analytics including the GA.
Other option is using vue-analytics . There is Easy to follow documentation from the author and the branch is here
This way any new pages added etc will be tracked automatically.
Hope this helps.
Legal context
Well... What is the targeted audience? If your main targets are privacy advocates or European people covered by the General Data Protection Regulation (GDPR), it would be a terrible idea to select Google Analytics as the solution. In France, for instance, we have a National Commission on Informatics and Liberty ("CNIL" in French). It is:
an independent French administrative regulatory body whose mission is
to ensure that data privacy law is applied to the collection, storage,
and use of personal data. (Source: Wikipedia)
The CNIL tends to strongly discourage the use of Google Analytics (see Q&A on the CNIL's formal notices concerning the use of Google Analytics). Therefore, be very careful about that, especially if you are targeting an international audience.
Recommended tool
When it comes to web analytics, you have a wide range of technical solutions. For obvious reasons, I will not put the complete list here, but I can give you a tool that I know and use: Matomo.
This one is recommended by the CNIL, it is open source (see GitHub) and GDPR-compliant. You can self-host Matomo (On-Premise), which is completely free. You only pay if you use Matomo Cloud.
Matomo has a technical website dedicated to developers and exposes an API which should allow you to do what you want. Doing some research, I even found a thread on the official forum where the OP seems to have a similar problem to yours. Since it is a requirement in your question, you can also find information about user permissions.
On this page, you can read that Matomo developers use Vue for the front-end code. There is also a plugin called vue-matomo, which may be useful in your project.
Finally, to track single-page applications (SPAs), read on: Single-Page Application/Progressive Web App Tracking: Integrate - Matomo Analytics (formerly Piwik Analytics) - Developer Docs - v5
As I read from this source-
If you have a website that has dynamic content or multiple pages
and steps under one URL (or an all-flash website), you have to do a
little more work to get Google Analytics to effectively track your
pages.
Here are some more references you can check upon-
Here you can read, how Google Analytics works.
Here you can find its pricing plans which are free for small businesses.
Here you can read how Google Analytics performs for dynamic links.
Here is another answer given by #Ruslan Konygin on a similar problem.
Here is a complete guide on Tracking Dynamic Sites and Pages with Google Analytics.
I want to fetch my Google Analytics report to my local through an api.
I have created a Report in Google Analytics explore : this image.
Here I can download the csv which I want to read the data in my local through the csv.
Now for that I am trying to follow this document :
Quick Start
Which I am not sure this is correct approach or not. Also got stuck on some point like
I have downloaded Credentials.json where not sure what to change in client_email.
What is the best approach to achieve that. Any programming language is fine. Can anybody help me here to achieve this.
That api you're using is called the Reporting API or the Data API as they've remarketed it for GA4. And generally, it would be a good practice to use it and there are plenty of libraries and examples of its usage in multiple languages, including Python, here is a good example. I'd suggest either going with a third party article detailling the setup process, or even just using a third party library that can abstract a lot of roughness Google introduced in the API.
But that api is new, and the product, overall, is effectively just a beta with quite a number of bugs and documentation holes. There's a simpler way to get the data out. First, link the GA4 property to BQ. This will effectively set up daily (or streaming) exports to BQ.
Once the data is in BQ, you're able to see the raw data and understand it way better than in GA4 explorer. From there, it will be much easier for you to export the data to your local from BQ. Here is a library that could be handy. Having the raw data in BQ makes it way easier to debug your data reports.
On my website, users enter some personal information, including ZIP code. This information will be passed to a function that will determine the display of the next page.
The problem is that the function utilizes an underlying statistical model, for which zip codes have too many possible values (~43,000) to be useful. I want to map zip codes to something broader, like designated market area (DMA has around 200 possible values).
But using Google Analytics and BigQuery, I already have the user's DMA before they even enter their ZIP code. Is there a way to access that information while they are still on the page so I can input it to the function?
In case you are wondering if you can use Google Analytics Information in realtime (not quite clear from the question), that will not work - GA does not work in realtime; data processing time is announced as 4 hours for the "premium" version to 24 hours to the standard version, and even if it's often faster you probably do not want to build you business on an undocumented feature that might or might not work as expected.
Also API limits make realtime data retrieval unfeasible even for smaller sites.
If however you have a stash of precomputed data that can be linked to the current user via an identifier (clientId or similar) it would probably be best to export this to external storage as suggested by Willian Fuks.
Since you mentioned personal data, keep in mind that this must not be stored within Google Analytics as per Google's TOS.
(not quite an answer, more like some thoughts of mine)
I don't think that running queries in BQ for each user you find in production is a good approach.
Costs will increase considerably, performance will not be satisfactory by any means in this scenario and you might start hitting quotas limits for jobs against a single table.
One possibility that might work is having your back-end use some google analytics client for retrieving data from G.A. Still, you should check if the quotas are appropriate for you.
Another possibility (I suspect this might be the best option) that you may consider for your scenario is using Google Datastore. It might suit your needs quite well; you could have some table from BigQuery being exported to Datastore and have your back-end system query it directly for the user DMA.
I am looking to extend a web application using the new real time API in order to support collaboration (javascript). For that purpose I would also like to include a chat which will be available to users collaborating on same document. After extensive search I cannot find parts of the Drive API that can be used for this. Furthermore none of the open-source examples provided by google implement chat functionality.
Is there existing services/code that I can use to intergrade
chat into my application or I would need to implement it?
As mentioned before, the chat should be available to those collaborating on same document.
p.s I do not require any special features, just a simple chat as the one found in google docs etc.
Right now I don't know of any out-of-the-box solution to this problem. Some people have implemented chat in a realtime document by just placing the chat messages in the realtime data model.
This works fine so long as you don't also want to use undo/redo feature. If you are using undo/redo, then people would end up undoing the chat messages.
We are interested in adding some better support for this eventually, but no promises on anything in the short term.
You will need to build your own chat system, as Google just killed off XMPP support on the Chat API (which largely kills off the use of the API). The new Hangouts API does not provide access to Chat ( though some additional methods may come in handy: https://developers.google.com/+/hangouts/ for the dev API).
You are therefore left to your own devices. Fear not, however, writing a chat system is pretty easy. I am about to release a (mostly free) service to do so, so if you want to not have to write the code for it, I can keep you posted.
If, however, you'd prefer to build the code for it, you will most likely want to look into either socket.io or postal.js. both provide the same thing: a pub/sub model. From there, you will need to implement a choice of either:
Long polling: supported by all browsers but a bit clunky
Websockets: not supported by IE8 and below
This will serve as your data transfer.
Two other possible options are paid services: you can retrofit RabbitMQ to do what you want to do (this, however, will seem clunky). You can also retrofit the Meteord daemon, which does what you want natively, but has an outdated JS library.
The keyword of all this is pub/sub, though.
Does anyone know how disqus works?
It manages comments on a blog, but the comments are all held on third-party site. Seems like a neat use of cross-site communication.
The general pattern used is JSONP
Its actually implemented in a fairly sophisticated way (at least on the jQuery site) ... they defer the loading of the disqus.js and thread.js files until the user scrolls to the comment section.
The thread.js file contains json content for the comments, which are rendered into the page after its loaded.
You have three options when adding Disqus commenting to a site:
Use one of the many integrated solutions (WordPress, Blogger, Tumblr, etc. are supported)
Use the universal JavaScript code
Write your own code to communicate with the Disqus API
The main advantage of the integrated solutions is that they're easy to set up. In the case of WordPress, for example, it's as easy as activating a plug-in.
Having the ability to communicate with the API directly is very useful, and offers two advantages over the other options. First, it gives you as the developer complete control over the markup. Secondly, you're able to process comments server-side, which may be preferable.
Looks like that using easyXDM library, which uses the best available way for current browser to communicate with other site.
Quoting Anton Kovalyov's (former engineer at Disqus) answer to the same question on a different site that was really helpful to me:
Disqus is a third-party JavaScript application that runs in your browser and injects itself on publishers' websites. These publishers need to install a small snippet of JavaScript code that makes the first request to our servers and loads initial JavaScript loader. This loader then creates all necessary iframe elements, gets the data from our servers, renders templates and injects the result into some element on the page.
As you can probably guess there are quite a few different technologies supporting what seems like a simple operation. On the back-end you have to run and scale a gigantic web application that serves millions of requests (mostly read). We use Python, Django, PostgreSQL and Redis (for our realtime service).
On the front-end you have to minimize your payload, make sure your app is super fast and that it doesn't break in extremely hostile environments (you will be surprised how screwed up publisher websites can be). Cross-domain communication—ability to send messages from hosting website to your servers—can be tricky as well.
Unfortunately, it is impossible to explain how everything works in a comment on Quora, or even in an article. So if you're interested in the back-end side of Disqus just learn how to write, run and operate highly-scalable websites and you'll be golden. And if you're interested in the front-end side, Ben Vinegar and myself (both front-end engineers at Disqus) wrote a book on the topic called Third-party JavaScript (http://thirdpartyjs.com/).
I'm planning to read the book he mentioned, I guess it will be quite helpful.
Here's also a link to the official answer to this question on the Disqus site.
short answer? AJAX, you get your own url eg "site.com/?comments=ID" included via javascript... but with real time updates like that you would need a polling server.
I think they keep the content on their site and your site will only send & receive the data to/from disqus. Now I wonder what happens if you decide that you want to bring your commenting in house without losing all existing comments!. How easy would you get to your data I wonder? They claim that the data belongs to you, but they have the control over it, and there is not much explanation on their site about this.
I'm always leaving comment in disqus platform. Sometimes, comment seems to be removed once you refreshed it and sometimes it's not. I think the one that was removed are held for moderation without saying it.