I was wondering if there was an adapter library for Javascript to make integrating tools like Google Analytics, Kissmetrics, mixpanel, Chartbeat, gaug.es, etc easier. Something like SLF4J, but for event tracking. We are evaluating different tools, and it seems like it would be useful if it was possible to quickly swap between them.
We recently built analytics.js to solve this problem. It's a javascript wrapper library for Google Analytics, KISSmetrics, Mixpanel, Gauges, Chartbeat, GoSquared, and 16 other services.
The library does pretty much everything you mentioned:
Loads all the javascript snippets asynchronously
Abstracts the event-views and other API methods across all the libraries
And exposes each service's global object in case you need to do something custom
You can check out the open source library on Github: http://segmentio.github.com/analytics.js
Or for an easier solution that also supports the same abstraction on the server-side for Python, Ruby, Node, Java and .NET, check out https://segment.io
It turns out there is a library that does much of what I was looking for, known as Gatling.
It greatly simplifies tracking multiple analytics libraries at once, all asynchronously. In addition, you can even expose your GA 'gat' object for triggering page views and events manually.
It doesn't seem to use the Google Analytics 'gaq' object, and it doesn't abstract event-views across multiple libraries, which I think would be useful features, but it is pretty close to what I was looking for.
This area is sometimes called "Universal Tag" or "Tag Management" - the last time I looked there was a handful of big-ticket vendors, like Tealium, BrightTag & Tagman, but not much in the open source world.
In mid-2011, QuiBit (qubitproducts.com) launched their open-source OpenTag, but this seems to be an open/closed play, where the on-page javascript is open-source, but it is powered by their proprietary management system which does the tag injection - while they don't display terms & pricing on their site, I seem to recall they had a sizeable no-contract, no-pay starting band, which maybe you could use while you work out which solution(s) is best, then embed them directly before production ramps up to the paying levels of traffic.
I'm not sure there could be such a standard container, because every site has very specific needs in term of tracking (since features and usage are often unique).
But you could do the following:
design precisely the data model of what you want to track and build JavaScript objects or an API exposing it to the tracking code
design event handlers for every interaction / transaction you want to track
use a sort of plug-in system to map each tracking action to the core language of each tracking provider
Related
I'm building a website where users can create new pages with dynamic URLs. I would like to add traffic analytics for each of these pages individually and allow users to view a dashboard of the data collected for the page they have created (just view, they shouldn't be able to edit the dashboard they are given). This should also be automated so that when a new user registers and creates their own page, this is tracked automatically.
If possible, this should work in Vue.js, but anything that works with vanilla JS should be fine too.
Is this possible with a tool like Google Analytics or a similar alternative? Is this expensive?
Most of the Web applications now a days are single page apps or Client side web applications. This means most of the analytics platforms like GA etc will work with your web application.
However, You will need to implement the analytics in a specific way for you to capture the required metrics. I think using Google Tag manager and creating a DataLayer will make it easier in the long run. Once the datalayer is available you can use this data with any analytics tool.
In order to track a screenview change, you can use any custom trigger available with your application that notifies of the screen view change or you can use the History change as mentioned here: Google Tag Manager and Single Page apps.
You can also use the App Rote changes as mentioned here on simple routing to populate the datalayer and use the values from the datalayer in any analytics including the GA.
Other option is using vue-analytics . There is Easy to follow documentation from the author and the branch is here
This way any new pages added etc will be tracked automatically.
Hope this helps.
Legal context
Well... What is the targeted audience? If your main targets are privacy advocates or European people covered by the General Data Protection Regulation (GDPR), it would be a terrible idea to select Google Analytics as the solution. In France, for instance, we have a National Commission on Informatics and Liberty ("CNIL" in French). It is:
an independent French administrative regulatory body whose mission is
to ensure that data privacy law is applied to the collection, storage,
and use of personal data. (Source: Wikipedia)
The CNIL tends to strongly discourage the use of Google Analytics (see Q&A on the CNIL's formal notices concerning the use of Google Analytics). Therefore, be very careful about that, especially if you are targeting an international audience.
Recommended tool
When it comes to web analytics, you have a wide range of technical solutions. For obvious reasons, I will not put the complete list here, but I can give you a tool that I know and use: Matomo.
This one is recommended by the CNIL, it is open source (see GitHub) and GDPR-compliant. You can self-host Matomo (On-Premise), which is completely free. You only pay if you use Matomo Cloud.
Matomo has a technical website dedicated to developers and exposes an API which should allow you to do what you want. Doing some research, I even found a thread on the official forum where the OP seems to have a similar problem to yours. Since it is a requirement in your question, you can also find information about user permissions.
On this page, you can read that Matomo developers use Vue for the front-end code. There is also a plugin called vue-matomo, which may be useful in your project.
Finally, to track single-page applications (SPAs), read on: Single-Page Application/Progressive Web App Tracking: Integrate - Matomo Analytics (formerly Piwik Analytics) - Developer Docs - v5
As I read from this source-
If you have a website that has dynamic content or multiple pages
and steps under one URL (or an all-flash website), you have to do a
little more work to get Google Analytics to effectively track your
pages.
Here are some more references you can check upon-
Here you can read, how Google Analytics works.
Here you can find its pricing plans which are free for small businesses.
Here you can read how Google Analytics performs for dynamic links.
Here is another answer given by #Ruslan Konygin on a similar problem.
Here is a complete guide on Tracking Dynamic Sites and Pages with Google Analytics.
In my current job I have to import an analytics framework into our clients pages (we're a 3rd party also, our services are provided by including our tag in the client's pages) for counting basic metrics such as: Visitors,Visits,Views,Avg. Visit duration,Bounce Rate,Pages/Visit .
I've looked at several opensource solution such as piwik etc.. but as we handle allot of traffic this will come at a pretty hefty price, so obviously GA seems like the most reasonable solution for us. the problem is that the bulk of my clients already have GA on their pages and everybody has a different version of it (analytics.js,ga.js,dc.js etc..) where each version has a different method of working with multiple trackers on page and some don't support that all together. so using a pixel reporting technique seems like the safest method to use here but .. I can't find any documentation on google or anywhere else for that matter explaining what functionality I will loose when using this approach.
The API documentation does a decent job of showing me how to do it myself and there's also nojsstats (which i prefer not to use) but has anyone had any experience with this kind of a scenario and can eloborate on what functionality will be lost as a result of this approach? or has found a different solution for this kind of problem?
Thanks in advance,
Amnon
You can look into the parameter reference for the measurement protocol - that's what you can get, provided you pass all the parameters (so everything that's not in there you lose, most notably all Geo information which will be set to the location of your server).
The biggest challenge would be that you'd have to maintain a session id in your respective clients applications, else you'll only register single page visits with 100% bounce rate (since Google will not know which visits belong to the same user). That seems a lot more cumbersome than additional javascript trackers.
However since they already have Google Analytics you do not need to implement any tracker at all, you can query the existing accounts via a serverside language of your choice (or via Google Apps Script). Which is actually what the documentation you've linked to describes (Core reporting API is not about collecting data, but about querying existing data).
I am looking to extend a web application using the new real time API in order to support collaboration (javascript). For that purpose I would also like to include a chat which will be available to users collaborating on same document. After extensive search I cannot find parts of the Drive API that can be used for this. Furthermore none of the open-source examples provided by google implement chat functionality.
Is there existing services/code that I can use to intergrade
chat into my application or I would need to implement it?
As mentioned before, the chat should be available to those collaborating on same document.
p.s I do not require any special features, just a simple chat as the one found in google docs etc.
Right now I don't know of any out-of-the-box solution to this problem. Some people have implemented chat in a realtime document by just placing the chat messages in the realtime data model.
This works fine so long as you don't also want to use undo/redo feature. If you are using undo/redo, then people would end up undoing the chat messages.
We are interested in adding some better support for this eventually, but no promises on anything in the short term.
You will need to build your own chat system, as Google just killed off XMPP support on the Chat API (which largely kills off the use of the API). The new Hangouts API does not provide access to Chat ( though some additional methods may come in handy: https://developers.google.com/+/hangouts/ for the dev API).
You are therefore left to your own devices. Fear not, however, writing a chat system is pretty easy. I am about to release a (mostly free) service to do so, so if you want to not have to write the code for it, I can keep you posted.
If, however, you'd prefer to build the code for it, you will most likely want to look into either socket.io or postal.js. both provide the same thing: a pub/sub model. From there, you will need to implement a choice of either:
Long polling: supported by all browsers but a bit clunky
Websockets: not supported by IE8 and below
This will serve as your data transfer.
Two other possible options are paid services: you can retrofit RabbitMQ to do what you want to do (this, however, will seem clunky). You can also retrofit the Meteord daemon, which does what you want natively, but has an outdated JS library.
The keyword of all this is pub/sub, though.
Preamble
So, this question has already been answered, but as it was my first question for this project, I'm going to continue to reference it in other questions I ask for this project.
For anyone who came from another question, here is the basic idea: Create a web app that can make it much easier to create other web applications or websites. To do this, you would basically create a modular site with "widgets" and then combine them into the final display pages. Each widget would likely have its own set of functions combined in a Class if you use Prototype or .prototype.fn otherwise.
Currently
I am working on getting the basics down: editing CSS, creating user JavaScript functions and dynamically finding their names/inputs, and other critical technical aspects of the project. Soon I will create a rough timeline of the features I wish to create. Soon after I do this, I intent to create a Blog of sorts to keep everyone informed of the project's status.
Original Question
Hello all, I am currently trying to formalize an idea I have for a personal project (which may turn into a professional one later on). The concept is a reflective web application. In other words, a web application that can build other web applications and is actively used to build and improve itself. Think of it as sort of a webapp IDE for creating webapps.
So before I start explaining it further, my question to all of you is this: What do you think would be some of the hardest challenges along the way and where would be the best place to start?
Now let me try to explain some of the aspects of this concept briefly here. I want this application to be as close to a WYSIWYG as possible, in that you have a display area which shows all or part of the website as it would appear. You should be free to browse it to get to the areas you want to work on and use a JavaScript debugger/console to ask "what would happen if...?" questions.
I intend for the webapps to be built up via components. In other words, the result would be a very modular webapp so that you can tweak things on a small or large scale with a fair amount of ease (generally it should be better than hand coding everything in <insert editor of choice>).
Once the website/webapp is done, this webapp should be able to produce all the code necessary to install and run the created website/webapp (so CSS, JavaScript, PHP, and PHP installer for the database).
Here are the few major challenges I've come up with so far:
Changing CSS on the fly
Implementing reflection in JavaScript
Accurate and brief DOM tree viewer
Allowing users to choose JavaScript libraries (i.e. Prototype, jQuery, Dojo, extJS, etc.)
Any other comments and suggestions are also welcome.
Edit 1: I really like the idea of AppJet and I will check it out in detail when I get the time this weekend. However, my only concern is that this is supposed to create code that can go onto others webservers, so while AppJet might be a great way for me to develop this app more rapidly, I still think I will have to generate PHP code for my users to put on their servers.
Also, when I feel this is ready for beta testers, I will certainly release it for free for everyone on this site. But I was thinking that out of beta I should follow a scheme similar to that of git: Free for open source apps, costs money for private/proprietary apps.
Conceptually, you would be building widgets, a widget factory, and a factory making factory.
So, you would have to find all the different types of interactions that could be possible in making a widget, between widgets, within a factory, and between multiple widget making factories to get an idea.
Something to keep on top of how far would be too far to abstract?
**I think you would need to be able to abstract a few layers completely for the application space itself. Then you'd have to build some management tool for it all. **
- Presentation, Workflow and the Data tier.
Presentation: You are either receiving feedback, or putting in input. Usually as a result of clicking, or entering something. A simple example is making dynamic web forms in a database. What would you have to store in a database about where it comes/goes from? This would probably make up the presentation layer. This would probably be the best exercise to start with to get a feel for what you may need to go with.
Workflow: it would be wise to build a simple workflow engine. I built one modeled on Windows Workflow that I had up and running in 2 days. It could set the initial event that should be run, etc. From a designer perspective, I would imagine a visio type program to link these events. The events in the workflow would then drive the presentation tier.
Data: You would have to store the data about the application as much as the data in the application. So, form, event, data structures could possibly be done by storing xml docs depending on whether you need to work with any of the data in the forms or not. The data of the application could also be stored in empty xml templates that you fill in, or in actual tables. At that point you'd have to create a table creation routine that would maintain a table for an app to the spec. Google has something like this with their google DB online.
Hope that helps. Share what you end up coming up with.
Why use PHP?
Appjet does something really similar using 100% Javascript on the client and server side with rhino.
This makes it easier for programmers to use your service, and easier for you to deploy. In fact even their data storage technique uses Javascript (simple native objects), which is a really powerful idea.
With the increased power of JavaScript frameworks like YUI, JQuery, and Prototype, and debugging tools like Firebug, doing an application entirely in browser-side JavaScript looks like a great way to make simple applications like puzzle games and specialized calculators.
Is there any downside to this other than exposing your source code? How should you handle data storage for this kind of program?
Edit: yes, Gears and cookies can be used for local storage, but you can't easily get access to files and other objects the user already has around. You also can't save data to a file for a user without having them invoke some browser feature like printing to PDF or saving page as a file.
I've written several application in JS including a spreadsheet.
Upside:
great language
short code-run-review cycle
DOM manipulation is great for UI design
clients on every computer (and phone)
Downside:
differences between browsers (especially IE)
code base scalability (with no intrinsic support for namespaces and classes)
no good debuggers (especially, again, for IE)
performance (even though great progress has been made with FireFox and Safari)
You need to write some server code as well.
Bottom line: Go for it. I did.
Another option for developing simple desktop like applications or games in JavaScript is Adobe AIR. You can build your app code in either HTML + JavaScript or using Flash/Flex or a combination of both. It has the advantage of being cross-platform (actually cross-platform, Linux, OS X, and Windows. Not just Windows and OS X).
Heck, it may be the only time in your career as a developer that you can write a web page and ONLY target ONE browser.
SproutCore is a wholly JavaScript-hosted application framework, borrowing concepts particularly from Cocoa (such as KVO) and Ruby on Rails (such as using a CLI generator for your models, views and controllers). It includes Prototype, but builds plenty of stuff such as sophisticated controls on top of that. Its Photos demo is arguably impressive (especially in Safari 3.1).
Greg already pointed you to Gears; in addition, HTML 5 will come with a standardized means of local storage. Safari 3.1 ships with an implementation where you have a per-site SQLite database with user-settable size maximums, as well as a built-in database browser with SQL querying. Unfortunately, it will be a long time until we can expect broad browser support. Until then, Gears is indeed an alternative (but not for Safari… yet!). For simpler storage, there is of course always cookies.
The downside to this would be that you are at the mercy of them having js enabled. I'm not sure that this is a big deal now. Virtually every browser supports js and has it enabled by default.
Of course the other downside would be performance. You are again at the mercy of the client handling all the intensive work. This also may not be that big of a deal, and would be dependent on the type of app you are building.
I've never used Gears, but it looks like it is worth a shot. The backup plan would be to run some server side script through ajax that dumps your data somewhere.
Not completely client side, but oh well.
Nihilogic (not my site) does a lot of stuff with Javascript. They even have several games that they've made in Javascript.
I've also seen a neat roguelike game made in Javascript. Unfortunately, I can't remember what it was called...
If you want to write a standalone JavaScript application, look at XULrunner. It's what Firefox is built on, but it is also built so that you can distribute it as an application runtime. You will write some of the interface in JavaScript and use JavaScript for your code.
Gears might provide the client-side persistent data storage you need. There isn't a terribly good way of not exposing your source code, though. You could obfuscate it but that only helps somewhat.
I've done simple apps like this for stuff like a Sudoku solver.
You might run into performance issues given that you're completely at the mercy of the client's Javascript interpreter. Gears would be a nice way of data storage, but I don't think it has penetrated the market that much. You could just use cookies if you're not fussy about that kind of thing.
I'm with ScottKoon here, Adobe AIR is great. I've really only made one really nice (imho) widget thus far, but I did so using jQuery and Prototype.js, which floored in such wonderful ways because I didn't have to learn a whole new event model. Adobe AIR is really sweet, the memory foot print isn't too bad, upgrading to a new version is built into AIR so it's almost automatic, and best of all it's cross-platform...they even have an alpha-version for Linux, but it works pretty well already on my Eee.
Standalone games in GWT:
http://gpokr.com/
http://kdice.com/
In regard to saving files from a javascript application:
I am really excited about the possibilities of client-side applications. Flash 10 introduced the ability to create files for save right in the browser. I thought it was super cool, so I built a javascript+flash component to wrap the saving feature. Right now it only works for creating text based files (vcard, ical, xml, html, css, etc.)
Downloadify Home Page
Source Code & Documentation on Github
See It In Use at Starter for jQuery
I am looking to add support for non-text files soon, but this is a start.
My RSS feeds have served me well- I found that Javascript roguelike!
It's called The Tombs of Asciiroth.
Given that you're going to be writing some server code anyway, it makes sense to keep storage on the server for a lot of domains (address books, poker scores, gui configuration, etc.,.) For anything the size of what you'll get in Webkit or Gears, you can probably also keep it on your server.
The advantage of keeping it on your server is two-fold:
You can integrate it fairly simply as a Model layer in a typical MVC framework, and,
Users get a consistent view without being tied to their browser/PC, or in a less-than-ideal environment (Internet Cafés).
The server code for handling this can also be fairly trivial, particularly if it's written with this task in mind, so it's not a huge cognitive burden.
Go with qooxdoo. They recently realsed 1.0, although most users of it say it was ripe for 1.0 at least two versions ago.
I compared qooxdoo with YUI and ext, and I think qooxdoo is the way to go for programmers - YUI isn't that polished as qooxdoo, from a programmer's point of view and ext has a not so friendly licensing model.
A few of the strong points (for me) of qooxdoo are:
extremely clean code
the nicest OO programming model I've seen among Javascript frameworks
an extremely rich UI widget library
It also features a test runner for unit tests, an API doc generator and reader, a logging facility, and several useful features for debugging, grouped under something called Inspector.
The only downside is that there aren't readymade themes (something like skins) for qooxdoo. But creating your own theme is quite easy.