I work on a web application which uses cloud storage as the primary save mechanism for users. We don't have our own cloud storage service, instead we leverage the REST APIs for Google Drive, Dropbox, etc.
Recently we have noticed an increasingly number of users in China, unfortunately none of the cloud services we currently integrate with are available in China. The alternative forms of saving in the app have limitations which cannot be resolved, and developing our own cloud storage system that functions in China would be a very large engineering undertaking.
As such I have been trying to find a service which we could add that works in China, but have had very little success. Nearly all of the sites are in Chinese, and even with Google Translate I haven't been able to find any company that claims to offer these services. Most have some form of Cloud storage for users, but I haven't been able to find a REST API that would allow us to integrate with them. Any leads someone could provide would be much appreciated.
Requirements:
HTTP API that can be used from the browser similar to Dropbox HTTP documentation
Can be accessed in the UK ( for development ) and China
Preferably some documentation in English...
Ideally free and commonly used in China
I have implemented client-side code to work with Chinese storage cloud services (REST API). However this already dates back quite some time. Basically, there is often no English documentation and sadly you have to rely on translation. Also I suggest using a VPN (if that helps) because accessing these services outside from China is often terrible slow. This is also valid for most of the developer docs.
What direction I can point to is Baidu PCS (sadly all links are 404 so take a look here) which still exist. But as the biggest player this is the first thing I would try. There are probably other providers but the question is how long they will exist because others I have implemented are already gone (e.g. Kanbox).
This is just my experience and it might not fully apply today but maybe it helps.
Update 1: What I also found in a quick search is Weiyun which has a reverse engineered API but yeah that's far from reliable and official supported, but worth a look.
Update 2: Added alternative link for Baidu PCS documentation.
Related
I'm building a website where users can create new pages with dynamic URLs. I would like to add traffic analytics for each of these pages individually and allow users to view a dashboard of the data collected for the page they have created (just view, they shouldn't be able to edit the dashboard they are given). This should also be automated so that when a new user registers and creates their own page, this is tracked automatically.
If possible, this should work in Vue.js, but anything that works with vanilla JS should be fine too.
Is this possible with a tool like Google Analytics or a similar alternative? Is this expensive?
Most of the Web applications now a days are single page apps or Client side web applications. This means most of the analytics platforms like GA etc will work with your web application.
However, You will need to implement the analytics in a specific way for you to capture the required metrics. I think using Google Tag manager and creating a DataLayer will make it easier in the long run. Once the datalayer is available you can use this data with any analytics tool.
In order to track a screenview change, you can use any custom trigger available with your application that notifies of the screen view change or you can use the History change as mentioned here: Google Tag Manager and Single Page apps.
You can also use the App Rote changes as mentioned here on simple routing to populate the datalayer and use the values from the datalayer in any analytics including the GA.
Other option is using vue-analytics . There is Easy to follow documentation from the author and the branch is here
This way any new pages added etc will be tracked automatically.
Hope this helps.
Legal context
Well... What is the targeted audience? If your main targets are privacy advocates or European people covered by the General Data Protection Regulation (GDPR), it would be a terrible idea to select Google Analytics as the solution. In France, for instance, we have a National Commission on Informatics and Liberty ("CNIL" in French). It is:
an independent French administrative regulatory body whose mission is
to ensure that data privacy law is applied to the collection, storage,
and use of personal data. (Source: Wikipedia)
The CNIL tends to strongly discourage the use of Google Analytics (see Q&A on the CNIL's formal notices concerning the use of Google Analytics). Therefore, be very careful about that, especially if you are targeting an international audience.
Recommended tool
When it comes to web analytics, you have a wide range of technical solutions. For obvious reasons, I will not put the complete list here, but I can give you a tool that I know and use: Matomo.
This one is recommended by the CNIL, it is open source (see GitHub) and GDPR-compliant. You can self-host Matomo (On-Premise), which is completely free. You only pay if you use Matomo Cloud.
Matomo has a technical website dedicated to developers and exposes an API which should allow you to do what you want. Doing some research, I even found a thread on the official forum where the OP seems to have a similar problem to yours. Since it is a requirement in your question, you can also find information about user permissions.
On this page, you can read that Matomo developers use Vue for the front-end code. There is also a plugin called vue-matomo, which may be useful in your project.
Finally, to track single-page applications (SPAs), read on: Single-Page Application/Progressive Web App Tracking: Integrate - Matomo Analytics (formerly Piwik Analytics) - Developer Docs - v5
As I read from this source-
If you have a website that has dynamic content or multiple pages
and steps under one URL (or an all-flash website), you have to do a
little more work to get Google Analytics to effectively track your
pages.
Here are some more references you can check upon-
Here you can read, how Google Analytics works.
Here you can find its pricing plans which are free for small businesses.
Here you can read how Google Analytics performs for dynamic links.
Here is another answer given by #Ruslan Konygin on a similar problem.
Here is a complete guide on Tracking Dynamic Sites and Pages with Google Analytics.
I have recently came across this term "Serverless Stack" and while I did a little research about it, I found it helps us in creating serverless web applications using framework like ReactJS and DynamoDB for backend which can be hosted over cloud using AWS.
However when it comes to the scalability of applications, there is very little to no information available across various blogs.
Had anyone tried this stack in your applications?.
I would want to hear,
What are the other tech stack (languages, frameworks) which can be used as part of this new Serverless Stack
Does it scale well? (Particularly when a website has more viewers)
Can someone shed some light?
In my experience I can tell you that what you are talking about is also known as no-backend applications (resource).
The principle of this approach is that you can abstract a number of features that traditionally are implemented into the server tier, and move them into decoupled services exposed as SaaS.
As you mentioned, a famous example are the smartphone hybrid applications that rely only onto Firebase that provides them authentication, authorization and other few backend features.
If you need another kind of feature, like emails, you can do it within your frontend code by using a proper email service provider.
In terms of scalability what you have to do is simply to scale the services that you are using, for example using a bigger Firebase plan.
In terms of security you have to understand that in a web application your code is always visible so all your business logic could be red, analized and easly hacked. This is why the no-backend approach fits better the mobile application needs, since they are wrapped into proper containers designed in order to grant a better level of obscuration about what your application is doing.
Hope this could help you
Some of the existing frameworks:
apex: lets you build, deploy, and manage AWS Lambda functions with ease
chalice: Python Serverless Microframework for AWS
claudia: makes it easy to deploy Node.js projects to AWS Lambda and API Gateway
serverless.com: helps building apps on AWS Lambda
Search "serverless" on github to find more.
AWS-based services seem to scale well but have a look at the competition too:
https://azure.microsoft.com/en-us/services/functions/
https://cloud.google.com/functions/
In my current job I have to import an analytics framework into our clients pages (we're a 3rd party also, our services are provided by including our tag in the client's pages) for counting basic metrics such as: Visitors,Visits,Views,Avg. Visit duration,Bounce Rate,Pages/Visit .
I've looked at several opensource solution such as piwik etc.. but as we handle allot of traffic this will come at a pretty hefty price, so obviously GA seems like the most reasonable solution for us. the problem is that the bulk of my clients already have GA on their pages and everybody has a different version of it (analytics.js,ga.js,dc.js etc..) where each version has a different method of working with multiple trackers on page and some don't support that all together. so using a pixel reporting technique seems like the safest method to use here but .. I can't find any documentation on google or anywhere else for that matter explaining what functionality I will loose when using this approach.
The API documentation does a decent job of showing me how to do it myself and there's also nojsstats (which i prefer not to use) but has anyone had any experience with this kind of a scenario and can eloborate on what functionality will be lost as a result of this approach? or has found a different solution for this kind of problem?
Thanks in advance,
Amnon
You can look into the parameter reference for the measurement protocol - that's what you can get, provided you pass all the parameters (so everything that's not in there you lose, most notably all Geo information which will be set to the location of your server).
The biggest challenge would be that you'd have to maintain a session id in your respective clients applications, else you'll only register single page visits with 100% bounce rate (since Google will not know which visits belong to the same user). That seems a lot more cumbersome than additional javascript trackers.
However since they already have Google Analytics you do not need to implement any tracker at all, you can query the existing accounts via a serverside language of your choice (or via Google Apps Script). Which is actually what the documentation you've linked to describes (Core reporting API is not about collecting data, but about querying existing data).
I am looking to extend a web application using the new real time API in order to support collaboration (javascript). For that purpose I would also like to include a chat which will be available to users collaborating on same document. After extensive search I cannot find parts of the Drive API that can be used for this. Furthermore none of the open-source examples provided by google implement chat functionality.
Is there existing services/code that I can use to intergrade
chat into my application or I would need to implement it?
As mentioned before, the chat should be available to those collaborating on same document.
p.s I do not require any special features, just a simple chat as the one found in google docs etc.
Right now I don't know of any out-of-the-box solution to this problem. Some people have implemented chat in a realtime document by just placing the chat messages in the realtime data model.
This works fine so long as you don't also want to use undo/redo feature. If you are using undo/redo, then people would end up undoing the chat messages.
We are interested in adding some better support for this eventually, but no promises on anything in the short term.
You will need to build your own chat system, as Google just killed off XMPP support on the Chat API (which largely kills off the use of the API). The new Hangouts API does not provide access to Chat ( though some additional methods may come in handy: https://developers.google.com/+/hangouts/ for the dev API).
You are therefore left to your own devices. Fear not, however, writing a chat system is pretty easy. I am about to release a (mostly free) service to do so, so if you want to not have to write the code for it, I can keep you posted.
If, however, you'd prefer to build the code for it, you will most likely want to look into either socket.io or postal.js. both provide the same thing: a pub/sub model. From there, you will need to implement a choice of either:
Long polling: supported by all browsers but a bit clunky
Websockets: not supported by IE8 and below
This will serve as your data transfer.
Two other possible options are paid services: you can retrofit RabbitMQ to do what you want to do (this, however, will seem clunky). You can also retrofit the Meteord daemon, which does what you want natively, but has an outdated JS library.
The keyword of all this is pub/sub, though.
We want out website to support international geocoding. We are wary of using APIs that throttle and/or cap requests, which could leave us with our pants down as the service gains volume. How do modern websites such as Facebook implement geocoding? Are there tools to implement accurate in-house scalable geocoding solutions for the entire world?
I don't know about APIs for international geoconding. However, maybe Cassandra http://cassandra.apache.org/ can add something of value for you. Although it is a storage-centric solution, it surely can scale.
Twitter is using it for storing geolocation and places of interest data http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html as well as https://simplegeo.com/
I know that this is an older question. You could always use a service, then cache the relevant result in a local database... simply look local first before falling back to the service. After a while, most of your requests will be hits against your local database, with very few fallbacks.
For a U.S. based project, we started with a seeded U.S. zipcode database, and using Bing as a fallback, never once hit the 5,000/day (iirc) limit. YMMV with this solution, but it isn't an unreasonable one.