I am looking to extend a web application using the new real time API in order to support collaboration (javascript). For that purpose I would also like to include a chat which will be available to users collaborating on same document. After extensive search I cannot find parts of the Drive API that can be used for this. Furthermore none of the open-source examples provided by google implement chat functionality.
Is there existing services/code that I can use to intergrade
chat into my application or I would need to implement it?
As mentioned before, the chat should be available to those collaborating on same document.
p.s I do not require any special features, just a simple chat as the one found in google docs etc.
Right now I don't know of any out-of-the-box solution to this problem. Some people have implemented chat in a realtime document by just placing the chat messages in the realtime data model.
This works fine so long as you don't also want to use undo/redo feature. If you are using undo/redo, then people would end up undoing the chat messages.
We are interested in adding some better support for this eventually, but no promises on anything in the short term.
You will need to build your own chat system, as Google just killed off XMPP support on the Chat API (which largely kills off the use of the API). The new Hangouts API does not provide access to Chat ( though some additional methods may come in handy: https://developers.google.com/+/hangouts/ for the dev API).
You are therefore left to your own devices. Fear not, however, writing a chat system is pretty easy. I am about to release a (mostly free) service to do so, so if you want to not have to write the code for it, I can keep you posted.
If, however, you'd prefer to build the code for it, you will most likely want to look into either socket.io or postal.js. both provide the same thing: a pub/sub model. From there, you will need to implement a choice of either:
Long polling: supported by all browsers but a bit clunky
Websockets: not supported by IE8 and below
This will serve as your data transfer.
Two other possible options are paid services: you can retrofit RabbitMQ to do what you want to do (this, however, will seem clunky). You can also retrofit the Meteord daemon, which does what you want natively, but has an outdated JS library.
The keyword of all this is pub/sub, though.
Related
I'm working on a project where I want 1-to-n number of people to be able to access the same instance of a web page simultaneously.
An example of this would be something like iscribble.net where a person can create a drawing board, and other users can enter this board and draw simultaneously on the same board and chat etc...
what is this called? I can't find the right search term to look up a tutorial or something
You might try "real-time collaboration," which is often achieved through WebSockets or similar technology.
Socket.IO is a popular JavaScript implementation of WebSockets.
What you're looking for is indeed websockets. However, push notifications, depending on complexity and interaction of your idea may be what you really need. There are different services that offer these things, as well as many open source solutions that come packaged with such capability.
Parse.com is great for a beginner for a small push enabled app with a limited number of requests. (I do not work for Parse).
A websocket app in my opinion is for an advanced developer even when the websockets are handled by a server side game solution with functionality ready to go. I'd check out push on parse.com first in a prototype to get your feet wet and see if it works.
I'm trying to create a website like Ustream using ruby on rails.
I want users to be able to turn on their webcams and broadcasts live. I also want them to be able to send out a link to their live broadcast. The broadcast will need to work cross-browser also.
How can I do this effectively using ruby on rails?
Please be as detailed as possible. I'm looking for most simple and efficient solution.
Thanks in advance.
Your question is pretty vague, so I apologies if this is not as specific as you may hope
Live Streaming & Rails
Rails really isn't designed for live-streaming
It's MVC structure is best used to interact with large data-sets, and is better suited to handling things like authentication, API's, data-driven applications, etc
The live-streaming functionality you seek is more in the realm of node.js & socket.io, more specifically websocket architecture, whereby two connected devices can share data across a single connection
There are a number of options available, but they are limited unless you go down the proprietary route:
TokBox
TokBox is the safest bet for Rails apps - it uses a third-party API to connect the devices, and implements the connection on the front-end with Javascript & flash. We have actually implemented this before, and it's very simple to do - it's all explained here
Tokbox is now owned by Telefonica, and I believe are investing heavily to make their technology better available & higher quality to developers. So we'll have to see how it goes
WebRTC
This is more like a driver, but is the best quality of all the options. The only issue is that implementing this technology is actually pretty hard. Here is an overview for you:
WebRTC is an open-source project enabling plugin-free, Real Time
Communications (RTC) in the browser. It includes the fundamental
building blocks for high-quality communications such as network,
audio, and video components used in voice and video chat applications.
Recommendations
Having implemented TokBox before, I'd recommend you look at that. You can see a tutorial about it here
Has anyone an idea for the following scenario?
I have a RIA-Webapplication (realized in ExtJs). What I want to implement is the possibility to use local ressources like card readers or fingerprint readers or other serial devices and filesystem access.
I thought about implementing this with a local websocket service which has to be installed by our customer before using our RIA the first time. When the webapp is loading it should scan the local machine if a websocket service is available and connect to it.
After that, local events (like new card is beeing read or recognized new finger) should be passed to the browser via websocket connection.
Any ideas how to get started with such a solution?
I have made something like that. Besides the obvious things such as read/write/poll data from the card-readers and so on, you would have to either implement everything yourself or, use a library for technology you are going to employ for your web-server. So, if you use a LAMP solution, i think there are some web-socket libraries for PHP that you can use. However, if you do everything by yourself then you have to implement everything from hand-shaking, to creating data packets. I have done everything from scratch by using .NET and it provides a number of useful libraries such as hashing. Java also would be a good option and have those kind of libraries as well. In general if you doing everything by yourself i would say the trickiest thing would be to split the data you want to transmit among various data packet. It is not that hard to do things from scratch. The RFC (https://www.rfc-editor.org/rfc/rfc6455) helped me a lot. Hopefully, this helps.
I am building a chat feature inside a website, something like stackoverflow chat but simpler, are there any best practices or standards I need to follow while creating this feature using MVC .net and javascript? are there any articles documents or books talked about this in details?
Disclaimer: I am by no means an expert, but I'll do my best. If someone thinks this would better serve as a community wiki, I'll change it to such, just let me know.
As I see it, there are two general fields of chat services:
Chat Rooms (Like IRC)
Needs some sort of moderation hierarchy
Somewhat less need for privacy in the communication
as a note: you may want to have both private and public chat rooms,
in which case, privacy may become more of an issue
Minimal need for user registration
The only reasons you may need user registration in this case is for
moderation or privacy reasons
Like is often done on IRC networks, I'd make user registration
optional
One-on-One Chat (Like XMPP)
Typically needs encryption on all sides
Privacy is key
Authentication is very important (Unless you're doing a chat-roulette
type thing)
Moderation needs are minimal
About the only moderation tools you'll need are something to prevent
spam (which can be solved by simply not revealing user lists, and
requiring users to approve of someone before that person can chat
with them), and something to allow users to block other users
Okay, so now that we've figured out what type of service you want to
implement, we have to figure out the "how".
Pre-existing Standards
There's lots of code out there, but...
Typically the standards are more complex than something you could roll
yourself
The clear exception here is IRC, which is drop-dead simple to
understand, although may actually be somewhat lacking in features
for what you want to do
May contain extra things that you don't care about (such as file-transfers
in XMPP)
That doesn't mean you actually have to support them, but it's more pages
of rfc documents that you have to look through
Rolling Your Own Protocol
Potentially insecure, as you may not know what to look out for
Potentially lower up-front costs, higher long-term costs
What I mean by this is simply that when you roll it yourself, you save the
time of having to look through a bunch of standards, but as a result, you
may end up building a less robust system than you hoped
More easily customizable for your needs (granted you don't mess things up,
see previous point)
Some Other Odds-and-Ends
You may want to make bots
Be careful about opening up an API for others to make bots though. This
could be good or bad, depending on your community. People could use it for
spam, or for great things. With great power comes great responsibility.
It's possible to combine both a chatroom system and a one-to-one chat. IRC
supports one-to-one chats, and XMPP supports chatrooms.
Node.js is something you should consider for a chat service server.
This is why:
it's absurdly fast
easy to set up
you can make use of websockets (compatible with safari, ios safari, chrome, firefox, opera) to make things work super-fluent and fast
I'd definitely take a look at XMPP. There are pleanty of XMPP servers already available Tigase (Java), Ejabberd (Erlang), etc., which will get you up and going pretty quickly.
There is an XMPP spec for multi-user chats (MUC) http://xmpp.org/extensions/xep-0045.html which both Tigase and Ejabberd have implemented.
There is a JavaScript library for XMPP called Strophe (http://strophe.im/strophejs/) which you might find interesting as well. Strophe was used to write Speeqe (https://github.com/thepug/speeqe) as well. (demo: www.speeqe.com)
Since XMPP has been around for so long (used to be known as Jabber) there are ton of great books as well including one that talks about using Strophe (http://www.amazon.com/Professional-Programming-JavaScript-jQuery-Programmer/dp/0470540710 - written by the creator of ChessPark)
I'm writing an application in which I'd like to have near real time collaborative editing features for documents (Very similar to Google Documents style editing).
I'm aware of how to keep track of cursor position, that's simple. Just poll the server ever half second or second with the current user id, filename, line number and row number which can be stored in a database, and the return value of this polling request is the position of other user's cursors.
What I don't know how to do is update the document in such a way that it won't throw your cursor off and force a full reload as that would be far to slow for my purposes.
This really only has to work in Google Chrome, preferably Firefox as well. I don't need to support any other browser.
The algorithm used behind the scenes for merging collaborative edits from multiple peers is called operational transformation. It's not trivial to implement though.
See also this question for useful links.
Real time collaborative editing requires several things to be effective. Most of the other answers here focus on only one aspect of the problem; namely distributed state (aka shared-mutable-state). Operational Transformation (OT), Conflict-Free Replicated Data Types (CRDT), Differential Synchronization, and other related technologies are all approaches to achieving near-real-time distributed state. Most focus on eventual consistency, which allow temporary divergences of each of the participants state, but guarantee that each participants state will eventually converge when editing stops. Other answers have mentioned several implementations of these technologies.
However, once you have shared mutable state, you need several other features to provide a reasonable user experience. Examples of these additional concepts include:
Identity: Who the people you are collaborating with are.
Presence: Who is currently "here" editing with you now.
Communication: Chat, audio, video, etc., that allow users to coordinate actions
Collaborative Cueing: Features that give indications as to what the other participants are doing and/or are about to do.
Shared cursors and selections are examples of Collaborative Cueing (a.k.a Collaboration Awareness). They help users understand the intentions and likely next actions of the other participants. The original poster was partly asking about the interplay between shared mutable state and collaborative cueing. This is important because the location of a cursor or selection in a document is typically described via locations within the document. The issue is that the location of a cursor (for example) is dependent on the context of the document. When I say my cursor is at index 37, that means character 37 in the document I am looking at. The document you may have right now may be different than mine, due to your edits or those of other users, and therefore index 37 in your document may not be correct.
So the mechanism you use to distribute cursor locations must be somehow integrated into or at least aware of the mechanism of the system that provides concurrency control over the shared mutable state. One of the challenges today is that while there are many OT / CRDT, bidirectional messaging, chat, and other libraries out there, they are isolated solutions that are not integrated. This makes it hard to build an end user system that provides a good user experience, and often results in technical challenges left to the developer to figure out.
Ultimately, to implement an effective real time collaborative editing system, you need to consider all of these aspects; and we haven't even discussed history, authorization, application level conflict resolution, and many other facets. You must build or find technologies that support each of these concepts in a way that make sense for your use case. Then you must integrate them.
The good news is that applications that support collaborative editing are becoming much more popular. Technologies that support building them are maturing and new ones are becoming available every month. Firebase was one of the first solutions that tried to wrap in many of these concepts into an easy to use API. A new-comer Convergence (full disclosure, I am a founder of Convergence Labs), provides an all-in-one API that supports the majority of these collaborative editing facets and can significantly reduce the time, cost, and complexity of building real time collaborative editing apps.
You don't need xmpp or wave for this necessarily. Most of the work on an opensource implementation called infinote already have been done with jinfinote ( https://github.com/sveith/jinfinote). Jinfinote was recently also ported to python ( https://github.com/phrearch/py-infinote) to handle concurrency and document state centrally. I currently use both within the hwios project ( https://github.com/phrearch/hwios), which relies on websockets and json transport. You don't want really want to use polling for these kind of applications. Also xmpp seems to complicate things unnecessarily imo.
After coming upon this question and doing a more careful search, I think the best standalone application to check out would be Etherpad, which runs as a JS browser app and using Node.js on the server side. The technology behind this is known as operational transformation.
Etherpad was originally a pretty heavyweight application that was bought by Google and incorporated into Google Wave, which failed. The code was released as open source and the technology was rewritten in Javascript for Etherpad Lite, now renamed just "Etherpad". Some of the Etherpad technology was probably also incorporated into Google Docs.
Since Etherpad, there have been various versions to this technology, notably some Javascript libraries that allow for integrating this directly into your web app:
ShareJS
ot.js
I am the maintainer of the meteor-sharejs package for adding realtime editors directly to a Meteor app, which IMHO is the best of both worlds :)
As Gintautas pointed out, this is done by Operational Transformation. As I understand it, the bulk of the research and development on this feature was done as part of the now-defunct Google Wave project, and is known as the Wave Protocol. Fortunately, Google Wave is open-sourced, so you can get some good code samples at http://code.google.com/p/wave-protocol/
The Google Docs team did a little bit of a case study around how the real time collaboration worked, but I can't find the blog entry.
There is some decent stuff on the wikipedia page, though:
http://en.wikipedia.org/wiki/Collaborative_real-time_editor
I've recently published a repository with a working example of what seems you're trying to achieve:
https://quill-sharedb-cursors.herokuapp.com
It's based off ShareDB (OT) working as the backend and Quill rich text editor on the frontend.
Basically just wires all these things with some more code to draw the cursors. The code should be fairly simple to understand and to copy over to any specific solution.
Hope it helps with the endeavor.