Develop Reference

What is the best way to send push notification in angular webapp from mongodb? [duplicate]

I am building a small chat application for friends, but unsure about how to get information in a timely manner that is not as manual or as rudimentary as forcing a page refresh.
Currently, I am implementing this using simple AJAX, but this has the disadvantage of regularly hitting the server when a short timer elapses.
In researching long/short polling, I ran across HTML5 WebSockets. This seems easy to implement, but I'm not sure if there are some hidden disadvantages. For example, I think WebSockets is only supported by certain browsers. Are there other disadvantages to WebSockets that I should be aware of?
Since it seems like both technologies do the same thing, in what sorts of scenarios would one prefer to use one over the other? More specifically, has HTML5 WebSockets made AJAX long/short polling obsolete, or are there compelling reasons to prefer AJAX over WebSockets?

WebSockets is definitely the future now.
Long polling is a dirty workaround to prevent creating connections for each request like AJAX does - but long polling was created when WebSockets didn't exist. Now due to WebSockets,
long polling is going away no more.
WebRTC allows for peer-to-peer communication.
I recommend learning WebSockets.
Comparison:
of different communication techniques on the web
AJAX - request → response. Creates a connection to the server, sends request headers with optional data, gets a response from the server, and closes the connection.
Supported in all major browsers.
Long poll - request → wait → response. Creates a connection to the server like AJAX does, but maintains a keep-alive connection open for some time (not long though). During connection, the open client can receive data from the server. The client has to reconnect periodically after the connection is closed, due to timeouts or data eof. On server side it is still treated like an HTTP request, same as AJAX, except the answer on request will happen now or some time in the future, defined by the application logic.
support chart (full) | wikipedia
WebSockets - client ↔ server. Create a TCP connection to the server, and keep it open as long as needed. The server or client can easily close the connection. The client goes through an HTTP compatible handshake process. If it succeeds, then the server and client can exchange data in both directions at any time. It is efficient if the application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server, so data is simply encrypted.
support chart (very good) | wikipedia
WebRTC - peer ↔ peer. Transport to establish communication between clients and is transport-agnostic, so it can use UDP, TCP or even more abstract layers. This is generally used for high volume data transfer, such as video/audio streaming, where reliability is secondary and a few frames or reduction in quality progression can be sacrificed in favour of response time and, at least, some data transfer. Both sides (peers) can push data to each other independently. While it can be used totally independent from any centralised servers, it still requires some way of exchanging endPoints data, where in most cases developers still use centralised servers to "link" peers. This is required only to exchange essential data for establishing a connection, after which a centralised server is not required.
support chart (medium) | wikipedia
Server-Sent Events - client ← server. Client establishes persistent and long-term connection to server. Only the server can send data to a client. If the client wants to send data to the server, it would require the use of another technology/protocol to do so. This protocol is HTTP compatible and simple to implement in most server-side platforms. This is a preferable protocol to be used instead of Long Polling. support chart (good, except IE) | wikipedia
Advantages:
The main advantage of WebSockets server-side, is that it is not an HTTP request (after handshake), but a proper message based communication protocol. This enables you to achieve huge performance and architecture advantages. For example, in node.js, you can share the same memory for different socket connections, so they can each access shared variables. Therefore, you don't need to use a database as an exchange point in the middle (like with AJAX or Long Polling with a language like PHP).
You can store data in RAM, or even republish between sockets straight away.
Security considerations
People are often concerned about the security of WebSockets. The reality is that it makes little difference or even puts WebSockets as better option. First of all, with AJAX, there is a higher chance of MITM, as each request is a new TCP connection that is traversing through internet infrastructure. With WebSockets, once it's connected it is far more challenging to intercept in between, with additionally enforced frame masking when data is streamed from client to server as well as additional compression, which requires more effort to probe data. All modern protocols support both: HTTP and HTTPS (encrypted).
P.S.
Remember that WebSockets generally have a very different approach of logic for networking, more like real-time games had all this time, and not like http.

One contending technology you've omitted is Server-Sent Events / Event Source. What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet? has a good discussion of all of these. Keep in mind that some of these are easier than others to integrate with on the server side.

For chat applications or any other application that is in constant conversation with the server, WebSockets are the best option. However, you can only use WebSockets with a server that supports them, so that may limit your ability to use them if you cannot install the required libraries. In which case, you would need to use Long Polling to obtain similar functionality.

XHR polling A Request is answered when the event occurs (could be straight away, or after a delay). Subsequent requests will need to made to receive further events.
The browser makes an asynchronous request of the server,
which may wait for data to be available before responding. The
response can contain encoded data (typically XML or JSON) or
Javascript to be executed by the client. At the end of the processing
of the response, the browser creates and sends another XHR, to await
the next event. Thus the browser always keeps a request outstanding
with the server, to be answered as each event occurs. Wikipedia
Server Sent Events Client sends request to server. Server sends new data to webpage at any time.
Traditionally, a web page has to send a request to the server to
receive new data; that is, the page requests data from the server.
With server-sent events, it's possible for a server to send new data
to a web page at any time, by pushing messages to the web page. These
incoming messages can be treated as Events + data inside the web page. Mozilla
WebSockets After the initial handshake (via HTTP protocol). Communication is done bidirectionally using the WebSocket protocol.
The handshake starts with an HTTP request/response, allowing servers
to handle HTTP connections as well as WebSocket connections on the
same port. Once the connection is established, communication switches
to a bidirectional binary protocol which does not conform to the HTTP
protocol. Wikipedia

For a push notification, is a websocket mandatory?

I have PHP on the server side, and HTML and javascript on the client side.
I am making an app where a stakeholder types a message that is broadcasted to multiple recievers of a group in real time.
I did some research on google and I understand I need to use WebSockets or Comet for real time push notifications. Is WebSocket or Comet mandatory for sending mass notifications to users?
Is my understanding correct? Any references to start with?

If the client is a browser, then the ONLY two ways a standard browser can connect to a server is via an Ajax (e.g. http) request or a webSocket connection. So, if you want a client to get notified of something from the outside world it has to use one of those two mechanisms.
HTTP requests are transitory. The client makes a request of a server, the server responds. HTTP requests are perfect for the client requesting information from the server. They are not very good at the server sending information to the client because normally the client is not connected. There are hacks and work-arounds where the client "polls" the server on some interval and maybe even the server uses longer running requests to try to simulate a "push" type system, but they are sub-optimal hacks at best.
webSockets are continuous connections. The client connects and the connection remains in place for as long as both sides want. This allows either side the ability to send a message to the other side whenever they want. That means the server can "push" data to the client whenever it wants. webSockets are efficient for push connections and are recommended (this is one of the main things they were designed for).
Comet is a library that was originally built for using HTTP to try to "hack" or "simulate" push before webSockets were invented and then before they were widely supported. I can think of no reason why one would want to use Comet instead of a webSocket unless you had such an old browser that webSocket was not supported.
So, if you are trying to do "realtime server push" to a browser, then you must have a continuously connected socket from the client which means webSocket (or something built on top of webSocket like socket.io).
For phone apps where you have access to the phone SDK, you can use the "push" system built into the OS to push some messages from server to client. This isn't quite the same as the two way webSocket channel, but since you asked about "push notifications", the OS push services available in both Android and IOS could also be an option for pushing notifications from server to client. Here's info on iOS notifications and Google Cloud Messaging
As of 2016, one can also use Server-sent events in all modern browsers except Microsoft browsers (not supported yet in Edge or IE) to push data from server to client. Here's a browser compatibility table. Server-sent events use a long lasting HTTP connection, a special MIME type and a supporting client in order to be able to send events from server to client at any time. Unlike webSockets, server-sent events are one way only (from server to client). A client would then use a traditional Ajax call in order to be able to send data to a server (whereas with a webSocket data can be sent either way over the same webSocket connection).
Here's a good description of how server-sent events work: How do server-sent events actually work?

Is your client application a SPA? (Single Page application)?
It's very important because if not, you have to consider that everytime a client change page, connection with websocket server will be lost.
In this case you have to manage a queue because if stakeholder send a multicast request when one client is disconnected, client won't receive nothing.
Polling won't solve this situation too and it's an orrible solution because mobile clients (for example) with typical internet plan, will consume megabytes for unuseful "ping" traffic.
A real example of polling is a child in a car asking his dad every minute if they are arrived to a destination!
So, Is there a solution without using spa?
Yes, using a "shared storage" between stakeholder and clients, and using websocket only for "wake up" online clients saying: Hey there is something new, go to check!
Everytime a client open a page it will receive from backend also not-read notifications, taken from the storage.
When a stakeholder want to notify something, it will just store the notification message in the shared storage and send a "pulse" to notification server.
Notification server will forward the "pulse" to online clients (just in case someone is stuck reading a page).
If a "pulse" is lost because a client is changing page there is no problem because the client will bring notifications from the storage.
Every page will contain this logic:
Retrive number or unread notifications (server side)
Connect to the notification server after 5 seconds (javascript side).
Hope it helps.

I would suggest that using webSockets is a more efficient way compared to other options, why is this? Well when a client receives a notification that there's a change in the server there is no need to create an AJAX call to the server to get that change, it can be sent to the client with the same webSocket connection more easily than AJAX. This means efficient code and a faster running App!

In what situations would AJAX long/short polling be preferred over HTML5 WebSockets?

I am building a small chat application for friends, but unsure about how to get information in a timely manner that is not as manual or as rudimentary as forcing a page refresh.
Currently, I am implementing this using simple AJAX, but this has the disadvantage of regularly hitting the server when a short timer elapses.
In researching long/short polling, I ran across HTML5 WebSockets. This seems easy to implement, but I'm not sure if there are some hidden disadvantages. For example, I think WebSockets is only supported by certain browsers. Are there other disadvantages to WebSockets that I should be aware of?
Since it seems like both technologies do the same thing, in what sorts of scenarios would one prefer to use one over the other? More specifically, has HTML5 WebSockets made AJAX long/short polling obsolete, or are there compelling reasons to prefer AJAX over WebSockets?

WebSockets is definitely the future now.
Long polling is a dirty workaround to prevent creating connections for each request like AJAX does - but long polling was created when WebSockets didn't exist. Now due to WebSockets,
long polling is going away no more.
WebRTC allows for peer-to-peer communication.
I recommend learning WebSockets.
Comparison:
of different communication techniques on the web
AJAX - request → response. Creates a connection to the server, sends request headers with optional data, gets a response from the server, and closes the connection.
Supported in all major browsers.
Long poll - request → wait → response. Creates a connection to the server like AJAX does, but maintains a keep-alive connection open for some time (not long though). During connection, the open client can receive data from the server. The client has to reconnect periodically after the connection is closed, due to timeouts or data eof. On server side it is still treated like an HTTP request, same as AJAX, except the answer on request will happen now or some time in the future, defined by the application logic.
support chart (full) | wikipedia
WebSockets - client ↔ server. Create a TCP connection to the server, and keep it open as long as needed. The server or client can easily close the connection. The client goes through an HTTP compatible handshake process. If it succeeds, then the server and client can exchange data in both directions at any time. It is efficient if the application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server, so data is simply encrypted.
support chart (very good) | wikipedia
WebRTC - peer ↔ peer. Transport to establish communication between clients and is transport-agnostic, so it can use UDP, TCP or even more abstract layers. This is generally used for high volume data transfer, such as video/audio streaming, where reliability is secondary and a few frames or reduction in quality progression can be sacrificed in favour of response time and, at least, some data transfer. Both sides (peers) can push data to each other independently. While it can be used totally independent from any centralised servers, it still requires some way of exchanging endPoints data, where in most cases developers still use centralised servers to "link" peers. This is required only to exchange essential data for establishing a connection, after which a centralised server is not required.
support chart (medium) | wikipedia
Server-Sent Events - client ← server. Client establishes persistent and long-term connection to server. Only the server can send data to a client. If the client wants to send data to the server, it would require the use of another technology/protocol to do so. This protocol is HTTP compatible and simple to implement in most server-side platforms. This is a preferable protocol to be used instead of Long Polling. support chart (good, except IE) | wikipedia
Advantages:
The main advantage of WebSockets server-side, is that it is not an HTTP request (after handshake), but a proper message based communication protocol. This enables you to achieve huge performance and architecture advantages. For example, in node.js, you can share the same memory for different socket connections, so they can each access shared variables. Therefore, you don't need to use a database as an exchange point in the middle (like with AJAX or Long Polling with a language like PHP).
You can store data in RAM, or even republish between sockets straight away.
Security considerations
People are often concerned about the security of WebSockets. The reality is that it makes little difference or even puts WebSockets as better option. First of all, with AJAX, there is a higher chance of MITM, as each request is a new TCP connection that is traversing through internet infrastructure. With WebSockets, once it's connected it is far more challenging to intercept in between, with additionally enforced frame masking when data is streamed from client to server as well as additional compression, which requires more effort to probe data. All modern protocols support both: HTTP and HTTPS (encrypted).
P.S.
Remember that WebSockets generally have a very different approach of logic for networking, more like real-time games had all this time, and not like http.

One contending technology you've omitted is Server-Sent Events / Event Source. What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet? has a good discussion of all of these. Keep in mind that some of these are easier than others to integrate with on the server side.

For chat applications or any other application that is in constant conversation with the server, WebSockets are the best option. However, you can only use WebSockets with a server that supports them, so that may limit your ability to use them if you cannot install the required libraries. In which case, you would need to use Long Polling to obtain similar functionality.

XHR polling A Request is answered when the event occurs (could be straight away, or after a delay). Subsequent requests will need to made to receive further events.
The browser makes an asynchronous request of the server,
which may wait for data to be available before responding. The
response can contain encoded data (typically XML or JSON) or
Javascript to be executed by the client. At the end of the processing
of the response, the browser creates and sends another XHR, to await
the next event. Thus the browser always keeps a request outstanding
with the server, to be answered as each event occurs. Wikipedia
Server Sent Events Client sends request to server. Server sends new data to webpage at any time.
Traditionally, a web page has to send a request to the server to
receive new data; that is, the page requests data from the server.
With server-sent events, it's possible for a server to send new data
to a web page at any time, by pushing messages to the web page. These
incoming messages can be treated as Events + data inside the web page. Mozilla
WebSockets After the initial handshake (via HTTP protocol). Communication is done bidirectionally using the WebSocket protocol.
The handshake starts with an HTTP request/response, allowing servers
to handle HTTP connections as well as WebSocket connections on the
same port. Once the connection is established, communication switches
to a bidirectional binary protocol which does not conform to the HTTP
protocol. Wikipedia

Is there any good trick for server to handle more requests if I don't have to sent any data back?

I want to handle a lot of (> 100k/sec) POST requests from javascript clients with some kind of service server. Not many of this data will be stored, but I have to process all of them so I cannot spend my whole server power for serving requests only. All the processing need to be done in the same server instance, otherwise I'll need to use database for synchronization between servers which will be slower by orders of magnitude.
However I don't need to send any data back to the clients, and they don't even expect them.
So far my plan was to create few proxy servers instances which will be able to buffer the request and send them to main server in bigger packs.
For example let's say that I need to handle 200k requests / sec and each server can handle 40k. I can split load between 5 of them. Then each one will be buffering requests and sending them back to main server in packs of 100. This will result in 2k requests / sec on the main server (however, each message will be 100 times bigger - which probably means around 100-200kB). I could even send them back to the server using UDP to decrease amount of needed resources (then I need only one socket on main server, right?).
I'm just thinking if there is no other way to speed up the things. Especially, when as I said I don't need to send anything back. I have full control over javascript clients also, but unlucky javascript is unable to send data using UDP which probably would be solution for me (I don't even care if 0.1% of data will be lost).
Any ideas?
Edit in response to answers given me so far.
The problem isn't with server being to slow at processing events from the queue or with putting events in the queue itself. In fact I plan to use disruptor pattern (http://code.google.com/p/disruptor/) which was proven to process up to 6 million requests per second.
The only problem which I potentially can have is need to have 100, 200 or 300k sockets open at the same time, which cannot be handled by any of the mainstream servers. I know some custom solutions are possible (http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-3) but I'm wondering if there is no way to even better utilization of fact that I don't have to replay to clients.
(For example some way to embed part of the data in initial TCP packet and handle TCP packets as they would be UDP. Or some other kind of magic ;))

Make a unique and fast (probably in C) function that get's all requests, from a very fast server (like nginx). The only job of this function is to store the requests in a very fast queue (like redis if you got enought ram).
In another process (or server), depop the queue and do the real work, processing request one by one.

If you have control of the clients, as you say, then your proxy server doesn't even need to be an HTTP server, because you can assume that all of the requests are valid.
You could implement it as a non-HTTP server that simply sends back a 200, reads the client request until it disconnects, and then queues the requests for processing.

I think what you're describing is an implementation of a Message Queue. You also will need something to hand off these requests to whatever queue you use (RabbitMQ is quite good, there are many alternatives).
You'll also need something else running which can do whatever processing you actually want on the requests. You haven't made that very clear, so I'm not too sure exactly what would be right for you. Essentially the idea will be that incoming requests are dumped as quickly as simply as possible into the queue by your web server, and then the web server is free to go back to serving more requests. When the system has some resources, it uses them to process the queue, but when it's busy the queue just keeps growing.
Not sure what platform you're on, but might want to look at something like Lighttpd for serving the POSTs. You might (if same-domain restrictions don't shoot you down) get away with having Lighttpd running on a subdomain of your application (so post.myapp.com). Failing that you could put a proper load balancer in front of your webservers altogether (so all requests go to www.myapp.com and the load balancer decides whether to forward them to the web server or the queue processor).
Hope that helps

Consider using MongoDB for persisting your requests, it's fire and forget mechanism can help your servers to response faster.

How keep-alive of HTTP can/is play role in AJAX application

"keep-alive" its there in HTTP. Some says it good should be used, but i am unable to get to any conclusion.
So please provide your input/answer/views so i can get some ground for this,
What it does?
Scenario where it should and should not be done?
How it can make AJAX application better ?
Risks DO's and DONT's if any?
Thank you all for inputs.

First off if your connection to the server is using HTTP/1.1 then you are most likely already using "keep-alive".
What is it? Logically HTTP is a connectionless protocol. That is each request/response to the server creates a new connection, does its business and drops the connection. However in HTTP/1.1 the default behaviour is to keep the connection open for use by subsequent requests to the server. The "keep-alive" header was added to HTTP/1.0 to allow this behaviour to be opted into, in HTTP/1.1 the server needs to opt-out by closing the connection itself and/or sending a "connection-close" header in response.
Why is it beneficial? Creating a connection especially one that needs to be authenticated can take some time. By re-using an existing connection the setup and authentication effort is much reduced.
How can it make your AJAX app better? You are probably already benefitting from it.
What are the risks? When making a connection through a shared appliance which may make a connection to the server in the clients behalf it is possible for other clients to re-use the connection, however that also makes it possible for other clients to use a connection that the server has authenticated for a different user.

It keeps the TCP socket to the client open, so you have not reestablish connection to send another HTTP request;
Keep-alive improves http performance when there are many requests in a row. It should not been used however if the requests are rare (server normally closes connection if there are no more requests coming from a client during some period of time).
Well, if your AJAX application sends a lot of requests to the server keep-alives improve its performance.
There is a risk of sockets depletion on server side, so server has rights to interrupt even keep-alive connections.

Really it boils down to questions of performance and resource.
Using a high(er) keep alive reduces latency on requests. This is particularly an issue if you are running over SSL where there additional handshakes to establish a connection.
OTOH, this will mean that there additional server processes.threads sitting idle, waiting for a subsequent request or the keep-alive to expire. This can hog memory and therefore slow down your server.
So you really need to play around and see what's acceptable in terms in browser performance vs server load.
Authentication (basic/digest/session based) is not relevant - it is the request which is authenticated - not the socket connection.
Note that the last time I did a fresh Apache install it came with a default setting of 5 seconds for the keep alive. This is ridiculously long for a non-ajax site.
C.