Develop Reference

Is there a way to make Using server-sent events persistent?

I need to run a script that updates the user's browser once every second. My need is a one way communication from the server to the client.
To do that, I implemented server-sent events polling from my server to the user's client.
The problem is that the user is allow to open multiple connection to the server "by opening multiple browser tabs" witch is a problem as it adds an overhead on the server.
I would like to limit the connection that is coming from a single user to 1. Meaning a single connection to the server across all their tabs. One connection between a client and the server even if the user have 10 browser tabs open.
This is my server-sent event implementation
var evtSource = new EventSource('getMyMessages.php');
evtSource.addEventListener("getMessagingQueue", function(e) {
console.log(e);
var data = JSON.parse(e.data);
processServerData(data);
}, false);
evtSource.onerror = function(e) {
evtSource.close();
};
Is there a way to make this method of communication persistent?

(This is an indirect answer: how to prevent the overhead of multiple tabs being open. You'll need some communication method between tabs if all of them getting fed the data is desirable, and the shared worker idea looks good for that.)
At the application level: Use a PHP session.
When you use a PHP session the PHP script locks it. This means no other scripts can access the session. In your case that is what you want. (In another recent question, Server-Sent Events Polling causing long delays, that was the cause of a problem.)
This is even better if you require they login to the session to be able to use it. (Otherwise someone who disables cookies can still access multiple tabs.)
The problem is that their other tabs just sit there, and (I believe) the PHP process is using up resources on the server, while it waits for the session to unlock.
You can get around that resource-usage problem by using a session variable to record that they have a current connection, and making sure you release the session lock (as described in the other question). You can then give the user a friendly error message when they try to connect more than once.
At the proxy/firewall level: Set a rule to not allow the same something to have more than one connection at a time to the same URL (your SSE script in this case). Redirect to them an error page explaining why they are not allowed to.
The obvious, but bad, choice for the something is their IP address. That sucks, because multiple users can be coming from the same IP address. You could use their user agent, but just as unreliable. Giving them a cookie, and looking for that is better. (You could even look for the PHP session cookie.)

For a push notification, is a websocket mandatory?

I have PHP on the server side, and HTML and javascript on the client side.
I am making an app where a stakeholder types a message that is broadcasted to multiple recievers of a group in real time.
I did some research on google and I understand I need to use WebSockets or Comet for real time push notifications. Is WebSocket or Comet mandatory for sending mass notifications to users?
Is my understanding correct? Any references to start with?

If the client is a browser, then the ONLY two ways a standard browser can connect to a server is via an Ajax (e.g. http) request or a webSocket connection. So, if you want a client to get notified of something from the outside world it has to use one of those two mechanisms.
HTTP requests are transitory. The client makes a request of a server, the server responds. HTTP requests are perfect for the client requesting information from the server. They are not very good at the server sending information to the client because normally the client is not connected. There are hacks and work-arounds where the client "polls" the server on some interval and maybe even the server uses longer running requests to try to simulate a "push" type system, but they are sub-optimal hacks at best.
webSockets are continuous connections. The client connects and the connection remains in place for as long as both sides want. This allows either side the ability to send a message to the other side whenever they want. That means the server can "push" data to the client whenever it wants. webSockets are efficient for push connections and are recommended (this is one of the main things they were designed for).
Comet is a library that was originally built for using HTTP to try to "hack" or "simulate" push before webSockets were invented and then before they were widely supported. I can think of no reason why one would want to use Comet instead of a webSocket unless you had such an old browser that webSocket was not supported.
So, if you are trying to do "realtime server push" to a browser, then you must have a continuously connected socket from the client which means webSocket (or something built on top of webSocket like socket.io).
For phone apps where you have access to the phone SDK, you can use the "push" system built into the OS to push some messages from server to client. This isn't quite the same as the two way webSocket channel, but since you asked about "push notifications", the OS push services available in both Android and IOS could also be an option for pushing notifications from server to client. Here's info on iOS notifications and Google Cloud Messaging
As of 2016, one can also use Server-sent events in all modern browsers except Microsoft browsers (not supported yet in Edge or IE) to push data from server to client. Here's a browser compatibility table. Server-sent events use a long lasting HTTP connection, a special MIME type and a supporting client in order to be able to send events from server to client at any time. Unlike webSockets, server-sent events are one way only (from server to client). A client would then use a traditional Ajax call in order to be able to send data to a server (whereas with a webSocket data can be sent either way over the same webSocket connection).
Here's a good description of how server-sent events work: How do server-sent events actually work?

Is your client application a SPA? (Single Page application)?
It's very important because if not, you have to consider that everytime a client change page, connection with websocket server will be lost.
In this case you have to manage a queue because if stakeholder send a multicast request when one client is disconnected, client won't receive nothing.
Polling won't solve this situation too and it's an orrible solution because mobile clients (for example) with typical internet plan, will consume megabytes for unuseful "ping" traffic.
A real example of polling is a child in a car asking his dad every minute if they are arrived to a destination!
So, Is there a solution without using spa?
Yes, using a "shared storage" between stakeholder and clients, and using websocket only for "wake up" online clients saying: Hey there is something new, go to check!
Everytime a client open a page it will receive from backend also not-read notifications, taken from the storage.
When a stakeholder want to notify something, it will just store the notification message in the shared storage and send a "pulse" to notification server.
Notification server will forward the "pulse" to online clients (just in case someone is stuck reading a page).
If a "pulse" is lost because a client is changing page there is no problem because the client will bring notifications from the storage.
Every page will contain this logic:
Retrive number or unread notifications (server side)
Connect to the notification server after 5 seconds (javascript side).
Hope it helps.

I would suggest that using webSockets is a more efficient way compared to other options, why is this? Well when a client receives a notification that there's a change in the server there is no need to create an AJAX call to the server to get that change, it can be sent to the client with the same webSocket connection more easily than AJAX. This means efficient code and a faster running App!

Prevent recursive calls of XmlHttpRequest to server

I've been googling for hours for this issue, but did not find any solution.
I am currently working on this app, built on Meteor.
Now the scenario is, after the website is opened and all the assets have been loaded in browser, the browser constantly makes recursive xhr calls to server. These calls are made at the regular interval of 25 seconds.
This can be seen in the Network tab of browser console. See the Pending request of the last row in image.
I can't figure out from where it originates, and why it is invoked automatically even when the user is idle.
Now the question is, How can I disable these automatic requests? I want to invoke the requests manually, i.e. when the menu item is selected, etc.
Any help will be appriciated.
[UPDATE]
In response to the Jan Dvorak's comment:
When I type "e" in the search box, the the list of events which has name starting with letter "e" will be displayed.
The request goes with all valid parameters and the Payload like this:
["{\"msg\":\"sub\",\"id\":\"8ef5e419-c422-429a-907e-38b6e669a493\",\"name\":\"event_Coll_Search_by_PromoterName\",\"params\":[\"e\"]}"]
And this is the response, which is valid.
a["{\"msg\":\"data\",\"subs\":[\"8ef5e419-c422-429a-907e-38b6e669a493\"]}"]
The code for this action is posted here
But in the case of automatic recursive requests, the request goes without the payload and the response is just a letter "h", which is strange. Isn't it? How can I get rid of this.?

Meteor has a feature called
Live page updates.
Just write your templates. They automatically update when data in the database changes. No more boilerplate redraw code to write. Supports any templating language.
To support this feature, Meteor needs to do some server-client communication behind the scenes.
Traditionally, HTTP was created to fetch dead data. The client tells the server it needs something, and it gets something. There is no way for the server to tell the client it needs something. Later, it became needed to push some data to the client. Several alternatives came to existence:
polling:
The client makes periodic requests to the server. The server responds with new data or says "no data" immediately. It's easy to implement and doesn't use much resources. However, it's not exactly live. It can be used for a news ticker but it's not exactly good for a chat application.
If you increase the polling frequency, you improve the update rate, but the resource usage grows with the polling frequency, not with the data transfer rate. HTTP requests are not exactly cheap. One request per second from multiple clients at the same time could really hurt the server.
hanging requests:
The client makes a request to the server. If the server has data, it sends them. If the server doesn't have data, it doesn't respond until it does. The changes are picked up immediately, no data is transferred when it doesn't need to be. It does have a few drawbacks, though:
If a web proxy sees that the server is silent, it eventually cuts off the connection. This means that even if there is no data to send, the server needs to send a keep-alive response anyways to make the proxies (and the web browser) happy.
Hanging requests don't use up (much) bandwidth, but they do take up memory. Nowadays' servers can handle multiple concurrent TCP connections, so it's less of an issue than it was before. What does need to be considered is the amount of memory associated with the threads holding on to these requests - especially when the connections are tied to specific threads serving them.
Browsers have hard limits on the number of concurrent requests per domain and in total. Again, this is less of a concern now than it was before. Thus, it seems like a good idea to have one hanging request per session only.
Managing hanging requests feels kinda manual as you have to make a new request after each response. A TCP handshake takes some time as well, but we can live with a 300ms (at worst) refractory period.
Chunked response:
The client creates a hidden iFrame with a source corresponding to the data stream. The server responds with an HTTP response header immediately and leaves the connection open. To send a message, the server wraps it in a pair of <script></script> tags that the browser executes when it receives the closing tag. The upside is that there's no connection reopening but there is more overhead with each message. Moreover, this requires a callback in the global scope that the response calls.
Also, this cannot be used with cross-domain requests as cross-domain iFrame communication presents its own set of problems. The need to trust the server is also a challenge here.
Web Sockets:
These start as a normal HTTP connection but they don't actually follow the HTTP protocol later on. From the programming point of view, things are as simple as they can be. The API is a classic open/callback style on the client side and the server just pushes messages into an open socket. No need to reopen anything after each message.
There still needs to be an open connection, but it's not really an issue here with the browser limits out of the way. The browser knows the connection is going to be open for a while, so it doesn't need to apply the same limits as to normal requests.
These seem like the ideal solution, but there is one major issue: IE<10 doesn't know them. As long as IE8 is alive, web sockets cannot be relied upon. Also, the native Android browser and Opera mini are out as well (ref.).
Still, web sockets seem to be the way to go once IE8 (and IE9) finally dies.
What you see are hanging requests with the timeout of 25 seconds that are used to implement the live update feature. As I already said, the keep-alive message ("h") is used so that the browser doesn't think it's not going to get a response. "h" simply means "nothing happens".
Chrome supports web sockets, so Meteor could have used them with a fallback to long requests, but, frankly, hanging requests are not at all bad once you've got them implemented (sure, the browser connection limit still applies).

Socket.IO detection of client's browser sometimes failing

I am working on a real time tracking application that uses Node.js and Socket.IO. In my tracking code that goes on a target site I have some code that grabs the user agent string of the browser and sends it back to the server. This USUALLY works fine but there are a few times where this data is set to undefined (this is where it happens).
For now, I just have a huge try/catch block on the server so it doesn't crash when running a method I've defined to detect what browser it is (it crashes when it tries to run the match() method). I'm assuming this is happening either from bots or from some other browser that has no user agent or has been tampered with. Am I wrong on that? Are there other reasons?
Does Socket.IO provide anything for browser detection? Either way I know I need to make the browser detection feature more robust but I'm just getting this project off the ground.
If there's no better way to do this, am I better off just checking to see if the data that was sent to the server is undefined and consider it as an "Other" browser?
See the difference in total connections and total browser numbers? At the moment, there's a difference of a little over 100. If this browser tracking issue wasn't happening the numbers should be exactly the same (because EVERY connection would have a browser, resolution, operating system, and a URL).

I'm not very familiar with socket.io, but it looks like you can get the request headers in socket.handshake.headers (Socket.io's Wiki – Authorizing). I don't know which browsers run JavaScript but don't have navigator.userAgent set, but maybe they'll send the User-Agent HTTP Header?

This actually had nothing to do with bots or tampered data like I thought it did. My problem was due to a rare race condition where the client would connect to the server but disconnect before sending the data to the server (the "beacon" event I have set up is where the client sends the data and the server receives it). So when the client went to disconnect, my server would look up the client but would return a undefined result because the data was never sent and stored in the first place. A rewarding experience but what a pain!

HTTP Body Streaming with Javascript

I am writing a debug/admin node server that allows users to execute a long-running process on the machine. I want to stream the output of the child process to the form they began the action from.
I can do this with sockets, but I have to have the client subscribe to a channel, and I have to post messages to the whole channel when they only have to do with the one client.
I'd prefer to be able to stream the http body down to the client. I can do this fairly easily with node: just keep writing to the request's socket, call end when I'm done.
Is there any way to use XhrHttpRequest to call a web service, have it fire events whenever new data is available, and a final event when it closes? Possible with jQuery?
Note that this isn't really the same use case as normal real-time updates, for which sockets are a good choice. This is a single request. I just want to get the response in pieces.

What I was hoping isn't possible: you can't make an xhr http request and keep it open, parsing chunks at a time.
Here is a summary of people's suggestions
Use socket.io anyway, and change your architecture to support pushing events.
Use socket.io, but make requests through it, as if you were hitting urls. Make a little url router on the server side of socket.io and stream stuff down all you want.
Keep the initial html page open and parse it as you go (not feasible for my implementation)
(3), but in a hidden iframe.
I went with 2.

As an update to this question, nowadays, you can use Sever-sent events (SSE). That way, you don't need to do anything particularly special on the server side, or setup websockets, which is overkill when you don't need full duplex. And XHR will keep the entire data in memory, which is non-ideal for large files. I had the same question, and I answered it here:
How to process streaming HTTP GET data?

some years ago i used "javascript" streaming over open http response. (years before ajax appeared)
the idea here : write chunks of
<script type="text/javascript">do js stuff here</script>
for each step of the process you want the client to react on.
it may still work.