I have PHP on the server side, and HTML and javascript on the client side.
I am making an app where a stakeholder types a message that is broadcasted to multiple recievers of a group in real time.
I did some research on google and I understand I need to use WebSockets or Comet for real time push notifications. Is WebSocket or Comet mandatory for sending mass notifications to users?
Is my understanding correct? Any references to start with?
If the client is a browser, then the ONLY two ways a standard browser can connect to a server is via an Ajax (e.g. http) request or a webSocket connection. So, if you want a client to get notified of something from the outside world it has to use one of those two mechanisms.
HTTP requests are transitory. The client makes a request of a server, the server responds. HTTP requests are perfect for the client requesting information from the server. They are not very good at the server sending information to the client because normally the client is not connected. There are hacks and work-arounds where the client "polls" the server on some interval and maybe even the server uses longer running requests to try to simulate a "push" type system, but they are sub-optimal hacks at best.
webSockets are continuous connections. The client connects and the connection remains in place for as long as both sides want. This allows either side the ability to send a message to the other side whenever they want. That means the server can "push" data to the client whenever it wants. webSockets are efficient for push connections and are recommended (this is one of the main things they were designed for).
Comet is a library that was originally built for using HTTP to try to "hack" or "simulate" push before webSockets were invented and then before they were widely supported. I can think of no reason why one would want to use Comet instead of a webSocket unless you had such an old browser that webSocket was not supported.
So, if you are trying to do "realtime server push" to a browser, then you must have a continuously connected socket from the client which means webSocket (or something built on top of webSocket like socket.io).
For phone apps where you have access to the phone SDK, you can use the "push" system built into the OS to push some messages from server to client. This isn't quite the same as the two way webSocket channel, but since you asked about "push notifications", the OS push services available in both Android and IOS could also be an option for pushing notifications from server to client. Here's info on iOS notifications and Google Cloud Messaging
As of 2016, one can also use Server-sent events in all modern browsers except Microsoft browsers (not supported yet in Edge or IE) to push data from server to client. Here's a browser compatibility table. Server-sent events use a long lasting HTTP connection, a special MIME type and a supporting client in order to be able to send events from server to client at any time. Unlike webSockets, server-sent events are one way only (from server to client). A client would then use a traditional Ajax call in order to be able to send data to a server (whereas with a webSocket data can be sent either way over the same webSocket connection).
Here's a good description of how server-sent events work: How do server-sent events actually work?
Is your client application a SPA? (Single Page application)?
It's very important because if not, you have to consider that everytime a client change page, connection with websocket server will be lost.
In this case you have to manage a queue because if stakeholder send a multicast request when one client is disconnected, client won't receive nothing.
Polling won't solve this situation too and it's an orrible solution because mobile clients (for example) with typical internet plan, will consume megabytes for unuseful "ping" traffic.
A real example of polling is a child in a car asking his dad every minute if they are arrived to a destination!
So, Is there a solution without using spa?
Yes, using a "shared storage" between stakeholder and clients, and using websocket only for "wake up" online clients saying: Hey there is something new, go to check!
Everytime a client open a page it will receive from backend also not-read notifications, taken from the storage.
When a stakeholder want to notify something, it will just store the notification message in the shared storage and send a "pulse" to notification server.
Notification server will forward the "pulse" to online clients (just in case someone is stuck reading a page).
If a "pulse" is lost because a client is changing page there is no problem because the client will bring notifications from the storage.
Every page will contain this logic:
Retrive number or unread notifications (server side)
Connect to the notification server after 5 seconds (javascript side).
Hope it helps.
I would suggest that using webSockets is a more efficient way compared to other options, why is this? Well when a client receives a notification that there's a change in the server there is no need to create an AJAX call to the server to get that change, it can be sent to the client with the same webSocket connection more easily than AJAX. This means efficient code and a faster running App!
Related
I am very new to websockets and trying to create an omegle as an example. A centralized chat server to which every client can connect through a websocket and the the server matches people based on interests. The people can chat through the websocket connection thereafter.
I am just curious about one thing: Once my site goes live and different clients keep connecting, essentially in the background they are connecting to my central server through the websocket. Can't any client run javascript on its chrome console and inject a malicious script or get access to the clients connected to the server already since the connection has been established and its a stateful connection? I am not sure if there is a way to do that. And if there is, what security mechanisms i need to take care of?
The response to your question is "Yes" and "No"!
it depends on so many things. First, is your chat a public one? I mean can every body join the chat room without login?
In this case i guess everybody can chat so a guy who want to use your websockets can do so a it is public.
In the case your chat is private it is your own to protect your websockets events from being accessed by everybody. This start by registering users after a robust login process.
As all others Internet (and computer based) tech, websocket is subject to hacking. You chould protect your websockets as you protect your web based developments.
Typing "websocket common attacks" in Google will return you links that talk of websocket security.
Yes, a client can definitely inject a malicious script... but it will only affect the client's own browser.
No. There isn't a way to do that unless you explicitly created a vulnerability.
You see, first of all, websockets are not peer-to-peer connections. They require a server to transmit messages between end users, so if the server doesn't do anything about an incoming message request, the user the first user intended to send a message to would not receive anything. Essentially, this means that websocket clients can only connect to users if the server lets them.
However, even if websockets were p2p connections that would still not be possible, or what kind of a web would we have?! I hope you understand that any data sent to a client isn't automatically executed as code. It usually is in a string format and it will never be able to run any code unless you actually eval it or something. (Or you forget about XSS and don't properly escape data that would be rendered as HTML.)
So, to answer your question, no and no. Security measures? Don't eval incoming data.
However, what you do have to be aware of in websockets is that any client can connect (well, at least try) to a websocket server. See, because websockets usually are used to send data between clients, when a server receives a message, it usually will broadcast that message to everyone in that connection/room except for the sender. This means that a user with malicious intent may enter your room and send data, as well as receive data that's probably meant to be private.
To prevent this, ensure you are setting up proper authentication. You can do this through cookies, perhaps, used with session or JWT (I usually use the latter). When a user joins a room or connects, check if that user is authorized and allowed to be in the room. Otherwise, reject the connection.
Here's a bit broader explanation that might help you.
Web sockets work on TCP connections. That means the socket connects to a specific destination and transmits data ("messages"). Those messages can flow both ways (in and out). Users connect to your server using clients (most commonly a browser).
The connections are unique, meaning no two clients share the same connection. Therefore, no one else knows what others send to or receive from the server.
Messages can be of various types, but the most commonly used is a string. Typically people encode some JSON objects in those strings to perform specific actions. Something along the lines of:
{
message: "Hello!",
channel: "general"
}
Now here comes your role. How to act on those messages, and what to send to whom. Standard libraries provide functionalities like targeting a single socket or sending messages to many sockets. Imagine it like a list of connections. They can send specific commands to join/leave channels, so you add more meta data to each. In the previous example, you might want to send Hello! to every connection that has "joined" a channel (by sending a specific command to do so).
You decide what the commands are:
{
command: "join/leave",
channel: "general"
}
You might want to spread every message you receive to others or keep it to yourself (and execute some custom functionality). Nothing happens out of the box - the server receives the message, and that's it.
All of the above means that whatever users do with their clients won't affect you or others. The only way you can know of their actions is if they send you a message. Those messages you must be careful with, as they can contain any malicious code.
Therefore I'd highly recommend you use some library that deals with web sockets in case you'd like to ship something to people. If it's for learning purposes - stick to the plain functionalities.
I am adding a WebSocket handler to my Azure Webapp. Lets say that I am writing chat server (though I'm not), where the server will keep a list of all open connections in memory so that all connected clients can talk to each other.
Now, I have auto-scaling turned on so my server can some point in time scale out to 2 instances. Now clients connected to instance 1 cannot talk to clients who happen to be connected to instance 2.
To solve this I am trying to find out if it is possible to target a specific instance using Azures ARRAffinity cookie, but what I found so far, this is only possible:
For ordinary web requests (not "new WebSocket()")
From server to server using node.js
So my question, is it in fact possible in some way to this with websockets? Do you have some other suggestions?
EDIT 2017-11-02: This is still an interesting question, but I have solved it in a different (mostly positive) way: Since all messages going through the websocket is already serialized, it was easy to utilize my already present Redis cache publish/subscribe functionality.
So what I do now is:
Message comes through websocket from client A - targeted at client B.
Message arrives at webserver X (one of many possible webservers depending on scale out)
Message is published to a Redis cache channel
All webservers subscribes to Redis cache channel and receives the message
The one webserver who happens to hold a websocket connection to the target client delivers the message, all other webservers ignore the message
I am building a small chat application for friends, but unsure about how to get information in a timely manner that is not as manual or as rudimentary as forcing a page refresh.
Currently, I am implementing this using simple AJAX, but this has the disadvantage of regularly hitting the server when a short timer elapses.
In researching long/short polling, I ran across HTML5 WebSockets. This seems easy to implement, but I'm not sure if there are some hidden disadvantages. For example, I think WebSockets is only supported by certain browsers. Are there other disadvantages to WebSockets that I should be aware of?
Since it seems like both technologies do the same thing, in what sorts of scenarios would one prefer to use one over the other? More specifically, has HTML5 WebSockets made AJAX long/short polling obsolete, or are there compelling reasons to prefer AJAX over WebSockets?
WebSockets is definitely the future now.
Long polling is a dirty workaround to prevent creating connections for each request like AJAX does - but long polling was created when WebSockets didn't exist. Now due to WebSockets,
long polling is going away no more.
WebRTC allows for peer-to-peer communication.
I recommend learning WebSockets.
Comparison:
of different communication techniques on the web
AJAX - request → response. Creates a connection to the server, sends request headers with optional data, gets a response from the server, and closes the connection.
Supported in all major browsers.
Long poll - request → wait → response. Creates a connection to the server like AJAX does, but maintains a keep-alive connection open for some time (not long though). During connection, the open client can receive data from the server. The client has to reconnect periodically after the connection is closed, due to timeouts or data eof. On server side it is still treated like an HTTP request, same as AJAX, except the answer on request will happen now or some time in the future, defined by the application logic.
support chart (full) | wikipedia
WebSockets - client ↔ server. Create a TCP connection to the server, and keep it open as long as needed. The server or client can easily close the connection. The client goes through an HTTP compatible handshake process. If it succeeds, then the server and client can exchange data in both directions at any time. It is efficient if the application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server, so data is simply encrypted.
support chart (very good) | wikipedia
WebRTC - peer ↔ peer. Transport to establish communication between clients and is transport-agnostic, so it can use UDP, TCP or even more abstract layers. This is generally used for high volume data transfer, such as video/audio streaming, where reliability is secondary and a few frames or reduction in quality progression can be sacrificed in favour of response time and, at least, some data transfer. Both sides (peers) can push data to each other independently. While it can be used totally independent from any centralised servers, it still requires some way of exchanging endPoints data, where in most cases developers still use centralised servers to "link" peers. This is required only to exchange essential data for establishing a connection, after which a centralised server is not required.
support chart (medium) | wikipedia
Server-Sent Events - client ← server. Client establishes persistent and long-term connection to server. Only the server can send data to a client. If the client wants to send data to the server, it would require the use of another technology/protocol to do so. This protocol is HTTP compatible and simple to implement in most server-side platforms. This is a preferable protocol to be used instead of Long Polling. support chart (good, except IE) | wikipedia
Advantages:
The main advantage of WebSockets server-side, is that it is not an HTTP request (after handshake), but a proper message based communication protocol. This enables you to achieve huge performance and architecture advantages. For example, in node.js, you can share the same memory for different socket connections, so they can each access shared variables. Therefore, you don't need to use a database as an exchange point in the middle (like with AJAX or Long Polling with a language like PHP).
You can store data in RAM, or even republish between sockets straight away.
Security considerations
People are often concerned about the security of WebSockets. The reality is that it makes little difference or even puts WebSockets as better option. First of all, with AJAX, there is a higher chance of MITM, as each request is a new TCP connection that is traversing through internet infrastructure. With WebSockets, once it's connected it is far more challenging to intercept in between, with additionally enforced frame masking when data is streamed from client to server as well as additional compression, which requires more effort to probe data. All modern protocols support both: HTTP and HTTPS (encrypted).
P.S.
Remember that WebSockets generally have a very different approach of logic for networking, more like real-time games had all this time, and not like http.
One contending technology you've omitted is Server-Sent Events / Event Source. What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet? has a good discussion of all of these. Keep in mind that some of these are easier than others to integrate with on the server side.
For chat applications or any other application that is in constant conversation with the server, WebSockets are the best option. However, you can only use WebSockets with a server that supports them, so that may limit your ability to use them if you cannot install the required libraries. In which case, you would need to use Long Polling to obtain similar functionality.
XHR polling A Request is answered when the event occurs (could be straight away, or after a delay). Subsequent requests will need to made to receive further events.
The browser makes an asynchronous request of the server,
which may wait for data to be available before responding. The
response can contain encoded data (typically XML or JSON) or
Javascript to be executed by the client. At the end of the processing
of the response, the browser creates and sends another XHR, to await
the next event. Thus the browser always keeps a request outstanding
with the server, to be answered as each event occurs. Wikipedia
Server Sent Events Client sends request to server. Server sends new data to webpage at any time.
Traditionally, a web page has to send a request to the server to
receive new data; that is, the page requests data from the server.
With server-sent events, it's possible for a server to send new data
to a web page at any time, by pushing messages to the web page. These
incoming messages can be treated as Events + data inside the web page. Mozilla
WebSockets After the initial handshake (via HTTP protocol). Communication is done bidirectionally using the WebSocket protocol.
The handshake starts with an HTTP request/response, allowing servers
to handle HTTP connections as well as WebSocket connections on the
same port. Once the connection is established, communication switches
to a bidirectional binary protocol which does not conform to the HTTP
protocol. Wikipedia
I have X amount of activity sensors connected to a server that inserts data to a database everytime a sensor is triggered. What I'm trying to do is create a web interface with a blue print of the facility (svg) and whenever a sensor is triggered, besides the db insert, I want it to show some sort of alert in my blue print. For that I need to keep an open connection to the server I think.
I was thinking of using web sockets, but it might be overkill since I only need to retrieve data from the server. But running an ajax call every second doesn't sound very efficient either. Are there any other alternatives?
Thank you
Some potential choices include:
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling
Which actual transport you end up using will depend on the your requirements for browser support and what technology you are using on the server to handle these requests. The transport choice may also depend on your network topology - what types of load balancers you need to integrate with, proxies, etc.
There are many libraries available on both the client and server sides, many of which support more than one of these transports.
For example (not an exhaustive list):
socket.io for nodejs
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling
SignalR for an asp/.net backend
WebSockets
Server-Sent Events
ForeverFrame
Long Polling
Atmosphere for a java backend
WebSockets
Server Side Events (SSE)
Long-Polling
Forever frame
JSONP
IMO - Websockets is NOT overkill for this type of problem and would lend itself nicely to this type of application.
Without specifically discussing frameworks or knowing what is running in the backend of your server(s), we have a few options to consider for the frontend:
Websockets
Websockets are designed for bidirectional communication, although it is kind of shocking how many users are surfing the web in a browser that doesn't support websockets. I always recommend a fallback for this, such as the other methods listed below.
SSE
SSE is an HTML5 spec and is still shaky at best. Try scrolling on a page while when an SSE event fires... It may be a little easier on the backend, put it sometimes hangs on the client side since it runs inside the same thread that the DOM is running in.
Long Polling
Keeps your connection open. It doesn't scale well with PHP, but performs swimmingly with Python+Twisted on the backend, or Node.Js
Good Old Ajax
Keep your requests small, and you still have a scalable solution. Yes, a full GET request is the most expensive, but is supported in just about every browser rolled out the past ten years. It is also worth noting that GET requests are easy to scale horizontally with more hardware.
In a perfect world:
You would break up your application into a few components, operating behind a reverse proxy such as Nginx. Then use Node.Js + Socket.IO handle the realtime aspects of your app.
Another option would be to use small Ajax requests, and offer websocket support for the browsers that support it. This is advice specifically for PHP in the backend.
WebSocket is certainly not overkill. On the contrary. With websockets, you have a bi-directional communication channel; this means, that the server can initiate communication whenever it seems fit (e.g. when sensor data changes).
In a previous project, I have used node.js together with socket.io, to monitor 50+ sensors. Data was updated in real-time in a browser. The data was visualized using smoothie.js.
Whenever a sensor value was updated, it was communicated to the browser. Some sensors only updated once a minute, others once a second, ...
Polling would have been overkill, because it would retrieve all data for all sensors, even from those that were not updated yet.
I had a similar problem and did a lot of research on this. As I understand it, there are three main options:
Short polling: Have an endpoint that your javascript client pings every second. This is the worst option, because the pings add latency up to one second to your communication, and depending on how you implement, the endpoint could query the database every second, adding unnecessary overhead.
Long polling: Have an endpoint that your javascript client pings that holds the connection until a) the event occurs or b) the connection times out. If the endpoint returns a response, the client gets the event information. If the endpoint does not return a response, no event has occurred, and the client sends a new request. This is a good option because the events can immediately trigger the response to the client, assuming you have an asynchronous interprocess communication layer (like 0MQ) to send the message without any sort of polling.
Websocket: Have your javascript client connect to a websocket server, which will send a message to your client immediately upon the event trigger.
I think a websocket is your best option, because it accommodates immediate communication of the event without all the request/response overhead. And most importantly, this is exactly what websockets are designed to do! As such, you will probably have to write the least amount of custom code with this solution.
There are two great commercial services that might work for you.
Firebase - a javascript hierarchical database and realtime
messaging/ synchronization platform, uses websockets and has other fallbacks
PubNub - a real time message passing and queue system, uses websockets
I am building a small chat application for friends, but unsure about how to get information in a timely manner that is not as manual or as rudimentary as forcing a page refresh.
Currently, I am implementing this using simple AJAX, but this has the disadvantage of regularly hitting the server when a short timer elapses.
In researching long/short polling, I ran across HTML5 WebSockets. This seems easy to implement, but I'm not sure if there are some hidden disadvantages. For example, I think WebSockets is only supported by certain browsers. Are there other disadvantages to WebSockets that I should be aware of?
Since it seems like both technologies do the same thing, in what sorts of scenarios would one prefer to use one over the other? More specifically, has HTML5 WebSockets made AJAX long/short polling obsolete, or are there compelling reasons to prefer AJAX over WebSockets?
WebSockets is definitely the future now.
Long polling is a dirty workaround to prevent creating connections for each request like AJAX does - but long polling was created when WebSockets didn't exist. Now due to WebSockets,
long polling is going away no more.
WebRTC allows for peer-to-peer communication.
I recommend learning WebSockets.
Comparison:
of different communication techniques on the web
AJAX - request → response. Creates a connection to the server, sends request headers with optional data, gets a response from the server, and closes the connection.
Supported in all major browsers.
Long poll - request → wait → response. Creates a connection to the server like AJAX does, but maintains a keep-alive connection open for some time (not long though). During connection, the open client can receive data from the server. The client has to reconnect periodically after the connection is closed, due to timeouts or data eof. On server side it is still treated like an HTTP request, same as AJAX, except the answer on request will happen now or some time in the future, defined by the application logic.
support chart (full) | wikipedia
WebSockets - client ↔ server. Create a TCP connection to the server, and keep it open as long as needed. The server or client can easily close the connection. The client goes through an HTTP compatible handshake process. If it succeeds, then the server and client can exchange data in both directions at any time. It is efficient if the application requires frequent data exchange in both ways. WebSockets do have data framing that includes masking for each message sent from client to server, so data is simply encrypted.
support chart (very good) | wikipedia
WebRTC - peer ↔ peer. Transport to establish communication between clients and is transport-agnostic, so it can use UDP, TCP or even more abstract layers. This is generally used for high volume data transfer, such as video/audio streaming, where reliability is secondary and a few frames or reduction in quality progression can be sacrificed in favour of response time and, at least, some data transfer. Both sides (peers) can push data to each other independently. While it can be used totally independent from any centralised servers, it still requires some way of exchanging endPoints data, where in most cases developers still use centralised servers to "link" peers. This is required only to exchange essential data for establishing a connection, after which a centralised server is not required.
support chart (medium) | wikipedia
Server-Sent Events - client ← server. Client establishes persistent and long-term connection to server. Only the server can send data to a client. If the client wants to send data to the server, it would require the use of another technology/protocol to do so. This protocol is HTTP compatible and simple to implement in most server-side platforms. This is a preferable protocol to be used instead of Long Polling. support chart (good, except IE) | wikipedia
Advantages:
The main advantage of WebSockets server-side, is that it is not an HTTP request (after handshake), but a proper message based communication protocol. This enables you to achieve huge performance and architecture advantages. For example, in node.js, you can share the same memory for different socket connections, so they can each access shared variables. Therefore, you don't need to use a database as an exchange point in the middle (like with AJAX or Long Polling with a language like PHP).
You can store data in RAM, or even republish between sockets straight away.
Security considerations
People are often concerned about the security of WebSockets. The reality is that it makes little difference or even puts WebSockets as better option. First of all, with AJAX, there is a higher chance of MITM, as each request is a new TCP connection that is traversing through internet infrastructure. With WebSockets, once it's connected it is far more challenging to intercept in between, with additionally enforced frame masking when data is streamed from client to server as well as additional compression, which requires more effort to probe data. All modern protocols support both: HTTP and HTTPS (encrypted).
P.S.
Remember that WebSockets generally have a very different approach of logic for networking, more like real-time games had all this time, and not like http.
One contending technology you've omitted is Server-Sent Events / Event Source. What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet? has a good discussion of all of these. Keep in mind that some of these are easier than others to integrate with on the server side.
For chat applications or any other application that is in constant conversation with the server, WebSockets are the best option. However, you can only use WebSockets with a server that supports them, so that may limit your ability to use them if you cannot install the required libraries. In which case, you would need to use Long Polling to obtain similar functionality.
XHR polling A Request is answered when the event occurs (could be straight away, or after a delay). Subsequent requests will need to made to receive further events.
The browser makes an asynchronous request of the server,
which may wait for data to be available before responding. The
response can contain encoded data (typically XML or JSON) or
Javascript to be executed by the client. At the end of the processing
of the response, the browser creates and sends another XHR, to await
the next event. Thus the browser always keeps a request outstanding
with the server, to be answered as each event occurs. Wikipedia
Server Sent Events Client sends request to server. Server sends new data to webpage at any time.
Traditionally, a web page has to send a request to the server to
receive new data; that is, the page requests data from the server.
With server-sent events, it's possible for a server to send new data
to a web page at any time, by pushing messages to the web page. These
incoming messages can be treated as Events + data inside the web page. Mozilla
WebSockets After the initial handshake (via HTTP protocol). Communication is done bidirectionally using the WebSocket protocol.
The handshake starts with an HTTP request/response, allowing servers
to handle HTTP connections as well as WebSocket connections on the
same port. Once the connection is established, communication switches
to a bidirectional binary protocol which does not conform to the HTTP
protocol. Wikipedia