Develop Reference

Data from database real-time. Use websockets or not? [duplicate]

I'm interested in building a small real-time multiplayer game, using HTML5/JavaScript for the client and probably Java for the server software.
I looked into WebSockets a bit, but it appears I had misconceptions on what WebSockets actually are. I had initially thought of WebSockets as just JavaScript's way of handling TCP sockets, just as they are used in Java and other languages, but it appears there is a whole handshaking process that must take place, and each transmission includes much HTTP overhead (and in that case, the benefits over Ajax do not seem as exciting as at a first glance)?
On a related topic, are there any better alternatives to WebSockets for this purpose (real-time multiplayer games in JavaScript)?

WebSockets are the best solution for realtime multiplayer games running in a web browser. As pointed out in the comments there is an initial handshake where the HTTP connection is upgraded but once the connection is established WebSockets offer the lowest latency connection mechanism for bi-directional communication between a server and a client.
I'd recommend you watch this: https://www.youtube.com/watch?v=_t28OPQlZK4&feature=youtu.be
Have a look at:
http://browserquest.mozilla.org/ code available here: https://github.com/mozilla/BrowserQuest
https://chrome.com/supersyncsports/
The only raw TCP solution would be to use a plugin which supports some kind of TCPClient object. I'd recommend you try out WebSockets.
You can find a number of options here. Just search for WebSockets within the page.
Also take a look at WebRTC. Depending on the purpose of your game and whether you need your server to manage game state, you could use this technology for peer-to-peer communication. You may still need a solution to handle putting players into groups - in that case WebSockets is the fastest/best solution.

Basically, you have 3 options at the time of this writing:
WebSockets
WebSockets is a lightweight messaging protocol that utilizes TCP, rather than a Javascript implementation of TCP sockets, as you've noted. However, beyond the initial handshake, there are no HTTP headers being passed to and fro beyond that point. Once the connection is established, data passes freely, with minimal overhead.
Long-polling
Long-polling, in a nutshell, involves the client polling the server for new information periodically with HTTP requests. This is extremely expensive in terms of CPU and bandwidth, as you're sending a hefty new HTTP header each time. This is essentially your only option when it comes to older browsers, and libraries such as Socket.io use long-polling as a fallback in these cases.
WebRTC
In addition to what has been mentioned already, WebRTC allows for communication via UDP. UDP has long been used in multiplayer games in non web-based environments because of its low overhead (relative to TCP), low latency, and non-blocking nature.
TCP "guarantees" that each packet will arrive (save for catastrophic network failure), and that they will always arrive in the order that they were sent. This is great for critical information such as registering scores, hits, chat, and so on.
UDP, on the other hand, has no such guarantees. Packets can arrive in any order, or not at all. This is actually useful when it comes to less critical data that is sent at a high frequency, and needs to arrive as quickly as possible, such as player positions or inputs. The reason being that TCP streams are blocked if a single packet gets delayed during transport, resulting in large gaps in game state updates. With UDP, you can simply ignore packets that arrive late (or not at all), and carry on with the very next one you receive, creating a smoother experience for the player.
At the time of this writing, WebSockets are probably your best bet, though WebRTC adoption is expanding quickly, and may actually be preferable by the time you're done with your game, so that's something to consider.

I'm not sure if WebSockets are still the best tool for networking
a real-time multiplayer these days (2017). WebRTC is a newer technology
which offers the potential of much higher performance. And these
days, WebRTC is also easier to work with thanks to the following libraries:
node-webrtc simplifies server-side networking
webrtc-native which also provides a server-side library, and could be faster as its name suggests
electron-webrtc provides an implementation which is a good match if you want to package your game using electron
Alternatively, if you want to be spared the actual details of networking implementation, and you're looking for a library which provides a higher-level multiplayer interface, take a look at Lance.gg. (disclaimer: I am one of the contributors).

Multiplayer games requires the server to send periodic snapshots of the world state to the client. In the context of a browser HTML/js application you have little choices: polling, websocket or write your own plugin to extend browser capabilities.
The HTTP polling such as BOSH or Bayeux are sophisticated but introduces network overhead and latency. The websocket was designed to overcome their limitation and is definitely more responsive.
Libraries, such as cometd or socket io, provide an abstraction of the transport and solve the browser compatibility issues for you. On top of that, it allows to switch between the underlying transports and compare their performance without effort.
I coded multiplayer arcade game with socket.io and usual measure 2ms latency with a websocket and around 30ms with xhr-polling on lan. It's enough for a multiplayer games.
I suggest you to have a look to nodejs and socket.io in order to be able to share code between the client and the server, you also car borrow some multiplayer code at [3].

If you are planing to use JavaScript for your game (as you are) then WebSocket is the best choice for you. And if you want to support older version of Internet Explorer then think of Signal R system Microsoft developed. They are using WebSocket under the hood, but they also have a few fall back options...so protocol will use the best available solution available.
http://signalr.net/

High CPU Utilization for Meteor.js

A meteor.js 0.82 app is running on an Ubuntu 14.04 server with 2GB memory and 2 cpu cores. It was deployed using mup. However the CPU utilization is very high, htop reports 2.72 load average.
Question: How do I find out which part of the app is causing such a high CPU utilization? I used Kadira but it does not reveal anything taking up alot of CPU load afaik.
Does Meteor only use a single core?

I had a similar problem before with Meteor 0.8.2-0.8.3. Here are what I have done to reduce the CPU usage, hope you may find it useful.
double check your functions, ensure all function has proper return, and does properly catch errors
try to use a replicaSet and oplog mongo convert standalone to replica set
write scripts to auto kill and resprawn a node process if it exceeds 100% cpu usage
utilize multi-core capability by starting 2 processes (edit you have done already) and configure and setup load-balance and reverse proxy
make sure to review your publish and subscription and limit what data to be sent to client (simply avoid something like Collection.find();)
Personally I recommend Phusion Passenger, it makes deploying Meteor applications an ease, and I have used it for several projects without any major problems.
One more thing, avoid running the processes in root (or privilege user), you should be running your apps in another user like www-data. This is for obvious security reason.
P.S. and multiple mongo processes showing in htop are threads under a master process, you can view it in tree mode by pressing F5.

How does Nodejs performance scale when using http.request?

I'm writing an application that makes heavy use of the http.request method.
In particular, I've found that sending 16+ ~30kb requests simultaneously really bogs down a Nodejs instance on a 512mb RAM machine.
I'm wondering if this is to be expected, or if Nodejs is just the wrong platform for outbound requests.

Yes, this behavior seems perfectly reasonable.
I would be more concerned if it was doing the work you described without any noticeable load on the system (in which case it would take a very long time). Remember that node is just an evented I/O runtime, so you can have faith that it is scheduling your I/O requests (about) as quickly as the underlying system can, hence it's using the system to it's (nearly) maximum potential, hence the system being "really bogged down".

One thing you should be aware of is the fact that http.request does not create a new socket for each call. Each request occurs on an object called an "agent" which contains a pool of up to 5 sockets. If you are using the v0.6 branch, then you can up this limit by using.
http.globalAgent.maxSockets = Infinity
Try that and see if it helps

Are WebSockets suitable for real-time multiplayer games?

I'm interested in building a small real-time multiplayer game, using HTML5/JavaScript for the client and probably Java for the server software.
I looked into WebSockets a bit, but it appears I had misconceptions on what WebSockets actually are. I had initially thought of WebSockets as just JavaScript's way of handling TCP sockets, just as they are used in Java and other languages, but it appears there is a whole handshaking process that must take place, and each transmission includes much HTTP overhead (and in that case, the benefits over Ajax do not seem as exciting as at a first glance)?
On a related topic, are there any better alternatives to WebSockets for this purpose (real-time multiplayer games in JavaScript)?

WebSockets are the best solution for realtime multiplayer games running in a web browser. As pointed out in the comments there is an initial handshake where the HTTP connection is upgraded but once the connection is established WebSockets offer the lowest latency connection mechanism for bi-directional communication between a server and a client.
I'd recommend you watch this: https://www.youtube.com/watch?v=_t28OPQlZK4&feature=youtu.be
Have a look at:
http://browserquest.mozilla.org/ code available here: https://github.com/mozilla/BrowserQuest
https://chrome.com/supersyncsports/
The only raw TCP solution would be to use a plugin which supports some kind of TCPClient object. I'd recommend you try out WebSockets.
You can find a number of options here. Just search for WebSockets within the page.
Also take a look at WebRTC. Depending on the purpose of your game and whether you need your server to manage game state, you could use this technology for peer-to-peer communication. You may still need a solution to handle putting players into groups - in that case WebSockets is the fastest/best solution.

Basically, you have 3 options at the time of this writing:
WebSockets
WebSockets is a lightweight messaging protocol that utilizes TCP, rather than a Javascript implementation of TCP sockets, as you've noted. However, beyond the initial handshake, there are no HTTP headers being passed to and fro beyond that point. Once the connection is established, data passes freely, with minimal overhead.
Long-polling
Long-polling, in a nutshell, involves the client polling the server for new information periodically with HTTP requests. This is extremely expensive in terms of CPU and bandwidth, as you're sending a hefty new HTTP header each time. This is essentially your only option when it comes to older browsers, and libraries such as Socket.io use long-polling as a fallback in these cases.
WebRTC
In addition to what has been mentioned already, WebRTC allows for communication via UDP. UDP has long been used in multiplayer games in non web-based environments because of its low overhead (relative to TCP), low latency, and non-blocking nature.
TCP "guarantees" that each packet will arrive (save for catastrophic network failure), and that they will always arrive in the order that they were sent. This is great for critical information such as registering scores, hits, chat, and so on.
UDP, on the other hand, has no such guarantees. Packets can arrive in any order, or not at all. This is actually useful when it comes to less critical data that is sent at a high frequency, and needs to arrive as quickly as possible, such as player positions or inputs. The reason being that TCP streams are blocked if a single packet gets delayed during transport, resulting in large gaps in game state updates. With UDP, you can simply ignore packets that arrive late (or not at all), and carry on with the very next one you receive, creating a smoother experience for the player.
At the time of this writing, WebSockets are probably your best bet, though WebRTC adoption is expanding quickly, and may actually be preferable by the time you're done with your game, so that's something to consider.

I'm not sure if WebSockets are still the best tool for networking
a real-time multiplayer these days (2017). WebRTC is a newer technology
which offers the potential of much higher performance. And these
days, WebRTC is also easier to work with thanks to the following libraries:
node-webrtc simplifies server-side networking
webrtc-native which also provides a server-side library, and could be faster as its name suggests
electron-webrtc provides an implementation which is a good match if you want to package your game using electron
Alternatively, if you want to be spared the actual details of networking implementation, and you're looking for a library which provides a higher-level multiplayer interface, take a look at Lance.gg. (disclaimer: I am one of the contributors).

Multiplayer games requires the server to send periodic snapshots of the world state to the client. In the context of a browser HTML/js application you have little choices: polling, websocket or write your own plugin to extend browser capabilities.
The HTTP polling such as BOSH or Bayeux are sophisticated but introduces network overhead and latency. The websocket was designed to overcome their limitation and is definitely more responsive.
Libraries, such as cometd or socket io, provide an abstraction of the transport and solve the browser compatibility issues for you. On top of that, it allows to switch between the underlying transports and compare their performance without effort.
I coded multiplayer arcade game with socket.io and usual measure 2ms latency with a websocket and around 30ms with xhr-polling on lan. It's enough for a multiplayer games.
I suggest you to have a look to nodejs and socket.io in order to be able to share code between the client and the server, you also car borrow some multiplayer code at [3].

If you are planing to use JavaScript for your game (as you are) then WebSocket is the best choice for you. And if you want to support older version of Internet Explorer then think of Signal R system Microsoft developed. They are using WebSocket under the hood, but they also have a few fall back options...so protocol will use the best available solution available.
http://signalr.net/

Performance testing on node.js "net"

Does anyone have any recommendations on how to get started with node.js "net" performance testing?
I want to see how my app will scale and want to test 10,000+ concurrent connections!
EDIT: I want to know so I can see if my Ubuntu server configs are correct, etc.

Professional performance testing tools are agnostic to your underlying technology (node.js / .NET), and see just the output (HTTP Requests and responses), so any tools can do.
There's HP's LoadRunner and a lot of others. I have used WebLOAD, which is more cost effective, and a bit easier to use.

10,000 concurrent connections. Hmmm. I would think that such a load would have to be tied to a user population for your app somewhere in the 500,000-2,000,000 range with a 2% to .5% level of concurrency respectively. If this was an internal facing corporate app then your user population expectations would be somewhere in the 83,333(12%) - 125,000 (8%). These concurrency models come from 15 years of observations in corporate and internet facing applications for levels of concurrency vs the defined user population for a given application facing model (internal corporate vs public internet).
The reason why I bring up the above is that you may be over stressing your component for its defined use and as a result you could have some engineering ghosts that you chase down to fix. This can impact your budget and availability to hit other issues that may show up in production use.
Just food for thought,
James Pulley

From the video it seems that memory usage doesn't budge because it doesn't spawn new processes, which is precisely the reason it has picked up a huge following. That's what event driven/non blocking can do