I am creating a gaming service in which players will be paired and compete against one another in real-time. I am building this is node/websockets and react. My question is very high level:
Where should I store data like the number of users online, list of online users, etc. I am sure I can store it in a DB but I know it will change frequently. Is this best suited for one of nodes memory stores like "data-store" or just have a set of variables on the server accessible to the websockets (what I have now)?
Or should I just put it in a DB anyway???
If you just have a single server and the data is all of the type you've listed that is temporal (doesn't need to survive a server restart), then just using a set of variables and keeping the data in memory is perfectly fine, is simplest to implement and will perform the best. There is really no reason to involve the additional overhead of a traditional disk-based store unless the data is overly large (which it does not seem to be).
If you cluster your server, then frequently changed data that does not need to be persistently stored across server restarts can be kept in an in-memory database in it's own process such as redis. Each server in your cluster can then query redis any time it needs the latest copy of the data. Because it's an in-memory data store, it's efficient for data that changes a lot while also being available to multiple processes.
Related
Beginner question: I built a simple draggable to-do list that caches the state in a single object (tasks, containers and index) - currently, it's storing it in local storage. I am working on the server side using express and node.js, but I am confused as to where I would simply store the object. Would a database like mongodb be a good choice...or is there an even simpler option? I assume I can keep the project static and have the server side just receive and serve up JSON? Thanks!
If you plan to integrate it with backend server, it is actually a good idea to store the object in a database. The benefit is, you can still maintain the state of your to-do-list no matter on which machine you are logging in. If you access your to-do-list app from the browser of your smartphone or desktop, they both still point to a single source of truth, which is your database. Think of it as a Trello board that is in-sync on every device. In your database, you may record the task status, task ID, description, etc. If you want to go further, you can group this information per user, so every user will have their own to-do-list. (which is not possible if you rely on conventional local storage). With database, you can extend the functionality beyond simple to-do-list. Alternatively, you may consider a much simpler solution by recording the object as JSON file and storing it in your server. This solution is feasible albeit limited flexibility.
I would recommend MongoDB Atlas and Firebase Realtime Database as both are beginner friendly and easy to use. Both are free-of-charge on limited usage and hosted in the cloud.
I have two options when it comes to a server I am developing.
Option a) when the server start, extract all data from database and store it on memory as objects.
Option b) everytime a data is required, extract it from database, transform it into object and return that. Dont store all data on memory.
I'm kind of afraid that if there is too much data stored in memory the app crashes. But I'm not sure how this is suppose to be handle.
Which is the right way to handle this?
Fill free to extend another option.
You can do a bit of both.
You can store some data in memory. For example if your database holds some settings which are some sort of constants and are not thousands of lines, then you can load them in memory. If you have some data that are accessed very frequently and are not being updated very often, you can also load them at runtime and keep them in memory.
If you have some data that expect to change frequently and/or are going to grow extensively, such as user data, or content. Load them from db everytime they are required.
If your application is a large scale app with high traffic, you might want to add some caching system which will basically store some data in memory or file and update it when it is changed or when it expires with a time to live property.
I am considering using node.js to build an API like service but am having trouble understanding whether to use a data structure vs. storing information in a database/text file.
Basically the program would allow for a user to come on line and collect that users geo-coded location. Then the service would store that information in either a javascript data structure or store it into a database or text file. Then another user would log on and I would connect them with a user who is closes to them.
My question is, if I have a datastructure (some sort of custom implemented sorted list based off of geo-codes) would all of that information be volatile and I would loose it if the program crashed?
Would it be more preferable to store the information in a text file or database even though the access and write of that information would take longer?
Also, if I was using the data structure approach, would that make it more difficult to scale the application if I needed to expand to additional servers?
Any thoughts?
My question is, if I have a datastructure (some sort of custom
implemented sorted list based off of geo-codes) would all of that
information be volatile and I would loose it if the program crashed?
Yes, it would be volatile and you would lose it if the program crashed. All Javascript data is kept in RAM.
Would it be more preferable to store the information in a text file or
database even though the access and write of that information would
take longer?
When exactly to save data to a persistent store is highly dependent upon the details of the situation. You could have only a disk store or you could have a RAM store that is periodically stored to disk or you could have a combination (a RAM cache in front of a persistent store). Using a database will generally do a lot of caching for you.
Also, if I was using the data structure approach, would that make it
more difficult to scale the application if I needed to expand to
additional servers?
If you want to share a live data store among multiple servers, then you have to use something other than just Javascript data stored in node.js memory. The usual solution is to go with some sort of external database which itself can be either in-memory (like Redis) or disk-store (like Mongo or Couchbase) which all the different servers can then access.
Progressing along with my isomorphic javascript crusade, I put Meteor on a hold while I played a bit more with the MEAN stack. To ward off any further procastination, I've decided to finish my original prototype community application. Now, my biggest issue with Meteor isn't reactivity, it's session/common data.
I know Meteor's native session system is based off of the reactive concept, and cookies don't "exist" because Meteor operates on "the wire". Though let's say I were building an application on the LAMP or mean STACK, and I was creating a user interface. I'd use cookies/sessions to control user activity. If Meteor operates off of reactivity, how do I maintain persistence?
I have searched through atmosphere for packages that fit my criteria, and I ran into a couple of packages that store "presistent sessions". Though these interfaces operate off of the client, not the server; hence my code would be exposed client, therefor setting the application up for exploitaton.
All that being stated, I know Meteor has it's standard user interface. What I'm trying to do here is understand Meteor, and gain experience for future endevours.
Meteor has a built-in login system that keeps track of the logged-in user, which is one of the main reasons people use cookies. If you want to store other data on the client in a persistent way, you can use the HTML5 localStorage API.
I think what you're referring to is that something like PHP lets you store data in a "SESSION" variable that is actually stored on the server, but persisted between different requests from the same client.
If this is what you are looking for, there are several approaches that will give you similar functionality:
Store data associated with a particular user, and use the userId that Meteor gives you to only publish it for that user (using Meteor.publish)
Have a randomly generated client ID that is stored in localStorage, and pass that in when calling subscriptions or methods to authenticate as that client. This will work in the case where the user is not logged in, and will give you a very similar result to cookies/session in PHP. You will still store the actual data in the database on the server, but you will know which data is associated with a particular client by the unique ID.
It's true that Meteor's Session variable is named in a way that can be confusing if you are coming from PHP where SESSION means something totally different.
Does this answer your question?
I am developing a simple login system, and I'll have to store two cookies. One indicating which user is logged, and other indicating the "plan" (database) that the user belongs. Well, at client side the first cookie (us_auth) and the second (pl_auth), will store hashes (md5) to make a key that I can check on server side.
Now at the server side I'll store these two keys to assign to them values to route the user correctly. A simple authentication system.
Well, here the question comes. At PHP we have $_SESSION, and at Node we have modules that make the same thing, but I want something that I have more control, to work directly with the data. I have two choices:
Memcached, or a simple global var that store an object with key->value (in my case, hash->value).
What's the best choice in my case? At simple point of view, the second alternative seems to be more fast, pratical and simple.
A simple global variable will fail if you replicate your application. Variables are not shared between processes (or even machines).
If you don’t plan on replicating your app, just restart your process, the memory allocated to that process will be lost, so will your in-memory session variables.
A memcached server can help you with that.
This is not specific to nodejs. You’ll have the same problem with PHP or any other technology.