I am building an app using socket.io
I'm using socket.io's rooms feature, there are 5 "topics" a user can subscribe to. Each message broadcast in that topic has a message type, of which there are 100. A user will only receive the messages of the types they are allowed to receive, which could be between 30 and 70.
My question: is it feasible to create a room for every topic + message type combination, which will be 5 x 100 rooms? Will socket.io perform well like this, or is there a better way to approach this problem? Would emitting individual messages to each individual socket, instead of using rooms, be better?
Thanks for your help.
socket.io rooms are a lightweight data structure. They are simply an array of connections that are associated with that room. You can have as many as you want (within normal memory usage limits). There is no heavyweight thing that makes a room expensive in terms of resources. It's just a list of sockets that wishes to be associated with that room. Emitting to a room is nothing more than iterating through the array of sockets in the room and sending to each one.
A room costs only a little bit of memory to keep the array of sockets that are in each room. Other than that, there is no additional cost.
Further, if your alternative is to just maintain an array of sockets for each topic anyway, then your alternative is probably not saving you much or anything.
My question: is it feasible to create a room for every topic + message
type combination, which will be 5 x 100 rooms?
Yes, that is easily feasible.
Will socket.io perform well like this, or is there a better way to
approach this problem?
There's no issue with having this many rooms. Whether it performs well or not depends entirely upon what you're doing with that many rooms. If you're reguarly sending lots of messages to lots of rooms that each have lots of sockets in them, then you'll have to benchmark if that has a performance issue or not.
Would emitting individual messages to each individual socket, instead
of using rooms, be better?
There won't be an appreciable difference. A room is just a convenience tool. Emitting to a room, just has to iterate through each socket in the room and sending to it anyway - the same as you proposed doing yourself. May as well use the built-in rooms capability rather than reimplement yourself.
Related
So the gist of my question. Imagine you have a service that handles 2-3-4-10 actions. And to communicate in several components, you have 2-3-4-10 Subjects.
So, is it better to have 1 subject, and pass in on next an object identifying which of the actions it relates to, and filter inside your subscription...or have the lot of them and subscribe separately?
How many subjects is too many? They more or less remain active all at once throughout.
Kind of curious in an abstract a sense as possible, rather than my own usecase and whether or not it could be done better.
I work on large angular applications that use hundreds if not thousands of subjects (Subject, BehaviorSubject, ReplaySubject, AsyncSubject).
Is there a performance hit for using many subjects?
To this, I'd say it's not the subjects that matter, since they are only taking up memory space. It's the pipeline that you attach to them, which places them into the cpu's computation queue that matters. This is dependent on the pipelines themselves, and not the subjects. You could have a single subject that connects to a long computational heavy pipeline, which if done incorrectly, would slow your program down since javascript runs on a single thread of execution (you could use web workers to avoid this problem).
Therefore, the number of subjects is irrelavent here if we are talking about how "performant" your application is. It's the pipelines that determine if your application is slow. Ie, data moving down a pipe and having operators manipulate it.
StackBlitz single pipe that is computationally heavy to prove my point.
Is it better to have 1 subject, and pass in on next an object identifying which of the actions it relates to, and filter inside your subscription?
I would say this is more of a design decision to have a bus of information ("a single subject") passing all your data along, instead of breaking them up into their respective streams. This could be handy if your data is interconnected, meaning your events depend on each other, and if the order they appear within the stream matters (like navigation events: started, performing, ended, etc).
I would be unhappy if a dev used one giant bin to place all their data into instead of breaking it up into respective streams. Ie, if I have a user object, company information, and notifications, I'd expect these to have separation of concerns, and to not be delivered through a bus system (a single subject), but to instead be delivered through different services, with their own respective subjects and pipelines.
How many subjects is too many? They more or less remain active all at once throughout.
If you're doing trivial maps and filtering, then don't worry about how many subjects you're using. Worry about if your streams of data make logical/logistical sense, and that they are structured/siloed correctly.
StackBlitz program combining 1 million behavior subjects to prove my point.
I am beginner in redis and had used it in my node.js project and its providing good results when I see the caching mechanism it's been spinning
So basically in world where MySql,firebase and mongodb are top in there perspective, where would redis fit? Can we use redis for better optimization replacing any of these most popular databases or can have greater application role with specific technologies ? Maybe it should be used with javascript and its framework(eg. node.js has good analogy with redis) more?
Redis is widely used for caching. Meaning, in a high availability infrastructure, when some data has to be accessed many times, you would store it in your database and then store it in redis with some unique key which you could rebuild easily with parameters. When the data is updated, you just clear that key in redis and add it again with the new data.
Example:
You have thousands of users.
They all connect many many times and go on their profile.
You might want to store their profile info in redis with a key {userid}_user_info.
The user tries to access his profile:
first check if data exists in redis,
if yes return it,
else get it from db and insert it in redis
The user updates his profile info, just refresh the redis value.
etc.
There is also another way redis is used, it's for queuing tasks and synchronising websockets broadcasts across machines. Here is a useful article about it
http://fiznool.com/blog/2016/02/24/building-a-simple-message-queue-with-redis/
As per using redis as a database, well for simple data it can be used, like settings where a simple key/value is enough. For storing complex data, it's a bit of a hassle, specially if you want to use relational functionalities, or searching features. Redis is fast because it does not have all these features, and keeps data in memory (not only, but it does contribute).
I need to gather information when user sees an article. User will browse through 1-30 articles in a minute (maybe even more if user just scrolls through everything looking something specific). I was wondering which way i can keep my server costs at minimum:
At client side javascript i push article id's into an array and send it to server when there is 30-60 id's. At server i loop through all the id's and insert them into database.
Every single time when user sees an article i will send one article id to server. In some cases this can cause over 60 requests in a minute. At server i insert the id into database.
In most of the cases, there is always a trade-off. And a lot of times, the optimal solution lies somewhere in the middle. I feel you should support both and use them interchangeably depending on the situation. Please go through following scenarios:
Will your end-user have bandwidth issues? If yes, it may make sense to go with option 2 or reduce the number of articles to a number such that it can be easily fetched at lower bandwidth as well.
Assuming the user does not has bandwidth issues such that loading of 30-60 articles won't take a lot of time for user, you can go with option 1 and keep using this option for subsequent fetch as well.
A lot of times it will make sense to go with option 1 for initial fetch and then fetch a lower number of articles after that.
Regarding server cost, it will make sense to send 30-60 articles together provided user reads them all. If you feel he won't read them all, find an optimal number using your app's analytics and send those number of articles in one go, provided bandwidth won't be an issue for user.
tl;dr; In data you should trust. Use your intuition, existing app usage patterns, and bandwidth availability of the user to make an informed decision. Also, server cost is not the only thing. Experience matters more, I think.
I'm writing a multi chatroom application that requires persistent storage of the conversations (ie new users should be able to see old messages). I'm using socket.io if that helps.
Currently, when a user logs into a room, my node app checks to see if someone has been to that room yet (and each room has a hierarchical parent, for instance the room called Pets may belong to North America since there'd be a separate European Pets room).
If this is the first time a person has been in the room for a while, it loads all messages from redis for that room. (Eventually redis stored conversations make their way into MySQL).
So I have a multidimensional array called messages["PARENT"]["ROOM"], such that messages["North America"]["Pets"] will be an array that has all the messages for that room. Aside from misunderstanding how arrays in JS work (as explained in this question: javascript push multidimensional array), it feels like I'm over complicating the situation. My reasoning for using the MD array was that it didn't make sense to be round trips requesting all the messages from Redis for a room that was active.
What would be the most logical approach to what I'm trying to accomplish? Should I just be using Redis and forgo this? How about some message queue or maybe a pubsub server? (I'm trying to not complicate this as it's just a prototype for a friend).
Thank you,
From an architectural point of view, this is a poor design. What if tomorrow you wanted to scale this application by setting up more servers? Surely these multi-dimensional arrays would be specific to each node instance. Decoupling the storage has its own advantages, one being scaling out - the storage is now shared among several servers. It all depends on what you want to achieve. You may also run out of memory if your MD increases in size, thus hampering your applications performance.
I'm currently sending data to browsers using socket io. There are many devices sending data to server which is then broadcasted to browsers. The data that is pushed from server is way too frequent like once every second per device and change in data is causing angular digest loop to kick off. This I guess will have a performance impact on mobile devices. I've changed the code to push data at 10 seconds frequency from server once with most recent data for all devices. I'd like to get it more real time, is there any best practices to deal with push model and being real time within angular? I know this is a performance related question without any hard numbers - I'd be happy to run some tests to give you numbers if you want.
Use $evalAsnyc. I pushing more than 100 update per second and there is no performance problem.