Develop Reference

Newb question: How to maintain a data list for all users in my webapp?

Disclaimer: I'm finding my way and not sure how to ask the question. This is what I want to do:
I am getting data from twitter API in my app (this is working).
I then want to store that data (as an array), and serve it to whichever user is accessing the app (so that I don't need to query the API everytime, just poll every 10 mins).
What do I need to be able to do this? (external database? or can I just save to a file on the server in someway? or something else)
For ref I'm building with sveltekit, and deploying with vercel.

If you are using Twitter's API directly within your own app, every user of your app needs to query the API at least once to get some data. You cannot serve the results that are returned to one user to other users without having your own back-end server and handling this accordingly. However, you can save a copy of the data returned to each user to that user's localStorage so that specific user does not have to query the API every time.
You can save the data on the client's localStorage and save an expiry timestamp that allows you to query the API again after the timestamp has passed.
Here is a tutorial on how to use localStorage with SvelteKit

Sending HTTP messages to a specific client connection/web session using cURL/NodeJs scripts with SSEs

Hi so first I apologize if my query may seem unclear, it’s first trying to do what I’m doing and I haven’t full idea around the intricacies and lingo lol.
So basically I’m running a NodeJs web server with React handling my front end. I’ve got Express to help manipulate user sessions and I just came by Server-Sent-Events as a way to send one-way messages(which is what I need to do). So far I’m able to send updates and messages via cURL on the terminal and running JS scripts, however these updates/messages go to every active client session but I want/need to be able to send these messages to specific active client sessions/connections.
Example: 5 client connections are established (session IDs A,B,C,D,E), now I want to send an alert message to session E only and manually.
I’m still green with NodeJs/Express and the concept of SSEs however I’m learning as I go for this pet project.
Send help

What you want is how SSE works. It is a dedicated connection between a client and a server process.
however these updates/messages go to every active client session
If that is what you see then your node script is running the exact same code for each client.
I think your question might be higher up - how to organize the data messaging? That is too big a topic for a single StackOverflow question, because it will depend on so many factors specific to your use case.
But one way would be to have an SQL database, with one record for each user. The node script polls that database table and if the record for the current user changes, it sends the new data to them. Then to send data to user E, you just edit the database record for user E.

How do I push mongodb data?

So I have developed a web app as a hobby on Handlebars, Express and Mongoose/MongoDB.
The app let's users create an account and then post advertisements for other users to see and respond to.
All the ads posted by users show up on the index page. So it is common view for all the users on this web app. I am relatively new to web development so to build such a simple app it took me a while but boy I learned a lot!
Now the issue I am facing is, when a user A posts an Ad while the user B is logged in and is currently on the index page (a page that lists all the ads posted) it won't show up for user B unless user B refreshes the page. Rightly so actually because only when the index page's route is hit it will query all the ads and refreshing is basically hitting the index route I get that. But I don't want it that way. I want it to show the new ad on user B's index and pretty much every user's index if there's new ad by any user.
So I did a little research/reading and I learned that I can do it by learning to work with triggers on mongodb and like create some kind of trigger that when a new ad is posted do something. I like the idea but failed to find resources to learn how to use such a thing.
The other option I was suggested was to use socket.io but that too I can't grasp how can I make an entire Ad document work as a socket. I am lost and implementing this feature of dynamically loading ads for all users will complete this hobby project of mine and will help me find a junior dev job in local community.
I request stackoverflow's community to guide me how do I go about doing this and what resources I can use to learn about it.

The socket.io seems to be the best solution for your case. What you will want to do with socket.io is every time a user posts an Ad you use socket.io to notify the rest of the users that there is an update.
If you don't want to send the entire document using the socket you can use the socket to notify the clients and on the client side every time you receive such a notice from the server you will either
a) Refresh the page(not suggested as it will make the user experience unpleasant) which is easier to implement
OR
b) You can use an Ajax request to get the new data from your server and update the fields on the fly(which makes for a better user experience).

Best Way You can come with using Short Polling concept from client side to ask for new data after 1 or 2 seconds (whatsoever count to need ) . Gmail for new inbox mails also uses sync method in a particular fashion . Just ask from server for new data
OR Second option to go through below
On Server Side
Serve index.html page to User A (which is logged in now).Some User B inserts data
Maintain a function or a cron job (checks the count of Total Ads ) Lets say after every 1 minute or so
If there is change in count from the previous total_count , update it and get new mlab documents and send it to function , Let's say push_new_ads which will be sync via socket.io to client
On Client Side
Sync your client_total_count with server_local_count push_new_ads using socket.io and If there is change in count , make a simple fetch api call to get data and appends it to previously fetched array
There is no such way to directly listen the changes in mongodb But you can trigger some changes from oplog using tailable cursors

Best practice for storing data in web chat application

I'm building web chat application using Code Igniter. In this chat app, there is only one channel / room. I'd like to know what's the best practice to store data whether using database or file in order to save bandwith and page load.
p.s : I use javascript setInterval to load chat div every x seconds.

if you want the file you could use reverse-ajax/comet for it is faster and take lesser bandwidth cause it uses long pooling
if you use database and use ajax it is slower because it always updates for new chat message in your database get it from there take the rows and displays it which takes time and I think more bandwidth

How to pull new post and comments

I am currently developing a website where many users post topics and the associated topics' comments are shown in the page. Currently I'm developing using cake php.
The very first time a user clicks the website, all the topics and comments are displayed. But when other users add new topic or comment to a topic, I need to show the update in the same page. I am confused as in how am I able to retrieve new contents and update accordingly in a page. For instance how facebook does it where when your friends adds status or comments on your status, it updates without refreshing the page.
I know that AJAX technology is used but how is it done. Any source that I can refer? Hope someone can help as I have been doing research for the past one week but no answers so far.

You can go two routes in this.
Server Push
http://en.wikipedia.org/wiki/Push_technology
This technique is perhaps the most efficient as the server notifies the client of any updates. However this technique usually requires a bit more work than a simple polling system. You can use something like nodejs or Comet to push updates. If you're using nodejs, I highly recommend using SocketIO to handle the client side. With Socket.io you can have the client side listening to the server on a channel so that the server can notify the client whenever an update happens.
Client polls server
In this version, the client (new visitors browser) constantly polls the server for updates. You can set whatever gap you want, but keep in mind that if you make the polling gap too small your server might take a performance hit as each new user will create many requests. This method is as simple as setting up a setInterval() call in JS coupled with an AJAX call.