Develop Reference

Form handling using cache-and-network policy, how can I handle this case?

I'm using urql and Svelte for a new web app.
I ran into a logical problem that I can't figure out how to best solve.
Suppose I have many todos and the list of these todos is already cached on my phone browser.
While I'm on the subway (very slow network) I open one to edit it (then the app loads the edit form with the todo from the cache) but since I use the "cache-and-network" policy and shortly before my wife changed it the todo re-updates itself undoing all my changes that I was writing in the textarea.
How do you think this situation can be fixed?
Should I wait for the cache-and-network to finish before I can edit the form? (This is not great for a maybe-offline UX);
should I warn the user that there is an updated todo?
should I warn the user that there is an updated todo only when I'm saving it in the DB in the backend?
other suggestions?

You can use navigator.onLine to check if your user is online or offline. if he is offline, save his changes to localStorage. when the status changes again; you can compare your current state to localStorage and update the form accordingly.
most of the logic I'm talking about is explained here
if you need help with saving a form in localStorage, you can use this template:
[].forEach.call(document.querySelector('#form-id').elements, function(el) {
localStorage.setItem(el.name, el.value);
});
// then refresh the page and run this to restore your form values:
[].forEach.call(document.querySelector('#form-id').elements, function(el) {
el.value = localStorage.getItem(el.name);
});

You describe a classic race condition where 2 users create a different operation over the same resource (in our use-case - a record in db)
Short answer: record versioning - manage a unique version number for each record. changes can only be executed when deliver with the same version value
Long answer: Suppose we are using a no-sql db where every list is saved in lists collection where every record looks something like this:
{
id: 'uuid-4',
items: [
{ name: 'item #1', checked: false },
{ name: 'item #2', checked: true },
{ name: 'item #3', checked: false }
],
version: 1648727895753
}
In the example above, I've use a timestamp as a record version (using the new Date().getTime() method). Every time this record is updated - the version changes as well.
Now - let's describe your scenario again:
2 users downloading the same todo list from server. At this point in time - both users have the same version value (let's say, version A). Now, let's assume that one user made an update while the other user is offline (in your example - due to traveling). In order for this pattern to work, we change the version value after each successful update so now the record has a new version value (let's say, version B). When the other user will try to make a change - it will be blocked because he still got the old version number (in our example A). In order to make a change - the newest version (B) should be downloaded first, then the user will update his own edits and write his changes to db (this process will result with an updated record with version C)
So in order to make this work, one should create the following mechanism:
Server Side (db logic)
on record update
make update only if request version value equals to record version value
on fetch record
return record with current version value
Client side (app logic)
on app load
fetch current todo list record (by id field)
on record update
send update to server with current version value
if update fails:
save current todo list record as old version
fetch current todo list record (by id field)
merge changes (you will decide how to reflect it to user)
send update to server with current version value (it's now the most updated one)
Technical Notes
Merging between 2 record versions (e.g. - between to JSON-based object) is a whole universe by its own. You may use a 3rd-party library to spot the differences (such as json-diff) or decide to apply a default behavior (for instance: if an item from list was deleted in db it will have a different style apply so the user understand that this item was removed)
Even now we still risk to have a race condition (well.. highly unlikely when using a simple todo list but may occur in application with high loads). For production ready setup - please consider using a message queue in order to apply all actions by a specific order (first in first out). Again - it's an overkill to your use-case but since we are solving here a more general problem - I think it's worth mentioning

Is there anyway i can update a firestore document automatically even if app is closed, is there a listeener or something

i have a firestore and project that needs to be updated automatically without user interaction but i do not know how to go about it, any help would be appreciated. take a look at the json to understand better
const party = {
id: 'bgvrfhbhgnhs',
isPrivate: 'true',
isStarted: false,
created_At: '2021-12-26T05:20:29.000Z',
start_date: '2021-12-26T02:00:56.000Z'
}
I want to update the isStarted field to true once the current time is equal to start_date

I think you will need Firebase Cloud Function, although I don't understand exactly what you mean.
With Cloud Functions, you can automatically run (add, delete, update, everything) codes on Google's Servers without the need for application and user interaction.
For example, in accordance with your example, it can automatically set "isStarted" to true when it hits the "start_date" time. If you want to code a system that does not require user interaction and should work automatically, you should definitely use Cloud Functions. Otherwise, you cannot do this on the application side.
For more info visit Cloud Functions

Ok, I managed to find a workaround to updating my documents automatically without user interaction since the google billing service won’t accept my card to enable cloud functions for my project, I tried what I could to make my code work and I don’t know if other peeps would follow my idea or if my idea would solve similar issues.
What I did was that in my nextjs file I created an API endpoint to fetch and update documents after installing the firebase admin SDK, so I fetched all documents and converted all start_date fields in each document to time and checked for documents whose start date is less than or equal to current date, so after getting the document, I ran a firestore function to Update the document.
Tho this will only run when you make a request to my domain.com/api/update-parties and never run again
In other to make it run at scheduled intervals, I signed up for a free tier account at https://www.easycron.com and added my API endpoint to EASYCRON to make requests to my endpoint at a minute interval, so when the request hits my endpoint, it runs my code like other serverless functions😜. Easy peezy.

Best way to have a Node.JS server keep updated with a FireBase database in real time?

I currently have a Node.JS server set up that is able to read and write data from a FireBase database when a request is made from a user.
I would like to implement time based events that result in an action being performed at a certain date or time. The key thing here though, is that I want to have the freedom to do this in seconds (for example, write a message to console after 30 seconds have passed, or on Friday the 13th at 11:30am).
A way to do this would be to store the date/time an action needs be performed in the database, and read from the database every second and compare the current date/time with events stored so we know if an action needs to be performed at this moment. As you can imagine though, this would be a lot of unnecessary calls to the database and really feels like a poor way to implement this system.
Is there a way I can stay synced with the database without having to call every second? Perhaps I could then store a version of the events table locally and update this when a change is made to the database? Would that be a better idea? Is there another solution I am missing?
Any advice would be greatly appreciated, thanks!
EDIT:
How I currently initialise the database:
firebase.initializeApp(firebaseConfig);
var database = firebase.database();
How I then get data from the database:
await database.ref('/').once('value', function(snapshot){
snapshot.forEach(function(childSnapshot){
if(childSnapshot.key === userName){
userPreferences = childSnapshot.val().UserPreferences;
}
})
});

The Firebase once() API reads the data from the database once, and then stops observing it.
If you instead us the on() API, it will continue observing the database after getting the initial value - and call your code whenever the database changes.

It sounds like you're looking to develop an application for scheduling. If that's the case you should check out node-schedule.
Node Schedule is a flexible cron-like and not-cron-like job scheduler
for Node.js. It allows you to schedule jobs (arbitrary functions) for
execution at specific dates, with optional recurrence rules. It only
uses a single timer at any given time (rather than reevaluating
upcoming jobs every second/minute).
You then can use the database to keep a "state" of the application so on start-up of the application you read all the upcoming jobs that will be expected and load them into node-schedule and let node-schedule do the rest.

The Google Cloud solution for scheduling a single item of future work is Cloud Tasks. Firebase is part of Google Cloud, so this is the most natural product to use. You can use this to avoid polling the database by simply specifying exactly when some Cloud Function should run to do the work you want.
I've written a blog post that demonstrates how to set up a Cloud Task to call a Cloud Functions to delete a document in Firestore with an exact TTL.

How to batch create users in Parse via REST api

I have roughly 1.5 million users I need to import into my parse _User table (we are migrating from our old sql db to Parse).
I've tried using a Background Job, but it is slow and gets killed every 15 minutes. As a result, we have only import ~300K users in 4 days.
So I decided to go the REST api route...and tried the PythonPy lib. It works, except for being able to do batch creation of users. From digging more into this, it seems that the REST api forces you to do a User.signup() call.
This of course makes the whole thing be serialized.
Is there anyway to do batch user creation?

Batch creating / signing up User objects is not supported in Parse. It is mentioned in this question:
https://parse.com/questions/what-user-operations-are-supported-in-batch

Update value in DB every five minutes

I am building a webapp where user have a ranking based on their twitter activity and their activity on my website.
Therefore I'd like to update their rank every five minutes, pulling their latest activity from twitter and update it in my database. I was thinking of using something like this:
var minutes = 5, the_interval = minutes * 60 * 1000;
setInterval(function() {
// my update here
}, the_interval);
However, I have several questions about this code:
where should I save it to make sure it is run?
will it slow my program or is it a problem to pull data out of twitter every five minute? Should I use their streaming API instead?
Note: I am using mongoDB

I'd suggest that you create a scheduled task/chron job/etc. (depends on your host OS) to call a separate Node.JS application that performs the specific tasks you want to do periodically and then it would exit when complete. (Or you could use a ChildProcess potentially as well).
While Node.JS is async, there's no need, given the description you provided, to perform this work within the same application process that is serving a web application. In fact, as it sounds like "busy work", it would be best handled by a distinct process to avoid impacting directly any of your interactive web users.

The placement shouldn't really matter as Node is asynchronous.