Develop Reference

Firestore : why using serverTimestamp gives different results?

I am having a hard time understanding serverTimestamp in firestore.
When I save a document in database in a firebase function using Fieldvalue.serverTimestamp() or in a javascript client code using serverTimestamp() it sometimes doesn't save the same thing in the database.
See screenshots below :
Sometime I get an object with {nanoseconds: xxx, seconds: xxx} and sometimes I get a timestamp formatted date...
The problem is when I try to query my orders using query(collectionRefOrders, orderBy('createdAt', 'desc'), limit(10)).
The orders with the object appears before the others ones even if they are created after...
Any clue why this happens ? What am I doing wrong ?
Thanks a lot.
EDIT :
Here is the code I use to add documents in the my firebase function (it is a request function I call in a website) :
const { getFirestore, FieldValue } = require('firebase-admin/firestore');
const firebaseDB = getFirestore();
exports.createOrderFromTunnel = functions.region('europe-west3')
.runWith({
timeoutSeconds: 10,
memory: "4GB",
})
.https
.onRequest(async (req, res) => {
cors(req, res, async () => {
try {
const { apiKey } = req.body;
const project = await getProjectFromApiKey(apiKey);
if (!project) {
return res.json({
success: false,
error: 'Unauthorized: invalid or missing api key'
});
}
const contactData = {
address: {},
createdAt: FieldValue.serverTimestamp()
};
const orderData = {
accounting: {
totalHT: 0,
totalTTC: 0,
totalTVA: 0,
},
createdAt: FieldValue.serverTimestamp(),
status: 'NEW',
};
const refProject = firebaseDB
.collection('projects')
.doc(project.id);
const colOrder = firebaseDB.collection(`projects/${project.id}/orders`)
const refOrder = colOrder.doc();
const colContact = firebaseDB.collection(`projects/${project.id}/contacts`)
const refContact = colContact.doc();
await firebaseDB.runTransaction(async transaction => {
const snapProject = await transaction.get(refProject);
const dataProject = snapProject.data();
const sequenceContact = dataProject.sequenceContact;
const sequenceOrder = dataProject.sequenceOrder;
contactData.sequence = sequenceContact;
orderData.sequenceNumber = sequenceOrder;
await transaction.set(refContact, contactData);
orderData.customer.id = refContact.id;
orderData.customer.sequence = sequenceContact;
await transaction.set(refOrder, orderData);
await transaction.update(refProject, {
sequenceContact: sequenceContact + 1,
sequenceOrder: sequenceOrder + 1,
totalContacts: dataProject.totalContacts + 1,
totalOrders: dataProject.totalOrders + 1,
});
return refOrder.id;
});
return res.json({
success: true
});
} catch (err) {
functions.logger.error(err);
return res.json({
success: false,
err
});
}
});
});
Here is the code I use to add documents in my client code (it is a web app in javascript) :
const createOrder = async (projectId) => {
try {
const orderData = {
accounting: {
totalHT: 0,
totalTTC: 0,
totalTVA: 0,
},
createdAt: serverTimestamp(),
status: 'NEW',
surface: 0,
};
const refProject = doc(firebaseDB, 'projects', projectId);
const colOrder = collection(firebaseDB, `projects/${projectId}/orders`)
const refOrder = doc(colOrder);
return await runTransaction(firebaseDB, async (transaction) => {
const snapProject = await transaction.get(refProject);
if (!snapProject.exists()) {
throw "Document does not exist!";
}
const dataProject = snapProject.data();
const sequence = dataProject.sequenceOrder;
orderData.sequenceNumber = sequence;
transaction.set(refOrder, orderData);
transaction.update(refProject, { sequenceOrder: sequence + 1, totalOrders: dataProject.totalOrders + 1 });
return refOrder.id;
});
} catch (e) {
console.error(e);
return null;
}
};

How to push data to firestore document?

Basically what I want is that if a document doesn't exist then create a new one (it works fine now) but if a document does exist, push a new object to the existing array.
I was able to get data from documents and console.log them, but don't know how to push new ones to the existing document.
My FB structure looks like this:
favorites
someUserID
Videos [
0: {
name: SomeName
url: SomeUrl
},
/* I would like to push new objects like this: */
1: {
name: data.name
url: data.url
},
]
This is my current code:
const { user } = UserAuth();
const UserID = user.uid;
const favoritesRef = doc(db, "favorites", UserID);
const test = async (data) => {
try {
await runTransaction(db, async (transaction) => {
const sfDoc = await transaction.get(favoritesRef);
if (!sfDoc.exists()) {
setDoc(favoritesRef, {
Videos: [{name: data.name}]
});
}
/* I got my document content here */
const newFavorites = await getDoc(favoritesRef);
console.log("Document data:", newFavorites.data());
/* And would like to push new Data here */
transaction.update(favoritesRef, { name: data.name});
});
console.log("Transaction successfully committed!");
} catch (e) {
console.log("Transaction failed: ", e);
}
}

To update the array Firestore now has a function that allows you to update an array without writing your code again:
Update elements in an array
If your document contains an array field, you can use arrayUnion()
and arrayRemove() to add and remove elements. arrayUnion() adds
elements to an array but only elements not already present.
arrayRemove() removes all instances of each given element.
import { doc, updateDoc, arrayUnion, arrayRemove } from "firebase/firestore";
const washingtonRef = doc(db, "cities", "DC");
// Atomically add a new region to the "regions" array field.
await updateDoc(washingtonRef, {
regions: arrayUnion("greater_virginia")
});
// Atomically remove a region from the "regions" array field.
await updateDoc(washingtonRef, {
regions: arrayRemove("east_coast")
});

Not sure if this helps but what i usually do is:
I'm adding every user that signs in to an array.
const snapshot = await getDoc(doc(db, "allUsers", "list"));
const currentUsers = snapshot.data().users;
await setDoc(doc(db, "allUsers", "list"), {
users: [...currentUsers, { name, uid: userId, avatar: photo }],
});
I first get the items that exist in the list, and them i create a new one that has every previous item and the ones i'm adding. The currentUsers is the current list in that caso. Maybe you should try thist instead of Videos: [{name: data.name}]
setDoc(favoritesRef, {
Videos: [...currentVideos, {name: data.name}]
})

I just figured it out like this:
const { user } = UserAuth();
const UserID = user.uid
const favoritesRef = doc(db, "favorites", UserID)
const test = async (data) => {
try {
await runTransaction(db, async (transaction) => {
const sfDoc = await transaction.get(favoritesRef);
if (!sfDoc.exists()) {
await setDoc(favoritesRef, {
favs: [
{
name: data.name,
ytb: data.ytb,
url: data.url
}]})
}
const doesExists = sfDoc.data().favs.some((fav) => fav.name === data.name)
console.log(doesExists)
if (doesExists === true)
{
console.log("AlreadyExist")
}
else {
const currentData = sfDoc.data().favs
transaction.update(favoritesRef, {
favs: [...currentData,
{
name: data.name,
ytb: data.ytb,
url: data.url
}]}
)}
});
console.log("Transaction successfully committed!");
} catch (e) {
console.log("Transaction failed: ", e);
}
}

How can I update nested array in Firestore?

Is there a way to update the likes and replies with firebase firestore?
I have reached that moment when I press on like on a single comment but I have only the reference to the document of the whole post , so I don't have a reference to the comments itself. I also reached the comment with the find method of the arrays but it's not a reference its the comment itself.
Thanks in advance, I am thinking I just have to reconsider how i structure my DB using firestore?
const addLikeToComment = async (
postID: string,
commentID: string,
userID: string
): Promise<void> => {
const currentPost = await getPostById(postID)
const currentPostDocId = currentPost[0].docID!
const currentPostRef = doc(db, 'posts', currentPostDocId)
const commentToAddLikeTo = currentPost[0].comments.find(
(currComment) => currComment.commentID === commentID
)
console.log(commentToAddLikeTo)
if (!commentToAddLikeTo?.likes.includes(userID)) {
await updateDoc(currentPostRef, {
// add the like to comments - currentComment - likes array inside it push the userID
})
} else {
await updateDoc(currentPostRef, {
// remove the like
})
}
}

const addLikeToComment = async (
postID: string,
commentID: string,
userID: string
): Promise<void> => {
const currentPost = await getPostById(postID)
const currentPostDocId = currentPost[0].docID!
const currentPostRef = doc(db, 'posts', currentPostDocId)
const commentToAddLikeTo = currentPost[0].comments.find(
(currComment) => currComment.commentID === commentID
)
const newLikes = commentToAddLikeTo?.likes
if (!commentToAddLikeTo?.likes.includes(userID)) {
newLikes?.push(userID)
} else {
const indexOfUserID = newLikes?.indexOf(userID)
newLikes?.splice(indexOfUserID!, 1)
}
await updateDoc(currentPostRef, {
...currentPost[0],
likes: newLikes,
})
}
This is how I solved this with some help from a co-worker. Basically I am manipulating the likes array directly and then updating the whole post information.

Counter not increasing in async map function

I am working with mongodb and nodejs. I have an array of customers I have to create each inside database.
const promises2 = customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: "Customer" });
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
}
});
await Promise.all([...promises2]);
The issue is counter is not increasing every time. I am getting same counter in all the created customers. What is the issue here?
Issue is something like this but don't have an answer.

The problem is that all the calls overlap. Since the first thing they each do is get the current counter, they all get the same counter, then try to use it. Fundamentally, you don't want to do this:
const counter = await Counter.findOne({ type: "Customer" });
// ...
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
...because it creates a race condition: overlapping asynchronous operations can both get the same sequence value and then both issue an update to it.
You want an atomic operation for incrementing and retrieving a new ID. I don't use MongoDB, but I think the findOneAndUpdate operation can do that for you if you add the returnNewDocument option. If so, the minimal change would be to swap over to using that:
const promises2 = customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
});
await Promise.all([...promises2]);
...but there's no reason to create an array and then immediately copy it, just use it directly:
await Promise.all(customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
}));
The overall operation will fail if anything fails, and only the first failure is reported back to your code (the other operations then continue and succeed or fail as the case may be). If you want to know everything that happened (which is probably useful in this case), you can use allSettled instead of all:
// Gets an array of {status, value/reason} objects
const results = await Promise.allSettled(customers.map(async customer => {
if (!customer.customerId) {
const counter = await Counter.findOneAndUpdate(
{ type: "Customer" },
{ $inc: { sequence_value: 1 } },
{ returnNewDocument: true }
);
console.log({counter});
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
}
}));
const errors = results.filter(({status}) => status === "rejected").map(({reason}) => reason);
if (errors.length) {
// Handle/report errors here
}
Promise.allSettled is new in ES2021, but easily polyfilled if needed.
If I'm mistaken about the above use of findOneAndUpdate in some way, I'm sure MongoDB gives you a way to get those IDs without a race condition. But in the worst case, you can pre-allocate the IDs instead, something like this:
// Allocate IDs (in series)
const ids = [];
for (const customer of customers) {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: "Customer" });
await Counter.findOneAndUpdate({ type: "Customer" }, { $inc: { sequence_value: 1 } });
ids.push(counter.sequence_value);
}
}
// Create customers (in parallel)
const results = await Promise.allSettled(customers.map(async(customer, index) => {
const customerId = ids[index];
try {
await Customer.create({
customerId
});
} catch (e) {
// Failed, remove the counter, but without allowing any error doing so to
// shadow the error we're already handling
try {
await Counter.someDeleteMethodHere(/*...customerId...*/);
} catch (e2) {
// ...perhaps report `e2` here, but don't shadow `e`
}
throw e;
}
});
// Get just the errors
const errors = results.filter(({status}) => status === "rejected").map(({reason}) => reason);
if (errors.length) {
// Handle/report errors here
}

Your map function is not returning a promise.
Try this :
const promises2 = [];
customers.map((customer) => {
return new Promise(async (resolve) => {
if (!customer.customerId) {
const counter = await Counter.findOne({ type: 'Customer' });
console.log({ counter });
const payload = {
customerId: counter.sequence_value,
};
await Customer.create(payload);
await Counter.findOneAndUpdate({ type: 'Customer' }, { $inc: { sequence_value: 1 } });
}
resolve();
});
});
await Promise.all(promises2);

How to run fetch() in a loop?

I am new to nodejs and promise based request. I want to fetch the data from a remote server in a loop, and then create a JSON object from all fetched data.
const fetch = require('node-fetch');
const users = [];
const ids = await fetch('https://remote-server.com/ids.json');
console.log(ids);
// [1,2,3]
ids.forEach(id => {
var user = await fetch(`https://remote-server.com/user/${id}.json`);
users.push(user);
});
console.log(users);
expected output
[
{
name: 'user 1',
city: 'abc'
},
{
name: 'user 2',
city: 'pqr'
},
{
name: 'user 3',
city: 'xyz'
}
]

So to launch in parallel:
const ids = await fetch('https://remote-server.com/ids.json');
const userPromises = ids.map(id => fetch(`https://remote-server.com/user/${id}.json`));
const users = await Promise.all(userPromises);
to launch in sequence:
const users = [];
const ids = await fetch('https://remote-server.com/ids.json');
for(const id of ids){
const user = await fetch(`https://remote-server.com/user/${id}.json`);
users.push(user);
}

You forgot to add async in the forEach:
ids.forEach(async (id) => { // your promise is in another function now, so you must specify async to use await
var user = await fetch(`https://remote-server.com/user/${id}.json`);
users.push(user);
});