I'd like to perform a batch update using Knex.js
For example:
'UPDATE foo SET [theValues] WHERE idFoo = 1'
'UPDATE foo SET [theValues] WHERE idFoo = 2'
with values:
{ name: "FooName1", checked: true } // to `idFoo = 1`
{ name: "FooName2", checked: false } // to `idFoo = 2`
I was using node-mysql previously, which allowed multiple-statements. While using that I simply built a mulitple-statement query string and just send that through the wire in a single run.
I'm not sure how to achieve the same with Knex. I can see batchInsert as an API method I can use, but nothing as far as batchUpdate is concerned.
Note:
I can do an async iteration and update each row separately. That's bad cause it means there's gonna be lots of roundtrips from the server to the DB
I can use the raw() thing of Knex and probably do something similar to what I do with node-mysql. However that defeats the whole knex purpose of being a DB abstraction layer (It introduces strong DB coupling)
So I'd like to do this using something "knex-y".
Any ideas welcome.
I needed to perform a batch update inside a transaction (I didn't want to have partial updates in case something went wrong).
I've resolved it the next way:
// I wrap knex as 'connection'
return connection.transaction(trx => {
const queries = [];
users.forEach(user => {
const query = connection('users')
.where('id', user.id)
.update({
lastActivity: user.lastActivity,
points: user.points,
})
.transacting(trx); // This makes every update be in the same transaction
queries.push(query);
});
Promise.all(queries) // Once every query is written
.then(trx.commit) // We try to execute all of them
.catch(trx.rollback); // And rollback in case any of them goes wrong
});
Assuming you have a collection of valid keys/values for the given table:
// abstract transactional batch update
function batchUpdate(table, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(table)
.where('id', tuple.id)
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate('user', [...]);
Are you unfortunately subject to non-conventional column names? No worries, I got you fam:
function batchUpdate(options, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(options.table)
.where(options.column, tuple[options.column])
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate({ table: 'user', column: 'user_id' }, [...]);
Modern Syntax Version:
const batchUpdate = (options, collection) => {
const { table, column } = options;
const trx = await knex.transaction();
try {
await Promise.all(collection.map(tuple =>
knex(table)
.where(column, tuple[column])
.update(tuple)
.transacting(trx)
)
);
await trx.commit();
} catch (error) {
await trx.rollback();
}
}
You have a good idea of the pros and cons of each approach. I would recommend a raw query that bulk updates over several async updates. Yes you can run them in parallel, but your bottleneck becomes the time it takes for the db to run each update. Details can be found here.
Below is an example of an batch upsert using knex.raw. Assume that records is an array of objects (one obj for each row we want to update) whose values are the properties names line up with the columns in the database you want to update:
var knex = require('knex'),
_ = require('underscore');
function bulkUpdate (records) {
var updateQuery = [
'INSERT INTO mytable (primaryKeyCol, col2, colN) VALUES',
_.map(records, () => '(?)').join(','),
'ON DUPLICATE KEY UPDATE',
'col2 = VALUES(col2),',
'colN = VALUES(colN)'
].join(' '),
vals = [];
_(records).map(record => {
vals.push(_(record).values());
});
return knex.raw(updateQuery, vals);
}
This answer does a great job explaining the runtime relationship between the two approaches.
Edit:
It was requested that I show what records would look like in this example.
var records = [
{ primaryKeyCol: 123, col2: 'foo', colN: 'bar' },
{ // some other record, same props }
];
Please note that if your record has additional properties than the ones you specified in the query, you cannot do:
_(records).map(record => {
vals.push(_(record).values());
});
Because you will hand too many values to the query per record and knex will fail to match the property values of each record with the ? characters in the query. You instead will need to explicitly push the values on each record that you want to insert into an array like so:
// assume a record has additional property `type` that you dont want to
// insert into the database
// example: { primaryKeyCol: 123, col2: 'foo', colN: 'bar', type: 'baz' }
_(records).map(record => {
vals.push(record.primaryKeyCol);
vals.push(record.col2);
vals.push(record.colN);
});
There are less repetitive ways of doing the above explicit references, but this is just an example. Hope this helps!
The solution works great for me! I just include an ID parameter to make it dynamic across tables with custom ID tags. Chenhai, here's my snippet including a way to return a single array of ID values for the transaction:
function batchUpdate(table, id, collection) {
return knex.transaction((trx) => {
const queries = collection.map(async (tuple) => {
const [tupleId] = await knex(table)
.where(`${id}`, tuple[id])
.update(tuple)
.transacting(trx)
.returning(id);
return tupleId;
});
return Promise.all(queries).then(trx.commit).catch(trx.rollback);
});
}
You can use
response = await batchUpdate("table_name", "custom_table_id", [array of rows to update])
to get the returned array of IDs.
The update can be done in batches, i.e 1000 rows in a batch
And as long as it does it in batches, the bluebird map could be used.
For more information on bluebird map: http://bluebirdjs.com/docs/api/promise.map.html
const limit = 1000;
const totalRows = 50000;
const seq = count => Array(Math.ceil(count / limit)).keys();
map(seq(totalRows), page => updateTable(dbTable, page), { concurrency: 1 });
const updateTable = async (dbTable, page) => {
let offset = limit* page;
return knex(dbTable).pluck('id').limit(limit).offset(offset).then(ids => {
return knex(dbTable)
.whereIn('id', ids)
.update({ date: new Date() })
.then((rows) => {
console.log(`${page} - Updated rows of the table ${dbTable} from ${offset} to ${offset + batch}: `, rows);
})
.catch((err) => {
console.log({ err });
});
})
.catch((err) => {
console.log({ err });
});
};
Where pluck() is used to get ids in array form
Related
I am developping an app to order food online. As backend service I am using firestore to store the data and files. The user can order dishes and there are limited stocks. So every time a user order a dish and create a basket I update the stock of the corresponding ordered dishes. I am using a firebase function in order to perform this action. To be honest it is the first I am creating firebase function.
Into the Basket object, there is a list of ordered Dishes with the corresponding database DishID. When the basket is created, I go through the DishID list and I update the Quantity in the firestore database. On my local emulator it works perfectly and very fast. But online it takes minutes to perform the first update. I can deal with some seconds. Even if it takes a few seconds (like for cold restart) it's okay. But sometimes it can take 3 minutes and someone else can order a dish during this time.
Here is my code:
//Update the dishes quantities when a basket is created
exports.updateDishesQuantity = functions.firestore.document('/Baskets/{documentId}').onCreate(async (snap, context) => {
try{
//Get the created basket
const originalBasket = snap.data();
originalBasket.OrderedDishes.forEach(async dish => {
const doc = await db.collection('Dishes').doc(dish.DishID);
console.log('Doc created');
return docRef = doc.get()
.then((result) =>{
console.log('DocRef created');
if(result.exists){
console.log('Result exists');
const dishAvailableOnDataBase = result.data().Available;
console.log('Data created');
const newQuantity = { Available: Math.max(dishAvailableOnDataBase - dish.Quantity, 0)};
console.log('Online doc updated');
return result.ref.set(newQuantity, { merge: true });
}else{
console.log("doc doesnt exist");
}
})
.catch(error =>{
console.log(error);
return null;
});
});
}catch(error){
console.log(error);
}
});
I have a couple of logs output to debug the outputs on the server. It's the doc.get() function that takes 2 minutes to execute as you can see on the logger below:
Firebase logger
Thanks for your help,
Thansk for your help. I just edited a little bit your code to make it work. I post my edited code. Thanks a lot, now it takes just 4 seconds to update the quantities.
Kid regards
//Update the dishes quantities when a basket is created
exports.updateDishesQuantity = functions.firestore.document('/Baskets/{documentId}').onCreate(async (snap, context) => {
try {
//Get the created basket
const originalBasket = snap.data();
const promises = [];
const quantities = [];
originalBasket.OrderedDishes.forEach(dish => {
promises.push(db.collection('Dishes').doc(dish.DishID).get());
quantities.push(dish.Quantity);
});
const docSnapshotsArray = await Promise.all(promises);
console.log("Promises", promises);
const promises1 = [];
var i = 0;
docSnapshotsArray.forEach(result => {
if (result.exists) {
const dishAvailableOnDataBase = result.data().Available;
const newQuantity = { Available: Math.max(dishAvailableOnDataBase - quantities[i], 0) };
promises1.push(result.ref.set(newQuantity, { merge: true }));
}
i++;
})
return Promise.all(promises1)
} catch (error) {
console.log(error);
return null;
}
});
You should not use async/await within a forEach() loop, see "JavaScript: async/await with forEach()" and "Using async/await with a forEach loop".
And since your code executes, in parallel, a variable number of calls to the asynchronous Firebase get() and set() methods, you should use Promise.all().
You should refactor your Cloud Function along the following lines:
//Update the dishes quantities when a basket is created
exports.updateDishesQuantity = functions.firestore.document('/Baskets/{documentId}').onCreate(async (snap, context) => {
try {
//Get the created basket
const originalBasket = snap.data();
const promises = [];
originalBasket.OrderedDishes.forEach(dish => {
promises.push(db.collection('Dishes').doc(dish.DishID).get());
});
const docSnapshotsArray = await Promise.all(promises);
const promises1 = [];
docSnapshotsArray.forEach(snap => {
if (result.exists) {
const dishAvailableOnDataBase = result.data().Available;
const newQuantity = { Available: Math.max(dishAvailableOnDataBase - dish.Quantity, 0) };
promises1.push(result.ref.set(newQuantity, { merge: true }));
}
})
return Promise.all(promises1)
} catch (error) {
console.log(error);
return null;
}
});
Note that instead of looping and calling push() you could use the map() method for a much concise code. However, for SO answers, I like the clarity brought by creating an empty array, populating it with a forEach() loop and passing it to Promise.all()...
Also note that since you are updating quantities in a basket you may need to use a Transaction.
I have a Cloud Firestore DB with the following structure:
users
[uid]
name: "Test User"
posts
[id]
content: "Just some test post."
timestamp: (Dec. 22, 2017)
uid: [uid]
There is more data present in the actual DB, the above just illustrates the collection/document/field structure.
I have a view in my web app where I'm displaying posts and would like to display the name of the user who posted. I'm using the below query to fetch the posts:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
const postDocs = docSnaps.docs;
for (let i in postDocs) {
loadedPosts[postDocs[i].id] = postDocs[i].data();
}
});
// Render loadedPosts later
What I want to do is query the user object by the uid stored in the post's uid field, and add the user's name field into the corresponding loadedPosts object. If I was only loading one post at a time this would be no problem, just wait for the query to come back with an object and in the .then() function make another query to the user document, and so on.
However because I'm getting multiple post documents at once, I'm having a hard time figuring out how to map the correct user to the correct post after calling .get() on each post's user/[uid] document due to the asynchronous way they return.
Can anyone think of an elegant solution to this issue?
It seems fairly simple to me:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
db.collection('users').child(doc.data().uid).get().then((userDoc) => {
loadedPosts[doc.id].userName = userDoc.data().name;
});
})
});
If you want to prevent loading a user multiple times, you can cache the user data client side. In that case I'd recommend factoring the user-loading code into a helper function. But it'll be a variation of the above.
I would do 1 user doc call and the needed posts call.
let users = {} ;
let loadedPosts = {};
db.collection('users').get().then((results) => {
results.forEach((doc) => {
users[doc.id] = doc.data();
});
posts = db.collection('posts').orderBy('timestamp', 'desc').limit(3);
posts.get().then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
loadedPosts[doc.id].userName = users[doc.data().uid].name;
});
});
After trying multiple solution I get it done with RXJS combineLatest, take operator. Using map function we can combine result.
Might not be an optimum solution but here its solve your problem.
combineLatest(
this.firestore.collection('Collection1').snapshotChanges(),
this.firestore.collection('Collection2').snapshotChanges(),
//In collection 2 we have document with reference id of collection 1
)
.pipe(
take(1),
).subscribe(
([dataFromCollection1, dataFromCollection2]) => {
this.dataofCollection1 = dataFromCollection1.map((data) => {
return {
id: data.payload.doc.id,
...data.payload.doc.data() as {},
}
as IdataFromCollection1;
});
this.dataofCollection2 = dataFromCollection2.map((data2) => {
return {
id: data2.payload.doc.id,
...data2.payload.doc.data() as {},
}
as IdataFromCollection2;
});
console.log(this.dataofCollection2, 'all feeess');
const mergeDataFromCollection =
this.dataofCollection1.map(itm => ({
payment: [this.dataofCollection2.find((item) => (item.RefId === itm.id))],
...itm
}))
console.log(mergeDataFromCollection, 'all data');
},
my solution as below.
Concept: You know user id you want to get information, in your posts list, you can request user document and save it as promise in your post item. after promise resolve then you get user information.
Note: i do not test below code, but it is simplify version of my code.
let posts: Observable<{}[]>; // you can display in HTML directly with | async tag
this.posts = this.listenPosts()
.map( posts => {
posts.forEach( post => {
post.promise = this.getUserDoc( post.uid )
.then( (doc: DocumentSnapshot) => {
post.userName = doc.data().name;
});
}); // end forEach
return posts;
});
// normally, i keep in provider
listenPosts(): Observable<any> {
let fsPath = 'posts';
return this.afDb.collection( fsPath ).valueChanges();
}
// to get the document according the user uid
getUserDoc( uid: string ): Promise<any> {
let fsPath = 'users/' + uid;
return this.afDb.doc( fsPath ).ref.get();
}
Note: afDb: AngularFirestore it is initialize in constructor (by angularFire lib)
If you want to join observables instead of promises, use combineLatest. Here is an example joining a user document to a post document:
getPosts(): Observable<Post[]> {
let data: any;
return this.afs.collection<Post>('posts').valueChanges().pipe(
switchMap((r: any[]) => {
data = r;
const docs = r.map(
(d: any) => this.afs.doc<any>(`users/${d.user}`).valueChanges()
);
return combineLatest(docs).pipe(
map((arr: any) => arr.reduce((acc: any, cur: any) => [acc].concat(cur)))
);
}),
map((d: any) => {
let i = 0;
return d.map(
(doc: any) => {
const t = { ...data[i], user: doc };
++i;
return t;
}
);
})
);
}
This example joins each document in a collection, but you could simplify this if you wanted to just join one single document to another.
This assumes your post document has a user variable with the userId of the document.
J
I have a Cloud Firestore DB with the following structure:
users
[uid]
name: "Test User"
posts
[id]
content: "Just some test post."
timestamp: (Dec. 22, 2017)
uid: [uid]
There is more data present in the actual DB, the above just illustrates the collection/document/field structure.
I have a view in my web app where I'm displaying posts and would like to display the name of the user who posted. I'm using the below query to fetch the posts:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
const postDocs = docSnaps.docs;
for (let i in postDocs) {
loadedPosts[postDocs[i].id] = postDocs[i].data();
}
});
// Render loadedPosts later
What I want to do is query the user object by the uid stored in the post's uid field, and add the user's name field into the corresponding loadedPosts object. If I was only loading one post at a time this would be no problem, just wait for the query to come back with an object and in the .then() function make another query to the user document, and so on.
However because I'm getting multiple post documents at once, I'm having a hard time figuring out how to map the correct user to the correct post after calling .get() on each post's user/[uid] document due to the asynchronous way they return.
Can anyone think of an elegant solution to this issue?
It seems fairly simple to me:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
db.collection('users').child(doc.data().uid).get().then((userDoc) => {
loadedPosts[doc.id].userName = userDoc.data().name;
});
})
});
If you want to prevent loading a user multiple times, you can cache the user data client side. In that case I'd recommend factoring the user-loading code into a helper function. But it'll be a variation of the above.
I would do 1 user doc call and the needed posts call.
let users = {} ;
let loadedPosts = {};
db.collection('users').get().then((results) => {
results.forEach((doc) => {
users[doc.id] = doc.data();
});
posts = db.collection('posts').orderBy('timestamp', 'desc').limit(3);
posts.get().then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
loadedPosts[doc.id].userName = users[doc.data().uid].name;
});
});
After trying multiple solution I get it done with RXJS combineLatest, take operator. Using map function we can combine result.
Might not be an optimum solution but here its solve your problem.
combineLatest(
this.firestore.collection('Collection1').snapshotChanges(),
this.firestore.collection('Collection2').snapshotChanges(),
//In collection 2 we have document with reference id of collection 1
)
.pipe(
take(1),
).subscribe(
([dataFromCollection1, dataFromCollection2]) => {
this.dataofCollection1 = dataFromCollection1.map((data) => {
return {
id: data.payload.doc.id,
...data.payload.doc.data() as {},
}
as IdataFromCollection1;
});
this.dataofCollection2 = dataFromCollection2.map((data2) => {
return {
id: data2.payload.doc.id,
...data2.payload.doc.data() as {},
}
as IdataFromCollection2;
});
console.log(this.dataofCollection2, 'all feeess');
const mergeDataFromCollection =
this.dataofCollection1.map(itm => ({
payment: [this.dataofCollection2.find((item) => (item.RefId === itm.id))],
...itm
}))
console.log(mergeDataFromCollection, 'all data');
},
my solution as below.
Concept: You know user id you want to get information, in your posts list, you can request user document and save it as promise in your post item. after promise resolve then you get user information.
Note: i do not test below code, but it is simplify version of my code.
let posts: Observable<{}[]>; // you can display in HTML directly with | async tag
this.posts = this.listenPosts()
.map( posts => {
posts.forEach( post => {
post.promise = this.getUserDoc( post.uid )
.then( (doc: DocumentSnapshot) => {
post.userName = doc.data().name;
});
}); // end forEach
return posts;
});
// normally, i keep in provider
listenPosts(): Observable<any> {
let fsPath = 'posts';
return this.afDb.collection( fsPath ).valueChanges();
}
// to get the document according the user uid
getUserDoc( uid: string ): Promise<any> {
let fsPath = 'users/' + uid;
return this.afDb.doc( fsPath ).ref.get();
}
Note: afDb: AngularFirestore it is initialize in constructor (by angularFire lib)
If you want to join observables instead of promises, use combineLatest. Here is an example joining a user document to a post document:
getPosts(): Observable<Post[]> {
let data: any;
return this.afs.collection<Post>('posts').valueChanges().pipe(
switchMap((r: any[]) => {
data = r;
const docs = r.map(
(d: any) => this.afs.doc<any>(`users/${d.user}`).valueChanges()
);
return combineLatest(docs).pipe(
map((arr: any) => arr.reduce((acc: any, cur: any) => [acc].concat(cur)))
);
}),
map((d: any) => {
let i = 0;
return d.map(
(doc: any) => {
const t = { ...data[i], user: doc };
++i;
return t;
}
);
})
);
}
This example joins each document in a collection, but you could simplify this if you wanted to just join one single document to another.
This assumes your post document has a user variable with the userId of the document.
J
Im trying to figure this out.
I want to get all my users from my database, cache them
and then when making a new request I want to get those that Ive cached + new ones that have been created.
So far:
const batchUsers = async ({ user }) => {
const users = await user.findAll({});
return users;
};
const apolloServer = new ApolloServer({
schema,
playground: true,
context: {
userLoader: new DataLoader(() => batchUsers(db)),// not sending keys since Im after all users
},
});
my resolver:
users: async (obj, args, context, info) => {
return context.userLoader.load();
}
load method requiers a parameter but in this case I dont want to have a specific user I want all of them.
I dont understand how to implement this can someone please explain.
If you're trying to just load all records, then there's not much of a point in utilizing DataLoader to begin in. The purpose behind DataLoader is to batch multiple calls like load(7) and load(22) into a single call that's then executed against your data source. If you need to get all users, then you should just call user.findAll directly.
Also, if you do end up using DataLoader, make sure you pass in a function, not an object as your context. The function will be ran on each request, which will ensure you're using a fresh instance of DataLoader instead of one with a stale cache.
context: () => ({
userLoader: new DataLoader(async (ids) => {
const users = await User.findAll({
where: { id: ids }
})
// Note that we need to map over the original ids instead of
// just returning the results of User.findAll because the
// length of the returned array needs to match the length of the ids
return ids.map(id => users.find(user => user.id === id) || null)
}),
}),
Note that you could also return an instance of an error instead of null inside the array if you want load to reject.
Took me a while but I got this working:
const batchUsers = async (keys, { user }) => {
const users = await user.findAll({
raw: true,
where: {
Id: {
// #ts-ignore
// eslint-disable-next-line no-undef
[op.in]: keys,
},
},
});
const gs = _.groupBy(users, 'Id');
return keys.map(k => gs[k] || []);
};
const apolloServer = new ApolloServer({
schema,
playground: true,
context: () => ({
userLoader: new DataLoader(keys => batchUsers(keys, db)),
}),
});
resolver:
user: {
myUsers: ({ Id }, args, { userLoader }) => {
return userLoader.load(Id);
},
},
playground:
{users
{Id
myUsers
{Id}}
}
playground explained:
users basically fetches all users and then myusers does the same thing by inhereting the id from the first call.
I think I choose a horrible example here since I did not see any gains in performence by this. I did see however that the query turned into:
SELECT ... FROM User WhERE ID IN(...)
I have a Cloud Firestore DB with the following structure:
users
[uid]
name: "Test User"
posts
[id]
content: "Just some test post."
timestamp: (Dec. 22, 2017)
uid: [uid]
There is more data present in the actual DB, the above just illustrates the collection/document/field structure.
I have a view in my web app where I'm displaying posts and would like to display the name of the user who posted. I'm using the below query to fetch the posts:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
const postDocs = docSnaps.docs;
for (let i in postDocs) {
loadedPosts[postDocs[i].id] = postDocs[i].data();
}
});
// Render loadedPosts later
What I want to do is query the user object by the uid stored in the post's uid field, and add the user's name field into the corresponding loadedPosts object. If I was only loading one post at a time this would be no problem, just wait for the query to come back with an object and in the .then() function make another query to the user document, and so on.
However because I'm getting multiple post documents at once, I'm having a hard time figuring out how to map the correct user to the correct post after calling .get() on each post's user/[uid] document due to the asynchronous way they return.
Can anyone think of an elegant solution to this issue?
It seems fairly simple to me:
let loadedPosts = {};
posts = db.collection('posts')
.orderBy('timestamp', 'desc')
.limit(3);
posts.get()
.then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
db.collection('users').child(doc.data().uid).get().then((userDoc) => {
loadedPosts[doc.id].userName = userDoc.data().name;
});
})
});
If you want to prevent loading a user multiple times, you can cache the user data client side. In that case I'd recommend factoring the user-loading code into a helper function. But it'll be a variation of the above.
I would do 1 user doc call and the needed posts call.
let users = {} ;
let loadedPosts = {};
db.collection('users').get().then((results) => {
results.forEach((doc) => {
users[doc.id] = doc.data();
});
posts = db.collection('posts').orderBy('timestamp', 'desc').limit(3);
posts.get().then((docSnaps) => {
docSnaps.forEach((doc) => {
loadedPosts[doc.id] = doc.data();
loadedPosts[doc.id].userName = users[doc.data().uid].name;
});
});
After trying multiple solution I get it done with RXJS combineLatest, take operator. Using map function we can combine result.
Might not be an optimum solution but here its solve your problem.
combineLatest(
this.firestore.collection('Collection1').snapshotChanges(),
this.firestore.collection('Collection2').snapshotChanges(),
//In collection 2 we have document with reference id of collection 1
)
.pipe(
take(1),
).subscribe(
([dataFromCollection1, dataFromCollection2]) => {
this.dataofCollection1 = dataFromCollection1.map((data) => {
return {
id: data.payload.doc.id,
...data.payload.doc.data() as {},
}
as IdataFromCollection1;
});
this.dataofCollection2 = dataFromCollection2.map((data2) => {
return {
id: data2.payload.doc.id,
...data2.payload.doc.data() as {},
}
as IdataFromCollection2;
});
console.log(this.dataofCollection2, 'all feeess');
const mergeDataFromCollection =
this.dataofCollection1.map(itm => ({
payment: [this.dataofCollection2.find((item) => (item.RefId === itm.id))],
...itm
}))
console.log(mergeDataFromCollection, 'all data');
},
my solution as below.
Concept: You know user id you want to get information, in your posts list, you can request user document and save it as promise in your post item. after promise resolve then you get user information.
Note: i do not test below code, but it is simplify version of my code.
let posts: Observable<{}[]>; // you can display in HTML directly with | async tag
this.posts = this.listenPosts()
.map( posts => {
posts.forEach( post => {
post.promise = this.getUserDoc( post.uid )
.then( (doc: DocumentSnapshot) => {
post.userName = doc.data().name;
});
}); // end forEach
return posts;
});
// normally, i keep in provider
listenPosts(): Observable<any> {
let fsPath = 'posts';
return this.afDb.collection( fsPath ).valueChanges();
}
// to get the document according the user uid
getUserDoc( uid: string ): Promise<any> {
let fsPath = 'users/' + uid;
return this.afDb.doc( fsPath ).ref.get();
}
Note: afDb: AngularFirestore it is initialize in constructor (by angularFire lib)
If you want to join observables instead of promises, use combineLatest. Here is an example joining a user document to a post document:
getPosts(): Observable<Post[]> {
let data: any;
return this.afs.collection<Post>('posts').valueChanges().pipe(
switchMap((r: any[]) => {
data = r;
const docs = r.map(
(d: any) => this.afs.doc<any>(`users/${d.user}`).valueChanges()
);
return combineLatest(docs).pipe(
map((arr: any) => arr.reduce((acc: any, cur: any) => [acc].concat(cur)))
);
}),
map((d: any) => {
let i = 0;
return d.map(
(doc: any) => {
const t = { ...data[i], user: doc };
++i;
return t;
}
);
})
);
}
This example joins each document in a collection, but you could simplify this if you wanted to just join one single document to another.
This assumes your post document has a user variable with the userId of the document.
J