Develop Reference

Firebase Query and how to read the output

Above is my Js code
I have a database that has information regarding the location of an apartment, I am trying to search for a specific property to see if it exists in the database. The user will be able to key into the search box to perform the search.
"propertiesRef" is used to store the user input.
I tried storing the data into "q" that I received from querying the database. But I have no idea how to read the result.
This is the console log for "q", but I don't quite understand the information that is shown, I want to know which output in the console should I be looking at and how do I access them?

The query() function just creates an instance Query. You need to use getDocs() function to actually fetch data from Firestore.
const search = (property) => {
const propertiesRef = collection(db, "flats-table");
const q = query(propertiesRef, where("name", "==", property))
return getDocs(q).then((qSnap) => {
const data = qSnap.docs.map(d => ({ id: d.id, ...d.data() }))
console.log(data);
return data;
})
// or use async-await
// const qSnap = await getDocs(q);
}
Checkout the documentation for more examples.

A firestore query not working in react-native

I am using react-native and I want to get specific data so I used a query, the problem is that it didn't work, but if I remove the where and do an if statement it works. How can I fix this ?
This is my implementation:
let query = firestore().collection('conversations');
query = query.where('users', 'array-contains', myId);
// query = query.where('postId', '==', postId);
query
.orderBy('timestamp', 'desc')
.limit(limit)
.get()
.then((snapshot) => {
const offersObjects = {};
if (snapshot.docs.length) {
snapshot.docs.forEach((doc) => {
if (postId === doc.data().postId) {
offersObjects[doc.id] = { ...doc.data(), pendingMessages: [] };
}
});
dispatch({
type: chatTypes.SET_OFFERS,
payload: { offers: offersObjects },
});
}
})
Where the code is commented is where this query doesn't work which is weird since the one above it works fine. How can I fix this ?

If you have more than one condition on a query, it may need a composite index that is not automatically created. If that is the case, executing the query (e.g. by calling get() on it) will raise an error, but you'r not hancling errors. I recommend adding a catch clause after your then and logging the error.
If the problem is caused by a missing index, the error message contains a URL that opens the Firebase console on the page to create the exact index that is needed, with all fields already filled in. Create the index, wait for it to be completely created, and run the query again.

Trouble batch setting a document field by docId in Firestore

I have been using firebase (firestore) for a while but I'm a little stuck and was wondering if anyone can think of a solution.
On the firestore DB I have a single collection of users, each user has an email address and several other fields. In this instance I am checking if a user email exists and if it does, I want to create a list field for that particular user with a listUid. I am referencing the users by email, grabbing the docId for those users and then trying to set a list field for each of them.
I am not getting any error's from firestore, it's simply not updating in the DB for some reason and I can't figure out where I am going wrong. Thanks in advance
export const addListUidToExistingUserList = (
{ firestore },
emailArray,
listUid
) => {
return async () => {
let docIds = [];
emailArray.forEach((emailAddress) => {
//find users by email (works)
const query = db
.collection("users")
.where("email", "==", emailAddress);
//get docId's for user with matching email (works)
query.get().then((querySnapshot) => {
querySnapshot.forEach((doc) => {
docIds.push(doc.id);
});
});
//add a new list with corresponding listUid (does not work)
docIds.forEach((id) => {
let userRef = db.collection("users").doc(id);
batch.set(userRef, { lists: [{ listUid }] });
});
});
return await batch.commit();
};
};

You are running into this issue because your docIds array is always empty at the time you call docIds.forEach.
That's because query.get().then runs asynchronously, and so docIds.forEach is not waiting for it to complete.
You could either:
await query.get().then; or
Add the docIds.forEach function INSIDE the then callback of query.get.
Here are your possible fixes:
await query.get().then
//get docId's for user with matching email (works)
await query.get().then((querySnapshot) => {
querySnapshot.forEach((doc) => {
docIds.push(doc.id);
});
});
OR:
docIds.forEach inside then
//get docId's for user with matching email (works)
query.get().then((querySnapshot) => {
querySnapshot.forEach((doc) => {
docIds.push(doc.id);
});
docIds.forEach((id) => {
let userRef = db.collection("users").doc(id);
batch.set(userRef, { lists: [{ listUid }] });
});
});
Note: Of course, you could also add batch.set directly into your first iteration of querySnapshot.docs.forEach to prevent an unnecessary iteration.

need help accessing Firestore sub-collection?

I'm a novice when it comes to coding (started teaching myself ~year ago), any help will be much appreciated, thank you in advance.
I saw that there is 3-4 other post on stack overflow on how to access Firestore's sub-collections.
I tried them all and had no luck, hence why I'm posting this as a new question.
right now I have my data set is up as: collection/document.array. And that was fine till now because I just needed to read that data from the array to draw it out in my little React project with .reduce and .map and push new data to that array on input.
this is the code I have right now for getting data from Firestore:
--- in firebase.js ----------------------------------------------------------------------
export const fire = firebase.initializeApp(config);
export const db = fire.firestore()
_________________________________________________________________________________________
--- in events-context.js ----------------------------------------------------------------
const fetchEvents = async () => {
try {
const data = await db.collection('events').get();
setEvents(data.docs.map(doc => ({ ...doc.data(), id: doc.id })));
} catch ({ message }) {
alert(`Error # fetchEvents, Error:${message}`);
}
};
But now I want to add edit and a remove feature, but in order to do that, I need to carry out my array of data into a sub-collection so each individual element from that array had its own id so that I could later target it. so it needs to be set up something like this: collection/document/sub-collection

To access a document inside a collection, you must know the document ID from another source. This can be done by managing the names inside an array of strings or maps that you can then process within your app per your design.
For example: once created, you will have a snapshot of the reference of which you can store the document id inside the parent document:
db.collection("events").doc(id).update({
payers: firebase.firestore.FieldValue.arrayUnion(paySnapshot.ref.id)
})`
Once you have this information, you can append it to the relevant document path using one of the following techniques.
db.collection("events").doc(id).collection("payers").doc(pay_id).get()
db.doc(\events/${id}/payers/${pay_id}`).get()`
I strongly advise against using .get() on a collection without limit() and where() conditions to reduce the reads that can occur.

Try this, it works for me :)
Insert data >>>
const q = query(collection(this.fire, "events"));
const querySnapshot = await getDocs(q);
const queryData = querySnapshot.docs.map((details) => ({
...details.data(),
id: details.id,
}));
console.log(queryData);
queryData.map(async (v, id) => {
await setDoc(doc(this.fire, `events/${auth}/more`, events.title), {
'title': events.title,
'uid': auth,
//your data here
})
})
Read Data >>>
const querySnapshot = await getDocs(collection(this.fire,
`/events/${auth}/more/`)); querySnapshot.forEach((doc) => { //
doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data()); });
return querySnapshot;

Batch update in knex

I'd like to perform a batch update using Knex.js
For example:
'UPDATE foo SET [theValues] WHERE idFoo = 1'
'UPDATE foo SET [theValues] WHERE idFoo = 2'
with values:
{ name: "FooName1", checked: true } // to `idFoo = 1`
{ name: "FooName2", checked: false } // to `idFoo = 2`
I was using node-mysql previously, which allowed multiple-statements. While using that I simply built a mulitple-statement query string and just send that through the wire in a single run.
I'm not sure how to achieve the same with Knex. I can see batchInsert as an API method I can use, but nothing as far as batchUpdate is concerned.
Note:
I can do an async iteration and update each row separately. That's bad cause it means there's gonna be lots of roundtrips from the server to the DB
I can use the raw() thing of Knex and probably do something similar to what I do with node-mysql. However that defeats the whole knex purpose of being a DB abstraction layer (It introduces strong DB coupling)
So I'd like to do this using something "knex-y".
Any ideas welcome.

I needed to perform a batch update inside a transaction (I didn't want to have partial updates in case something went wrong).
I've resolved it the next way:
// I wrap knex as 'connection'
return connection.transaction(trx => {
const queries = [];
users.forEach(user => {
const query = connection('users')
.where('id', user.id)
.update({
lastActivity: user.lastActivity,
points: user.points,
})
.transacting(trx); // This makes every update be in the same transaction
queries.push(query);
});
Promise.all(queries) // Once every query is written
.then(trx.commit) // We try to execute all of them
.catch(trx.rollback); // And rollback in case any of them goes wrong
});

Assuming you have a collection of valid keys/values for the given table:
// abstract transactional batch update
function batchUpdate(table, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(table)
.where('id', tuple.id)
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate('user', [...]);
Are you unfortunately subject to non-conventional column names? No worries, I got you fam:
function batchUpdate(options, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(options.table)
.where(options.column, tuple[options.column])
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate({ table: 'user', column: 'user_id' }, [...]);
Modern Syntax Version:
const batchUpdate = (options, collection) => {
const { table, column } = options;
const trx = await knex.transaction();
try {
await Promise.all(collection.map(tuple =>
knex(table)
.where(column, tuple[column])
.update(tuple)
.transacting(trx)
)
);
await trx.commit();
} catch (error) {
await trx.rollback();
}
}

You have a good idea of the pros and cons of each approach. I would recommend a raw query that bulk updates over several async updates. Yes you can run them in parallel, but your bottleneck becomes the time it takes for the db to run each update. Details can be found here.
Below is an example of an batch upsert using knex.raw. Assume that records is an array of objects (one obj for each row we want to update) whose values are the properties names line up with the columns in the database you want to update:
var knex = require('knex'),
_ = require('underscore');
function bulkUpdate (records) {
var updateQuery = [
'INSERT INTO mytable (primaryKeyCol, col2, colN) VALUES',
_.map(records, () => '(?)').join(','),
'ON DUPLICATE KEY UPDATE',
'col2 = VALUES(col2),',
'colN = VALUES(colN)'
].join(' '),
vals = [];
_(records).map(record => {
vals.push(_(record).values());
});
return knex.raw(updateQuery, vals);
}
This answer does a great job explaining the runtime relationship between the two approaches.
Edit:
It was requested that I show what records would look like in this example.
var records = [
{ primaryKeyCol: 123, col2: 'foo', colN: 'bar' },
{ // some other record, same props }
];
Please note that if your record has additional properties than the ones you specified in the query, you cannot do:
_(records).map(record => {
vals.push(_(record).values());
});
Because you will hand too many values to the query per record and knex will fail to match the property values of each record with the ? characters in the query. You instead will need to explicitly push the values on each record that you want to insert into an array like so:
// assume a record has additional property `type` that you dont want to
// insert into the database
// example: { primaryKeyCol: 123, col2: 'foo', colN: 'bar', type: 'baz' }
_(records).map(record => {
vals.push(record.primaryKeyCol);
vals.push(record.col2);
vals.push(record.colN);
});
There are less repetitive ways of doing the above explicit references, but this is just an example. Hope this helps!

The solution works great for me! I just include an ID parameter to make it dynamic across tables with custom ID tags. Chenhai, here's my snippet including a way to return a single array of ID values for the transaction:
function batchUpdate(table, id, collection) {
return knex.transaction((trx) => {
const queries = collection.map(async (tuple) => {
const [tupleId] = await knex(table)
.where(`${id}`, tuple[id])
.update(tuple)
.transacting(trx)
.returning(id);
return tupleId;
});
return Promise.all(queries).then(trx.commit).catch(trx.rollback);
});
}
You can use
response = await batchUpdate("table_name", "custom_table_id", [array of rows to update])
to get the returned array of IDs.

The update can be done in batches, i.e 1000 rows in a batch
And as long as it does it in batches, the bluebird map could be used.
For more information on bluebird map: http://bluebirdjs.com/docs/api/promise.map.html
const limit = 1000;
const totalRows = 50000;
const seq = count => Array(Math.ceil(count / limit)).keys();
map(seq(totalRows), page => updateTable(dbTable, page), { concurrency: 1 });
const updateTable = async (dbTable, page) => {
let offset = limit* page;
return knex(dbTable).pluck('id').limit(limit).offset(offset).then(ids => {
return knex(dbTable)
.whereIn('id', ids)
.update({ date: new Date() })
.then((rows) => {
console.log(`${page} - Updated rows of the table ${dbTable} from ${offset} to ${offset + batch}: `, rows);
})
.catch((err) => {
console.log({ err });
});
})
.catch((err) => {
console.log({ err });
});
};
Where pluck() is used to get ids in array form