I am developing the backend of an application using Node JS, Sequelize and Postgres database.
When the course is registered, the user must inform which organizations, companies and teachers will be linked to it.
The organization IDs are passed through an array to the backend, I am trying to do a check to make sure that the passed IDs exist.
What I've done so far is this:
const { organizations } = req.body;
const organizationsArray = organizations.map(async (organization) => {
const organizationExists = await Organization.findByPk(organization);
if (!organizationExists) {
return res
.status(400)
.json({ error: `Organization ${organization} does not exists!` });
}
return {
course_id: id,
organization_id: organization,
};
});
await CoursesOrganizations.bulkCreate(organizationsArray);
This link has the complete controller code, I believe it will facilitate understanding.
When !OrganizationExists is true, I am getting the return that the organization does not exist. The problem is when the organization exists, I am getting the following message error.
The Array.map() is returning an array of promises that you can resolve to an array using Promise.all(). Inside the map you should use throw new Error() to break out of the map - this error will be raised by Promise.all() and you can then catch it and return an error to the client (or swallow it, etc).
This is a corrected version of your pattern, resolving the Promise results.
const { organizations } = req.body;
try {
// use Promise.all to resolve the promises returned by the async callback function
const organizationsArray = await Promise.all(
// this will return an array of promises
organizations.map(async (organization) => {
const organizationExists = await Organization.findByPk(organization, {
attributes: ['id'], // we only need the ID
raw: true, // don't need Instances
});
if (!organizationExists) {
// don't send response inside the map, throw an Error to break out
throw new Error(`Organization ${organization} does not exists!`);
}
// it does exist so return/resolve the value for the promise
return {
course_id: id,
organization_id: organization,
};
})
);
// if we get here there were no errors, create the records
await CoursesOrganizations.bulkCreate(organizationsArray);
// return a success to the client
return res.json({ success: true });
} catch (err) {
// there was an error, return it to the client
return res.status(400).json({ error: err.message });
}
This is a refactored version that will be a bit faster by fetching all the Organizations in one query and then doing the checks/creating the Course inserts.
const { Op } = Sequelize;
const { organizations } = req.body;
try {
// get all Organization matches for the IDs
const organizationsArray = await Organization.findAll({
attributes: ['id'], // we only need the ID
where: {
id: {
[Op.in]: organizations, // WHERE id IN (organizations)
}
},
raw: true, // no need to create Instances
});
// create an array of the IDs we found
const foundIds = organizationsArray.map((org) => org.id);
// check to see if any of the IDs are missing from the results
if (foundIds.length !== organizations.length) {
// Use Array.reduce() to figure out which IDs are missing from the results
const missingIds = organizations.reduce((missingIds, orgId) => {
if (!foundIds.includes(orgId)){
missingIds.push(orgId);
}
return missingIds;
}, []); // initialized to empty array
throw new Error(`Unable to find Organization for: ${missingIds.join(', ')}`);
}
// now create an array of courses to create using the foundIds
const courses = foundIds.map((orgId) => {
return {
course_id: id,
organization_id: orgId,
};
});
// if we get here there were no errors, create the records
await CoursesOrganizations.bulkCreate(courses);
// return a success to the client
return res.json({ success: true });
} catch (err) {
// there was an error, return it to the client
return res.status(400).json({ error: err.message });
}
If you have an array of Ids and you want to check if they exist you should you use the (in) operator, this makes it so that you are hitting the DB only once and getting all the records in one hit (instead of getting them one by one in a loop), after you get these records you can check their lengths to determine if they all exist or not.
const { Op } = require("sequelize");
let foundOrgs = await Organization.findAll({
where: {
id: {
[Op.in]: organizationsArray,
}
}
});
Related
I want to unit test the below function that uses Prisma and is in my service.
How would I go about writing the unit test case for it in the spec file?
Do I have to test all the prisma database(findUnique,deleteMany,createMany) calls and all the functionality within the function including the errors thrown?
async rules(options: {
data: UserAssociateRulesDto;
}): Promise<FetchRule> {
try {
// Check if rule present
const role = await this.db.rules.findUnique({
where: {
rule_name: options.data['rule_name'],
},
});
if (!rule) {
throw new BadRequestException('No such user role exists');
}
// Map Permissions to Rule
const rulePermissionAssociations = [];
for (const permission of options.data['permissions']) {
rulePermissionAssociations.push({
rule_id: rule['id'],
permission_id: permission['permission_id'],
});
}
// Delete all records
await this.db.rulepermission_association.deleteMany({
where: {
rule_id: rule['id'],
},
});
// Add rule-permissions mapping
await this.db.rule_association.createMany({
data: rulePermissionAssociations,
});
return this.db.rules.findUnique();
} catch (error) {
throw new HttpException(error.message, error.status);
}
}
Im guessing this problem is because I don't know how to use async await effectively. I still dont get it and I've been trying to understand for ages. sigh.
Anyway, heres my function:
app.post("/declineTrades", async (request, response) => {
//---------------------------------------------
const batch = db.batch();
const listingID = request.body.listingID;
const tradeOfferQuery = db
//---------------------------------------------
//Get trade offers that contain the item that just sold
//(therefore it cannot be traded anymore, I need to cancel all existing trade offers that contain the item because this item isn't available anymore)
//---------------------------------------------
.collection("tradeOffers")
.where("status", "==", "pending")
.where("itemIds", "array-contains", listingID);
//---------------------------------------------
//Function that gets all trade offers that contain the ID of the item.
async function getIdsToDecline() {
let tempArray = [];
tradeOfferQuery.get().then((querySnapshot) => {
querySnapshot.forEach((doc) => {
//For each trade offer found
let offerRef = db.collection("tradeOffers").doc(doc.id);
//Change the status to declined
batch.update(offerRef, { status: "declined" });
//Get the data from the trade offer because I want to send an email
//to the who just got their trade offer declined.
const offerGet = offerRef.get().then((offer) => {
const offerData = offer.data();
//Check the items that the receiving person had in this trade offer
const receiverItemIds = Array.from(
offerData.receiversItems
.reduce((set, { itemID }) => set.add(itemID), new Set())
.values()
);
//if the receiver item id's array includes this item that just sold, I know that
//I can get the sender ID (users can be sender or receiver, so i need to check which person is which)
if (receiverItemIds.includes(listingID)) {
tempArray.push(offerData.senderID);
}
});
});
});
//With the ID's now pushed, return the tempArray
return tempArray;
}
//---------------------------------------------
//Call the above function to get the ID's of people that got declined
//due to the item no longer being available
const peopleToDeclineArray = await getIdsToDecline();
//Update the trade offer objects to declined
const result = await batch.commit();
//END
response.status(201).send({
success: true,
result: result,
idArray: peopleToDeclineArray,
});
});
Im guessing that my return tempArray is in the wrong place? But I have tried putting it in other places and it still returns an empty array. Is my logic correct here? I need to run the forEach loop and add to the array before the batch.commit happens and before the response is sent.
TIA Guys!
As #jabaa pointed out in their comment, there are problems with an incorrectly chained Promise in your getIdsToDecline function.
Currently the function initializes an array called tempArray, starts executing the trade offer query and then returns the array (which is currently still empty) because the query hasn't finished yet.
While you could throw in await before tradeOfferQuery.get(), this won't solve your problem as it will only wait for the tradeOfferQuery to execute and the batch to be filled with entries, while still not waiting for any of the offerRef.get() calls to be completed to fill the tempArray.
To fix this, we need to make sure that all of the offerRef.get() calls finish first. To get all of these documents, you would use the following code to fetch each document, wait for all of them to complete and then pull out the snapshots:
const itemsToFetch = [ /* ... */ ];
const getAllItemsPromise = Promise.all(
itemsToFetch.map(item => item.get())
);
const fetchedItemSnapshots = await getAllItemsPromise;
For documents based on a query, you'd tweak this to be:
const querySnapshot = /* ... */;
const getSenderDocPromises = [];
querySnapshot.forEach((doc) => {
const senderID = doc.get("senderID");
const senderRef = db.collection("users").doc(senderID);
getSenderDocPromises.push(senderRef.get());
}
const getAllSenderDocPromise = Promise.all(getSenderDocPromises);
const fetchedSenderDataSnapshots = await getAllSenderDocPromise;
However neither of these approaches are necessary, as the document you are requesting using these offerRef.get() calls are already returned in your query so we don't even need to use get() here!
(doc) => {
let offerRef = db.collection("tradeOffers").doc(doc.id);
//Change the status to declined
batch.update(offerRef, { status: "declined" });
//Get the data from the trade offer because I want to send an email
//to the who just got their trade offer declined.
const offerGet = offerRef.get().then((offer) => {
const offerData = offer.data();
//Check the items that the receiving person had in this trade offer
const receiverItemIds = Array.from(
offerData.receiversItems
.reduce((set, { itemID }) => set.add(itemID), new Set())
.values()
);
//if the receiver item id's array includes this item that just sold, I know that
//I can get the sender ID (users can be sender or receiver, so i need to check which person is which)
if (receiverItemIds.includes(listingID)) {
tempArray.push(offerData.senderID);
}
});
}
could be replaced with just
(doc) => {
// Change the status to declined
batch.update(doc.ref, { status: "declined" });
// Fetch the IDs of items that the receiving person had in this trade offer
const receiverItemIds = Array.from(
doc.get("receiversItems") // <-- this is the efficient form of doc.data().receiversItems
.reduce((set, { itemID }) => set.add(itemID), new Set())
.values()
);
// If the received item IDs includes the listed item, add the
// sender's ID to the array
if (receiverItemIds.includes(listingID)) {
tempArray.push(doc.get("senderID"));
}
}
which could be simplified to just
(doc) => {
//Change the status to declined
batch.update(doc.ref, { status: "declined" });
// Check if any items that the receiving person had in this trade offer
// include the listing ID.
const receiversItemsHasListingID = doc.get("receiversItems")
.some(item => item.itemID === listingID);
// If the listing ID was found, add the sender's ID to the array
if (receiversItemsHasListingID) {
tempArray.push(doc.get("senderID"));
}
}
Based on this, getIdsToDecline actually queues declining the invalid trades and returns the IDs of those senders affected. Instead of using the batch and tradeOfferQuery objects that are outside of the function that make this even more unclear, you should roll them into the function and pull it out of the express handler. I'll also rename it to declineInvalidTradesAndReturnAffectedSenders.
async function declineInvalidTradesAndReturnAffectedSenders(listingID) {
const tradeOfferQuery = db
.collection("tradeOffers")
.where("status", "==", "pending")
.where("itemIds", "array-contains", listingID);
const batch = db.batch();
const affectedSenderIDs = [];
const querySnapshot = await tradeOfferQuery.get();
querySnapshot.forEach((offerDoc) => {
batch.update(offerDoc.ref, { status: "declined" });
const receiversItemsHasListingID = offerDoc.get("receiversItems")
.some(item => item.itemID === listingID);
if (receiversItemsHasListingID) {
affectedSenderIDs.push(offerDoc.get("senderID"));
}
}
await batch.commit(); // generally, the return value of this isn't useful
return affectedSenderIDs;
}
This then would change your route handler to:
app.post("/declineTrades", async (request, response) => {
const listingID = request.body.listingID;
const peopleToDeclineArray = await declineInvalidTradesAndReturnAffectedSenders(listingID);
response.status(201).send({
success: true,
result: result,
idArray: peopleToDeclineArray,
});
});
Then adding the appropriate error handling, swapping out the incorrect use of HTTP 201 Created for HTTP 200 OK, and using json() instead of send(); you now get:
app.post("/declineTrades", async (request, response) => {
try {
const listingID = request.body.listingID;
const affectedSenderIDs = await declineInvalidTradesAndReturnAffectedSenders(listingID);
response.status(200).json({
success: true,
idArray: affectedSenderIDs, // consider renaming to affectedSenderIDs
});
} catch (error) {
console.error(`Failed to decline invalid trades for listing ${listingID}`, error);
if (!response.headersSent) {
response.status(500).json({
success: false,
errorCode: error.code || "unknown"
});
} else {
response.end(); // forcefully end corrupt response
}
}
});
Note: Even after all these changes, you are still missing any form of authentication. Consider swapping the HTTPS Event Function out for a Callable Function where this is handled for you but requires using a Firebase Client SDK.
I am trying to run a transaction which requires to get the data from a query:
firestore
.runTransaction((transaction) => {
const query = firestore
.collection("Lottories")
.doc("R3DYubrqFbbVfQNjYXfh")
.collection("sets")
.where("isAvailable", "==", false)
.limit(1);
return transaction.get(query).then((snapshot) => {
const ticketDoc = snapshot.docs[0];
const ticketDocData = ticketDoc.data();
const lottoUpdate = firestore
.collection("Lottories")
.doc("R3DYubrqFbbVfQNjYXfh")
.collection("sets")
.doc(ticketDoc.id);
const ticketUpdate = firestore
.collection("UserLotto")
.doc(userId)
.collection("tickets")
.doc("abc");
const countUpdate = firestore
.collection("UserData")
.doc(userId);
transaction.update(lottoUpdate, { isAvailable: true });
transaction.update(countUpdate, {
ticketCount: ticketCount - 2,
});
transaction.set(ticketUpdate, {
ticketId: ticketDoc.id,
lottoId: "abc",
claimed: false,
});
return ticketDocData;
});
})
.then((ticketDocData) => {
console.log(
"Transaction successfully committed!",
ticketDocData
);
setPopScreen("ticketPurchased");
})
.catch((error) => {
console.log("Transaction failed:", error);
});
For my application I need to run a query in order to complete my transaction. I Get the error:
Transaction failed: [FirebaseError: Function Transaction.get() requires its first argument to be a DocumentReference, but it was: a custom t object]
I understand transaction requires a document reference. Is there a work around this?
The nearest equivalent is to do the query with a higher limit, then in the transaction, get the doc again and check again for the required input condition....
// this will do the OP sets/updates, but *doesn't* assume the input
// doc meets the required condition (isAvailable==true)
// return a bool indicating success
function reserveDoc(lottoDoc) {
return firestore.runTransaction(transaction => {
return transaction.get(lottoDoc.ref).then(ticketDoc => {
// important, check the original condition again here
if (ticketDoc.data().isAvailable) {
// OP code to set/update goes here
return true
} else {
return false
}
})
})
}
// try a transaction on the first doc in the array. return if successful
// otherwise, try recursively on the remainder of the array
// return a bool indicating success
function reserveOneOf(lottoDocs) {
if (lottoDocs.length === 0) return false
return reserveDoc(lottoDocs[0]).then(success => {
// did it work? if not, try another doc
return success ? success : reserveOneOf(lottoDocs.slice(1))
})
}
function originalOPFunction() {
const query = firestore
.collection("Lottories")
.doc("R3DYubrqFbbVfQNjYXfh")
.collection("sets")
.where("isAvailable", "==", true) // note, I assume the OP query had a typo, checking for false
.limit(10);
return query.get().then(snapshot => {
return reserveOneOf(snapshot.docs)
}).then(success => {
// if !success here, the app must deal with NO docs currently meeting the criterion
// the OP needed to handle this circumstance anyway (if the limit(1) query returned no docs
})
}
The first param of Transaction get really is a document reference, not a query. This is confusing because documentReference.get() and transaction.get(documentReference) and `query.get(), kind of look and sound the same, but transaction is only atomic on a single doc, not on a set from a query, even one limited to 1.
transaction.get(query) the query must be a DocumentReference. So you need something like this:
db.collection("Lottories/R3DYubrqFbbVfQNjYXfh/sets")
.where("isAvailable", "==", false)
.limit(1)
.get()
.then((docs) => {
db.runTransaction((transaction) => transaction.get(docs[0]).then((doc) => {
if (doc.exists) //do something
}));
});
I am using Firebase authentication to store users. I have two types of users: Manager and Employee. I am storing the manager's UID in Firestore employee along with the employee's UID. The structure is shown below.
Firestore structure
Company
|
> Document's ID
|
> mng_uid: Manager's UID
> emp_uid: Employee's UID
Now I want to perform a query like "Retrieve employees' info which is under the specific manager." To do that I tried to run the below code.
module.exports = {
get_users: async (mng_uid, emp_uid) => {
return await db.collection("Company").where("manager_uid", "==", mng_uid).get().then(snaps => {
if (!snaps.empty) {
let resp = {};
let i = 0;
snaps.forEach(async (snap) => {
resp[i] = await admin.auth().getUser(emp_uid).then(userRecord => {
return userRecord;
}).catch(err => {
return err;
});
i++;
});
return resp;
}
else return "Oops! Not found.";
}).catch(() => {
return "Error in retrieving employees.";
});
}
}
Above code returns {}. I tried to debug by returning data from specific lines. I got to know that the issue is in retrieving the user's info using firebase auth function which I used in forEach loop. But it is not returning any error.
Thank you.
There are several points to be corrected in your code:
You use async/await with then() which is not recommended. Only use one of these approaches.
If I understand correctly your goal ("Retrieve employees' info which is under the specific manager"), you do not need to pass a emp_uid parameter to your function, but for each snap you need to read the value of the emp_uid field with snap.data().emp_uid
Finally, you need to use Promise.all() to execute all the asynchronous getUser() method calls in parallel.
So the following should do the trick:
module.exports = {
get_users: async (mng_uid) => {
try {
const snaps = await db
.collection('Company')
.where('manager_uid', '==', mng_uid)
.get();
if (!snaps.empty) {
const promises = [];
snaps.forEach(snap => {
promises.push(admin.auth().getUser(snap.data().emp_uid));
});
return Promise.all(promises); //This will return an Array of UserRecords
} else return 'Oops! Not found.';
} catch (error) {
//...
}
},
};
I am trying to do TWO things here.
1)Send a notification to all employees. 2)Copy a Specific ref to the
Employees id ref. if no Special ref exists i will copy General ref.
The program runs without errors. Infact its perfect. But sometimes i get a Timed out error with the Notifications code part.
Error: fcm.googleapis.com network timeout. Please try again.
The code that copys one reference to another, always works, never ever received an error there.
I feel the error is due to not correctly handling promises with forEach. Could you help me get this code to excecute flawlessly, with correctly placed Promises?
exports.myFunc = functions.https.onRequest( (request, response) => {
admin.database().ref('/Employees').once('value').then(function(snap) {
snap.forEach(function (snapshot) {
var obj = snapshot.val();
if(obj.department){//only go ahead if dept is present
console.log(' : ' + obj.department);
var id, tkid, dept;
id = obj.empId; tkid = obj.tokenId; dept = obj.department;
var welcomeStr="hello! Welcom to our Department";
//================================================================notifications
var payload = {
data: {
greeting: welcomeStr,
to_who: id
}
};
admin.messaging().sendToDevice(tkid,payload)
.then(function(response){
console.log("Successfully sent message: ", response);
})
.catch(function(error){
console.log("Error sending message: ", error);
})
//===================================================Ref copying
var destinationRef = admin.database().ref('/Employees/' + id);//final destination
var option2Ref = admin.database().ref('/Company/General');//when special doesnt exist
var option1Ref = admin.database().ref('/Company/Special');//if special exists
option1.once('value', function(snapshot1){
if (snapshot1.exists()){//copy from straing from option11 to Employees/id
option1.once('value', function(snap) {
destinationRef.set( snap.val(), function(error) {
if( error && typeof(console) !== 'undefined' && console.error ) { console.error(error); }
console.log('DONE .... ' + id);
});
});
}
else{//we need to copy from option2 to Employees/id
option2Ref.once('value', function(snap) {
newRef.set( snap.val(), function(error) {
if( error && typeof(console) !== 'undefined' && console.error ) { console.error(error); }
console.log('DONE .... ' + id);
});
});
}
});
}
else{
console.log('No Department: ' + obj.dept);
return;
}
});
});
response.send("WOKAY!");
});
here i've rewritten your code in hopes to clean up the complicated promise chains
dropped promises are one of the most common and difficult problems to debug, i've seen my fair share
important changes to your code:
modern async syntax
so that the promises are cleaner to organize
use Promise.all instead of forEach
this way every promise is awaited without being forgotten
(hopefully) all of the promises are returned properly
all snapshot operations are run concurrently, and the onRequest handler should wait until they're all finished before terminating
using promises for once and set instead of callbacks
i'm not really sure what libraries these are
it looks like they accept promise-based usage
so i eliminated callback usage in favor of promises
please review the TODO mark
not really sure what's intended for that else block, so be sure to patch that up
async function handleSnapshot(snapshot) {
const {empId, tokenId, department} = snapshot.val()
// only go ahead if dept is present
if (!department) throw new Error("no department")
console.log("department:", department)
//================================================================notifications
const payload = {
data: {
greeting: "Hello! Welcome to our Department",
to_who: empId
}
}
const response = await admin.messaging().sendToDevice(tokenId, payload)
console.log("successfully sent message", response)
//===================================================Ref copying
const destinationRef = admin.database().ref('/Employees/' + empId) // final destination
const option2Ref = admin.database().ref('/Company/General') // when special doesnt exist
const option1Ref = admin.database().ref('/Company/Special') // if special exists
const snapshot1 = await option1Ref.once("value")
// copy from string from option1 to Employees/id
if (snapshot1.exists()) {
await destinationRef.set(snapshot1.val())
console.log("DONE1...", empId)
}
// TODO review this block
// we need to copy from option2 to Employees/id
else {
const snapshot2 = await option2Ref.once("value")
await destinationRef.set(snapshot2.val())
console.log("DONE2...", empId)
}
}
exports.myFunc = functions.https.onRequest(async(request, response) => {
const snapshots = await admin.database().ref('/Employees').once('value')
await Promise.all(snapshots.map(handleSnapshot))
response.send("WOKAY!")
})
To add one very important step to #ChaseMoskal answer.
For those using TypeScript with Firebase, since firebase server is not running v8+ in NodeJs, theres a great chance you might get this error:
"TypeError: snapshots.map is not a function"... on the line: await Promise.all(snapshots.map(handleSnapshot)).
Thats cause in your tsconfig.json its possibly "lib": ["es6"]. In that case just add this small snippet to the accepted answer, to get the Firebase Datasnapshot into an array that could be used with .map(...)
Longer Version:
exports.myFunc = functions.https.onRequest(async(request, response) => {
const snapshots = await admin.database().ref('/Employees').once('value')
var data_snap_arr = [];
snapshots.forEach(function(child_Snapshot) {
var stuff = child_Snapshot.val();
stuff.key = child_Snapshot.key;
data_snap_arr.push(stuff);
await Promise.all(data_snap_arr.map(handleSnapshot))
response.send("WOKAY!")
})
Shorter Version:
exports.myFunc = functions.https.onRequest(async(request, response) => {
const snapshots = await admin.database().ref('/Employees').once('value')
let data_snap_arr = Object.keys(snapshots.val() || {}) .map(k => snapshots.val()[k]);
await Promise.all(data_snap_arr.map(handleSnapshot))
response.send("WOKAY!")
})