I have a simple tree with ids that are keys to a Mongo collection. I'm using a node library called treewalker. As I walk each node of the tree, I'm trying to look up the name (using mongoose) and simply append it to the current node. If I don't do a callback to lookup the node name, and just use some fixed value, I get the value I'm expecting. Let me illustrate in code:
Here is my tree:
{
"categoryTree": [
{
"categoryId": "1",
"children": [
{
"categoryId": "2",
"children": [
{
"categoryId": "3",
"children": []
},
{
"categoryId": "4",
"children": []
}
]
},
{
"categoryId": "5",
"children": []
},
{
"categoryId": "6",
"children": []
}
]
},
{
"categoryId": "7",
"children": [
{
"categoryId": "8",
"children": []
}
]
}
]
}
Here is code that does what I want:
catTree.categoryTree.forEach(function(node){
var counter = 0;
tree.walkTree(node, 'children', function(obj){
obj.name = counter++;
});
});
//This tree has the names (as the counter above) in it as I expect
console.log(JSON.stringify(catTree));
However, as soon as I throw in a mongoose callback to get the category name, the category tree that's printed no longer has the names.
catTree.categoryTree.forEach(function(node){
tree.walkTree(node, 'children', function(obj){
//Cat is a mongoose model defined elsewhere
Cat.findById(obj.categoryId, {_id:0,name:1}).exec(function(err, value){
obj.name = value.name;
});
});
});
//This tree has NO names :(
console.log(JSON.stringify(catTree));
I know this is an issue of timing, but I can't figure out how to solve it. I've seen several SO Articles like this one that suggest tracking callbacks and continuing only after they've all been called. I can't figure out how to apply that pattern to my case because I'm walking a tree and not just iterating a flat list. I'm starting to think that my problem might that I'm using the treewalker library, vs. just writing my own algorithm with callbacks after each node is visited.
I really appreciate your help!
You database calls are asynchronous. That means they complete some time in the future, long after the .forEach() iteration has finished. If your database can handle a whole tree of queries being thrown at it at once (running all those queries essentially in parallel), then you can do something as simple as this:
let cntr = 0;
catTree.categoryTree.forEach(function(node){
tree.walkTree(node, 'children', function(obj){
//Cat is a mongoose model defined elsewhere
++cntr;
Cat.findById(obj.categoryId, {_id:0,name:1}).exec(function(err, value){
--cntr;
if (!err) {
obj.name = value.name;
}
// see if all requests are done
if (cntr === 0) {
console.log(JSON.stringify(catTree));
}
});
});
});
Anytime you're trying to coordinate more than one asynchronous operation, it usually makes sense to use promises (since that's exactly what they were built for) and mongoose has promises built in for queries. Here you collect a promise from each query into an array and then Promise.all() to tell you when they are all done.
let promises = [];
catTree.categoryTree.forEach(function(node){
tree.walkTree(node, 'children', function(obj){
//Cat is a mongoose model defined elsewhere
let p = Cat.findById(obj.categoryId, {_id:0,name:1}).exec().then(function(value) {
obj.name = value.name;
});
promises.push(p);
});
});
Promise.all(promises).then(function() {
console.log(JSON.stringify(catTree));
}).catch(function(err) {
// error
console.log(err);
});
Related
I would like to update a single value in a mongo document that is an array of arrays of arrays,
Here is the code that I have thus far... but it's not working. I think I need to use the $ operator on the chart to access the currently selected chorePerson... but not sure how that works.
I am really struggling with the mongoDB syntax and updating arrays
This is my data set, I have put in a t1, t2 and t3 in each of the respective arrays...
[
{
"_id": "5e3d891d956bb31c307d8146",
"t1": 0,
"affiliation": "800_800",
"year": 2020,
"month": "February",
"weekNumber": 6,
"weekStart": "02/02/2020",
"weekEnd": "02/08/2020",
"chart": [
{
"t2": 0,
"_id": "5e3d8a6358a3d92448e85aa5",
"ordinal": 5,
"ordinalString": "Friday",
"chorePerson": [
{
"completed": false,
"t3": 0,
"_id": "5e3d8a6358a3d92448e85aa7",
"person": "Frank",
"personID": "5e2891587678601434f6929c",
"phone": "8008008002",
"chore": "Another one",
"choreID": "5e39ca3949acee16fc280c98"
}
]
}
],
"date": "2020-02-07T16:03:47.770Z",
"__v": 0
}
]
I have been able to update t1 and t2 using this query:
{"_id":"5e3d891d956bb31c307d8146","chart._id":"5e3d9260fc365c2d080a32ce"}
and a set call of
{"$set":{"t1":1,"chart.$.t2":2}}
but I cannot figure out how to go down one more level in the arrays
Here is my query: I have hard coded the 'chart[ 1 ]' to try to force it to fetch only that record
{"_id":"5e3d891d956bb31c307d8146","chart._id":"5e3d8a6358a3d92448e85aa5","chart[1].chorePerson._id":"5e3d8a6358a3d92448e85aa7"}
This is my 'set' call
{"$set":{"t1":1,"chart.$.t2":2,"chart.$[].chorePerson.$.t3":3}}
When I run this in my program I get:
error Updating the path 'chart.$[].chorePerson.$.t3' would create a conflict at 'chart'
UPDATE 1
So I tried this:
{"_id":"5e3d93777f099b28b0fff2ae","chart._id":"5e3d953ed92a082738e8e2b9","chart.chorePerson._id":"5e3d953ed92a082738e8e2bb"}
{"$set":{"t1":1,"chart.$.t2":2,"chart.$.chorePerson.$.t3":3}}
which gave me:
error Too many positional (i.e. '$') elements found in path 'chart.$.chorePerson.$.t3
So I thought I would go back to the answer posted by whoami:
{"$set":{"t1":1,"chart.$.t2":2,"chart.$[].chorePerson.$.t3":3}}
Which gave me:
error Updating the path 'chart.$[].chorePerson.$.t3' would create a conflict at 'chart'
UPDATE 2
So I tried moving the [] in my set statement to the chart.$.chorePerson
{"$set":{"t1":1,"chart.$.t2":2,"chart.$.chorePerson.$[].t3":3}}
and selecting only the relevant chorePerson (5e3da771e08e3e31ac420004)
{"_id":"5e3da771e08e3e31ac41fffd","chart._id":"5e3da771e08e3e31ac420002","chart.chorePerson._id":"5e3da771e08e3e31ac420004"}
Which is getting me closer... now the data is being set in the correct chore chart and chart, but ALL of the chorePerson fields are being updated even though my select is supposed to only return the chore person '5e3da771e08e3e31ac420004'
After trial and error and lots of stack overflow articles, I have finally found the code that does what I need it to do:
var query = {"chart.chorePerson._id":idCP};
var update = {$set: {"chart.$.chorePerson.$[elem].completed":true, "chart.$.chorePerson.$[elem].completed_at": Date.now()}};
var options = {new: true, arrayFilters: [ { "elem._id": { $eq: idCP } } ]};
if(debugThis){
console.log("query " + JSON.stringify(query));
console.log("update " + JSON.stringify(update));
console.log("options " + JSON.stringify(options));
}
// see stackoverflow question:
// https://stackoverflow.com/questions/60099719/is-this-the-correct-use-of-the-operator-for-findoneandupdate/60120688#60120688
ChoreChart.findOneAndUpdate( query, update, options)
.then(chart => {
if(debugThis){
console.log('updated chart ' + JSON.stringify(chart));
}
return resolve(msgResponse);
})
.catch(err => {
msgResponse.message('error ' + err.message);
return resolve(msgResponse);
})
Which yields exactly what I want:
Many thanks to the people who wrote these articles:
https://docs.mongodb.com/manual/reference/method/db.collection.findOneAndUpdate/#behavior
Update nested subdocuments in MongoDB with arrayFilters
Updating a Nested Array with MongoDB
How to Update Multiple Array Elements in mongodb
arrayFilters in mongodb
https://github.com/Automattic/mongoose/issues/6386
I'm new to Node and Firebase.
I'm currently working on a crafting calculator for a game and have the game items stored in Firebase. Some items are composite items, for example:
1 Lumber Plank = 5 Logs
Based on such requirements, I've structured all the items as a single collection titled as items in Firebase.
Log would be persisted as:
{
"type": "basic",
"name": "log"
}
While lumber plank would be:
{
"type": "composite",
"name": "lumber plank",
"materials": ["log"],
"material_values": [5]
}
With such a structure, I'm trying to construct a crafting tree by recursively searching through the database. A final structure would look as such:
{
"name": "board",
"count": 1,
"materials": [
{
"name": "lumber plank",
"count": 1,
"materials": [
{
"name": "log",
"count": 5,
"materials": null
}
]
}
]
}
I'm having trouble with understanding the callbacks while debugging and this piece of code currently returns undefined followed by log (I'm assuming this comes from the console.log within the search function).
async function search(item, result, count) {
let calcItem = {
name: item,
count: count
};
db.collection("items")
.doc(item)
.get()
.then(doc => {
const data = doc.data();
if (data.type === basic) {
calcItem.materials = null;
result.push(calcItem);
return result;
} else {
let materials = data.materials;
let materialsCount = data.material_values;
calcItem.materials = [];
for (let i = 0; i < materials.length; i++) {
console.log(materials[i]);
search(materials[i], calcItem.materials, materialsCount[i]);
}
}
});
}
let item = "lumber plank";
search(item, [], 1).then(result => console.log(result));
Would appreciate any pointers/tips here. Thanks
Following feedback from Doug,
I've kinda refactored my code based on your comments and I'm seeing some progress.
function recursiveSearch(item, count, result) {
let calcItem = {
name: item,
count: count
};
dbSearch(item).then(function (doc) {
const data = doc.data();
console.log(data);
if (data.type === basic) {
calcItem.materials = null;
result.push(calcItem);
return result;
} else {
let materials = data.materials;
let materialsCount = data.material_values;
calcItem.materials = [];
for (let i = 0; i < materials.length; i++) {
recursiveSearch(materials[i], materialsCount[i], calcItem.materials);
}
}
});
}
function dbSearch(item) {
return Promise.resolve(db.collection("items")
.doc(item)
.get()
.then());
}
Log now outputs the search correctly.
{
material_values: [ 5 ],
materials: [ 'log' ],
name: 'lumber plank',
type: 'composite'
}
{
name: 'log',
type: 'basic'
}
However, if I understand it correctly, if I were to add in this line it's still going to return undefined, am I right?
console.log(recursiveSearch("lumber plank", 1, [])
If so, how do I actually log out the entire item structure whilst completing all the recursive searches?
Sorry if the question sounds kinda dumb. I primarily come from a Java background and dealing with promises/async/await is entirely new to me
You're not dealing with promises correctly. search doesn't actually return a "real" promise, but the caller is expecting it to do so. Since it's async, but doesn't return a value directly, it's actually returning is a promise that always resolves to undefined. It's also apparently intended to be a recursive function, which is making it harder to understand (did you mean to return the promise from the inner call to search?).
Minimally, you should start by making search return the promise chain that it establishes:
return db.collection("item")...get().then()
This will let you receive the value returned by the then callback.
I'll also point out that you're started to use async/await syntax, but never committed to using await to make this code more easy to read, which is a bit confusing.
I don't know if there is a word for this, guess there is, but right now I couldn't explain it better than "where array contains item(s) from array".
It might sound weird, but actually it not (I think), and I'm having a hard time figuring out how I can do this in Azure CosmosDB.
Here goes. I have a document like this (simplified):
{
"id": "2a62fcf4-988f-4ebe-aedc-fb0c664b85d8",
"Title": "Seks års fængsel for overgreb",
"ZipCodes": [
{
"Code": "6500",
"Name": "Vojens",
"FoundViaTerm": "Vojens"
},
{
"Code": "6400",
"Name": "Sønderborg",
"FoundViaTerm": "Sønderborg"
},
{
"Code": "6700",
"Name": "Esbjerg",
"FoundViaTerm": "Esbjerg"
}
],
"_rid": "k1sVAPf7SQAMAAAAAAAAAA==",
"_self": "dbs/k1sVAA==/colls/k1sVAPf7SQA=/docs/k1sVAPf7SQAMAAAAAAAAAA==/",
"_etag": "\"00001000-0000-0000-0000-5a14898e0000\"",
"_attachments": "attachments/",
"_ts": 1511295374
}
Ok, now I want to query documents like this and find all, where ZipCodes.Code is in a list of zipcodes, ex. ('6500', '2700').
I'm puzzle here...
I found the ARRAY_CONTAINS method and it works, if I only come in with one zipcode - my problem is I come with a list.
Hope somebody can help, thanks in advance.
Per my experience , expr in ARRAY_CONTAINS (arr_expr, expr [, bool_expr]) method is not supported list arguments.
According to your situation , I suggest you use UDF in Cosmos DB.
I created 3 sample documents as your description.
[
{
"id": "1",
"zip": [
{
"code": "1111"
},
{
"code": "2222"
}
]
},
{
"id": "2",
"zip": [
{
"code": "2222"
},
{
"code": "3333"
}
]
},
{
"id": "3",
"zip": [
{
"code": "4444"
},
{
"code": "1111"
},
{
"code": "2222"
}
]
}
]
Please refer to the snippet of UDF code as below :
function test(zipcode){
var arrayList = ["1111","2222"]
var ret = false ;
for(var i=0 ;i <zipcode.length;i++){
if(arrayList.indexOf(zipcode[i].code)){
ret= true;
}else{
ret = false;
break;
}
}
return ret;
}
You could select zip array (select c.zip from c) ,then loop the results and invoke the UDF above in your code with the zip[i] arguments.
Hope it helps you.
Just for summary:
Use the IN operator from Cosmos DB SQL APIs to query entry which is included in the list condition.
Like
SELECT * FROM c WHERE c.ZipCodes[0].Code IN ("6500", "6700")
Or
SELECT DISTINCT c FROM c JOIN zc IN c.ZipCodes WHERE zc.Code IN ("2720", "2610")
I would like to propose another solution to this problem.
Use EXISTS with ARRAY_CONTAINS in this way:
SELECT * FROM c
WHERE EXISTS
(SELECT VALUE z FROM z in c.ZipCodes WHERE ARRAY_CONTAINS(["6500","6700"], z))
You can do something like this:
For each item in ZipCodes, you get a zip and compare with the array of codes you are checking. This, IMHO, is much better than using UDF.
{
query: '
SELECT DISTINCT value r
FROM root r
JOIN zip IN r.zipCodes
WHERE ARRAY_CONTAINS(#zipIds, zip, true)
',
parameters: [{name: "#zipIds", value: zipIds}]
}
The last param of ARRAY_CONTAINS tells the function to accept partial matches.
Apart from the fact that using UDF looks as the easier option, i would not use UDFs in your query's filter, since it compromises the performance of your query. I faced the same problem in my work environment, where things are designed to use UDFs to help in the queries, but the reality is that most of the times we are doing queries by using single values, and using UDF will actually result on the query not using the index. So in that case if you need to validate multiple values in the array, depending on the volume of values you need to validate, you can always write something like ARRAY_CONTAINS(c, 1) or ARRAY_CONTAINS(c, 2) or ....
It doesn't look so elegant solution, but will ensure that it will use the index and will do the best performance in your query.
Just started with Firebase and loving it.
Was watching this video https://www.youtube.com/watch?v=Idu9EJPSxiY and wanted some help with a particular query. If anyone can help?
In the schema/structure below if i wanted to query all the events that a person named "David" attended, what would it be?
{
"users": {
"key1": {
"name": "David",
"age": 33,
"profileImg": "http/......"
}
"key2": {
"name": "John",
"age": 25,
"profileImg": "http/......"
}
},
"events": {
"fm": {
"name": "event1",
"date": 12332443456
}
},
"eventAttendees": {
"fm": {
"key1": "David",
"key2": "John"
}
}
}
I'm kinda stuck and would like some help. Thanks.
DB structure
You can try something like this (I'm using node.js firebase module to show the example)
firebase.database().ref('eventAttendees').once('value', function (eventAttendeesList) {
var events = []; //array to store the events
eventAttendeesList.forEach(function (eachEvent) { //forEach list of eventAttendees node, Eg: fm
eachEvent.forEach(function (data) { //forEach value of current eventAttendees, Eg: jey1: "David", key2: "John"
if (data.val() == "David") {
firebase.database().ref('events').child(eachEvent.key).once('value', function (snapshot) { // get the event data
events.push(snapshot) // add to array of events David attended
})
} //end of if
})// end of eachEvent.forEach
})// end of eventAttendeesList.forEach
return events;
})
I might have some syntax error somewhere, but you get the idea
First is to query for the list of data under the eventAttendees node ("fm") and loop through each value
In each of your value for the eventAttendees node there are your key1, key2 which we need to do another loop to get the data ("David", "John"). Here we will check if the data matches the person you are looking for
If it matches we will query firebase for the event data under the events node with the key that matches eachEvent.key and add it to our events array
I have an app that makes requests to 3 different API REST servers, gather and manipulate the info received into objects and uses mongoose's model to send the info different Mongo collections (one for each API REST).
My objective is to have a function operate on the different sets of info right before or at the same time (or right after) it is sent to the MongoDB.
Here's an example with 2 concrete objects that are being retrieved and sent to the db:
Object1:
{
"name": "X",
"a": 10,
"b": 9,
"n": 1502642712481,
"iname": "ix",
"_id": "59907f34040eb562d8d11d8c"
}
Object2:
{
"name": "Y",
"n": 1502642712552,
"iname": "ix",
"_id": "59907f34040eb562d8d11d8d",
"b": "10",
"a": "9",
}
These have their own models and are correctly being inserted into the MongoDB with a save and I want to compare the "a" and "b" keys of each objects (FYI: "iname" is a common key for both objects and "n" is a timestamp created at the moment the request is being returned).
The type of operation I want to make on 'a' and 'b' are relatively basic mathematics (at the moment): ax - ay and bx -by
Any idea on how I could do that ?
Right now the code to save each object to the db are nested within an infinite loop and looks like:
async function loopY() {
setTimeout(
async function () {
var list = ['Y'];
var data = await y(list); //<---- This is the call to wait whatever the request of Y will return
data.forEach((object) => {
if (object.name === 'Y') {
var iname = 'ix'
} else {
var iname = 'N/A' };
var tick = new Ymodel({
mk: object.mk,
name: object.name,
a: object.a,
b: object.b,
n: object.n,
iname: iname,
});
// tick.sendToCompare(); //<---- HERE I THOUGHT OF USING A FUNCTION FROM THE MODEL BUT NOT SURE HOW TO HANDLE THE DIVISION BETWEEN THE INFORMATION
tick.save(function(err, tick) {
if (err) return console.log(err);
console.log(`[SUCCESS]: ${tick.name} added to db!`);
});
});
loopY();
}
}, 1000); // <------- THE TIMEOUT HELPS ME MANAGE THE REQUESTS TO THE API SERVERS AND AVOID A "TOO MANY REQUESTS" ERROR.
};
loopY();
I have the exact same function for "X" (loopX()) with the slight difference that it adds an "sn" key to the object.
I hope this was clear enough and respected the rules... Also, thanks a lot in advance for your help!