Develop Reference

Pass query criteria to mongoDB aggregation

our current setup is: SPA frontend, Azure functions with mongoose middleware, MongoDB
(Maybe first read the question***)
Since we have a lot of documents in our DB and our customer wants to query them we are facing the following problem:
The user is assigned to his organization. He wants to search for Doc1s he has not responded to.
Doc1
{
_id
organization -> partitionKey
content
}
By creating doc2 with reference to doc1 he can respond.
Doc2
{
_id
organization -> partitionKey
Doc1ref
content
}
We have a 1:n relationship.
At the moment we filter just by query criteria of doc1 with limit and skip options.
But the new requirement is to filter the same way by referring doc2s.
I was thinking of:
Doing it in my code => Problem: after we have read with limit=100 and I filter it by my code, the result is not 100 anymore.
Extending doc1 by doc2 arrays => Must be the last option
Dynamic aggregation, Prepared in the code and executed at runtime => Don't want to user dynamic aggregations and the benefits of mongoose are almost lost.
Create a MongoDB view with lookup aggregation (populating doc1 by doc1.respondedOrganizations) => Problem is see here is the performance. When searching a lot of documents and then joining them by a non partitionKey.
*** So, I come to my question:
Is it possible to pass a virtual (not existing) query criteria...
doc1.find({ alreadyResponded : my.organization } )
...and use it as input variable in an aggregation
{
$lookup: {
from: Doc2s,
localField: _id,
foreignField: Doc1ref,
as: < output array field >
pipeline: [{
$match: {
$organization: {
$eq: $$alreadyResponded
}]
}
}
It would reduce query performance extremly.
Thanks

Updating Query in Firestore

I am new to Firestore, need some help on firestore update.
I have following structure and wants to update "employee name" property. Not sure how to select and update.
Department:[
Name: Accounts
Employee:[
{Name :David,
Age :25},
{Name:Paul,
Age:27}
]
]
Here is what I was trying to do:
let depempCollectionRef = admin.firestore().collection('DepEmployee').doc('depempid')
depempCollectionRef.Department.Employee
.update({ name: 'Scott' },{merge:true})
.then(function() { console.log("Document successfully updated!"); })

Employee is just an embedded data structure in your Firestore document – so you can't address it through the reference directly. As far as Firestore is concerned, Employee is just an attribute on the Department document as Name is.
Before I propose a solution, let me point out two things:
If using update, you don't need {merge: true}. You use {merge: true} together with set to get an update-like behavior, if the document already exists.
I wouldn't use an Array of employees. It might make more sense to store the employees in their own collection in Firestore and then just list their reference IDs (= foreign keys) here. As a general rule of thumb: try to keep your data structure flat. Also use Arrays only, if you need to maintain a certain order of items.
A) If you have a separate collection for employees, updating the name is as easy as:
employeeCollection.doc('101').update({name: 'Scott'})
B) If you want to store employee data within your department document, I would still store them as a map with IDs (instead of an Array) and then access them with dot notation:
Department:[
Name: Accounts
Employees:{
101: {
Name :David,
Age :25
},
102: {
Name:Paul,
Age:27
}
}
]
depempCollectionRef.Department
.set({ ['101.name']: 'Scott' }, {merge:true})
C) And if you really want to store the data embedded in an Array, I believe you have to read and update the whole Array (not sure, if there is a better solution):
const employeesSnap = await depempCollectionRef.Department.get()
const updatedEmployees = changeNameOfScottInArray()
depempCollectionRef.Department
.update({ 'Employees': updatedEmployees })
I didn't test this code, but I hope you get the gist of it!
I'd recommend you flatten your data structure by creating a separate Employee collection and then just referencing them by their foreign key in your department (solution A).

Select all the fields in a mongoose schema

I want to obtain all the fields of a schema in mongoose. Now I am using the following code:
let Client = LisaClient.model('Client', ClientSchema)
let query = Client.findOne({ 'userclient': userclient })
query.select('clientname clientdocument client_id password userclient')
let result = yield query.exec()
But I want all the fields no matter if they are empty. As always, in advance thank you

I'm not sure if you want all fields in a SQL-like way, or if you want them all in a proper MongoDB way.
If you want them in the proper MongoDB way, then just remove the query.select line. That line is saying to only return the fields listed in it.
If you meant in a SQL-like way, MongoDB doesn't work like that. Each document only has the fields you put in when it was inserted. If when you inserted the document, you only gave it certain fields, that document will only have those fields, even if other documents in other collections have different fields.
To determine all available fields in the collection, you'd have to find all the documents, loop through them all and build an object with all the different keys you find.
If you need each document returned to always have the fields that you specify in your select, you'll just have to transform your object once it's returned.
const fields = ['clientname', 'clientdocument', 'client_id', 'password', 'userclient'];
let Client = LisaClient.model('Client', ClientSchema)
let query = Client.findOne({ 'userclient': userclient })
query.select(fields.join(' '))
let result = yield query.exec()
fields.forEach(field => result[field] = result[field]);
That forEach loop will set all the fields you want to either the value in the result (if it was there) or to undefined if it wasn't.

MongoDB is schemaless and does not have tables, each collection can have different types of items.Usually the objects are somehow related or have a common base type.
Retrive invidual records using
db.collectionName.findOne() or db.collectionName.find().pretty()
To get all key names you need to MapReduce
mapReduceKeys = db.runCommand({
"mapreduce": "collection_name",
"map": function() {
for (var key in this) {
emit(key, null);
}
},
"reduce": function(key, stuff) {
return null;
},
"out": "collection_name" + "_keys"
})
Then run distinct on the resulting collection so as to find all the keys
db[mapReduceKeys.result].distinct("_id") //["foo", "bar", "baz", "_id", ...]

Filtering user Ids from an array of users

I wish to find all users, not the current User. A pair of users are stored within a "Room" array under this collection structure:
structure of each room (from another html page)
var newRoom = Rooms.insert({
owner : Meteor.userId(),
receiver : receiver,
people : [ owner , receiver ],
});
Collection.js (using dburles collection helper)
Rooms.helpers({
receiverName: function() {
return Meteor.users.findOne({ _id: this.receiver }).username;
}
});
html
<!-- **allRooms.html** Works fine, names appear -->
{{#each rooms}} {{receiverName}}{{/each }}
<!-- **roomDetail.html** names dont show, this.receiver undefined -->
{{receiverName}}
roomDetail js template helper
self.subscribe('room', Router.current().params._id);
self.subscribe('users');
});
How do I return and display the user's Id thats not the current user from the people field which is an array? I hope to show it in the child page (roomDetail).

Assuming:
Rooms is a collection, and you already have a room document to search on.
You only want to fetch a single user.
Give this a try:
// The list of userIds in room minus the current user's id.
var userIds = _.without(room.People, Meteor.userId());
// Assuming we want only one user...
var user = Meteor.users.findOne({ _id: userIds[0] });
Some thoughts about your original code:
You can't include references to Rooms in your Meteor.users selector unless Rooms is a field of users. Mongo has no notion of joins.
$ne isn't that you want. If you had 100 users published, and your array only contained 2 users (one of which you didn't want), using $ne would return 99 users.
Based on your comments, it looks like you need this in a collection helper. Maybe something like this:
Rooms.helpers({
findUser: function() {
var userIds = _.without(this.People, Meteor.userId());
return Meteor.users.findOne({ _id: userIds[0] });
},
});
And then elsewhere in your code, for a given room instance you could do:
room.findUser()

mongo find selector causes unmatching results to be returns

userLink_titles = Entries.find({ _id:"bxSbMgszYxbCqDonF"})
returns:
docs: Object
CBqHrJvTE8xz7u2Rz: Object
_id: "CBqHrJvTE8xz7u2Rz"
author: "AHSwfYgeGmur9oHzu"
Q8m7PMbQr62E3A73f: Object
_id: "Q8m7PMbQr62E3A73f"
author: "AHSwfYgeGmur9oHzu"
bxSbMgszYxbCqDonF: Object
_id: "bxSbMgszYxbCqDonF"
author: "AHSwfYgeGmur9oHzu"
As you can see it returns the correct document but it also returns incorrect documents.
findOne: userLink_titles = Entries.findOne({ _id:"bxSbMgszYxbCqDonF"}) works as expected and only returns the correct document.
I would use the findOne except that the end intention is to make the _id selector an array such that find would return the documents for all documents that match the _id selectors in the array.
Bonus Points:
Let's say I wanted to retrieve the titles of a set of articles, where the references to those articles (the _ids of the Articles in their Articles collection) have been saved in a user collection.
So effectively I would retrieve the Article references from the user collection, and use those references to retrieve the titles of Articles from the Articles collection.
What would the code/pseudo code look like for that? Something like the following (I am assuming the below is a complete bastardization of some best practices for querying/retrieving records)
user_profile = Users.findOne({username : "Frank"});
user_saved_articles_ids = user_profile.findOne({saved_articles_ids});
userLinks = Articles.find({ _id:user_saved_articles_ids});
userLinksTitles = Articles.find({titles});

For those interested in the bonus question, you want MongoDB's $in operator.
If this will be used in a publish method on the server then you may want to check out publish-with-relations.
Something like the function below would be an efficient way to retrieve the information. It makes two calls to the database. Using the fields parameter prevents unnecessary data from being sent between the db and the webserver.
/**
* Titles of a User's saved articles
*
* #method getSavedTitles
* #param {String} username
* #return {Array} a list of article titles
*/
function getSavedTitles (username) {
var user,
ids,
linkedArticles,
titles;
user = Users.findOne({username: username},
{fields:{ _id:0, profile:1 }});
if (!user || !user.profile) {
throw new Meteor.Error(500, 'Missing user profile');
}
ids = user.profile.savedArticleIds;
if (!ids || !_.isArray(ids)) {
throw new Meteor.Error(500, 'Missing saved article ids');
}
linkedArticles = Articles.find({_id: {$in: ids}},
{fields:{ _id:0, title:1 }});
titles = _.pluck(linkedArticles.fetch(), "title");
return titles;
} // end getSavedTitles

If you want to inspect the results of a query in the console, use Entries.find(...).fetch()
Entries.find() returns a cursor, which is a construct used to render lists efficiently, so that only the entries that change need to be re-rendered. Returning cursors from helpers on which you use {{#each}} will lead to more responsive apps.