Develop Reference

Update database entry using mongoose

Hello i am using mongoose.
I have built this query that finds my desired project :
const projects = await ClientManagers.findOne({'project.contactPerson.work_email' : 'testing#email.com'} , { 'project.$.companyName': 1 });
this returns an object from my database like this :
{
'projectName' : 'x',
'companyName' : 'x bv'
}
How can i update the company name to be 'Y bv' instead of 'x bv'.

Assuming this is your document structure,
{
"_id" : ObjectId("5f2ae5a4b1549ac0460920dd"),
"projectName" : "A",
"project" : [
{
"companyName" : "T1",
"contactPerson" : {
"work_email" : "t1#gmail.com"
}
},
{
"companyName" : "T2",
"contactPerson" : {
"work_email" : "t2#gmail.com"
}
}
]
}
Single Update updateOne()
If you know email will be unique and want to update single document then use updateOne().
first is query part to find condition, email t1#gmail.com
second is set/update part, here $ is for array because project is an array, update companyName to T1 Company
await ClientManagers.updateOne(
{ 'project.contactPerson.work_email': 't1#gmail.com' },
{
$set: { "project.$.companyName": "T1 Companmy" }
}
)
Multiple Update updateMany()
If email is not unique and want to update everywhere then use updateMany(), it will update every matching documents.
await ClientManagers.updateMany(
{ 'project.contactPerson.work_email': 't1#gmail.com' },
{
$set: { "project.$.companyName": "T1 Company" }
}
)
Not suggesting update() method to use, because its deprecated in mongoose and will give Deprecation Warnings
, this function is replaced with updateOne(), updateMany() and replaceOne() methods.

Good start. Mongo has better documentation with examples. I suggest you to refer that also.
use update
db.collection.update({companyName:'x bv'}, {"$set":{"companyName":y}})
Mongo is case sensitive. So name should match exactly.
update updates one document. To update multiple, use updateMany or multi:true option with update or findOneAndMondify for one update for find and update case.

Update method returns weird data, not updated data

I have a problem with update method which returns this object when I run my endpoint
{ n: 1, nModified: 1, ok: 1 }
This is the code which I tried, and I tried with { new: true } but that doesnt help, i want to get updated data back.
router.put('/:username/experience/edit/:id', function(req, res) {
const { title, company, location, from, to, workingNow, description } = req.body;
User
.update({'experience._id': req.params.id},
{'$set': {
'experience.$.title': title,
'experience.$.company': company,
'experience.$.location': location,
'experience.$.from': from,
'experience.$.to': to,
'experience.$.workingNow': workingNow,
'experience.$.description': description,
}},
function(err, model) {
console.log(model);
if(err){
return res.send(err);
}
return res.json(model);
});
})

If you are on MongoDB 3.0 or newer, you need to use the .findOneAndUpdate() and use projection option to specify the subset of fields to return. You also need to set returnNewDocument to true. Of course you need to use the $elemMatch projection operator here because you cannot use a positional projection and return the new document.
As someone pointed out:
You should be using .findOneAndUpdate() because .findAndModify() is highlighed as deprecated in every official language driver. The other thing is that the syntax and options are pretty consistent across drivers for .findOneAndUpdate(). With .findAndModify(), most drivers don't use the same single object with "query/update/fields" keys. So it's a bit less confusing when someone applies to another language to be consistent. Standardized API changes for .findOneAndUpdate() actually correspond to server release 3.x rather than 3.2.x. The full distinction being that the shell methods actually lagged behind the other drivers ( for once ! ) in implementing the method. So most drivers actually had a major release bump corresponding with the 3.x release with such changes.
db.collection.findOneAndUpdate(
{
"_id": ObjectId("56d6a7292c06e85687f44541"),
"rankings._id" : ObjectId("46d6a7292c06e85687f55543")
},
{ $inc : { "rankings.$.score" : 1 } },
{
"projection": {
"rankings": {
"$elemMatch": { "_id" : ObjectId("46d6a7292c06e85687f55543") }
}
},
"returnNewDocument": true
}
)
From MongoDB 3.0 onwards, you need to use findAndModify and the fields options also you need to set new to true in other to return the new value.
db.collection.findAndModify({
query: {
"_id": ObjectId("56d6a7292c06e85687f44541"),
"rankings._id" : ObjectId("46d6a7292c06e85687f55543")
},
update: { $inc : { "rankings.$.score" : 1 } },
new: true,
fields: {
"rankings": {
"$elemMatch": { "_id" : ObjectId("46d6a7292c06e85687f55543") }
}
}
})
Both queries yield:
{
"_id" : ObjectId("56d6a7292c06e85687f44541"),
"rankings" : [
{
"_id" : ObjectId("46d6a7292c06e85687f55543"),
"name" : "Ranking 2",
"score" : 11
}
]
}

Sequelize "WHERE" Clause in subqueries

I'am trying to build my query using sequelize, in the where clause I need to give the conditional value from my front-end so i did it like this :
getResults(req) {
return parm
.findAll({
attributes: [
sequelize.literal('DISTINCT "id"')
],
where : {
name: req.query.parm.replace(/"/g, '').split(',')
} ,
raw: true
});
}
and it's working!
but now I need to write a subquery including where clause:
something like this :
SELECT tab1.name FROM
(SELECT name FROM "MYTABLE"
WHERE id = (value from the front-end) AND name IN (values from front-end)
) as tab1
Here is what i have tried :
getTest(req) {
if (req.query.parm != null) {
return parm .sequelize.query('SELECT id FROM "table_base" where id = $mid ); ',
{ type: sequelize.QueryTypes.SELECT ,
bind: { mid: [req.query.parm.replace(/"/g, '').split(',')] }} );
}
},
i tried to use raw query and i tested the binding parameters but i get this error when i execute this testing query :
Executing (default): SELECT id FROM "table_base" where id = $1 );

The answer to your question is YES it is indeed possible! SQL can pretty much do anything even if you are using sequelize. If you write the subquery and it doesn't work just post it back here so people can take a look and debug. Thanks

How to Speed Up Count on MongoDB View

I have been troubleshooting why a MongoDB view I created is so slow. The view targets the transactions collection, and returns records that have an openBalance that is greater than 0. I also run some additional aggregation stages to shape the data the way I want it.
In order to speed up the execution of the view it makes use of an index on the targeted collection by matching on the indexed field in stage one of the view's aggregation pipeline, like so:
// View Stage 1
{ "transactions.details.openBalance" : { "$exists" : true, "$gt" : 0.0 } }
After much investigation I have determined that the aggregation from the view returns data very quickly. What's slow is the count that's run as part of the endpoint:
let count = await db.collection('view_transactions_report').find().count();
So what I'm trying to figure out now is why the count is so much slower on the view than on the underlying collection, and what I can do to speed it up. Or, perhaps there's an alternative way to generate the count?
The underlying collection has something like 800,000 records, but the count returns quickly. But the count on the view, which only returns a filtered set of 10,000 of those initial 800,000 records, returns much more slowly. In terms of specifics, I'm talking about 3/4 of a second for the count on the collection to return, verses six seconds for the count on the mongo view to return.
So, first off, why is the count so much slower on the view (with it's much smaller data set) than on the underlying collection, and secondly, what can I do to address the speed of the count for the view?
I have a couple other aggregation queries I'm running, to determine totalCustomers and totalOpenBalance, that also seem to run slow (see code below).
The relevant part of my endpoint function code looks like this:
// previous code
let count = await db.collection('view_transaction_report').find(search).count();
let totalCustomers = await db.collection('view_transaction_report').find(search).count({
$sum: "customer._id"
});
let result = {};
if (totalCustomers > 0) {
result = await db.collection('view_transaction_report').aggregate([{
$match: search,
},
{
$group: {
_id: null,
totalOpenBalance: {
$sum: '$lastTransaction.details.openBalance'
}
}
}
]).next();
}
db.collection('view_transaction_report').find(search).skip(skip).limit(pagesize).forEach(function (doc) {
docs.push(doc);
}, function (err) {
if (err) {
if (!ioOnly) {
return next(err);
} else {
return res(err);
}
}
if (ioOnly) {
res({
sessionId: sessID,
count: count,
data: docs,
totalCustomers: totalCustomers,
totalOpenBalance: result.totalOpenBalance
});
} else {
res.send({
count: count,
data: docs,
totalCustomers: totalCustomers,
totalOpenBalance: result.totalOpenBalance
});
}
});
In terms of executionStats, this is what shows for the queryPlanner section of the generated view:
"queryPlanner" : {
"plannerVersion" : 1.0,
"namespace" : "vio.transactions",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [
{
"transactions.details.openBalance" : {
"$gt" : 0.0
}
},
{
"transactions.destails.openBalance" : {
"$exists" : true
}
}
]
},
"winningPlan" : {
"stage" : "CACHED_PLAN",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"transactions.details.openBalance" : {
"$exists" : true
}
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"transactions.details.openBalance" : 1.0
},
"indexName" : "openBalance",
"isMultiKey" : true,
"multiKeyPaths" : {
"transactions.details.openBalance" : [
"transactions",
"transactions.details"
]
},
"isUnique" : false,
"isSparse" : true,
"isPartial" : false,
"indexVersion" : 2.0,
"direction" : "forward",
"indexBounds" : {
"transactions.details.openBalance" : [
"(0.0, inf.0]"
]
}
}
}
},
"rejectedPlans" : [
]
}

In the comments, #Wan Bachtiar mentioned that "openBalance" looks to be a multikey index. To clarify, yes, in the targeted collection, the "openBalance" field is an embedded field within an array. This is the case even though, in the view, the data is shaped in such a way that "openBalance" is an embedded field that is not within an array.
The multikey index on the targeted collection is where the issue lies, because instead of a 1 for 1 document situation, Mongo needs to look through every array element pertaining to this "openBalance" field, which, logically, dramatically increases the scan time - because sometimes there are many, many array elements pertaining to this particular field.
After some further checking, I realized I can address this issue by changing how I populate "openBalance" to our mongo collection via the ETL. By making this change I'll be able to make "openBalance" a standard index, rather than a multikey index, which in turn will allow mongo to search a much smaller data set in order to return my counts.

Firebase: How do I retrieve records from my data for which a specific key exists?

I have data in firebase that looks like this:
"application": {
"companies": {
"firebase": {
"creation": {
"name": "Firebase Inc",
"location": "USA"
},
"google": {
"creattion": {
"name": "Google Inc",
"location": "USA"
}
}
"facebook": {
},
"apple": {
}
}
}
}
There are tens of thousands of records under companies key. How do i efficiently execute following queries?
How do I query only the records for which key creation is present under their name?
How do I query only the records that DO NOT have key creation present under their name?
I also want to call .on('child_added') on the returned result set so that I can process only those specific records later on. Is it possible?

EDIT: Simpler way without using an extra parameter
Queries
Here are the queries to do this without having to use an extra parameter:
Find the companies without creation:
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").equalTo(null);
Find the companies with creation:
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").startAt(!null);
You would add ".indexOn": "creation" to the rules.
Edit 2: I was curious, so I pushed 11,000 records to /companies2 (half with creation children, half without). I was able to retrieve 5500 matching records in ~4 seconds using the above queries (or one of the variants I've shown below).
Edit 3: If you're running these queries frequently, it might be worth it to separate children of /companies into two bins based the presence of creation. That way, you can read the two segments separately without having to rely on queries.
Factory
Here is what the revised factory would look like (I've revised the PLNKR to match):
app.factory("CompaniesFactory",function($q, fbUrl){
return function(hasCreation){
var deferred = $q.defer();
var ref = new Firebase(fbUrl+'/companies').orderByChild("creation");
var query;
if (hasCreation) {
query = ref.startAt(!null);
// or:
// query = ref.startAt(true);
} else {
query = ref.equalTo(null);
// or:
// query = ref.endAt(!null);
// query = ref.endAt(true);
}
query.once("value", function(dataSnapshot){
deferred.resolve(dataSnapshot.val());
}, function (error) {
deferred.reject(error);
});
return deferred.promise;
}
});
And yes, it is possible to call .on('child_added') on the returned dataSnapshot. See DataSnapshot.ref().
Original answer using an extra parameter:
(Keeping this for reference)
Another way to do it would be by adding another parameter called hasCreation to children of companies that have creation, and query by that.
Data
The query would then be var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
If hasCreation in the query is null, the query will return the companies without a hasCreation child.
If hasCreation in the query is true, the query will return the companies with hasCreation===true.
{
"company1" : {
"creation" : {
"name" : "company1"
},
"hasCreation" : true
},
"company2" : {
"name" : "company2"
},
"company3" : {
"name" : "company3"
},
"company4" : {
"creation" : {
"name" : "company4"
},
"hasCreation" : true
}
}
Rules
You would add the ".indexOn" : "hasCreation" to your rules like so:
"so:29179389":{
".read" : true,
".write" : true,
"companies" : {
".indexOn" : "hasCreation"
}
}
Companies Factory
app.factory("CompaniesFactory",function($q, fbUrl){
return function(hasCreation){
var deferred = $q.defer();
if (!hasCreation) {
hasCreation = null;
}
var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
ref.once("value", function(dataSnapshot){
deferred.resolve(dataSnapshot.val());
});
return deferred.promise;
}
});
Controller
app.controller('HomeController',function($scope,fbUrl,CompaniesFactory) {
$scope.getCompanies = function(hasCreation) {
var companies = new CompaniesFactory(hasCreation).then(function(data){
console.log(data);
$scope.companies = data;
});
}
});
HTML
<body ng-app="sampleApp">
<div ng-controller="HomeController">
<button ng-click="getCompanies(true)">Find with creation</button>
<button ng-click="getCompanies(false)">Find without creation</button>
<h2>Companies:</h2>
{{companies}}
</div>
</body>

What I would do, is I would set a condition to verify if your xxx.firebaseio.com/Application/companies/______/creation exists. In a empty blank, you can set for loop to irritate over the array of companies.Then, you can create two arrays with angular.forEach: one including those companies which do have 'creation', and the other array in which the elements do not include the 'creation'.
Hope that helps :)
Edit:
There is another approach to this question, in this thread:
Angularfire: how to query the values of a specific key in an array?