Let's say I have the following array of objects:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Sent',
order: 5
}
]
I was wondering, I would like to de-dupe objects within this array, but only de-dupe the "oldest" object with a duplicated status, that is, for the above array, I would like to end up with the following de-duped array:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 4
},
{
status: 'Sent',
order: 5
}
]
That is, status Ordered 4 is "younger" than status Ordered 3 ... and not simply to just de-dupe...this could extended to the following:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
]
will result in:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
]
N.B. There is one caveat, I don't actually have the order key in the object, this is purely for demonstrative purposes to demonstrate what I mean for the purposes of this question..
Pretty simple reduce function. Basically we create an object (which can only have one value per key). The "last" entry in your array with a specific status will be preserved.
const input = [
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
];
const output = Object.values(input.reduce((a, e) => {
a[e.status] = e;
return a;
}, {}));
console.log(output);
A one-liner with a Map:
const arr = [
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
];
const deduped = [...new Map(arr.map(o => [o.status, o])).values()];
console.log(deduped);
JS doesn't include a dedup/distinct function natively so you have to make your own such as by using a filter to check for duplicates
the standard call back for the filter is (item, index, array)
this means that you can do
filter((item,idx,arr)=> arr.findIndex(other=>areEqual(item,other)) != idx)
this reads as remove all elements where the they have previously existed in the array because findIndex returns the location of the first match, so if they current index doesn't equal it then its not the first match
the reason for findindex rather than indexOf is that findIndex allows you to specify how to check for equality
this is where the areEqual callback comes in, it takes 2 items and returns true if they are equal
so could be
areEqual(a,b){
return a.field === b.field
}
of course this is overkill for just a.field === b.field
and could be rewritten as
filter((item,idx,arr)=> arr.findIndex(other=>item.field === other.field) != idx)
but you indicate that your test for equality might be more complex than a single known field so the callback to areEqual allows much more complex logic
unfortunately there isn't a findLastIndex but simply reversing the array before filtering will fix that, then reverse the result to regain the original order
Related
I have this one schema
{
_id: "123456",
id: "123",
inventory: [
{
id: "foo",
count: 0
},
{
id: "bar",
count: 3
}
]
}
I wanted every "count" keys in the inventory array to be "price" which will look like this at the end:
{
_id: "123456",
id: "123",
inventory: [
{
id: "foo",
price: 0
},
{
id: "bar",
price: 3
}
]
}
And I've tried this
Model.updateOne({ id: "123" }, { $unset: { inventory: [{ count: 1 }] } } )
But it seems to be deleting the "inventory" field itself
The first thing here is to try to use $rename but how the docs explain:
$rename does not work if these fields are in array elements.
So is necessary to look for another method. So you can use this update with aggregation query:
This query uses mainly $map, $arrayToObject and $objectToArray. The trick here is:
Create a new field called inventory (overwrite existing one)
Iterate over every value of the array with $map, and then for each object in the array use $objectToArray to create an array and also iterate over that second array using again $map.
Into this second iteration create fields k and v. Field v will be the same (you don't want to change the value, only the key). And for field k you have to change only the one whose match with your condition, i.e. only change from count to price. If this condition is not matched then the key remain.
db.collection.update({},
[
{
$set: {
inventory: {
$map: {
input: "$inventory",
in: {
$arrayToObject: {
$map: {
input: {$objectToArray: "$$this"},
in: {
k: {
$cond: [
{
$eq: ["$$this.k","count"]
},
"price",
"$$this.k"
]
},
v: "$$this.v"
}
}
}
}
}
}
}
}
])
Example here
I have a filter object that is returned by query params
url = /all?channels=calls,text&calls=voicemail,missed
const query = {
channels: 'calls,texts',
calls: 'voicemail,missed',
};
I then have an array of objects that come in from a socket.
const arr = [
{
id: 1,
channel: 'SMS',
sent: '2021-08-22T03:21:18.41650+0000',
sender: {
contactType: 'business',
},
recipients: [
{
contactType: 'corporate',
},
],
direction: 'INBOUND',
},
{
id: 2,
channel: 'VOICE',
sent: '2021-08-20T23:15:56.00000+0000',
sender: {
contactType: 'business',
},
recipients: [
{
contactType: 'corporate',
},
],
callDetails: {
answered: false,
voicemail: true,
},
direction: 'INBOUND',
},
{
id: 3,
channel: 'VOICE',
sent: '2021-08-20T23:15:56.00000+0000',
sender: {
contactType: 'business',
},
recipients: [
{
contactType: 'corporate',
},
],
callDetails: {
answered: true,
voicemail: false,
},
direction: 'INBOUND',
},
{
id: 4,
channel: 'VOICE',
sent: '2021-08-20T23:15:56.00000+0000',
sender: {
contactType: 'business',
},
recipients: [
{
contactType: 'corporate',
},
],
callDetails: {
answered: false,
voicemail: false,
},
direction: 'INBOUND',
},
];
I want to filter out the objects that match the filters but the query obj isn't friendly enough to just map the arr through.
With the query obj shared above, i should return the objects id:1 and id:2 and id:4 from arr, since those object meet the criteria of sms, voicemail, & missed
I assume i need a modified query obj that has to have various conditions available for each property, i.e calls: voicemail === callDetails.voicemail === true or calls: received === callDetails.answered === true
I've seen lots of examples on how to filter an array of objects with multiple match-criteria, but with the req of the property having multiple conditions, i've hit a wall.
thanks for the help
The main idea is to provide kind of a rosetta stone which does bridge/map the query specific syntax with any list item's specific data structure. Thus one will end up writing a map which takes a query's structure into account but ensures for each necessary query endpoint an item specific filter condition/function.
The query function should simply filter the item list by applying a list of logical OR conditions, thus using some for returning the boolean filter value.
Which leaves one of implementing a helper method which collects ... via Object.entries and Array.prototype.flatMap as well as via String.prototype.split and Array.prototype.map ... the function endpoints from the above introduced requirements configuration/map, based on the query object, provided by the system. Thus this helper might be named resolveQuery.
const sampleList = [{
id: 1,
channel: 'SMS',
direction: 'INBOUND',
}, {
id: 2,
channel: 'VOICE',
callDetails: {
answered: false,
voicemail: true,
},
direction: 'INBOUND',
}, {
id: 3,
channel: 'VOICE',
callDetails: {
answered: true,
voicemail: false,
},
direction: 'INBOUND',
}, {
id: 4,
channel: 'VOICE',
callDetails: {
answered: false,
voicemail: false,
},
direction: 'INBOUND',
}];
// prepare a `requirements` map which ...
// - on one hand maps `query`-syntax to a list items's structure
// - and on the other hand does so by providing an item specific
// filter condition/function for each necessary query endpoint.
const requirements = {
channels: {
texts: item => item.channel === 'SMS',
},
calls: {
voicemail: item => item.channel === 'VOICE' && !!item.callDetails.voicemail,
missed: item => item.channel === 'VOICE' && !item.callDetails.answered,
},
}
// const query = {
// channels: 'calls,texts',
// calls: 'voicemail,missed',
// };
function resolveQuery(requirements, query) {
const reject = item => false;
// create/collect a list of filter condition/functions
// which later will be applied as logical OR via `some`.
return Object
.entries(query)
.flatMap(([ groupKey, groupValue ]) =>
// e.g groupKey => 'channels',
// groupValue => 'calls,texts'
groupValue
.split(',')
.map(requirementKey =>
// e.g requirementKey => 'calls'
// or requirementKey => 'texts'
requirements?.[groupKey]?.[requirementKey?.trim()] ?? reject
)
);
}
function queryFromItemList(itemList, requirements, query) {
const conditionList = resolveQuery(requirements, query);
console.log(
'conditionList ... [\n ',
conditionList.join(',\n '),
'\n]'
);
return itemList.filter(item =>
conditionList.some(condition => condition(item))
);
}
console.log(
queryFromItemList(sampleList, requirements, {
channels: 'calls,texts',
calls: 'voicemail,missed',
})
);
.as-console-wrapper { min-height: 100%!important; top: 0; }
I'm trying to create a pipeline to add a field based in a condition:
I have a field called helpful which is an array that will contain a list of id's, what I want to do is add a field depending if a given ID is insided that array
an example of the data structure may be this:
{
helpful: [ 5ecd62230a180f0017dc5342 ],
verifiedPurchase: false,
_id: 5f789010e07e4033342c7307,
title: 'text',
body: 'text',
rating: 3,
user: {
_id: 5ecd62230a180f0017dc5342,
name: 'store11',
picture: 'pictureurl'
},
replies: [],
updatedAt: 2020-10-03T18:04:48.026Z,
createdAt: 2020-10-03T14:52:00.410Z,
helpfulCount: 1,
helpfulForMe: false
},
I already tried with this pipeline
{
$addFields:{
helpfulForMe: {
$cond: {
if: {"$in":[user, "$helpful"] } ,
then: true,
else: false,
}
}
}
},
and this one
"$addFields": {
"helpfulForMe" : {
"$in":[
['5ecd62230a180f0017dc5342'], "$helpful"
]
}
}
},
but both returned false even when I set a matching ID
I hope to get a good fix from you guys. Thanks
You can try if your input is array of ids,
$reduce to iterate loop of helpful array and check condition if id in user array then return true otherwise false
let user = ["5ecd62230a180f0017dc5342"];
{
$addFields: {
helpfulForMe: {
$reduce: {
input: "$helpful",
initialValue: false,
in: {
$cond: [{ $in: ["$$this", user] }, true, "$$value"]
}
}
}
}
}
Playground
I'm pushing an updateOne query to update (or create if not existent) a subdocument. This is my schema
{
accountData: {
userId: Number,
items: [
{
itemId: Number,
purchaseTime: Number
}
]
}
}
and this is an example document
{
accountData: {
userId: 1,
items: [
{
itemId: 2,
purchaseTime: 3
},
{
itemId: 3,
purchaseTime: 5
}
]
}
}
When I run the following bulk query for finding and updating purchaseTime value of itemId's 2 and 4 I get an error. This is the query
let bulkQuery = [];
bulkQuery.push({
updateOne: {
filter: {
'accountData.userId': 1,
'accountData.items.itemId': 2
},
update: {
$inc: {
'accountData.items.$.purchaseTime': 1
}
},
upsert: true,
setDefaultsOnInsert: true
}
});
bulkQuery.push({
updateOne: {
filter: {
'accountData.userId': 1,
'accountData.items.itemId': 4
},
update: {
$inc: {
'accountData.items.$.purchaseTime': 1
}
},
upsert: true,
setDefaultsOnInsert: true
}
});
MyModel.bulkWrite(bulkQuery);
The expected change on the document I provided is increasing purchaseTime of itemId=2 from 3 to 4 and adding a new item object with itemId=4 and purchaseTime=1. However, when I run this, I get the following error
BulkWriteError: The positional operator did not find the match needed
from the query. Unexpanded update: accountData.items.$.purchaseTime
Good afternoon all,
I am having a really tough time working with aggregation queries in MongoDB 3.4. I have a problem that is asking me to do push the results of my aggregation query into an empty array called categories which I have been able to do successfully using this code:
var categories = [];
database.collection("item").aggregate([{
$group : {
_id : "$category",
num : {$sum : 1}
}},
{$sort:{_id:1}}]).toArray(function(err, data){
categories.push(...data);
callback(categories);
console.log(categories);
})
}
categories looks like this:
[ { _id: 'Apparel', num: 6 },
{ _id: 'Books', num: 3 },
{ _id: 'Electronics', num: 3 },
{ _id: 'Kitchen', num: 3 },
{ _id: 'Office', num: 2 },
{ _id: 'Stickers', num: 2 },
{ _id: 'Swag', num: 2 },
{ _id: 'Umbrellas', num: 2 } ]
Next I have the following task:
In addition to the categories created by your aggregation query,
include a document for category "All" in the array of categories
passed to the callback. The "All" category should contain the total
number of items across all categories as its value for "num". The
most efficient way to calculate this value is to iterate through
the array of categories produced by your aggregation query, summing
counts of items in each category.
The problem is that it seems like inside my .toArray() method the data parameter sometimes acts like an array and sometimes not. For example if I wanted to add perhaps just the value of the num key to the categories array like so: categories.push(...data["num"]) I get an error stating undefined is not iterable.
Since I cannot iterate over each data.num key I cannot extract it's value and add it to a running total of all data.num values.
What am I not understanding about what is going on here?
You don't need to use application logic to group data, mongoDB aggregation is made for this task. Add another $group to your query with a new field All that $sum your $num field and $push all documents to a new field called categories :
db.item.aggregate([{
$group: {
_id: "$category",
num: { $sum: 1 }
}
}, { $sort: { _id: 1 } }, {
$group: {
_id: 1,
All: { $sum: "$num" },
categories: {
$push: {
_id: "$_id",
num: "$num"
}
}
}
}])
It gives :
{
"_id": 1,
"All": 23,
"categories": [{
"_id": "Swag",
"num": 2
}, {
"_id": "Office",
"num": 2
}, {
"_id": "Stickers",
"num": 2
}, {
"_id": "Apparel",
"num": 6
}, {
"_id": "Umbrellas",
"num": 2
}, {
"_id": "Kitchen",
"num": 3
}, {
"_id": "Books",
"num": 3
}, {
"_id": "Electronics",
"num": 3
}]
}
For consuming the output, data is an array, to access the first element use data[0] :
var categories = [];
database.collection("item").aggregate([{
$group: {
_id: "$category",
num: { $sum: 1 }
}
}, { $sort: { _id: 1 } }, {
$group: {
_id: 1,
All: { $sum: "$num" },
categories: {
$push: {
_id: "$_id",
num: "$num"
}
}
}
}]).toArray(function(err, data) {
var totalCount = data[0]["All"];
console.log("total count is " + totalCount);
categories = data[0]["categories"];
for (var i = 0; i < categories.length; i++) {
console.log("category : " + categories[i]._id + " | count : " + categories[i].num);
}
})
What I wanted to achieve was pushing or unshifting as we'll see in a moment an object that looked like this into my categories array:
var allCategory = {
_id: "All",
num: [sum of all data.num values]
}
I ended up messing with .reduce() method and used it on the categories array. I got lucky through some console.log-ing and ended up making this:
var categories = [];
database.collection("item").aggregate([{
$group : {
_id : "$category",
num : {$sum : 1}
}},
{$sort:{_id:1}}]).toArray(function(err, data){
categories.push(...data);
var sum = categories.reduce(function(acc, val){
// console.log(acc, val["num"])
return acc + val["num"]
},0);
var allCategory = {
_id: "All",
num: sum
}
categories.unshift(allCategory)
callback(categories);
})
First I use a spread operator to push all the objects from data into categories. Then declare sum which runs .reduce() on categories returning the accumulation of val["num"] which is really data.num (console log is life). I create the allCategory document/object then use .unshift to place it at the beginning of my categories array (this placement was a requirement) then use my callback.
I think it's a hacky way of accomplishing my goal and I had to go through some trial and error as to the correct order of methods and variables in my .toArray(). Yet it worked and I learned something. Thanks for the help #Bertrand Martel .