Develop Reference

de-dupe previous similar entries in an array

Let's say I have the following array of objects:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Sent',
order: 5
}
]
I was wondering, I would like to de-dupe objects within this array, but only de-dupe the "oldest" object with a duplicated status, that is, for the above array, I would like to end up with the following de-duped array:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 4
},
{
status: 'Sent',
order: 5
}
]
That is, status Ordered 4 is "younger" than status Ordered 3 ... and not simply to just de-dupe...this could extended to the following:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
]
will result in:
[
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
]
N.B. There is one caveat, I don't actually have the order key in the object, this is purely for demonstrative purposes to demonstrate what I mean for the purposes of this question..

Pretty simple reduce function. Basically we create an object (which can only have one value per key). The "last" entry in your array with a specific status will be preserved.
const input = [
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
];
const output = Object.values(input.reduce((a, e) => {
a[e.status] = e;
return a;
}, {}));
console.log(output);

A one-liner with a Map:
const arr = [
{
status: 'Draft',
order: 1
},
{
status: 'Accepted',
order: 2
},
{
status: 'Ordered',
order: 3
},
{
status: 'Ordered',
order: 4
},
{
status: 'Ordered',
order: 5
},
{
status: 'Confirmed',
order: 6
}
];
const deduped = [...new Map(arr.map(o => [o.status, o])).values()];
console.log(deduped);

JS doesn't include a dedup/distinct function natively so you have to make your own such as by using a filter to check for duplicates
the standard call back for the filter is (item, index, array)
this means that you can do
filter((item,idx,arr)=> arr.findIndex(other=>areEqual(item,other)) != idx)
this reads as remove all elements where the they have previously existed in the array because findIndex returns the location of the first match, so if they current index doesn't equal it then its not the first match
the reason for findindex rather than indexOf is that findIndex allows you to specify how to check for equality
this is where the areEqual callback comes in, it takes 2 items and returns true if they are equal
so could be
areEqual(a,b){
return a.field === b.field
}
of course this is overkill for just a.field === b.field
and could be rewritten as
filter((item,idx,arr)=> arr.findIndex(other=>item.field === other.field) != idx)
but you indicate that your test for equality might be more complex than a single known field so the callback to areEqual allows much more complex logic
unfortunately there isn't a findLastIndex but simply reversing the array before filtering will fix that, then reverse the result to regain the original order

Move elements within MongoDB document

Background:
A customer is an object that has a name field.
A line is an object that has the following fields:
inLine - an array of customers
currentCustomer - a customer
processed - an array of customers
The collection 'line' contains documents that are line objects.
Problem:
I'm trying to implement a procedure which would do the following:
Push currentCustomer to processed
Set currentCustomer to the 1st element in inLine
Pop the 1st element of inLine
Since the new value of a field depends on the previous value of another, atomicity is important here.
What I tried so far:
Naive approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, {
$set: {
currentCustomer: '$inLine.0',
},
$pop: {
inLine: -1,
},
$push: {
processed: '$currentCustomer',
},
});
However, currentCustomer is set to a string which is literally "$inLine.0" and processed has a string which is literally "$currentCustomer".
Aggregation approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
currentCustomer: '$inLine.0',
},
$pop: {
inLine: -1,
},
$push: {
processed: '$currentCustomer',
},
}]);
However, I got the following error:
MongoError: A pipeline stage specification object must contain exactly one field.
Multi-stage aggregation approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
currentCustomer: '$inLine.0',
},
}, {
$pop: {
inLine: -1,
},
}, {
$push: {
processed: '$currentCustomer',
},
}]);
However, $pop and $push are Unrecognized pipeline stage names.
I tried making it using only $set stages, but it ended up very ugly and I still couldn't get it to work.

Based on turivishal's answer, it was solved like so:
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
// currentCustomer = inLine.length === 0 ? null : inLine[0]
currentCustomer: {
$cond: [
{ $eq: [{ $size: '$inLine' }, 0] },
null,
{ $first: '$inLine' },
],
},
// inLine = inLine.slice(1)
inLine: {
$cond: [
{ $eq: [{ $size: '$inLine' }, 0] },
[],
{ $slice: ['$inLine', 1, { $size: '$inLine' }] },
],
},
// if currentCustomer !== null then processed.push(currentCustomer)
processed: {
$cond: [
{
$eq: ['$currentCustomer', null],
},
'$processed',
{
$concatArrays: [
'$processed', ['$currentCustomer'],
],
}
],
},
},
}]);

I don't think its possible with simple update using $push or $pop.
As per your experiment, the aggregation can not support direct $push, $pop stage in root level, so I have corrected your query,
currentCustomer check condition if size of inLine is 0 then return null otherwise get first element from inLine array using $arrayElemAt,
inLine check condition if size of inLine is 0 then return [] otherwise remove first element from inLine array using $slice and $size
processed concat both arrays using $concatArrays, $ifNull to check if field is null then return blank array, check condition if currentCustomer null then return [] otherwise return currentCustomer
db.collection('line').findOneAndUpdate(
{ _id: new ObjectId(lineId), },
[{
$set: {
currentCustomer: {
$cond: [
{ $eq: [{ $size: "$inLine" }, 0] },
null,
{ $arrayElemAt: ["$inLine", 0] }
]
},
inLine: {
$cond: [
{ $eq: [{ $size: "$inLine" }, 0] },
[],
{ $slice: ["$inLine", 1, { $size: "$inLine" }] }
]
},
processed: {
$concatArrays: [
{ $ifNull: ["$processed", []] },
{
$cond: [
{ $eq: ["$currentCustomer", null] },
[],
["$currentCustomer"]
]
}
]
}
}
}]
);
Playground

How to conditionally add a field based on match a field with an array Mongo

I'm trying to create a pipeline to add a field based in a condition:
I have a field called helpful which is an array that will contain a list of id's, what I want to do is add a field depending if a given ID is insided that array
an example of the data structure may be this:
{
helpful: [ 5ecd62230a180f0017dc5342 ],
verifiedPurchase: false,
_id: 5f789010e07e4033342c7307,
title: 'text',
body: 'text',
rating: 3,
user: {
_id: 5ecd62230a180f0017dc5342,
name: 'store11',
picture: 'pictureurl'
},
replies: [],
updatedAt: 2020-10-03T18:04:48.026Z,
createdAt: 2020-10-03T14:52:00.410Z,
helpfulCount: 1,
helpfulForMe: false
},
I already tried with this pipeline
{
$addFields:{
helpfulForMe: {
$cond: {
if: {"$in":[user, "$helpful"] } ,
then: true,
else: false,
}
}
}
},
and this one
"$addFields": {
"helpfulForMe" : {
"$in":[
['5ecd62230a180f0017dc5342'], "$helpful"
]
}
}
},
but both returned false even when I set a matching ID
I hope to get a good fix from you guys. Thanks

You can try if your input is array of ids,
$reduce to iterate loop of helpful array and check condition if id in user array then return true otherwise false
let user = ["5ecd62230a180f0017dc5342"];
{
$addFields: {
helpfulForMe: {
$reduce: {
input: "$helpful",
initialValue: false,
in: {
$cond: [{ $in: ["$$this", user] }, true, "$$value"]
}
}
}
}
}
Playground

Retrieve specific Record from an array after findOneAndUpdate Mongoose 5.x

Let's say I have a document like below:
{
_id: '1'
persons: [
{
userId: 'user1',
changed: false
},
{
userId: 'user2',
changed: false
},
]
}
I can simply do like below to update a specific record inside persons:
Model.findOneAndUpdate(
{
_id: '1',
'persons.userId': 'user1'
},
{
$set: { persons.$.changed: true }
},
{ new: true }
)
But the returned document will contain every element from persons array, is there any way so that I can get the specific array elements matching the condition 'persons.userId': 'user1'?

You can use projection as one of the parameters of findAndModify, try:
Model.findOneAndUpdate(
{
_id: "1",
"persons.userId": "user1"
},
{
$set: { "persons.$.changed": true }
},
{
new: true,
projection: {
persons: {
$elemMatch: { userId: "user1" }
}
}
}
)

Mongoose $slice and get orginal size array

I'm currently trying to get the total amount of items in my News object, and return a slice of the items as objects.
I found out how to use the $slice operator in my query, but I don't know how to get the original size of the array of items.
The code I'm currently using in NodeJS:
if (req.query.limit) {
limit = 5;
}
News.findOne({ connected: club._id }, {items: {$slice: limit}}).exec(function (err, news) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else if (!news || news.items.length === 0) {
res.jsonp([]);
} else {
const returnObj = { items: [], totalNumber: 0 };
const items = news.items.sort(function (a, b) {
return b.date - a.date
});
res.jsonp({
items: items,
totalNumber: news.items.length
});
}
});
The Mongo model:
var mongoose = require('mongoose'),
validator = require('validator'),
Schema = mongoose.Schema;
var NewsSchema = new Schema({
connected: {
type: Schema.Types.ObjectId,
required: 'Gelieve een club toe te wijzen.',
ref: 'Club'
},
items: [{
userFirstName: String,
action: String,
date: Date,
targetName: String
}],
created: {
type: Date,
default: Date.now
}
});
module.exports = mongoose.model('News', NewsSchema);
How would I do this efficiently?
Thanks!
EDIT: final code which works:
News.aggregate([{
$match: {
connected: club._id
}
}, {
$project: {
totalNumber: {
$size: '$items'
},
items: {
$slice: ['$items', limit]
}
}
}
]).exec(function (err, news) {
console.log(news);
if (!news || news[0].items.length === 0) {
res.jsonp([]);
} else {
res.jsonp(news[0]);
}
});

You cannot have both information at once using find and $slice.
The soluce you have :
Use aggregate to return the count and only the sliced values.
Like :
[{
$project: {
count: {
$size: "$params",
},
params: {
$slice: ["$params", 5],
},
},
}]
To help you out making aggregate, you can use the awesome mongodb-compass software and its aggregate utility tool.
Use a find without $slice, get the number of item there, and then slice in javascript the array before returning it.
EDIT :
[{
$sort: {
'items.date': -1,
},
}, {
$project: {
count: {
$size: "$items",
},
params: {
$slice: ["$items", 5],
},
},
}]