Related
Here's an example through JS code of what I'm trying to achieve:
let waiting = findSessions() // regular query for status "WAITING"
let results = [];
for (let w of waiting) {
// Only push it to results if the w.members[0] and TARGET_USER_ID have never matched before.
// Meaning, in the "session" collection, there are no documents that have these 2 IDs in the members field
if (!hasMatchedBefore(w.members[0], "TARGET_USER_ID")) {
results.push(w);
}
}
IGNORE MOST OF WHAT'S ABOVE
Just realized how poorly written the old question was. Let's start from the beginning.
Consider the collection "sessions":
[
{
status: "WAITING",
type: "TEXT",
members: [
"adam"
]
},
{
status: "WAITING",
type: "TEXT",
members: [
"john"
]
},
{
status: "WAITING",
type: "VOICE",
members: [
"alex"
]
},
{
status: "ENDED",
type: "VOICE",
members: [
"adam",
"alex"
]
},
{
status: "TIMEOUT",
type: "TEXT",
members: [
"adam",
"TARGET"
]
}
]
I'm making a match-matching system. Let's say "TARGET" wants to match. I'm trying to write a MongoDB aggregation that does the following.
Find all documents with query { type: "TEXT", status: "WAITING" }
Iterate through each document: check if members[0] and TARGET have ever matched before
If members[0] and TARGET have matched before (check entire collection, any type/status), then it will not be included in the final array
It should return the following:
[
{
status: "WAITING",
type: "TEXT",
members: [
"john"
]
},
]
Notice how there were 3 "WAITING" rooms in the collection. But TARGET had already matched with adam. And TARGET cannot match with alex, because alex is in a "VOICE" session. So in this case, john would be the only appropriate match.
One option is to use $lookup on the same collection:
db.sessions.aggregate([
{$match: {
status: "WAITING",
type: "TEXT",
"members.0": {$ne: target}
}},
{$lookup: {
from: "sessions",
let: {member: {$first: "$members"}},
pipeline: [{$match: {$expr: {$setIsSubset: [["$$member", target], "$members"]}}}],
as: "found"
}},
{$match: {"found.0": {$exists: false}}},
{$group: {
_id: 0,
members: {$push: {$arrayElemAt: ["$members", 0]}},
status: {$first: "$status"},
type: {$first: "$type"}
}}
])
See how it works on the playground example
I think, I have a solution for you.
Data
[
{
_id: "ObjectId1",
status: "WAITING",
"members": [
"ID1"
]
},
{
_id: "ObjectId2",
status: "WAITING",
"members": [
"ID2"
]
},
{
_id: "ObjectId3",
status: "ENDED",
"members": [
"ID1",
"ID2"
]
}
]
Query
db.collection.find({
status: "ENDED",
members: {
$elemMatch: {
$eq: "ID2"
}
}
})
Output
[
{
"_id": "ObjectId3",
"members": [
"ID1",
"ID2"
],
"status": "ENDED"
}
]
Note: Please check mongoplayground link.
Main Solution: https://mongoplayground.net/p/xkyyW8gsWV6
Other Solution: https://mongoplayground.net/p/1ndltdDU38-
I have a list of nested items in the JSON object. I have to filter only root nodes (parent).
The below is the JSON object.
const myJSON = [
{
__typename: 'Query', node:
{
__typename: 'Item', childItems: [Object], id: 'a', label: 'node1', url: '#', parent: null
}
},
{
__typename: 'Query', node:
{
__typename: 'Item', childItems: [Object], id: 'b', label: 'node2', url: '#', parent: null
}
},
{
__typename: 'Query', node:
{
__typename: 'Item', childItems: [Object], id: 'a', label: 'node3', url: '#', parent: 'node1'
}
}
]
this is my javascript code and the object is retrieved inside the object variable.
I want to filter only labels of parent nodes from the above object.
My desire output should be:
node1
node2
node3
node4
In order to obtain only the desired properties after the .filter() method, you can use the .map() method to transform the final array.
Note that I changed item.node.parent == null to !item.node.parent. Like this it doesn't look just for those which are null, but for those which are falsy. Change it again to null if that is the behaviour you are expecting.
As you can see in the snippet, using map can tell which property of the array I want to keep
EDIT: Answering your comment, of course you can select more than one property using the .map() method, as long as you format it as an object. The function filterParentAndObtainLabelAndUrl(input) returns label and url. As you can see, you can easily add as many as you want
const filterParentAndObtainLabelValue = (input) => {
return input.filter(element => !element.node.parent)
.map(element => element.node.label);
}
const filterParentAndObtainLabelAndUrl = (input) => {
return input.filter(element => !element.node.parent)
.map(element => {
return ({
label: element.node.label,
url: element.node.url
})
})
}
const inputJSON = [{ "__typename": "Query", "node": { "__typename": "Item", "childItems": [null], "id": "a", "label": "node1", "url": "#", "parent": null } }, { "__typename": "Query", "node": { "__typename": "Item", "childItems": [null], "id": "b", "label": "node2", "url": "#", "parent": null } }, { "__typename": "Query", "node": { "__typename": "Item", "childItems": [null], "id": "a", "label": "node3", "url": "#", "parent": "node1" } }]
console.log('Just the label: ', filterParentAndObtainLabelValue(inputJSON))
console.log('Label and url: ', filterParentAndObtainLabelAndUrl(inputJSON))
My Question schema looks like this:
const questionSchema = new mongoose.Schema({
content: String,
options: [{
content: String,
correct: Boolean
}]
});
I also have a Test schema, where I refer to Question:
const testSchema = new mongoose.Schema({
// ...
questions: [{
type: mongoose.Schema.Types.ObjectId,
ref: "Question"
}]
})
When I fetch Questions (using find(), findOne() or Test.find().populate("questions")) I'd like to add to the document a new boolean field multiple based to on how many options have correct === true. Expected output:
{
_id: "...",
_v: 1,
content: "What is a capital of Slovenia?"
options: [
{
content: "Bled",
correct: false
},
{
content: "Ljubljana",
correct: true
}
],
multiple: false
}
Is it possible to use some kind of a function which is called everytime I query Question and adds a new field to a fetched object or do I have to store multiple field permanently in Mongo?
Depending on your needs there are a couple of approaches here.
Mongoose Virtual Field
The most direct should be since you are using mongoose would be to add a virtual field to the schema which basically calculates it's value when it is accessed. You don't specify your logic in the question, but presuming something like "more than one true" means that multiple is true then you would do something like this:
const questionSchema = new Schema({
content: String,
options: [{
content: String,
correct: Boolean
}]
},
{
toJSON: { virtuals: true },
toObject: { virtuals: true }
});
questionSchema.virtual('multiple').get(function() {
return this.options.filter(e => e.correct).length > 1;
});
That's a basic "getter" which simply looks at the array content and returns if the number of true elements for the correct property are more than one within the array content. It can really be whatever logic you want in the function. Note the use of function() and not () => since "arrow functions" have a different scope of this and that is important to mongoose to determine the current instance value at the time of evaluation.
The toJSON and toObject options in the schema definition are optional, but basically their point is that you can access the "virtual" property directly ( i.e question.multiple === false ) but something like console.log(question) does NOT show the virtual properties unless that definition is added with those options.
MongoDB Projection
Another option is to just have MongoDB do the work to return the modified document from the server in results. This is done using the aggregation framework, which is basically the tool for any "result manipulation".
Here as an example we implement the same logic as presented in the virtual method, along with using $lookup in the same way a populate() would be done. Except of course this is one request to the server and not two as would be the case with populate(), which simply issues a separate query for the "related" data:
// Logic in aggregate result
let result = await Test.aggregate([
{ "$lookup": {
"from": Question.collection.name,
"let": { "questions": "$questions" },
"pipeline": [
{ "$match": {
"$expr": {
"$in": [ "$_id", "$$questions" ]
}
}},
{ "$addFields": {
"multiple": {
"$gt": [
{ "$size": {
"$filter": {
"input": "$options",
"cond": "$$this.correct"
}
}},
1
]
}
}}
],
"as": "questions"
}}
]);
Same sort of operations with $filter instead of Array.filter() and $size instead of Array.length. Again the main benefit is the "server join" here, so it's possibly better for you to implement the "virtual" logic there rather than on the schema.
Whilst it is "possible" to do things like using an aggregate() result with mongoose schema and methods, the default behavior is that aggregate() returns "plain objects"
and not the "mongoose document" instances which have the schema methods. You could re-cast the results and use schema methods, but that's probably going to mean defining "special" schema and model classes just for specific "aggregation" results, and probably not the most efficient thing to do.
Overall which one you implement depends on which suits your application needs the best.
And of course whilst you "could" also just store the same data in the MongoDB document instead of calculating each time it's retrieved, then the overhead basically shifts to the time of writing the data, where mostly this would depend on how you write data. For instance if you "add new options" to existing options then you basically need to read the whole document from MongoDB, inspect the content and then decide what to write back for the multiple value. So the same logic presented here ( more than one true in the array ) has no "atomic" write process that can be done without reading the document data first.
As a working example of these approaches, see the following listing:
const { Schema } = mongoose = require('mongoose');
const uri = 'mongodb://localhost:27017/test';
const opts = { useNewUrlParser: true };
mongoose.set('useFindAndModify', false);
mongoose.set('useCreateIndex', true);
mongoose.set('debug', true);
const questionSchema = new Schema({
content: String,
options: [{
content: String,
correct: Boolean
}]
},
{
toJSON: { virtuals: true },
toObject: { virtuals: true }
});
questionSchema.virtual('multiple').get(function() {
return this.options.filter(e => e.correct).length > 1;
});
const testSchema = new Schema({
questions: [{
type: Schema.Types.ObjectId,
ref: 'Question'
}]
});
const Question = mongoose.model('Question', questionSchema);
const Test = mongoose.model('Test', testSchema);
const log = data => console.log(JSON.stringify(data, undefined, 2));
(async function() {
try {
const conn = await mongoose.connect(uri, opts);
await Promise.all(
Object.entries(conn.models).map(([k,m]) => m.deleteMany())
);
// Insert some data
let questions = await Question.insertMany([
{
"content": "What is the a capital of Slovenia?",
"options": [
{ "content": "Bled", "correct": false },
{ "content": "Ljubljana", "correct": true }
]
},
{
"content": "Who are the most excellent people?",
"options": [
{ "content": "Bill", "correct": true },
{ "content": "Ted", "correct": true },
{ "content": "Evil Duke", "correct": false }
]
}
]);
await Test.create({ questions })
// Just the questions
let qresults = await Question.find();
log(qresults);
// Test with populated questions
let test = await Test.findOne().populate('questions');
log(test);
// Logic in aggregate result
let result = await Test.aggregate([
{ "$lookup": {
"from": Question.collection.name,
"let": { "questions": "$questions" },
"pipeline": [
{ "$match": {
"$expr": {
"$in": [ "$_id", "$$questions" ]
}
}},
{ "$addFields": {
"multiple": {
"$gt": [
{ "$size": {
"$filter": {
"input": "$options",
"cond": "$$this.correct"
}
}},
1
]
}
}}
],
"as": "questions"
}}
]);
log(result);
} catch(e) {
console.error(e)
} finally {
mongoose.disconnect()
}
})()
And it's output:
Mongoose: questions.deleteMany({}, {})
Mongoose: tests.deleteMany({}, {})
Mongoose: questions.insertMany([ { _id: 5cce2f0b83d75c2d1fe6f728, content: 'What is the a capital of Slovenia?', options: [ { _id: 5cce2f0b83d75c2d1fe6f72a, content: 'Bled', correct: false }, { _id: 5cce2f0b83d75c2d1fe6f729, content: 'Ljubljana', correct: true } ], __v: 0 }, { _id: 5cce2f0b83d75c2d1fe6f72b, content: 'Who are the most excellent people?', options: [ { _id: 5cce2f0b83d75c2d1fe6f72e, content: 'Bill', correct: true }, { _id: 5cce2f0b83d75c2d1fe6f72d, content: 'Ted', correct: true }, { _id: 5cce2f0b83d75c2d1fe6f72c, content: 'Evil Duke', correct: false } ], __v: 0 } ], {})
Mongoose: tests.insertOne({ questions: [ ObjectId("5cce2f0b83d75c2d1fe6f728"), ObjectId("5cce2f0b83d75c2d1fe6f72b") ], _id: ObjectId("5cce2f0b83d75c2d1fe6f72f"), __v: 0 })
Mongoose: questions.find({}, { projection: {} })
[
{
"_id": "5cce2f0b83d75c2d1fe6f728",
"content": "What is the a capital of Slovenia?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72a",
"content": "Bled",
"correct": false
},
{
"_id": "5cce2f0b83d75c2d1fe6f729",
"content": "Ljubljana",
"correct": true
}
],
"__v": 0,
"multiple": false,
"id": "5cce2f0b83d75c2d1fe6f728"
},
{
"_id": "5cce2f0b83d75c2d1fe6f72b",
"content": "Who are the most excellent people?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72e",
"content": "Bill",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72d",
"content": "Ted",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72c",
"content": "Evil Duke",
"correct": false
}
],
"__v": 0,
"multiple": true,
"id": "5cce2f0b83d75c2d1fe6f72b"
}
]
Mongoose: tests.findOne({}, { projection: {} })
Mongoose: questions.find({ _id: { '$in': [ ObjectId("5cce2f0b83d75c2d1fe6f728"), ObjectId("5cce2f0b83d75c2d1fe6f72b") ] } }, { projection: {} })
{
"questions": [
{
"_id": "5cce2f0b83d75c2d1fe6f728",
"content": "What is the a capital of Slovenia?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72a",
"content": "Bled",
"correct": false
},
{
"_id": "5cce2f0b83d75c2d1fe6f729",
"content": "Ljubljana",
"correct": true
}
],
"__v": 0,
"multiple": false,
"id": "5cce2f0b83d75c2d1fe6f728"
},
{
"_id": "5cce2f0b83d75c2d1fe6f72b",
"content": "Who are the most excellent people?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72e",
"content": "Bill",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72d",
"content": "Ted",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72c",
"content": "Evil Duke",
"correct": false
}
],
"__v": 0,
"multiple": true,
"id": "5cce2f0b83d75c2d1fe6f72b"
}
],
"_id": "5cce2f0b83d75c2d1fe6f72f",
"__v": 0
}
Mongoose: tests.aggregate([ { '$lookup': { from: 'questions', let: { questions: '$questions' }, pipeline: [ { '$match': { '$expr': { '$in': [ '$_id', '$$questions' ] } } }, { '$addFields': { multiple: { '$gt': [ { '$size': { '$filter': { input: '$options', cond: '$$this.correct' } } }, 1 ] } } } ], as: 'questions' } } ], {})
[
{
"_id": "5cce2f0b83d75c2d1fe6f72f",
"questions": [
{
"_id": "5cce2f0b83d75c2d1fe6f728",
"content": "What is the a capital of Slovenia?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72a",
"content": "Bled",
"correct": false
},
{
"_id": "5cce2f0b83d75c2d1fe6f729",
"content": "Ljubljana",
"correct": true
}
],
"__v": 0,
"multiple": false
},
{
"_id": "5cce2f0b83d75c2d1fe6f72b",
"content": "Who are the most excellent people?",
"options": [
{
"_id": "5cce2f0b83d75c2d1fe6f72e",
"content": "Bill",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72d",
"content": "Ted",
"correct": true
},
{
"_id": "5cce2f0b83d75c2d1fe6f72c",
"content": "Evil Duke",
"correct": false
}
],
"__v": 0,
"multiple": true
}
],
"__v": 0
}
]
I have result json file with 10000 of lines. inside the one array object there are some unnecessary json object i need remove. I have tried so many ways but it's didn't work for me. herewith the piece line of json file
[
{
"product_id": "easybridge",
"errors": []
},
{
"product_id": "learningstudio",
"errors": []
},
{
"product_id": "pearsontestprep",
"errors": []
},
{
"product_id": "productization",
"errors": []
},
{
"product_id": "equella",
"errors": [
{
"property": "instance.test_ids[1]",
"message": "requires property \"maintenance\"",
"schema": {
"$id": "#/properties/test_ids/items",
],
"properties": {
"trend": {
"$id": "#/properties/test_ids/items/properties/trend",
"examples": [
true
]
},
"display": {
"$id": "#/properties/test_ids/items/properties/display",
"type": "boolean",
"examples": [
true
]
},
"test_id": {
"$id": "#/properties/test_ids/items/properties/test_id",
"type": "string",
},
"test_name": {
"$id": "#/properties/test_ids/items/properties/test_name",
"type": "string",
},
"maintenance": {
"$id": "#/properties/test_ids/items/properties/maintenance",
"type": "boolean",
]
},
"instance": {
"trend": false,
"display": false,
"test_id": "8597ae3c-e2a9-45c7-b279-bde1710681be",
"test_name": "Equella Pearsonresearch Ping Test",
"nrAlertStatus": "enabled",
"test_locations": [
{
"alert_state": false,
"location_name": "AWS_US_WEST_2",
"location_label": "Portland, OR, USA",
"included_to_health": false
}
],
"included_to_health": false,
"critical_alert_threshold": 60
},
"name": "required",
"argument": "maintenance",
"stack": "instance.test_ids[1] requires property \"maintenance\""
{
"product_id": "easybridge",
"errors": []
},
I just need only
{
"product_id": "equella",
"errors": [
{
"property": "instance.test_ids[1]",
"message": "requires property \"maintenance\"",
}
},
if the errors json array is not empty. i don't need even this json how can i remove "schema" json object and other unnecessary json object and arrays specially "schema" json object using java script or java. please help
Loop through the array, look at each object, and create a new array by copying over the data you need.
For instance, I'm taking it you don't care about an object if its array of errors is empty, and that you don't care about the schema ever:
let newJSON = [];
//Assume the json variable is the parsed JSON file you posted.
for (let element of json) {
//Must have at least one error
if (element.errors.length > 0) {
//Create a new object
let newObj = {
"product_id" : element.product_id,
"errors" : []
};
//Add each errror
for (let error of element.errors) {
//Only copy across what we need
newObj.errors.push({
"property" : error.property,
"message" : error.message
});
}
//Add object to our new array of JSON
newJSON.push(newObj);
}
}
//newJSON is your processed JSON output
The easiest solution can be:
const records = [{
"product_id": "learningstudio",
"errors": []
},
{
"product_id": "pearsontestprep",
"errors": []
},
{
"product_id": "equella",
"errors": [{
"property": "instance.test_ids[1]",
"message": "requires property \"maintenance\"",
"schema": {
"$id": "#/properties/test_ids/items",
}
}]
}];
const filteredRecords = records.map((record) => {
record.errors = record.errors.map((error) => {
return {property: error. property, message: error.message};
});
return record;
});
console.log(filteredRecords);
You can use map and destructuring assignment to capture only desired properties
let json = [{"product_id": "equella", "errors": [{"property": "instance.test_ids[1]","message": "requires property \"maintenance\"",'xyz': 'not needed','useless': 'not needed',},{'xyz': 'not needed',}]},]
let op = json.map(({product_id,errors}) =>{
let { property, message } = errors[0]
return { product_id, errors: {property,message}}
})
console.log(op)
Below i mentioned the design document.
{
"_id": "_design/link",
"_rev": "62-0c0f00dd9dbedab5c2cca61c356bbff4",
"views": {
"link": {
"map": "function(doc) {\n if (doc.projects) { for (var i in doc.projects) { emit(doc._id, {_id: doc.projects[i].proj_id}); }} \n}"
},
"lists": {
"sample": "function(head, req) {while(row = getRow()){ send(row.doc.proj_name);} }"
}
}
}
The view result:
{
total_rows: 1,
offset: 0,
rows: [
{
id: "SCI130202",
key: "SCI130202",
value: {
_id: "PID00034"
},
doc: {
_id: "PID00034",
_rev: "1-0a363e98a605a72fd71bb4ac62e0b138",
client_id: "E000022",
client_name: "Edinburgh Steel",
type: "manage projects",
proj_id: "PID00034",
proj_name: "Global_upgrade_Oracle",
proj_domain: "Information Technology",
proj_start_date: "2014-10-08",
proj_end_date: "2015-07-07",
delivery_manager: null,
proj_standards: null,
proj_currency_type: "INR",
onsite: "No",
location: "Edinburgh",
proj_status: "Noy yet Start",
budgeted_margin: 45,
budgeted_hrs: 300,
projected_revenue: 200000,
billing_rate: 30,
unit_measure: "per month",
billing_cycle: "Milestone",
proj_core_tech_skills: [ ],
proj_secon_skills: [ ],
proj_sdlc_skills: [ ],
tag: "",
margin: [
{
desired_onsite: null,
desired_offshore: null,
lower_limit: null
}
]
}
}
]
}
I tried but the error comes like
function raised error: (new TypeError("row.doc is undefined", ""))
How to get the proj_name,proj_start_date and proj_end_date using couchdb list function?
You need to add the include_docs=true option to the URL you are using to query the view/list. Views do not automatically include the document.
And maybe you shouldn't use a list to filter your view result - just let the view emit what you need:
emit(doc._id, {
_id: doc.projects[i].proj_id
});
Turns into:
emit(doc.proj_id, {
proj_name: doc.proj_name,
proj_id: doc.proj_id,
proj_start_date: doc.proj_start_date,
proj_end_date: doc.proj_end_date
});
You don't need to emit the doc._id - it is automatically emitted for every row.