I'm using a Mongo DB. Have a two collections main_hikanshou and main_kokyaku. main_hikanshou contains only one column - phone_number. main_kokyaku contains about 10 columns where one is same phone_number. This two collections have about 150000 values, I would like to compare this two collections and get .csv output on matched values in computational-light way...
I'm using this js to just to print it, but it takes too long for using it this way...
var list = []
db.main_hikanshou.find().forEach(function(doc1){
var doc2 = db.main_kokyaku.findOne({phone_number: doc1.phone_number});
if (doc2) {
list.push(doc1);
}
});
print(list);
You can use $in
Try this:
db.main_hikanshou.find({},{phone_number:1} ).toArray(function (_err, docs) {//get all phone phone numbers
let phone_numberList = docs.map(v => { return v.phone_number });
var doc2 = db.main_kokyaku.find({ phone_number: { "$in": phone_numberList } });//search phone
print(doc2);
})
Join the matching phone_number to the main_hikanshou collection using aggregate
db.collection('main_hikanshou').aggregate([
{ $lookup:
{
from: 'main_kokyaku',
localField: 'phone_number',
foreignField: 'phone_number',
as: 'phoneNumber'
}
}
])
Related
Hi i have a mongo scheme called "payments" that has a 2 keys that optional:
userId or representativeId (if userId exist representativeId not exist and same about representativeId).
When i found the payments scheme based on cheque scheme that using the $match to filter the result based on my data, I bring all payments in the first lookup that match for my query , in the second and third lookup I want to bring the user data / representative data.
Maybe it will be mix of them its ok, its what I want to see if one of them does not exist the other must to be exist.
I want to get the final array that include the user or the representative or mix of them in the same array.
I am using aggregate to implement this.
the problem its give me back a empty array when 2 of the lookup show(user and representative),
but when I comment the lookup of the user or the representative, and i left with 2 lookup one for payment and after that last lookup user / representative and its work me like I want but just for user / representative just if i remove one of the lookup.
I want its bring my array with two of them.
const userAndRepData = await ChequeDB.aggregate<{[key: string]: any}>([
{
$match: {
$and: [
{
'chequeNumber': {
$in: chequeData.map(c => c.chequeNumber)
},
'accountNumber': {
$in: chequeData.map(c => c.accountNumber)
}
}
]
}
},
{
$lookup: {
from: 'payments',
localField: 'paymentId',
foreignField: '_id',
as: 'payment'
}
},
{
$unwind:'$payment'
},
{
$lookup: {
from: 'representatives',
localField: 'payment.representativeId',
foreignField: '_id',
as: 'representative'
}
},
{
$unwind: '$representative'
},
{
$lookup: {
from: 'users',
localField: 'payment.userId',
foreignField: '_id',
as: 'user'
}
},
{
$unwind: '$user'
},
{
$project: {
user: 1,
_id: 0,
representative: 1
}
}
]);
Unfortunately, you can't use data from one $lookup in the next one since the data from $lookup still hasn't been loaded at the moment query is executing, i.e. payment.representativeId will be null.
You can write resource intensive queries that would work around this, but the easiest and best (performance-wise) way to execute this query would be to have representativeId and userId stored on the Cheque collection.
I have a ONE TO MANY schema like this:
SHOP SCHEMA
const Shop = {
name: "Shop",
properties: {
_id: "objectId",
products:"Products[]"
}
}
PRODUCTS SCHEMA
const Products = {
name: "Products",
properties: {
_id: "objectId",
name : "string",
}
}
A shop has many products and as it can be seen 'pictorially' below
_id:'60f73ca7c1a70278596cc7d0',
products:[
{_id:1, name:'product1'},
{_id:2, name: 'product2'},
{_id:3, name: 'product3'}
]
Now, say I want to delete product2, How do I do it with mongodb realm?
What I have tried so far
const obj = realm.objects('Shop').filtered("_id == $0 AND products._id == $1", ObjectId('60f73ca7c1a70278596cc7d0'), ObjectId('2'))
realm.write(() => {
realm.delete(obj)
})
But this doesn't delete the item in the products array.
How can I achieve deleting a specific element in products array in this One to Many relationshiop using realm?
The code in the question is very close to being correct, you just need to filter for the product you want to delete instead of the shop. It's not clear if you know the product _id or name but you can filter by either one.
Here's the code to filter for products with an _id of 1 and then delete it (which will also remove it from any lists that contain a reference to it.
const prod = realm.objects('Products').filtered("_id == 1");
realm.write(() => {
realm.delete(prod);
prod == null;
})
The above is taken from the documentation Filter Query and Delete An Object
Keep in mind this will delete all products with id = 1 so as long as _id's are unique it's fine.
I've got a chat schema that looks like that:
var chatSchema = new mongoose.Schema({
users: [{
type: mongoose.Schema.Types.ObjectId,
required: true
}]
});
It contains array of user IDs.
Now I want to find one chat document that contains an array of two user IDs.
At the beginning I tried to do this:
Chat.findOne({ users: { $in: [req.user_id, receiver._id] }})
.then(chat => { })
But it seems that every time it gives me the chat that contains at least one of the IDs I mentioned in the query.
So I've tried to change it to this but with no luck:
Chat.findOne()
.where({ users: { $in: [req.user_id] }})
.where({ users: { $in: [receiver._id] }})
.then(chat => { })
I need to find the chat that contains both of the user ID's inside the users array otherwise I expect for a null value.
How can I achieve this goal?
Thanks!
This is the way $in works - returns the document when at least one value matches. You should use $all instead:
Chat.findOne({ users: { $all: [req.user_id, receiver._id] }})
I have two collections in my application that are parsed from two separate json files. I have inserted data from the two files into separate collections. The collections have corresponding numerical ID's and I want to match them up in a new collection. For example: the postmeta collection has a post_id value and the posts collection has a corresponding ID.
To explain this further here is a simple collections example. One thing to note is that there are over 730 collection posts and although there are matching ID's they are not sorted so when I view them they don't match each other.
The posts collection example:
{
"_id": "kTeQxenYZcQfPiaYv",
"ID": "44",
"post_content": "Today we talked about the letter Hh..."
}
The postsmeta collection example:
{
"_id": "otEGQYxvv6MkCABST",
"post_id": "44",
"meta_value": "http://www.mrskitson.ca/wp-content/uploads/2010/11/snackTime.jpg"
}
What I would like to do is parse through the collections and take for example posts collection where the ID matches the postsmeta collection. Once I find a match I want to insert the collections content (post_content & meta_value) into a new collection.
Here is all my code so far.
lib/collections/posts.js
Postsmeta = new Mongo.Collection('postsmeta');
Posts = new Mongo.Collection('posts');
server/publications.js
Meteor.publish('postsmeta', function() {
return Postsmeta.find();
});
Meteor.publish('posts', function() {
return Posts.find();
});
server/main.js
Meteor.startup(() => {
var postsmeta = JSON.parse(Assets.getText('postsmeta.json'));
var posts = JSON.parse(Assets.getText('posts.json'));
var length = postsmeta.length;
for(x=0; x < length; x++){
Posts.insert({
ID: posts[x].ID,
post_content: posts[x].post_content
});
Postsmeta.insert({
post_id: postsmeta[x].post_id,
meta_value: postsmeta[x].meta_value
});
}
});
Let's refactor your code a bit. We'll build the Postsmeta collection first and then jointly create the Posts and PostsCombined collections. Since Postsmeta will already exist we can just search inside it to find matching documents.
Meteor.startup(() => {
const postsmeta = JSON.parse(Assets.getText('postsmeta.json'));
postsmeta.forEach(doc => {
Postsmeta.insert({ post_id: doc.post_id, meta_value: doc.meta_value });
});
const posts = JSON.parse(Assets.getText('posts.json'));
posts.forEach(doc => {
const post = { ID: doc.ID, post_content: doc.post_content}
Posts.insert(post); // omit if you don't need the uncombined collection
const metadoc = Postsmeta.findOne({post_id: doc.ID}); // essentially a JOIN
if (metadoc) post.meta_value = metadoc.meta_value; // guard against no matching meta
PostsCombined.insert(post);
});
});
The following IDs are not present in your postsmeta data:
["56", "322", "521", "563", "583", "608", "625", "671", "707", "708",
"711", "713", "754", "758", "930", "1068", "1126", "1235", "1237", "1238",
"1239", "1246", "1249", "1256", "1263", "1355", "1375", "1678", "1680", "1763",
"1956", "2107", "2121", "2148", "2197", "2249"]
Do you want to put the collections together for consultation? because the insertion is correct for two different collections.
Tip one
If it is for query use the "find().map()", if you are using mongodb, within the function it will return the values of each row of the first collection and soon you can call the other collection and check the id of the collection and return a JSON or Array of what you need. I do not pretend to do it that way, but it's a way of putting the two collections together.
Best solution
The correct way is not thinking as if noSql was a relational database like the other postgres, mysql and etc ... think that it is a dynamic bank, where in the same collection you can have everything you need at that moment, so I think You create a new collection that would be the junction of the two, when save saves the data in this other collection, which would be the query collection, and in that it would weigh less the query and until it would return the data faster, but suppose a 5x more faster than the above example ...
I hope I have helped, any questions or doubts I will be here. Hugs!
I hit an API which follows 50 members' data in a game once a day, and use mongoose to convert the JSON into individual documents in a collection. Between days there is data which is consistent, for example each member's tag (an id for the member in game), but there is data which is different (different scores etc.). Each document has a createdAt property.
I would like to find the most recent document for each member, and thus have an array with each member's tag.
I an currently using the following query to find all documents where tags match, however they are returning all documents, not just one. How do I sort/limit the documents to the most recent one, whilst keep it as one query (or is there a more "mongodb way")?
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).lean().exec(function(err, members) {
res.json(members);
});
Thanks
You can query via the aggregation framework. Your query would involve a pipeline that has stages that process the input documents to give you the desired result. In your case, the pipeline would have a $match phase which acts as a query for the initial filter. $match uses standard MongoDB queries thus you can still query using $in.
The next step would be to sort those filtered documents by the createdAt field. This is done using the $sort operator.
The preceding pipeline stage involves aggregating the ordered documents to return the top document for each group. The $group operator together with the $first accumulator are the operators which make this possible.
Putting this altogether you can run the following aggregate operation to get your desired result:
memberTags = [1,2,3,4,5];
ClanMember.aggregate([
{ "$match": { "tag": { "$in": memberTags } } },
{ "$sort": { "tag": 1, "createdAt: -1 " } },
{
"$group": {
"_id": "$tag",
"createdAt": { "$first": "$createdAt" } /*,
include other necessary fields as appropriate
using the $first operator e.g.
"otherField1": { "$first": "$otherField1" },
"otherField2": { "$first": "$otherField2" },
...
*/
}
}
]).exec(function(err, members) {
res.json(members);
});
Or tweak your current query using find() so that you can sort on two fields, i.e. the tag (ascending) and createdAt (descending) attributes. You can then select the top 5 documents using limit, something like the following:
memberTags = [1,2,3,4,5];
ClanMember.find(
{ 'tag': { $in: memberTags } }, // query
{}, // projection
{ // options
sort: { 'createdAt': -1, 'tag': 1 },
limit: memberTags.length,
skip: 0
}
).lean().exec(function(err, members) {
res.json(members);
});
or
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).sort('-createdAt tag')
.limit(memberTags.length)
.lean()
.exec(function(err, members) {
res.json(members);
});
Ok, so, first, let's use findOne() so you get only one document out of the request
Then to sort by the newest document, you can use .sort({elementYouWantToSort: -1}) (-1 meaning you want to sort from newest to oldest, and 1 from the oldest to the newest)
I would recommend to use this function on the _id, which already includes creation date of the document
Which gives us the following request :
ClanMember.findOne({
'tag': {
$in: memberTags
}
}).sort({_id: -1}).lean().exec(function(err, members) {
res.json(members);
});