I have multiple subscriptions on the search results page of a Meteor app. So in the middle is a template for the search results of items, on the left is a template for trending items, and on the right is a template for related items.
On the server I publish a related items (on the right) by querying the Mongodb using a text search which is only possible on the server since minimongo doesn't have that functionality.
But I am also subscribing to a trending items (on the left) which grabs a different set of those same items. In isolation, I receive the correct results, that is, when I comment out the code for the trending items, I get the right results for the related items. And vice versa. But when both are left, it appears they are drawing from the same collection on the client and the results are distorted.
Is there any way to handle multiple subscriptions on the same page?
trendingItems.js
Meteor.subscribe('trendingItems');
Template.trendingItems.helpers ({
trendingItems: function() {
results = Items.find({}, {
fields : { follows : 1, title: 1, itemId: 1 },
sort : { follows : -1 },
limit : 5
}).fetch();
return results;
}
});
relatedItems.js
Template.relatedItems.helpers ({
relatedItems: function() {
return Items.find();
}
});
publications.js
Meteor.publish('relatedItems', function(searchString) {
return Items.find(
{ $text: { $search: searchString } }
);
});
Meteor.publish('trendingItems', function(options) {
results = Items.find({}, {
fields : { follows : 1, title: 1, itemId: 1 },
sort : { follows : -1 },
limit : 5
}).fetch();
return results;
});
A general solution to the problem of handling multiple subscriptions, rather than a specific solution that solves just this problem, is desirable.
Related
I'm using select2 load remote data way to render results (50 at a time) from an api. The response of the api might have duplicate values in any page response.
I have tried formatting response but unfortunately the method is having access only to the current page data.
Below is my code,
jQuery('#items').select2({
minimumInputLength : 2,
placeholder : '-- Select Items --',
ajax : {
url : '/api/v1/items',
quietMillis : 200,
dataType : 'json',
data : function (term, page) {
return {
term : term,
page : page,
page_limit : 50
};
},
results : function(data, page) {
//Here I'm getting only current page data. How can i get previous page data to check for duplicate values.
}
}
});
So, how can I filter the response and eliminate duplicate values by checking against the data fetched so far.
Any help would be appreciated.
It would be better if you post an example of your code. Let's say you have some data with duplicated entries:
var rawData = [
{
id: 'AL',
name: 'Alaska'
},
{
id: 'GE',
name: 'Georgia'
},
{
id: 'WY',
name: 'Wyoming'
},
{
id: 'GE',
name: 'Georgia'
}
];
function clearDuplicates(data) {
var temp = {};
for (var i = 0; i < data.length; i++) {
temp[data[i]['id']] = data[i];
}
return Object.values(temp);
}
var clearData = clearDuplicates(rawData);
console.log(clearData);
See output: duplicated entry 'Georgia' is now in one record. There can be a lot of ways to eliminate duplicates. This is just one simple example.
UPDATE:
If you use pagination (infinite scroll) in Select2, every page request is sent separately and you have to process result data and eliminate duplicates manually. it can be done by processResults parameter. (See example)
In that case, easiest way would be:
Handle every page request in processResults
Store all results in a global variable
Eliminate duplicates as described in the example above
Return desired result
Return:
return {
results: <YOUR_FILTERED_DATA>,
pagination: {
//paginatioin params
}
}
I have been troubleshooting why a MongoDB view I created is so slow. The view targets the transactions collection, and returns records that have an openBalance that is greater than 0. I also run some additional aggregation stages to shape the data the way I want it.
In order to speed up the execution of the view it makes use of an index on the targeted collection by matching on the indexed field in stage one of the view's aggregation pipeline, like so:
// View Stage 1
{ "transactions.details.openBalance" : { "$exists" : true, "$gt" : 0.0 } }
After much investigation I have determined that the aggregation from the view returns data very quickly. What's slow is the count that's run as part of the endpoint:
let count = await db.collection('view_transactions_report').find().count();
So what I'm trying to figure out now is why the count is so much slower on the view than on the underlying collection, and what I can do to speed it up. Or, perhaps there's an alternative way to generate the count?
The underlying collection has something like 800,000 records, but the count returns quickly. But the count on the view, which only returns a filtered set of 10,000 of those initial 800,000 records, returns much more slowly. In terms of specifics, I'm talking about 3/4 of a second for the count on the collection to return, verses six seconds for the count on the mongo view to return.
So, first off, why is the count so much slower on the view (with it's much smaller data set) than on the underlying collection, and secondly, what can I do to address the speed of the count for the view?
I have a couple other aggregation queries I'm running, to determine totalCustomers and totalOpenBalance, that also seem to run slow (see code below).
The relevant part of my endpoint function code looks like this:
// previous code
let count = await db.collection('view_transaction_report').find(search).count();
let totalCustomers = await db.collection('view_transaction_report').find(search).count({
$sum: "customer._id"
});
let result = {};
if (totalCustomers > 0) {
result = await db.collection('view_transaction_report').aggregate([{
$match: search,
},
{
$group: {
_id: null,
totalOpenBalance: {
$sum: '$lastTransaction.details.openBalance'
}
}
}
]).next();
}
db.collection('view_transaction_report').find(search).skip(skip).limit(pagesize).forEach(function (doc) {
docs.push(doc);
}, function (err) {
if (err) {
if (!ioOnly) {
return next(err);
} else {
return res(err);
}
}
if (ioOnly) {
res({
sessionId: sessID,
count: count,
data: docs,
totalCustomers: totalCustomers,
totalOpenBalance: result.totalOpenBalance
});
} else {
res.send({
count: count,
data: docs,
totalCustomers: totalCustomers,
totalOpenBalance: result.totalOpenBalance
});
}
});
In terms of executionStats, this is what shows for the queryPlanner section of the generated view:
"queryPlanner" : {
"plannerVersion" : 1.0,
"namespace" : "vio.transactions",
"indexFilterSet" : false,
"parsedQuery" : {
"$and" : [
{
"transactions.details.openBalance" : {
"$gt" : 0.0
}
},
{
"transactions.destails.openBalance" : {
"$exists" : true
}
}
]
},
"winningPlan" : {
"stage" : "CACHED_PLAN",
"inputStage" : {
"stage" : "FETCH",
"filter" : {
"transactions.details.openBalance" : {
"$exists" : true
}
},
"inputStage" : {
"stage" : "IXSCAN",
"keyPattern" : {
"transactions.details.openBalance" : 1.0
},
"indexName" : "openBalance",
"isMultiKey" : true,
"multiKeyPaths" : {
"transactions.details.openBalance" : [
"transactions",
"transactions.details"
]
},
"isUnique" : false,
"isSparse" : true,
"isPartial" : false,
"indexVersion" : 2.0,
"direction" : "forward",
"indexBounds" : {
"transactions.details.openBalance" : [
"(0.0, inf.0]"
]
}
}
}
},
"rejectedPlans" : [
]
}
In the comments, #Wan Bachtiar mentioned that "openBalance" looks to be a multikey index. To clarify, yes, in the targeted collection, the "openBalance" field is an embedded field within an array. This is the case even though, in the view, the data is shaped in such a way that "openBalance" is an embedded field that is not within an array.
The multikey index on the targeted collection is where the issue lies, because instead of a 1 for 1 document situation, Mongo needs to look through every array element pertaining to this "openBalance" field, which, logically, dramatically increases the scan time - because sometimes there are many, many array elements pertaining to this particular field.
After some further checking, I realized I can address this issue by changing how I populate "openBalance" to our mongo collection via the ETL. By making this change I'll be able to make "openBalance" a standard index, rather than a multikey index, which in turn will allow mongo to search a much smaller data set in order to return my counts.
My data structure looks like this (removed unnecessary parts):
{
"threads" : {
"PUSHID" : {
"info" : {
"members" : {
"uid" : true,
"uid2" : true
}
}
}
}
}
I'm trying to write some javascript to pull snapshots of threads a user is in, but I can't figure out a way for it to work without pulling snapshots of each thread. This is my code now that pulls each thread snapshot.
firebase.database().ref('threads').on('child_added', function(snapshot) {
if (snapshot.hasChild('info/members/' + userUid)) {
// Display thread info
}
});
I tried to make a query with .orderByChild('info/members/' + userUid) and removing null snapshots, but I would have to add a .indexOn for each userUid which is obviously not practical.
Your current structure makes it easy/efficient to look up the users for a thread. But your use case is to look up the threads for a user. You'll need to augment your data model to allow the use-case:
{
"user_threads" : {
"uid": {
"PUSHID": true,
"PUSHID2": true
},
"uid2": {
"PUSHID": true,
"PUSHID3": true
}
}
}
And then read it with:
firebase.database().ref('user_threads/'+userUid).on('child_added', function(snapshot) {
...
});
Modifying/expanding your data model to match the use-cases of your app is quite common when using NoSQL databases.
I'm using Meteor to return a list of Venues that are closest to the user's geolocation. I have the sort happening correctly from the server but the data is a little jumbled by the time it hits the client. It's my understanding that another sort needs to happen on the client once the data is received.
I have the following code in publish.js on the server:
Meteor.publish('nearestVenues', function(params){
var limit = !!params ? params.limit : 50;
if (!!params && !!params.coordinates){
return Venues.find(
{ 'location.coordinates':
{ $near :
{ $geometry :
{ type : "Point" ,
coordinates : params.coordinates
},
$maxDistance : 6000,
spherical: true
}
}
}, {limit: limit});
} else {
return Venues.find({}, {limit: limit});
}
});
And the following in a template helper for my view which returns nothing:
Template.VenueList.helpers({
venues: function(){
return Venues.find(
{ 'location.coordinates':
{ $near :
{ $geometry :
{ type : "Point" ,
coordinates : Session.get('currentUserCoords')
},
$maxDistance : 6000,
spherical: true
}
}
}, {limit: 10})
// return Venues.find({}, {limit: 5, sort: {_id: -1, createdAt: -1}});
}
EDIT: Removed extraneous params ? !!params : 50; code from beginning of publish statement.
Note: the commented out code at the bottom of the helper does in fact work so I know this is the correct place to do a client-side sort. So, how do I do a client side sort when the information is sorted by a Mongo geospatial method? There has to be a way to sort geolocation data from closest to farthest from a location— what am I missing here?
This might be a red herring but I notice that the third line in your first code snippet doesn't seem to do anything:
params ? !!params : 50;
What is that line supposed to do? Perhaps if that is fixed that will solve the problem?
The scenario is that I want to publish one whole collection and users' data (such as profile) who have relation with any document in the the first collection.
The problem is how can I publish that part of users collections?
Well, there are two ways, first is using package
https://atmospherejs.com/cottz/publish-with-relations
And second one - in publish function you can return multiple cursors, from docs
Meteor.publish("roomAndMessages", function (roomId) {
check(roomId, String);
return [
Rooms.find({_id: roomId}, {fields: {secretInfo: 0}}),
Messages.find({roomId: roomId})
];
});
After some research, I found reywood:publish-composite solved my problem completly.
Example:
Meteor.publishComposite('getItemsList', {
find: function() {
return Items.find({});
},
children: [
{
find: function(item) {
return Meteor.users.find(
{ _id: item.userId },);
}
}
]});
This will publish all the items documents with any user document that have a relation with it. ( Items.userId is mapped to Meteor.users._id )