In a Node.js application, i am using Mongoosastic to retrieve data from ElasticSearch :
Article.search({
"match_all": {}
}, function (err, results) {
console.log(results.hits.hits);
Post.search({
"match_all": {}
}, function (err, results) {
console.log(results.hits.hits);
return next();
});
});
Here i am making two requests to retrieve data from two different collections. I would like to know if it was the good way to do it ?
Is it possible to search data from the two collections (or multiple collections) in one single request ? (with the multi search API for example, but with Mongoosastic).
Thank you.
Looks like we can't do it with Mongoosastic.
In fact, the project define itself in the GitHub page as :
a mongoose plugin that can automatically index your models into
elasticsearch
So i use it to define my indexes, but to do search queries it works fine with the official javascript client.
The same question here : https://github.com/mongoosastic/mongoosastic/issues/41
Related
I have removed autopublish from my Meteor app. Now I'm publishing my collections manually. I have some related collections. I want to increase performance as much as possible.
If I'm, for instance, looking at a post and want to see all the comments related to this post, I have to query the database with both post: Posts.findOne(postId) AND comments: Comments.find({postId: postId}). I am querying the two collections in the data field with iron-router so they are present in my template but I'm also subscribing the publications in waitOn. Now I have found https://github.com/englue/meteor-publish-composite which lets me publish multiple collections at the same time. But I don't quite understand it. If I'm using Meteor.publishComposite('postAndComments', ...) in server/publish.js, subscribing postAndComments in waitOn, and setting both post and comments in data as I normally do, will I then have saved a demand on the database? To me it looks like I still query the database the same number of times. But is the queries done when publishing the only queries made while the queries done i data is only a way to retrieve what has already been queried from the database?
Besides, in example 1, it is shown how to publish top posts with the belonging comments and post/comment authors, but in the template, only the posts are outputted. How can I also output the comments and authors? Have I misunderstood the potentials of publishComments? I understand it as a kind of a join.
I used publishComposite successfully. In my example below I'm subscribing to Units matching to filter as well as Properties those units belong to.
Meteor.publishComposite('unitsWithProperties', function (filter) {
var filter = filter || {};
console.log(filter);
return {
find: function () {
var units;
units = Units.find(filter);
return units;
},
children: [
{
collectionName: 'properties',
find: function (unit) {
return Properties.find({ _id: unit.propertyId });
}
}
]
};
});
In your case I believe you can:
subscribe to Posts matching TopPosts criteria
subscribe to Comments and Authors of those posts in children: array of cursors
.
Hope this helps.
Alex
In the MEAN app I'm currently building, the client-side makes a $http POST request to my API with a JSON array of soundcloud track data specific to that user. What I now want to achieve is for those tracks to be saved to my app database under a 'tracks' table. That way I'm then able to load tracks for that user from the database and also have the ability to create unique client URLs (/tracks/:track)
Some example data:
{
artist: "Nicole Moudaber"
artwork: "https://i1.sndcdn.com/artworks-000087731284-gevxfm-large.jpg?e76cf77"
source: "soundcloud"
stream: "https://api.soundcloud.com/tracks/162626499/stream.mp3?client_id=7d7e31b7e9ae5dc73586fcd143574550"
title: "In The MOOD - Episode 14"
}
This data is then passed to the API like so:
app.post('/tracks/add/new', function (req, res) {
var newTrack;
for (var i = 0; i < req.body.length; i++) {
newTrack = new tracksTable({
for_user: req.user._id,
title: req.body[i].title,
artist: req.body[i].artist,
artwork: req.body[i].artwork,
source: req.body[i].source,
stream: req.body[i].stream
});
tracksTable.find({'for_user': req.user._id, stream: req.body[i].stream}, function (err, trackTableData) {
if (err)
console.log('MongoDB Error: ' + err);
// stuck here - read below
});
}
});
The point at which I'm stuck, as marked above is this: I need to check if that track already exists in the database for that user, if it doesn't then save it. Then, once the loop has finished and all tracks have either been saved or ignored, a 200 response needs to be sent back to my client.
I've tried several methods so far and nothing seems to work, I've really hit a wall and so help/advice on this would be greatly appreciated.
Create a compound index and make it unique.
Using the index mentioned above will ensure that there are no documents which have the same for_user and stream.
trackSchema.ensureIndex( {for_user:1, stream:1}, {unique, true} )
Now use the mongoDB batch operation to insert multiple documents.
//docs is the array of tracks you are going to insert.
trackTable.collection.insert(docs, options, function(err,savedDocs){
//savedDocs is the array of docs saved.
//By checking savedDocs you can see how many tracks were actually inserted
})
Make sure to validate your objects as by using .collection we are bypassing mongoose.
Make a unique _id based on user and track. In mongo you can pass in the _id that you want to use.
Example {_id : "NicoleMoudaber InTheMOODEpisode14",
artist: "Nicole Moudaber"
artwork: "https://i1.sndcdn.com/artworks-000087731284-gevxfm-large.jpg?e76cf77"
source: "soundcloud"
stream: "https://api.soundcloud.com/tracks/162626499/stream.mp3? client_id=7d7e31b7e9ae5dc73586fcd143574550"
title: "In The MOOD - Episode 14"}
_id must be unique and won't let you insert another document with the same _id. You could also use this to find the record later db.collection.find({_id : NicoleMoudaber InTheMOODEpisode14})
or you could find all tracks for db.collection.find({_id : /^NicoleMoudaber/}) and it will still use the index.
There is another method to this that I can explain if you dont' like this one.
Both options will work in a sharded environment as well as a single replica set. "Unique" indexes do not work in a sharded environment.
Soundcloud API provides a track id, just use it.
then before inserting datas you make a
tracks.find({id_soundcloud : 25645456}).exec(function(err,track){
if(track.length){ console.log("do nothing")}else {//insert}
});
In Mongoose, I can use a query populate to populate additional fields after a query. I can also populate multiple paths, such as
Person.find({})
.populate('books movie', 'title pages director')
.exec()
However, this would generate a lookup on book gathering the fields for title, pages and director - and also a lookup on movie gathering the fields for title, pages and director as well. What I want is to get title and pages from books only, and director from movie. I could do something like this:
Person.find({})
.populate('books', 'title pages')
.populate('movie', 'director')
.exec()
which gives me the expected result and queries.
But is there any way to have the behavior of the second snippet using a similar "single line" syntax like the first snippet? The reason for that, is that I want to programmatically determine the arguments for the populate function and feed it in. I cannot do that for multiple populate calls.
After looking into the sourcecode of mongoose, I solved this with:
var populateQuery = [{path:'books', select:'title pages'}, {path:'movie', select:'director'}];
Person.find({})
.populate(populateQuery)
.execPopulate()
you can also do something like below:
{path:'user',select:['key1','key2']}
You achieve that by simply passing object or array of objects to populate() method.
const query = [
{
path:'books',
select:'title pages'
},
{
path:'movie',
select:'director'
}
];
const result = await Person.find().populate(query).lean();
Consider that lean() method is optional, it just returns raw json rather than mongoose object and makes code execution a little bit faster! Don't forget to make your function (callback) async!
This is how it's done based on the Mongoose JS documentation http://mongoosejs.com/docs/populate.html
Let's say you have a BookCollection schema which contains users and books
In order to perform a query and get all the BookCollections with its related users and books you would do this
models.BookCollection
.find({})
.populate('user')
.populate('books')
.lean()
.exec(function (err, bookcollection) {
if (err) return console.error(err);
try {
mongoose.connection.close();
res.render('viewbookcollection', { content: bookcollection});
} catch (e) {
console.log("errror getting bookcollection"+e);
}
//Your Schema must include path
let createdData =Person.create(dataYouWant)
await createdData.populate([{path:'books', select:'title pages'},{path:'movie', select:'director'}])
I'm building a relatively big NodeJS application, and I'm currently trying to figure out how to fetch the data I need from the DB. Here is a part of my models :
One user has one role, which has access to many modules (where there's a table role_modules to link roles and modules).
In Rails, I would do something like user.role.modules to retrieve the list of the modules he has access to. In NodeJS it's a bit more complicated. I'm using node-orm2 along with PostgreSQL. Here is what I have so far:
req.models.user.find({email: req.body.user}, function(err, user) {
user[0].getRole(function(err, role) {
role.getModules(function(err, modules) {
var list_modules = Array();
modules.forEach(function(item) {
console.log(item);
list_modules.push(item.name);
})
But I can't do this, because item only contains role_id and module_id. If I want to have the name, I would have to do item.getModule(function() {...}), but the results would be asynchronous ... so I don't really see how I could end up with an array containing the names of the modules a user has access to ... have any idea?
Also, isn't that much slower than actually running a single SQL query with many JOIN? Because as I see it, the ORM makes multiple queries to get the data I want here...
Thank you!
I wrote an ORM called bookshelf.js that aims to simplify associations and eager loading relations between models in SQL. This is what your query would probably look like to load the role & modules on a user given your description:
var Module = Bookshelf.Model.extend({
tableName: 'modules'
});
var Role = Bookshelf.Model.extend({
tableName: 'roles',
modules: function() {
return this.belongsToMany(Module);
}
});
var User = Bookshelf.Model.extend({
tableName: 'users'
role: function() {
return this.hasOne(Role);
}
});
User.forge({email: req.body.user})
.fetch({
require: true,
withRelated: ['role.modules']
})
.then(function(user) {
// user, now has the role and associated modules eager loaded
console.log(user.related('role'));
console.log(user.related('role').related('modules'))
}, function(err) {
// gets here if no user was found.
});
Might be worth taking a look at.
Currently using node.js along with mongoose and express. I have two collections in my MongoDB and I can successfully retrieve data like this:
ActivityList.prototype = {
showActivities: function(req, res) {
point.find({}, function foundPoints(err, items) {
res.render('index2',{title: 'Credits' , points: items})
});
}
These data can be treated in my index.jade like this:
form(action="/asdad", method="post")
select(name = "item[point]")
each point in points
option(value='onlytesting') #{point.Activity}
input(type="submit", value="Update tasks")
As you can see I am using this to fill a drop down menu.. My issue is that i want to fill multiple drop down menus with data from other collections also.
The page is supposed to be my index.jade, such that a user is presented with multiple drop down menus which will be pulled directly from MongoDB.
My app.js calls:
app.get('/', activityList.showActivities.bind(activityList));
That works just fine, but I want to be able to fetch other data also with the "same" get..
Does anyone know how this can be accomplished?
Thanks for any tips