Develop Reference

Mongoose - When returning all items from a collection (with no search param) the returned items from the collection do not contain their mongo _id

I am having a bit of an issue with Mongoose/MongoDB this afternoon. I have a situation where I need to return all items from a collection, and doing so means that I do not pass in any search params to mongoose.find().
This is the controller that handles the get all request:
exports.get_all_posts = async (req, res, next) => {
const { params } = req;
const { sortby } = params;
//Sortby param takes two arguments for now: most_recent, oldest
try {
const getAllPosts = await BlogPost.find({}, { _id: 0 });
console.log(getAllPosts);
if (!getAllPosts) throw new Error('Could not get blog posts.');
res.json({
posts: date_.sort(getAllPosts, sortby)
});
} catch (error) {
next(error);
}
};
This is particularly where I think the issue is coming from:
const getAllPosts = await BlogPost.find({}, { _id: 0 });
I am passing an empty search parameter and then removing the _id so that it doesn't throw an error telling me that I need to provide the _id.
However I still need to be able to pull in all of the posts. My items from this collection return as normal, just without their _id's.
Here is my model for the blog posts:
const mongoose = require('mongoose');
const BlogPostSchema = new mongoose.Schema({
date: {
type: Date,
required: true
},
title: {
type: String,
required: true
},
author: {
type: String,
required: true
},
likes: {
type: Number,
required: false
},
post_body: {
type: String,
required: true
},
description: {
type: String,
required: true
},
tags: [
{
type: String,
required: false
}
],
featuredImage: {
type: String,
required: false
},
draft: {
type: Boolean,
required: true
}
});
module.exports = mongoose.model('BlogPost', BlogPostSchema);
One thing to note is that I have not defined an _id. Mongoose automatically adds in the _id field before saving a schema, so I think it is okay without it, as it has been in the past.
Thanks in advance for reading and any input!

Just as Joe has commented, { _id: 0 } as the second parameter is making your query not return the _id field.
Also as he said, there should be no problem whatsoever with using find({}).
Since other than what has already been stated, I couldn't figure out any mistake in the code snippets you provided, I guess this error could be coming from somewhere else in your project.

exports.get_all_posts = async (req, res, next) => { const { params } = req; const { sortby } = params;
try { const getAllPosts = await BlogPost.find({}); console.log(getAllPosts); if (!getAllPosts) throw new Error('Could not get blog posts.'); res.json({ posts: date_.sort(getAllPosts, sortby) }); } catch (error) { next(error); } };
no need to {_id:0} in the find() method because this method retrieve all the documents in the db collection

Populate Query Options with Async Waterfall

I'm trying mongoose populate query options but i don't know why the query options doesn't work.
I have user schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const UserSchema = new Schema(
{
username: { type: String, required: true },
email: { type: String },
name: { type: String },
address: { type: String }
},
{ timestamps: true }
);
module.exports = mongoose.model('User', UserSchema);
and feed schema:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const FeedSchema = new Schema(
{
user: { type: Schema.ObjectId, ref: 'User' },
notes: { type: String, required: true },
trx_date: { type: Date },
status: { type: Boolean, Default: true }
},
{ timestamps: true }
);
FeedSchema.set('toObject', { getters: true });
module.exports = mongoose.model('Feed', FeedSchema);
I want to find all feed by user id, i used async waterfall like the following code:
async.waterfall([
function(callback) {
User
.findOne({ 'username': username })
.exec((err, result) => {
if (result) {
callback(null, result);
} else {
callback(err);
}
});
},
function(userid, callback) {
// find user's feed
Feed
.find({})
// .populate('user', {_id: userid._id}) <== this one also doesn't work
.populate({
path: 'user',
match: { '_id': { $in: userid._id } }
})
.exec(callback);
}
], function(err, docs) {
if (err) {
return next(err);
}
console.log(docs);
});
With above code, i got all feeds, and it seems like the query option do not work at all, did i doing it wrong ?
Any help would be appreciate.

Not sure why you are looking to match "after" population when the value of _id is what is already stored in the "user" property "before" you even populate.
As such it's really just a simple "query" condition to .find() instead:
async.waterfall([
(callback) =>
User.findOne({ 'username': username }).exec(callback),
(user, callback) => {
if (!user) callback(new Error('not found')); // throw here if not found
// find user's feed
Feed
.find({ user: user._id })
.populate('user')
.exec(callback);
}
], function(err, docs) {
if (err) {
return next(err);
}
console.log(docs);
});
Keeping in mind of course that the .findOne() is returning the whole document, so you just want the _id property in the new query. Also note that the "juggling" in the initial waterfall function is not necessary. If there is an error then it will "fast fail" to the end callback, or otherwise pass through the result where it is not. Delate "not found" to the next method instead.
Of course this really is not necessary since "Promises" have been around for some time and you really should be using them:
User.findOne({ "username": username })
.then( user => Feed.find({ "user": user._id }).populate('user') )
.then( feeds => /* do something */ )
.catch(err => /* do something with any error */)
Or indeed using $lookup where you MongoDB supports it:
User.aggregate([
{ "$match": { "username": username } },
{ "$lookup": {
"from": Feed.collection.name,
"localField": "_id",
"foreignField": "user",
"as": "feeds"
}}
]).then( user => /* User with feeds in array /* )
Which is a bit different in output, and you could actually change it to look the same with a bit of manipulation, but this should give you the general idea.
Importantly is generally better to let the server do the join rather than issue multiple requests, which increases latency at the very least.

mongodb/mongoose aggregation that combine two table/collection

I am using mLab for database and mongoose in node js.I am using swagger which should not cause any problem. I have following schemas. when user request, I need to return movie and review together if (review = true) in query. One movie may have multiple reviews. first, I have to find all the movies in the database. when I find movies, I have to go through each of them, look for any reviews in another database and somehow attach them in movie. I need to return all the movies in one package because it will be used in getAll fucntion. No matter what I do it only returning movies without reviews.
var mongoose = require('mongoose');
var reviewSchema = new mongoose.Schema({
movie: {type: String, required: true},
reviewer: {type: String, required: true},
rating: {type: Number, required:true, min: 1, max: 5},
text: {type: String, required: true},
})
var movieSchema = new mongoose.Schema({
Title: {type: String, required: true, unique: true},
YearReleased: {type: Number, required: true},
Actors: [{
Name: {type: String, required: true}
}]
})
.
function getAll(req,res,next){
db.Movies.find({},function(err,movies){
if(err) throw {err:err};
if(req.swagger.params.review.value === false)
res.send({Movielist: movie});
else {
movies.forEach(function(movie, index){
db.Reviews.find({movie:movies[index].Title}, function (err, review) {
if(err) throw {err:err}
movies[index].Review = review // I am trying to populate each movie with reviews
});
});
res.send({Movielist: movies});
}
});
}

res.send is called before the db reviews result is received:
movies.forEach(function(movie, index){
// this call is asynchronous, res.send will run before the callback
db.Reviews.find({movie:movies[index].Title}, function (err, review) {
if(err) throw {err:err}
movies[index].Review = review // I am trying to populate each movie with reviews
});
});
res.send({Movielist: movies});
You can use promises to wait for the results. I'll give an example with async/await. Using promises without async/await is also an option.
async function getAll(req,res,next) {
try {
let movies = await getMovies();
res.send(movies);
} catch (err) {
console.log(err.stack);
}
}
async function getMovies () {
return new Promise(function (resolve, reject) {
db.Movies.find({},function(err,movies){
if(err) reject(err);
if(req.swagger.params.review.value === false)
resolve({Movielist: movie});
else {
movies.forEach(function(movie, index){
let review = await getReviews();
movies[index].Review = review
});
resolve({Movielist: movies});
}
});
});
}
async function getReviews (movie, index) {
return new Promise(function (resolve, reject) {
db.Reviews.find({movie:movies[index].Title}, function (err, review) {
if(err) reject({err:err});
resolve(review);
});
});
}
This could need some tweaks as I have not tested it. I hope it gives the general idea of how to fix the issue.
Without async/await you can call the promises and run ".then" to process the results. If you haven't used promises a few google searches should help understand how .then works.

First thank you #curtwphillips
however, I have found short way to do this. But for this to work, movie and review have to in same database as different collection. If you use different database, it doesn't work
function getAll(req,res,next) {
if (req.query.review === 'true') {
db.Movies.aggregate([
{
$lookup: {
from: "reviews",
localField: "Title",
foreignField: "movie",
as: "review"
}
}
], function (err, result) {
if (err) throw {err: err};
else {
res.send({Movielist: result});
}
});
}
else {
db.Movies.find({}, function (err, movies) {
if (err) throw {err: err};
res.send({Movielist: movies})
})
}
}

Mongoose find not executing

Here's what I've got in a file called Zone.js,
var mongoose = require('mongoose');
mongoose.set('debug', true);
var zoneSchema = new mongoose.Schema({
name: {type: String, required: true, default: '', required: true},
timestamp: {type: Date, default: Date.now, required: true},
zipCodes: {type: [String], default: [], required: true}
});
module.exports = mongoose.model('Zone', zoneSchema);
And then here's what I've got in a file called zoneController.js,
var Zone = require('../models/Zone');
module.exports = {
find: function(params, callback){
console.log('Finding zone');
Zone.find(params, function(err, zones){
console.log('Got results');
if (err){
callback(err, null);
return;
}
callback(null, zones);
});
}
}
And then, I have,
ZoneController = require('../controllers/zoneController');
ZoneController.find({}, function(err, results){
console.log('Zone results received');
}
The problem is that the .find() method doesn't give me anything. I get 'Finding zone' in my console, but absolutely nothing after that.
My folder structure is correct and I am referencing the correct files.

try this, I think that you must to return your find method in your controller.
let me know if it work.
module.exports = {
return {
find: function(params, callback) {
console.log('Finding zone');
Zone.find(params, function(err, zones) {
console.log('Got results');
if (err) {
callback(err, null);
return;
}
callback(null, zones);
});
}
}
}

Here is a .find example for Tweet model in one of my old projects.
Tweet.find({}).sort({date: -1}).populate('_creator').exec((err, tweets) => {
res.render('tweet-all', {title: `All Tweets`, tweets: tweets});
});
I think you must use .exec() in Model.

mongoose setting referenceone

Hello can someone help me. I can't set mongoose model field
Here my institute.js model file
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var instituteSchema = new Schema({
name: {type: String, required: true},
idApi: {type: String, required: true},
country: {
type: Schema.ObjectId,
ref: 'Country'
},
created_at: {type: Date, default: Date.now}
}, {collection: 'qel_par_institute', versionKey: false});
instituteSchema.methods.findByIdApi = function (id, callback) {
return mongoose.model('Institute').findOne().where('idApi').equals(id).exec(callback);
}
instituteSchema.methods.findByCountry = function (id, callback) {
return mongoose.model('Institute').find().where('country.id').equals(id).exec(callback);
}
mongoose.model('Institute', instituteSchema);
There the part of sync.js Persisting works except i can't either manually set the country ref neither another field
var instituteApi = new instituteModel({
name: item.name,
idFromApi: item.id
});
if (typeof item.country.id !== 'undefined') {
var country = new countryModel();
country.findByIdApi(item.country.id, function(err, res) {
if (err)
console.log(err);
if (res) {
instituteApi.country = res._id; //setting this way doesn't work
}
})
}
instituteApi.save(function(err) {
if (err)
console.log('something went wrong while saving!');
});

Couldn't set it because async call. Switched callback to promise with Q module. Everything works as wanted