Develop Reference

Mongoose is not populating (.populate()) on Production (Heroku), but works on Local

Essentially I am having one of those moments. My app's on Heroku, and the DB it uses is mLab (MongoDB).
It works on local (Cloud9), but not on production (Heroku).
I can't get .populate() to work on production.
Do you see any gaps in my code below (snippet) that may cause Heroku to fail, while it works on local?
Thank you.
(I have tried purging the DB (deleting the DB and making a new one. Also I've for similar questions on this site. Also I have tried 'heroku local --tail' command to debug and ran it on my local machine; it works on local... Just not on Heroku; appears buggy.)
People.find(id).populate("friends").exec(function(err, user){
if(err){
console.log("! Error retrieving user. " + err);
reject ("! Error retrieving user. " + err);
}
else {
console.log("0! Friends should be populated: " + user);
resolve(user);
}
});
My model:
var mongoose = require('mongoose');
var personSchema = mongoose.Schema({
name: String,
friends: [
{
id: {
type: mongoose.Schema.Types.ObjectId,
ref: "Person"
},
name: String
}
],
username: String,
password: String,
});
module.exports = mongoose.model("Person", personSchema);

Your API function looks ok.
I suspect your issue is with how your models are setup, or what is in your data-base. I had similar issues the first time I tried to use Heroku, because Localhost is more forgiving.
In order for your API to work, the following 3 things must be setup:
(1) Model file: people.js
must look like something like:
var mongoose = require("mongoose");
var Schema = mongoose.Schema;
var peopleSchema = new Schema({
name: {
type: String,
required: true,
trim: true
},
friends: [{
type: Schema.Types.ObjectId,
ref: "Friends"
}]
});
const People = mongoose.model('Peoples', peopleSchema);
module.exports = People;
And then you must have a 'Friends' model, that 'People' is referencing.
(2) Model file: friends.js
must look something like:
var mongoose = require("mongoose");
var Schema = mongoose.Schema;
// Create the Comment schema
var friendsSchema = new Schema({
friend_name: {
type: String,
required: true,
trim: true
},
});
const Friends = mongoose.model('Friends', friendsSchema);
module.exports = Friends;
And lastly, in order for .Populate to work, you need at least two docs in the database.
(3) Database must contain a Person doc and a Friend doc
must look something like:
people.js :
"_id": {
"$oid": "5bef3480f202a8000984b3c5"
},
"name": "Monica Geller"
"friends": [
{
"$oid": "5bef3480f202a8000984b5b4"
}
]
friends.js :
"_id": {
"$oid": "5bef3480f202a8000984b5b4"
},
"friend_name": "Rachel Green"
Hopefully this helps, or gets you closer to your answer.

It was a version issue.
Had to ensure that all platforms (mLab, and my local database) were using the same version of Mongoose.
npm install mongoose#5.4.8 --save

Preventing duplicate records in Mongoose

I'm fairly new to MongoDb / Mongoose, more used to SQL Server or Oracle.
I have a fairly simple Schema for an event.
EventSchema.add({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
I was looking at Mongoose Indexes which shows two ways of doing it, I used the field definition.
I also have a very simple API that accepts a POST and calls create on this collection to insert the record.
I wrote a test that checks that the insert of a record with the same pkey should not happen and that the unique:true is functioning. I already have a set of events that I read into an array so I just POST the first of these events again and see what happens, I expected that mongo DB would throw the E11000 duplicate key error, but this did not happen.
var url = 'api/events';
var evt = JSON.parse(JSON.stringify(events[0]));
// POST'ed new record won't have an _id yet
delete evt._id;
api.post(url)
.send(evt)
.end(err, res) {
err.should.exist;
err.code.should.equal(11000);
});
The test fails, there is no error and a duplicate record is inserted.
When I take a look at the collection I can see two records, both with the same pkey (the original record and the copy that I posted for the test). I do notice that the second record has the same creation date as the first but a later modified date.
(does mongo expect me to use the latest modified version record???, the URL is different and so is the ID)
[ { _id: 2,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/2',
modified: '2016-03-23T07:31:18.529Z',
created: '2016-03-23T07:31:18.470Z' },
{ _id: 1,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/1',
modified: '2016-03-23T07:31:18.470Z',
created: '2016-03-23T07:31:18.470Z' }
]
I had assumed that unique: true on the field definition told mongo db that this what you wanted and mongo enforced that for you at save, or maybe I just misunderstood something...
In SQL terms you create a key that can be used in URL lookup but you can build a unique compound index, to prevent duplicate inserts. I need to be able to define what fields in an event make the record unique because on a form data POST the submitter of a form does not have the next available _id value, but use the _id (done by "mongoose-auto-increment") so that the URL's use from other parts of the app are clean, like
/events/1
and not a complete mess of compound values, like
/events/Event%20name%2001%5fDevice%20X%5fTest%20Owner
I'm just about to start coding up the so for now I just wrote a simple test against this single string, but the real schema has a few more fields and will use a combination of them for uniqueness, I really want to get the initial test working before I start adding more tests, more fields and more code.
Is there something that I should be doing to ensure that the second record does not actually get inserted ?

It seems that you have done unique indexing(at schema level) after inserting some records in db.
please follow below steps to avoiding duplicates -
1) drop your db:
$ mongo
> use <db-name>;
> db.dropDatabase();
2) Now do indexing at schema level or db level
var EventSchema = new mongoose.Schema({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
It will avoid duplicate record insertion with same pKey value.
and for ensuring the index, use command db.db_name.getIndexes().
I hope it helps.
thank you

OK it looks like it has something to do with the index not having time to update before the second insert is posted (as there is only 9ms between them in my test suite).
need to do something about inserts waiting for "index"
needs to be API side as not all users of the API are web applications
I also found some other SO articles about constraints:
mongoose unique: true not work
Unique index not working with Mongoose / MongoDB
MongoDB/Mongoose unique constraint on Date field

on mongoose.connect add {useCreateIndex: true}
It should look like this
mongoose.connect(uri, {
useNewUrlParser: true,
useUnifiedTopology: true,
useCreateIndex: true
})

add:
EventSchema.index({ pkey: 1 }, { unique: true });

// Rebuild all indexes
await User.syncIndexes();
worked for me.
from https://masteringjs.io/tutorials/mongoose/unique

Nested mongoose Schema giving trouble when trying to query in controller

I'm working on a small project and I have a solution to this problem, but it involves creating a new Schema with a reference to the new Schema in the old Schema. I would like to avoid this if at all possible because it will mean spending a couple hours rewriting some code and messing with tests.
The project is a forum site, and there are three main Schemas that comprise it (in addition to cursory Schemas for the forums, notifications, settings and the schemas for the user and the users activities). The Board Schema (contains a list of all forum sections if that wasn't apparent) Is a Schema that makes a reference to the Threads Schema so it can get the threads for each Board. My problem is in the Thread Schema.
var ThreadSchema = new mongoose.Schema({
... other unrelated Schema stuff...
comments: [{
created: {
type: Date,
default: Date.now
},
creator: {
type: mongoose.Schema.ObjectId,
required: true,
ref: 'User'
},
content: {
type: String,
required: true,
get: escapeProperty
},
likes: [{
type: mongoose.Schema.ObjectId,
required: false,
ref: 'User'
}],
liked: {
type: Boolean,
default: false
},
saved: [{
type: mongoose.Schema.ObjectId,
required: false,
ref: 'User'
}]
}]
});
blah blah blah.
I'm trying to pull for the users profile only the comments that that user has posted. The threads were easy, but comment data is not coming through. The request to the server goes through as successful, but I don't get any data back. This is what I am trying.
obj.profileComments = function (req, res) {
var userId = req.params.userId;
var criteria = {'comments.creator': userId};
Thread.find(criteria)
.populate('comments')
.populate('comments.creator')
.skip(parseInt(req.query.page) * System.config.settings.perPage)
.limit(System.config.settings.perPage + 1)
.exec(function (err, threads) {
if (err) {
return json.bad(err, res);
}
json.good({
records: threads
}, res);
});
};
This is a controller, and the json.bad and json.good are helpers that I have created and exported they basically are res.send.
var good = function (obj, res) {
res.send({
success: 1,
res: obj
});
};
and the bad is very similar, it just handles errors in an obj.res.errors and puts them into messages.
So now that that is all out of the way, I'm a little lost on what to do?
Is this something I should try to handle with a method in my Schema? It seems like I might have a little bit more luck that way. Thank you for any help.

how to get stored options from `this.prompt` inside yeoman context?

Basically, yeoman force you to ask everything you need from developer. Although, it’s a good thing, that you can store something and in future runs these things will be autocompleted for developer. The point is that I want to not ask developer if he already answered on that questions.
here is example of basic yeoman generator (name will be saved and autocompleted later):
var yeoman = require('yeoman-generator');
module.exports = yeoman.generators.Base.extend({
init: function () {
var cb = this.async();
this.prompt([{
name: 'name',
message: 'your name:',
store: true,
}, {
name: 'moduleName',
message: 'module name:'
}], function (props) {
console.log(
props.name, // developer’s name
props.moduleName // module’s name
)
}.bind(this));
},
}
});
The question is how to get stored options from this.prompt inside yeoman context to do smth like this:
this.prompt([!this.name.stored && {
name: 'name', // so after first run this will never be asked again
message: 'your name:',
store: true,
}, {
name: 'moduleName',
message: 'module name:'
}], function (props) {
console.log(
props.name, // developer’s name
props.moduleName // module’s name
)
}.bind(this));

There's no public way to access the stored previous prompt answers.
If you want to cache some data and access it later, then use the storage functionality (this.config)
FWIW, the prompt cache is stored into the private this._globalConfig. I'm adding this detail for completeness, you probably shouldn't use it.

You can user default's value as function, where first argument is prev stored answers:
const answers = await this.prompt([
{
type: 'input',
name: 'projectName',
message: 'Your project id',
default: this.appname,
store: true
},
{
type: 'input',
name: 'projectTitle',
message: 'Your project title',
default: ({ projectName }) => projectName
},
])

You can add a config.js that will store and read the information in the users home directory as a config.json file and later the app can read this file that can be used as a default.
{
name: 'authorName',
message: 'What\'s your name?',
'default': self.defaultAuthorName
}
Check out https://www.npmjs.com/package/generator-yo-wordpress where the developer makes use of a config.js file.

How to add another field in a join sails js

Hey all just getting started with Sails js and mongoDB and im a bit confused.
I have two models with a many-to-many relationship:
User.js
module.exports = {
attributes: {
username: 'STRING',
password: 'STRING',
doors:{
collection: 'door',
via: 'users',
}
}
};
and Door.js
module.exports = {
attributes: {
name: 'STRING',
users:{
collection: 'user',
via: 'doors'
}
}
};
This works fine, I can create a user and a door and associate one with the other. However I'd like to have another field in the join, an expiry date (Say the user can only have access to a particular door until a particular date).
How would I go about doing that?

You need to create a many to many through association. However they are not officially supported as of yet.
You can manually do this however.
Now, in this example it may sometimes be a a little more difficult to get all the doors for a user and vice versa, because you have to preform a second look up. However you can do the following with this setup:
UserDoors
.find()
.where({expires:{'<':new Date()}})
.populate('doors')
.populate('users')
.exec(/*....*/)
Your models
User.js
module.exports = {
attributes: {
username: 'STRING',
password: 'STRING',
doors:{
collection: 'userDoors',
via: 'users',
}
}
};
userDoors.js
module.exports = {
attributes: {
doors:{
model: 'door'
},
users:{
model: 'user'
},
expires: 'datetime'
}
};
and Door.js
module.exports = {
attributes: {
name: 'STRING',
users:{
collection: 'userDoors',
via: 'doors'
}
}
};
Do a google search for sails.js many to many through to also help you find what your looking for.