Develop Reference

MongoDB - Mongoose - TypeError: save is not a function

I am attempting to perform an update to a MongoDB document (using mongoose) by first using .findById to get the document, then updating the fields in that document with new values. I am still a bit new to this so I used a tutorial to figure out how to get it working, then I have been updating my code for my needs. Here is the tutorial: MEAN App Tutorial with Angular 4. The original code had a schema defined, but my requirement is for a generic MongoDB interface that will simply take whatever payload is sent to it and send it along to MongoDB. The original tutorial had something like this:
exports.updateTodo = async function(todo){
var id = todo.id
try{
//Find the old Todo Object by the Id
var oldTodo = await ToDo.findById(id);
}catch(e){
throw Error("Error occured while Finding the Todo")
}
// If no old Todo Object exists return false
if(!oldTodo){
return false;
}
console.log(oldTodo)
//Edit the Todo Object
oldTodo.title = todo.title
oldTodo.description = todo.description
oldTodo.status = todo.status
console.log(oldTodo)
try{
var savedTodo = await oldTodo.save()
return savedTodo;
}catch(e){
throw Error("And Error occured while updating the Todo");
}
}
However, since I don't want a schema and want to allow anything through, I don't want to assign static values to specific field names like, title, description, status, etc. So, I came up with this:
exports.updateData = async function(update){
var id = update.id
// Check the existence of the query parameters, If they don't exist then assign a default value
var dbName = update.dbName ? update.dbName : 'test'
var collection = update.collection ? update.collection : 'testing';
const Test = mongoose.model(dbName, TestSchema, collection);
try{
//Find the existing Test object by the Id
var existingData = await Test.findById(id);
}catch(e){
throw Error("Error occurred while finding the Test document - " + e)
}
// If no existing Test object exists return false
if(!existingData){
return false;
}
console.log("Existing document is " + existingData)
//Edit the Test object
existingData = JSON.parse(JSON.stringify(update))
//This was another way to overwrite existing field values, but
//performs a "shallow copy" so it's not desireable
//existingData = Object.assign({}, existingData, update)
//existingData.title = update.title
//existingData.description = update.description
//existingData.status = update.status
console.log("New data is " + existingData)
try{
var savedOutput = await existingData.save()
return savedOutput;
}catch(e){
throw Error("An error occurred while updating the Test document - " + e);
}
}
My original problem with this was that I had a lot of issues getting the new values to overwrite the old ones. Now that that's been solved, I am getting the error of "TypeError: existingData.save is not a function". I am thinking the data type changed or something, and now it is not being accepted. When I uncomment the static values that were in the old tutorial code, it works. This is further supported by my console logging before and after I join the objects, because the first one prints the actual data and the second one prints [object Object]. However, I can't seem to figure out what it's expecting. Any help would be greatly appreciated.
EDIT: I figured it out. Apparently Mongoose has its own data type of "Model" which gets changed if you do anything crazy to the underlying data by using things like JSON.stringify. I used Object.prototype.constructor to figure out the actual object type like so:
console.log("THIS IS BEFORE: " + existingData.constructor);
existingData = JSON.parse(JSON.stringify(update));
console.log("THIS IS AFTER: " + existingData.constructor);
And I got this:
THIS IS BEFORE: function model(doc, fields, skipId) {
model.hooks.execPreSync('createModel', doc);
if (!(this instanceof model)) {
return new model(doc, fields, skipId);
}
Model.call(this, doc, fields, skipId);
}
THIS IS AFTER: function Object() { [native code] }
Which showed me what was actually going on. I added this to fix it:
existingData = new Test(JSON.parse(JSON.stringify(update)));
On a related note, I should probably just use the native MongoDB driver at this point, but it's working, so I'll just put it on my to do list for now.

You've now found a solution but I would suggest using the MongoDB driver which would make your code look something along the lines of this and would make the origional issue disappear:
// MongoDB Settings
const MongoClient = require(`mongodb`).MongoClient;
const mongodb_uri = `mongodb+srv://${REPLACE_mongodb_username}:${REPLACE_mongodb_password}#url-here.gcp.mongodb.net/test`;
const db_name = `test`;
let db; // allows us to reuse the database connection once it is opened
// Open MongoDB Connection
const open_database_connection = async () => {
try {
client = await MongoClient.connect(mongodb_uri);
} catch (err) { throw new Error(err); }
db = client.db(db_name);
};
exports.updateData = async update => {
// open database connection if it isn't already open
try {
if (!db) await open_database_connection();
} catch (err) { throw new Error(err); }
// update document
let savedOutput;
try {
savedOutput = await db.collection(`testing`).updateOne( // .save() is being depreciated
{ // filter
_id: update.id // the '_id' might need to be 'id' depending on how you have set your collection up, usually it is '_id'
},
$set: { // I've assumed that you are overwriting the fields you are updating hence the '$set' operator
update // update here - this is assuming that the update object only contains fields that should be updated
}
// If you want to add a new document if the id isn't found add the below line
// ,{ upsert: true }
);
} catch (err) { throw new Error(`An error occurred while updating the Test document - ${err}`); }
if (savedOutput.matchedCount !== 1) return false; // if you add in '{ upsert: true }' above, then remove this line as it will create a new document
return savedOutput;
}
The collection testing would need to be created before this code but this is only a one-time thing and is very easy - if you are using MongoDB Atlas then you can use MongoDB Compass / go in your online admin to create the collection without a single line of code...
As far as I can see you should need to duplicate the update object. The above reduces the database calls from 2 to one and allows you to reuse the database connection, potentially anywhere else in the application which would help to speed things up. Also don't store your MongoDB credentials directly in the code.

Model return values from another file not working with Sequelize.js

I have the following code that does not work currently.
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models.
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: true, force: false }).then(function() {
models.users.create({
name: 'joe',
loggedIn: true
}).then( task => {
console.log("saved user!!!!!");
});
process.exit();//close the nodeJS Script
}).catch(function(error) {
console.log(error);
});
sequelize-lib.js
var Sequelize = require('sequelize');
exports.getSequelizeConnection = function(stage){
var argv = require('minimist')(process.argv.slice(2)); //If this file is being used in a script, this will attempt to get information from the argument stage passed if it exists
//Change connection settings based on stage variable. Assume localhost by default.
var dbname = argv['stage'] ? argv['stage']+"_db" : 'localdb';
var dbuser = argv['stage'] ? process.env.RDS_USERNAME : 'admin';
var dbpass = argv['stage'] ? process.env.RDS_PASSWORD : 'local123';
var dbhost = argv['stage'] ? "database-"+argv['stage']+".whatever.com" : 'localhost';
//If state variable used during require overide any arguments passed.
if(stage){
dbname = stage+"_db";
dbuser = process.env.RDS_USERNAME
dbpass = process.env.RDS_PASSWORD
dbhost = "database-"+stage+".whatever.com"
}
var connection = new Sequelize(dbname,dbuser,dbpass, {
dialect: 'mysql',
operatorsAliases: false, //This gets rid of a sequelize deprecated warning , refer https://github.com/sequelize/sequelize/issues/8417
host: dbhost
});
return connection;
}
exports.setModels = function(connection){
//Import all the known models for the project.
const fs = require('fs');
const dir = __dirname+'/../models';
var models = {}; //empty model object for adding model instances in file loop below.
//#JA - Wait until this function finishes ~ hence readdirSync vs regular readdir which is async
fs.readdirSync(dir).forEach(file => {
console.log(file);
//Split the .js part of the filename
var arr = file.split(".");
var name = arr[0].toLowerCase();
//Create a modle object using the filename as the reference without the .js pointing to a created sequelize instance of the file.
models[name] = connection.import(__dirname + "/../models/"+file);
})
//Showcase the final model.
console.log(models);
return models; //This returns a model with reference to the sequelize models
}
I can't get the create command to work however with this setup. My guess is the variables must not be passing through correctly somehow. I'm not sure what I'm doing wrong?
The create command definitely works because if in the sequelize-lib.js I modify the setModels function to this...
exports.setModels = function(connection){
//Import all the known models for the project.
const fs = require('fs');
const dir = __dirname+'/../models';
var models = {}; //empty model object for adding model instances in file loop below.
//#JA - Wait until this function finishes ~ hence readdirSync vs regular readdir which is async
fs.readdirSync(dir).forEach(file => {
console.log(file);
//Split the .js part of the filename
var arr = file.split(".");
var name = arr[0].toLowerCase();
//Create a modle object using the filename as the reference without the .js pointing to a created sequelize instance of the file.
models[name] = connection.import(__dirname + "/../models/"+file);
models[name].create({
"name":"joe",
"loggedIn":true
});
})
//Showcase the final model.
console.log(models);
return models; //This returns a model with reference to the sequelize models
}
Then it works and I see the item added to the database! (refer to proof image below)
Take note, I am simply running create on the variable at this point. What am I doing wrong where the model object is not passing between files correctly? Weird part is I don't get any errors thrown in the main file?? It's as if everything is defined but empty or something and the command is never run and nothing added to the database.
I tried this in the main file also and no luck.
models["users"].create({
name: 'joe',
loggedIn: true
}).then( task => {
console.log("saved user!!!!!");
});
The purpose of this all is to read models automatically from the model directory and create instances that are ready to go for every model, even if new one's are added in the future.
UPDATE::
So I did another test that was interesting, it seems that the create function won't work in the .then() function of the sync command. It looks like it was passing it correctly though. After changing the front page to this...
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models using connection previosly created.
models["users"].create({
"name":"joe",
"loggedIn":true
});
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: true, force: false }).then(function() {
process.exit();//close the nodeJS Script
}).catch(function(error) {
console.log(error);
});
Doing this seems to get create to work. I'm not sure if this is good form or not though since the database might not be created at this point? I need a way to get it to work in the sync function.

Well I answered my question finally, but I'm not sure I like the answer.
var config = require('./libs/sequelize-lib.js');
var connection = config.getSequelizeConnection();//Choosing to not pass in variable this time since this should only run via script.
var models = config.setModels(connection);//Creates live references to the models using connection previosly created.
//Alter table as needed but do NOT force the change. If an error occurs we will fix manually.
connection.sync({ alter: false, force: false }).then( () => {
models["users"].create({
"name":"joe",
"loggedIn":true
}).then( user => {
console.log("finished, with user.name="+user.name);
process.exit();
}).catch( error => {
console.log("Error Occured");
console.log(error);
});
}).catch(function(error) {
console.log(error);
});
turns out that process.exit was triggering before create would occur because create happens async. This means that all my code will have to constantly be running through callbacks...which seems like a nightmare a bit. I wonder if there is a better way?

Replace a JSON document with another in MongoDB & Mongoose without knowing the properties of the JSON document beforehand

I've been at this problem for a few days now so I'm asking here now. I'm writing a poll app and the user can link a poll to their account if they're logged in. They can then edit the poll afterwards and how I've done it is copy the create a poll form but filled in the data already put in. The JSON document is made just fine, but the problem I am running into is attempting to replace the original document with the new one. I've made several solutions that don't work, but here's the closest one. In my Polls Mongoose Schema file, I have this router function:
var polls = db.collection('polls');
module.exports.replace = function(newPoll, callback){
polls.replaceOne(
{ $text: { $search: newPoll.question} }, {newPoll}
);
}
Unfortunately, this is the result:
{
"_id": 1,
"newPoll": {
[Contents of newPoll]
}
}
I want the contents of the newPoll to be "one step up" if that makes sense, but I couldn't figure that out. So I tried removing the old document's contents and looping through the JSON document and adding it to the old poll one by one, but the data wouldn't save.
for(var key in newPoll){
var value = newPoll[key];
polls.update(
{ $text: { $search: newPoll.question} }, {'$set' : { [key]: [value]}}, {strict: false}
);
}
I'm still new to this, so I'm sorry if the solution is obvious but I've tried and my Google-fu is yielding no results that have worked.

In this code:
var polls = db.collection('polls');
module.exports.replace = function(newPoll, callback){
polls.replaceOne(
{ $text: { $search: newPoll.question} }, {newPoll}
);
}
{newPoll} is equivalent to {newPoll: newPoll}. That is the reason for the structure of inserted document. Assuming newPoll is an object you should do it like this:
var polls = db.collection('polls');
module.exports.replace = function(newPoll, callback){
polls.replaceOne(
{ $text: { $search: newPoll.question} }, newPoll
);
}

Retrieving child record when parent key is unknown in Firebase

I have a data like this:
"customers": {
"aHh4OTQ2NTlAa2xvYXAuY29t": {
"customerId": "xxx",
"name": "yyy",
"subscription": "zzz"
}
}
I need to retrive a customer by customerId. The parent key is just B64 encoded mail address due to path limitations. Usually I am querying data by this email address, but for a few occasions I know only customerId. I've tried this:
getCustomersRef()
.orderByChild('customerId')
.equalTo(customerId)
.limitToFirst(1)
.once('child_added', cb);
This works nicely in case the customer really exists. In opposite case the callback is never called.
I tried value event which works, but that gives me whole tree starting with encoded email address so I cannot reach the actual data inside. Or can I?
I have found this answer Test if a data exist in Firebase, but that again assumes that you I know all path elements.
getCustomersRef().once('value', (snapshot) => {
snapshot.hasChild(`customerId/${customerId}`);
});
What else I can do here ?
Update
I think I found solution, but it doesn't feel right.
let found = null;
snapshot.forEach((childSnapshot) => {
found = childSnapshot.val();
});
return found;

old; misunderstood the question :
If you know the "endcodedB64Email", this is the way.:
var endcodedB64Email = B64_encoded_mail_address;
firebase.database().ref(`customers/${endcodedB64Email}`).once("value").then(snapshot => {
// this is getting your customerId/uid. Remember to set your rules up for your database for security! Check out tutorials on YouTube/Firebase's channel.
var uid = snapshot.val().customerId;
console.log(uid) // would return 'xxx' from looking at your database
// you want to check with '.hasChild()'? If you type in e.g. 'snapshot.hasChild(`customerId`)' then this would return true, because 'customerId' exists in your database if I am not wrong ...
});
UPDATE (correction) :
We have to know at least one key. So if you under some circumstances
only know the customer-uid-key, then I would do it like this.:
// this is the customer-uid-key that is know.
var uid = firebase.auth().currentUser.uid; // this fetches the user-id, referring to the current user logged in with the firebase-login-function
// this is the "B64EmailKey" that we will find if there is a match in the firebase-database
var B64EmailUserKey = undefined;
// "take a picture" of alle the values under the key "customers" in the Firebase database-JSON-object
firebase.database().ref("customers").once("value").then(snapshot => {
// this counter-variable is used to know when the last key in the "customers"-object is passed
var i = 0;
// run a loop on all values under "customers". "B64EmailKey" is a parameter. This parameter stores data; in this case the value for the current "snapshot"-value getting caught
snapshot.forEach(B64EmailKey => {
// increase the counter by 1 every time a new key is run
i++;
// this variable defines the value (an object in this case)
var B64EmailKey_value = B64EmailKey.val();
// if there is a match for "customerId" under any of the "B64EmailKey"-keys, then we have found the corresponding correct email linked to that uid
if (B64EmailKey_value.customerId === uid) {
// save the "B64EmailKey"-value/key and quit the function
B64EmailUserKey = B64EmailKey_value.customerId;
return B64UserKeyAction(B64EmailUserKey)
}
// if no linked "B64EmailUserKey" was found to the "uid"
if (i === Object.keys(snapshot).length) {
// the last key (B64EmailKey) under "customers" was returned. e.g. no "B64EmailUserKey" linkage to the "uid" was found
return console.log("Could not find an email linked to your account.")
}
});
});
// run your corresponding actions here
function B64UserKeyAction (emailEncrypted) {
return console.log(`The email-key for user: ${auth.currentUser.uid} is ${emailEncrypted}`)
}
I recommend putting this in a function or class, so you can easily call it up and reuse the code in an organized way.
I also want to add that the rules for your firebase must be defined to make everything secure. And if sensitive data must be calculated (e.g. price), then do this on server-side of Firebase! Use Cloud Functions. This is new for Firebase 2017.

I'm having a lot of trouble trying to modify a "this" object from within a pseudo javascript class when it's already been initialized

Scroll down to the bottom of this post to see a work around / possible solution.
This is probably easier just to explain in the source code with comments. The issue at hand is I cannot figure out how pseudo classes work together to perform the task I'm trying to do (explained in the code below).
The code is broken down into 3 files: lead.js, router.js, and db.js.
There are a decent amount of lines of code but most of it is comments.
[lead.js]
var bcrypt = require('bcrypt'),
validators = require('../lib/validators'),
utility = require('../lib/utility'),
document = {};
var Lead = module.exports = function (db) {
// Save a reference to the database.
this.db = db;
// Reference initial document.
// This is totally wrong, not sure how to 'send' a variable to the constructor of a class
// when I cannot add another param. Due to how I'm importing the db model, I won't know what
// the document is until I fill out the form. I've also tried 'document' instead of 'Lead.document'.
this.document = Lead.document;
// Setup the document if it exists.
// This also doesn't work.
// Basically I want to be able to set up a document variable outside of this module (line #100),
// Then pass it to this module after filling it up with values from a form.
// Then based on what's been filled in, it would fix up (trim, convert to lower case)
// some of the values automatically and default a few values that I'm not always going to pass.
if (!document) {
var salt = bcrypt.genSaltSync(10),
hash = bcrypt.hashSync(utility.generatePassword(), salt);
// Default values.
if (!document.meta.createdAt) { this.document.meta.createdAt = Date.now(); }
if (!document.login.password) { this.document.login.password = hash; }
if (!document.login.role) { this.document.login.role = 'User'; }
// Normalize a few values.
this.document.login.email = document.login.email.toLowerCase().trim();
this.document.contact.name.first = document.contact.name.first.trim();
this.document.contact.name.last = document.contact.name.last.trim();
this.document.contact.address.street = document.contact.address.street.trim();
this.document.contact.address.city = document.contact.address.city.trim();
this.document.contact.address.state = document.contact.address.state.trim();
this.document.contact.address.zip = document.contact.address.zip.trim();
this.document.contact.phone.home = document.contact.phone.home.trim();
}
// So in regards to the above code, the end result I'm looking for is...
// I want to append some properties to the this.document reference when the document is empty (when I'm updating it, I won't set the document),
// and on new documents it will append a few default values/normalize all the fields.
};
Lead.prototype.validate = function(fn) {
var errors = [];
// Some validation rules I cut out to make this shorter.
if (errors.length) return fn(errors);
fn();
};
Lead.prototype.save = function(fn) {
this.db.collection('leads', function(err, collection) {
if (err) { fn(new Error({message: err})); }
collection.insert(this.document, function(err, result) {
return fn(err, result);
});
});
};
---
[route.js file]
var db = require('../models/db');
app.post('/register', function(req, res) {
var data = req.body.lead || {};
// Fill the document.
var document = {
meta: {
host: req.headers.host,
referer: req.headers.referer,
createdIPAddress: req.connection.remoteAddress
},
login: {
email: data.email
},
contact: {
name: {
first: data.first,
last: data.last
},
address: {
street: data.street,
city: data.city,
state: data.state,
zip: data.zip
},
phone: {
home: data.phone
}
}
};
// Write the document.
db.lead.document = document;
db.lead.validate(function(err) {
if (err) {
req.session.error = err;
return res.redirect('back');
}
db.lead.save(function(err) {
res.redirect('/register/success');
});
});
});
---
[db.js]
var mongodb = require('mongodb'),
server = new mongodb.Server('localhost', 27017),
connection = new mongodb.Db('test', server);
connection.open(function(err, db) {});
module.exports = {
lead: new (require('./lead'))(connection)
};
When I run this, my validator always reports that the password is empty which makes sense. I'm sending the document initially to the class with an empty password (the password is randomly generated, not a form field) -- the problem is I have no idea what to do with the if (!document) ... code block to actually set the this.document properly.
I hope between the comments and code you can get an idea of what I'm trying to do. I've been stuck on this for a while.
EDIT
I changed the flow of it a bit to get a solution.
In the db.js, I exported the connection rather than instantiating the lead (and future models) directly.
In the router.js file, I require the db and lead file, then pass both the db connection and the document in the constructor of the Lead. Ex.
var lead = new Lead(db, document);
In the lead.js file, it becomes as simple as doing this.document = document (same as the db). When I submit a new lead, the values I don't send over from router.js get appended to the document (the created date, a random password, etc.) and everything is good.
Is this a decent way of handling this, or is there a better way to refactor this?

This is completely wrong way even if make this code work as you want. In this example you have singleton lead. By requesting /register url you want to set 'document' field to this singleton . (IMPORTANT) But requests work asynchronously. Absolutely no guarantee that you save the document, which has just validate. Because new request may overwrite it in lead object. You need to do this logic in request scope. One scope for one request. Not one for all.

You need to read up on object-oriented programming in Javascript.
The anonymous function you're defining near the top of your code is the constructor function, so with respect to the document property you want that is currently uninitialized, just type something like:
this.document = null;
Then some time later when you create a new object using this constructor, like so:
var myLead = new Lead(dbConnection);
You'll have the myLead.document property.
There are many other things wrong with your code, though. Why are you assuming that there is a global document variable with relevant data visible in your library when it's defined as {}? The code in that if statement at the end of your constructor should be run when the document property is set in your other file below, and should only expect this.document to exist.

You set var document = {} initially, and {} is not falsy. Better would be to set as a starting value document = null and then after checking for !document set document = {} before assigning whatever properties you need.