Preventing duplicate records in Mongoose - javascript

I'm fairly new to MongoDb / Mongoose, more used to SQL Server or Oracle.
I have a fairly simple Schema for an event.
EventSchema.add({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
I was looking at Mongoose Indexes which shows two ways of doing it, I used the field definition.
I also have a very simple API that accepts a POST and calls create on this collection to insert the record.
I wrote a test that checks that the insert of a record with the same pkey should not happen and that the unique:true is functioning. I already have a set of events that I read into an array so I just POST the first of these events again and see what happens, I expected that mongo DB would throw the E11000 duplicate key error, but this did not happen.
var url = 'api/events';
var evt = JSON.parse(JSON.stringify(events[0]));
// POST'ed new record won't have an _id yet
delete evt._id;
api.post(url)
.send(evt)
.end(err, res) {
err.should.exist;
err.code.should.equal(11000);
});
The test fails, there is no error and a duplicate record is inserted.
When I take a look at the collection I can see two records, both with the same pkey (the original record and the copy that I posted for the test). I do notice that the second record has the same creation date as the first but a later modified date.
(does mongo expect me to use the latest modified version record???, the URL is different and so is the ID)
[ { _id: 2,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/2',
modified: '2016-03-23T07:31:18.529Z',
created: '2016-03-23T07:31:18.470Z' },
{ _id: 1,
pkey: '6fea271282eb01467020ce70b5775319',
name: 'Event name 01',
owner: 'Test Owner',
device: 'Device X',
description: 'I have no idea what\'s happening',
__v: 0,
url: '/api/events/1',
modified: '2016-03-23T07:31:18.470Z',
created: '2016-03-23T07:31:18.470Z' }
]
I had assumed that unique: true on the field definition told mongo db that this what you wanted and mongo enforced that for you at save, or maybe I just misunderstood something...
In SQL terms you create a key that can be used in URL lookup but you can build a unique compound index, to prevent duplicate inserts. I need to be able to define what fields in an event make the record unique because on a form data POST the submitter of a form does not have the next available _id value, but use the _id (done by "mongoose-auto-increment") so that the URL's use from other parts of the app are clean, like
/events/1
and not a complete mess of compound values, like
/events/Event%20name%2001%5fDevice%20X%5fTest%20Owner
I'm just about to start coding up the so for now I just wrote a simple test against this single string, but the real schema has a few more fields and will use a combination of them for uniqueness, I really want to get the initial test working before I start adding more tests, more fields and more code.
Is there something that I should be doing to ensure that the second record does not actually get inserted ?

It seems that you have done unique indexing(at schema level) after inserting some records in db.
please follow below steps to avoiding duplicates -
1) drop your db:
$ mongo
> use <db-name>;
> db.dropDatabase();
2) Now do indexing at schema level or db level
var EventSchema = new mongoose.Schema({
pkey: { type: String, unique: true },
device: { type: String, required: true },
name: { type: String, required: true },
owner: { type: String, required: true },
description: { type: String, required: true },
});
It will avoid duplicate record insertion with same pKey value.
and for ensuring the index, use command db.db_name.getIndexes().
I hope it helps.
thank you

OK it looks like it has something to do with the index not having time to update before the second insert is posted (as there is only 9ms between them in my test suite).
need to do something about inserts waiting for "index"
needs to be API side as not all users of the API are web applications
I also found some other SO articles about constraints:
mongoose unique: true not work
Unique index not working with Mongoose / MongoDB
MongoDB/Mongoose unique constraint on Date field

on mongoose.connect add {useCreateIndex: true}
It should look like this
mongoose.connect(uri, {
useNewUrlParser: true,
useUnifiedTopology: true,
useCreateIndex: true
})

add:
EventSchema.index({ pkey: 1 }, { unique: true });

// Rebuild all indexes
await User.syncIndexes();
worked for me.
from https://masteringjs.io/tutorials/mongoose/unique

Related

How do i make specific users for only them to have access to their own specific data in mongodb and nodejs

I am working on a blog project. I am done with the functionality to create,edit and delete posts. I have also setup and login and signup with passport authentication and can already restrict pages to only logged in users. I want to also restrict editing and deleting of posts to only the users who created the post. I just can't seem to figure out how the link the two collections(one being the users collection and the other being the posts collection).
Below are the schemas for the two collections
posts schema
const articleSchema = new mongoose.Schema({
title: {
type: String,
required: true
},
description: {
type: String,
required: true
},
markdown: {
type: String,
required: true
},
createdAt: {
type: Date,
default: Date.now
},
slug: {
type: String,
required: true,
unique: true
},
sanitizedHTML: {
type: String,
required: true
}
})
users schema
const UserSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
});
In the articleSchema you need a field createdBy which will be linked with the id in userSchema. So, when the user is logged in you will get the user's id. Through which you can check if the post is created by that particular user. This is the best approach for your case right now.
If you want to have more extensive roles. For e.g. editors who can edit any articles. There are two ways to do it:
If you need multiple roles and their permissions are configurable
You need to create roleSchema which will have roleName, permissions etc. And in userSchema will be a field called role which corresponds to role_id for each user. This will allow you to change permissions and create new roles easily in future through UI.
If you just have few roles and confident that it won't change often
You can just define for yourself what these roles are: for e.g 1 -> Editor, 2 -> Subscriber, 3 -> Admin etc. And add a role field in userSchema to put the relevant role. This will allow you to have fixed permission for each role. Now when you are doing anything with article's you can get the role and filter accordingly.

To query on two fields

I have a Dynamoose (DynamoDB) model called PromoCode with a schema that looks like this:
{
promoCode: {
hashKey: true,
type: String,
},
previouslyUsed: {
type: Boolean,
default: false,
index: {
global: true,
name: 'previouslyUsedIndex',
},
},
promoGroup: {
type: String,
index: {
global: true,
name: 'promoGroupIndex',
},
},
}
Essentially, I have a table full of promo codes and I want to get a single promo code that hasn't been used and is part of a particular "group" of promo codes.
So, I want to query on both previouslyUsed and promoGroup fields and limit the results to a single results. This is what I came up with:
PromoCode.query('previouslyUsed').eq(false)
.and()
.filter('promoGroup').eq('friend').limit(1)
This returns no results, even though I know that the query should match a result. If I increase the limit to 10, then I get back four results. This makes me think that the limit is happenning before the and() thus the preceding filter() is only filtering on the 10 returned results where previouslyUsed=false.
How do I take a single result where the conditions previouslyUsed=false and promoGroup=friend are valid?
So, here's what I figured out (to answer my own question). Firstly, using filter will only filter the results that are pulled from the database. So, I wasn't experiencing some weird bug.
Secondly, what I really wanted was a range key setup. This will give me the following schema:
{
promoCode: {
hashKey: true,
type: String,
},
previouslyUsed: {
type: Boolean,
default: false,
index: {
global: true,
name: 'previouslyUsedIndex',
rangeKey: 'promoGroup',
},
},
promoGroup: {
type: String,
rangeKey: true,
index: true,
},
}
Note the use of both instances of rangeKey above. Evidently both are necessary to do the following query:
PromoCode.query('previouslyUsed').eq(false)
.where('promoGroup').eq('friend')
It's actually as "simple" as that. This let's me filter on two different fields.

Sequelize: how to do a WHERE condition on joined table with left outer join

My database model is as follows:
An employee drives one or zero vehicles
A vehicle can be driven by one or more employees
A vehicle has a model type that tells us it's fuel type amongst other things.
I'd like sequelize to fetch me all employees where they don't drive a vehicle, or if they do then the vehicle is not diesel.
So where VehicleID is null OR Vehicle.VehicleModel.IsDiesel = false
My current code is as follows:
var employee = sequelize.define('employee', {
ID: Sequelize.INTEGER,
VehicleID: Sequelize.INTEGER
});
var vehicle = sequelize.define('vehicle', {
ID: Sequelize.INTEGER,
ModelID: Sequelize.INTEGER
});
var vehicleModel = sequelize.define('vehicleModel', {
ID: Sequelize.INTEGER,
IsDiesel: Sequelize.BOOLEAN
});
employee.belongsTo(vehicle);
vehicle.belongsTo(vehicleModel);
If I run the following:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
}]
}];
employee
.findAll(options)
.success(function(results) {
// do stuff
});
Sequelize does a left outer join to get me the included tables. So I get employees who drive vehicles and who don't.
As soon as I add a where to my options:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
where: {
IsDiesel: false
}
}]
}];
Sequelize now does an inner join to get the included tables.
This means that I only get employees who drive a vehicle and the vehicle is not diesel. The employees who don't drive a vehicle are excluded.
Fundamentally, I need a way of telling Sequelize to do a left outer join and at the same time have a where condition that states the column from the joined table is false or null.
EDIT:
It turns out that the solution was to use required: false, as below:
options.include = [{
model: model.Vehicle,
attributes: ['ID', 'ModelID'],
include: [
{
model: model.VehicleModel,
attributes: ['ID', 'IsDiesel']
where: {
IsDiesel: false
},
required: false
}],
required: false
}];
I had already tried putting the first 'required:false' but I missed out on putting the inner one. I thought it wasn't working so I gave up on that approach. Dajalmar Gutierrez's answer made me realise I needed both for it to work.
When you add a where clause, sequelize automatically adds a required: true clause to your code.
Adding required: false to your include segment should solve the problem
Note: you should check this issue iss4019
Eager loading
When you are retrieving data from the database there is a fair chance that you also want to get associations with the same query - this is called eager loading. The basic idea behind that, is the use of the attribute include when you are calling find or findAll.
when you set
required: false
will do
LEFT OUTER JOIN
when
required: true
will do
INNER JOIN
for more detail docs.sequelizejs eager-loading

meteor make user's schema collection2 to login

I'd like use this schema as user and login or extend form users.I read documentation and I don't understand how extends from users. how I can make it?
Dipendenti = new Mongo.Collection('dipendenti');
DipendentiSchema = new SimpleSchema({
nome: {
type: String
},
cognome:{
type: String
},
codiceFiscale:{
type: String
},
telefono:{
type: String
},
indirizzo:{
type: String
}
});
I believe you are trying to extend/merge the schema listed above to the users collection. If so, you just need to attach the schema to the collection.
Meteor.users.attachSchema(DipendentiSchema);
UPDATE:
To use this new merged schema, you should be able to do something like:
Accounts.createUser({
username: 'test',
email: 'test#example.com',
password: 'password',
nome: 'Richard',
cognome: 'Ortiz',
codiceFiscale: 'EUR',
telefono: '+39 06 49911',
indirizzo: 'Piazzale Aldo Moro, 5, 00185 Roma, Italy'
});
If you want to make email address optional in your schema, you can add the following to it.
emails: {
optional: true,
type: [Object]
},
"emails.$.address": {
optional: true,
type: String
},
"emails.$.verified": {
optional: true,
type: Boolean
}
UPDATE 2:
Ensure that you are defining and attaching the schema wherever you are trying to make changes to the users collection. It is generally considered a best practice to do your database changes only on the server for security. You could write a method on the server using Meteor.methods({}); and then call it on the client with Meteor.call({}); and pass it your user data. You can read more about this approach the Meteor documentation.

Waterline queries similar to HQL

I have models in Sails as follows:
User Model
attributes: {
firstName: {
type: 'string',
required: true
},
lastName: {
type: 'string',
required: true
},
company: {
model: 'Company'
}
}
Company
name: {
type: 'string',
required: true,
unique: true
},
description: {
type: 'string',
required: true
}
In HQL queries, for getting a user working in a specific company, we use something like this:
Select * from User where company.name=?
How can I achieve same in Sails, Right now there are two queries which I am running, one to get User and then another to get company for that user. Is there any way both can be combined in one?
and one more thing, how does waterline deal with scenarios where in we need to get something out of a foreign key directly i.e. If I get user data, then can I get company details by just calling:
User.findOne({id:1}, function(err, res){
//Res contains user data with just companyId now,
//how can i get whole company object here
var companyName = res.company.name; //This is not working currently
})
Try something like this:
User.findOne({id: 1})
.populate('company')
.exec(function(err, user) {
if(err){
//handle errors
}else{
//do stuff
}
});
This should get the values from the association (foreign key).

Categories

Resources