Develop Reference

Sequelize bulkCreate controller not inserting into associated table

I am trying to insert multiple records into 2 associated tables/models using sequelize bulkCreate. I can only get data to insert into the parent table. I am using MVC framework.
I have each model in a separate file, with associations defined therein.
The function to carry out the bulkCreate is in a separate controller file.
I call the request using a route file.
Here are the models and the model index
Occurrence.js
module.exports = (sequelize, Sequelize) => {
const Occurrence = sequelize.define("occurrence", {
recordedBy : {
type : Sequelize.STRING,
allowNull : true
},
scientificName : {
type : Sequelize.STRING,
allowNull : true
}
});
Occurrence.associate = (models) => {
Occurrence.hasMany(models.materialSample, {
foreignKey: 'occurrenceTableID'
})
}
return Occurrence;
};
MaterialSample.js
module.exports = (sequelize, Sequelize) => {
const MaterialSample = sequelize.define("materialSample", {
materialSampleID : {
type : Sequelize.STRING,
allowNull : true
},
materialSampleType : {
type : Sequelize.STRING,
allowNull : true
}
numberCollected : {
type : Sequelize.INTEGER,
allowNull : true
}
});
MaterialSample.associate = (models) => {
MaterialSample.belongsTo(models.occurrence, {
foreignKey: 'occurrenceTableID'
})
}
return MaterialSample;
};
index.js
const dbConfig = require("../config/db.config.js");
const Sequelize = require("sequelize");
const { response } = require("express");
const sequelize = new Sequelize(dbConfig.DB, dbConfig.USER, dbConfig.PASSWORD, {
host: dbConfig.HOST,
dialect: dbConfig.dialect,
operatorsAliases: false,
pool: {
max: dbConfig.pool.max,
min: dbConfig.pool.min,
acquire: dbConfig.pool.acquire,
idle: dbConfig.pool.idle
}
});
const db = {};
db.Sequelize = Sequelize;
db.sequelize = sequelize;
db.project = require("./Project.js")(sequelize, Sequelize);
db.event = require("./Event.js")(sequelize, Sequelize);
db.occurrence = require("./Occurrence.js")(sequelize, Sequelize);
db.materialSample = require("./MaterialSample.js")(sequelize, Sequelize);
db.preservedSpecimen = require("./PreservedSpecimen.js")(sequelize, Sequelize);
db.transfer = require("./Transfer.js")(sequelize, Sequelize);
db.germplasmViabilityTest = require("./GermplasmViabilityTest.js")(sequelize, Sequelize);
db.viabilityTracking = require("./ViabilityTracking.js")(sequelize, Sequelize);
//set up table associations
Object.keys(db).forEach((modelName) => {
if ('associate' in db[modelName]){
//call the associate function and pass reference to all other models
db[modelName].associate(db)
}
})
module.exports = db;
Here is the controller file with the bulkCreate function (insertData.js)
const db = require("../models");
const MaterialSample = db.materialSample;
const Occurrence = db.occurrence;
async function insertData() {
//create some data
const data = [
{
recordedBy: "Botanist One",
MaterialSamples:
{
materialSampleType: "tissue",
numberCollected: 2
}
},
{
recordedBy: "Botanist Two",
MaterialSamples:
{
materialSampleType: "seed",
numberCollected: 72
}
}
]
//use bulkCreate with include to insert the data
const result = await Occurrence.bulkCreate(data, {
include : [
{
model: MaterialSample
}
]
})
}
module.exports = {
insertData
}
and the route
const express = require("express");
const router = express.Router();
const insertDatasController= require("../controllers/insertData")
let routes = (app) => {
//views
//index
app.get('/', (req, res) => {
res.render("index");
});
//POST /api/materialSamples
router.post("/materialSample", insertDataController.insertData)
app.use("/api", router);
};
module.exports = routes;
Both records are inserted into the occurrence table, but nothing gets inserted into the materialSample table.
I will also note that when I send POST the request via PostMan, it seems to hang in "Sending request..."

this fixed it. it was the capitalization of the table name in the controller function
async function insertData(req, res) {
//create some data
const data = [{
recordedBy: "Botanist One",
materialSamples: {
materialSampleType: "tissue",
numberCollected: 2
}
},
{
recordedBy: "Botanist Two",
materialSamples: {
materialSampleType: "seed",
numberCollected: 72
}
}
]
//use bulkCreate with include to insert the data
const result = await Occurrence.bulkCreate(data, {
include: [{
model: MaterialSample
}]
})
.then(() => {
res.send()
})
.catch((err) => {
console.log(err);
})
}

Prisma find many and count in one request

I have a pagination in my category service, and I have to return obj with total count of categories and data
But there's can be some parameters. As example, I should return categories that was created by certain user:
async findAll(
{ onlyParents }: ParamsCategoryDto,
user: ITokenPayload | undefined,
): Promise<IFilterRes> {
const categories = await this.prisma.category.findMany({
where: {
user_id: user?.id,
},
});
return {
pagination: {
total: this.prisma.category.count({
where: { // <- duplicate
user_id: user?.id,
},
}),
},
data: categories,
};
}
I should duplicate where in both query. Which is not very nice. Is there any option to do it in one request.
P.S. I can make some var for where, but in this way I lose typification, which I also don't like.

This is my example code to acheive it with a single transaction, no duplicate code and not losing type autocomplete
import { Prisma } from '#prisma/client';
import { PrismaClient } from '#prisma/client'
const prisma = new PrismaClient()
const findAll = async (userId: String) => {
const query: Prisma.categoriesFindManyArgs = {
where: {
user_id: userId,
}
};
const [categories, count] = prisma.$transaction([
prisma.categories.findMany(query),
prisma.categories.count({ where: query.where })
]);
return {
pagination: {
total: count
},
data: categories
};
};

Why could the findOrCreate plugin create several documents at once?

I'm currently working on a MERN application with following/follower function for the users. I decided to create separate schemas for following and follower relationships detached from user schema.
Follower schema
const mongoose = require('mongoose');
const findOrCreate = require('mongoose-findorcreate');
const ObjectId = mongoose.Schema.Types.ObjectId;
const followerSchema = mongoose.Schema({
userId: {
type: ObjectId,
ref: 'User'
},
follower: {
type: [ObjectId],
ref: 'User'
}
});
followerSchema.plugin(findOrCreate);
const Follower = mongoose.model('Follower', followerSchema);
module.exports = { followerSchema, Follower };
Following schema
const mongoose = require('mongoose');
const findOrCreate = require('mongoose-findorcreate');
const ObjectId = mongoose.Schema.Types.ObjectId;
const followingSchema = mongoose.Schema({
userId: {
type: ObjectId,
ref: 'User'
},
following: {
type: [ObjectId],
ref: 'User'
}
});
followingSchema.plugin(findOrCreate);
const Following = mongoose.model('Following', followingSchema);
module.exports = { followingSchema, Following };
The problem however starts in my service where followings are created as supposed to. However, for followers mongoose create 6-8 documents at once with userIds that don't even exist in my db.
Here's the code of the followerService (it's the first function)
const { Follower } = require('../models/followerModel');
const { Following } = require('../models/followingModel');
const { User } = require('../models/userModel');
const mongoose = require('mongoose');
exports.changeFollowStatus = async (user, userId) => {
console.log({ userId: userId, user: user._id });
const newFollowing = await Following.findOrCreate({ userId: user._id }, (err, following, created) => {
console.log({following: following});
if (!err && !isFollowing(following, userId)) {
following.following.push(mongoose.Types.ObjectId(userId));
following.save();
User.findByIdAndUpdate(mongoose.Types.ObjectId(userId), {$inc: {follower: 1}});
} else {
const followingIndex = following.following.indexOf(mongoose.Types.ObjectId(userId));
following.following.splice(followingIndex, 1);
following.save();
User.findByIdAndUpdate(mongoose.Types.ObjectId(userId), { $inc: { follower: -1 } });
}
});
const newFollower = await Follower.findOrCreate({ userId: mongoose.Types.ObjectId(userId) }, (err, follower, created) => {
console.log({follower: follower});
if (!err && !isFollower(follower, user._id)) {
follower.follower.push(user._id);
follower.save();
User.findByIdAndUpdate(user._id, { $inc: { following: 1 } });
} else {
const followerIndex = follower.follower.indexOf(user._id);
follower.follower.splice(followerIndex, 1);
follower.save();
User.findByIdAndUpdate(user._id, { $inc: { following: -1 } });
}
});
};
exports.showFollowings = async (userId) => {
const followings = await Following.findOrCreate({ userId: mongoose.Types.ObjectId(userId) });
return followings.following;
};
exports.showFollowers = async (userId) => {
const followers = await Follower.findOrCreate({ userId: mongoose.Types.ObjectId(userId) });
return followers.follower;
};
const isFollowing = (newFollowing, userId) => {
return newFollowing.following.includes(mongoose.Types.ObjectId(userId));
};
const isFollower = (newFollower, userId) => {
return newFollower.follower.includes(userId);
}
Now, my following adding code and follower adding code look almost identical, but for some reason for followers, mongoose creates many more documents. The strange thing is that there is a follower document with the correct userId, but many other with random userIds get created which doesn't happen with followings which works as supposed to.
I also checked whether I pass the correct values and everything seems to be fine. But just for reference, here's the controller function from which I pass the values.
exports.changeFollowingStatus = async (req, res, next) => {
passport.authenticate('jwt', async (err, user, info) => {
if (err) {
console.error({ authError: err });
};
if (info !== undefined) {
console.error({ authError: info.message });
res.status(403).send(info.message);
} else {
console.log({params: req.params});
const userToFollow = req.params.id;
await FollowerService.changeFollowStatus(user, userToFollow);
res.status(200).send({ message: 'success' })
}
})(req, res, next);
};
Could anyone help me with this bug or at least navigate me towards the right direction? I can't seem to find solution to my problem. Thank you all in advance!

Mongoose find by a subdocument's value

I have 2 schemas
const schema = Schema({
headLine: {
type: String,
required: false
},
availableDays: [{
type: Schema.Types.ObjectId,
ref: AvailableDay
}]
}, {collection: 'providers', timestamps: true});
module.exports = mongoose.model("Provider", schema);
const schema = Schema({
day: {
type: String,
enum: ['Mondays','Tuesdays','Wednesdays','Thursdays','Fridays','Saturdays','Sundays']
},
timeFrom: String,
timeTo: String
}, {collection: 'availableDays', timestamps: true});
module.exports = mongoose.model("AvailableDay", schema);
Then in a route I call to a repository like this
router.get('/', async (req, res) => {
const match = {};
const sort = {};
const options = {};
// Arrange sort
if(req.query.sortBy){
const sortArray = JSON.parse(req.query.sortBy);
sortArray.map(e => sort[e[0]] = e[1] && e[1] === 'desc' ? -1 : 1);
options['sort'] = sort
}
// Get the pagination: limit how many, skip where it starts
if(req.query.limit) {
options['limit'] = parseInt(req.query.limit);
}
if(req.query.skip) {
options['skip'] = parseInt(req.query.skip);
}
const docs = await ProviderRepository.findBy(match, {}, options);
res.status(200).json(docs)
});
So what I need here is to filter providers for an AvailableDay monday and return the docs and count the total docs for pagination. I'm doing something like this without success
const findBy = async (params, projection = "", options = {}, callback) => {
const data = () => {
Provider.find(params, projection, options)
.populate([{path: 'user', match: {gender: 'F'}}]).exec((error, e) => {
if (error) {
console.log('error:', error)
return {error: error}; // returns error in json
}
return e.filter(i => i.user);
});
};
const total = await Provider.countDocuments(params).exec();
return {data(), total}
}
Thanks in advance

Use mongoose-aggregate-paginate-v2 and update your schema. If you use that package then you have to convert your queries from populate to aggregate style.
STEP 1: Update schema. Sample Schema:
const mongoose = require('mongoose');
const mongoosePaginate = require('mongoose-aggregate-paginate-v2');
const Schema = mongoose.Schema;
let definition = {
headLine: {
type: String,
required: false
},
availableDays: [{
type: Schema.Types.ObjectId,
ref: AvailableDay
}]
};
let options = {
collection: 'providers'
};
let providerSchema = new Schema(definition, options);
providerSchema.plugin(mongoosePaginate);
module.exports = mongoose.model('providers', providerSchema);
STEP 2: Update controller. Sample code in controller:
router.get('/', async (req, res) => {
const match = {}
const sort = {
// Fill it based on your sort logic.
}
const paginateOptions = {
page: req.query.page, // Page number like: 1, 2, 3...
limit: req.query.limit // Limit like: 10, 15, 20...
};
ProviderRepository
.findBy(match, {}, sort, paginateOptions)
.then(() => {
res.status(200).json(docs)
})
.catch(() => {
res.status(HTTP_ERROR_CODE).json({ "error": "Your error message" })
})
});
STEP 3: Update manager. Sample code in manager:
const findBy = (match, projection, sort, paginateOptions) => {
if (!paginateOptions) {
paginateOptions = {
pagination: false
};
}
let providerAggregate = providerSchema.aggregate([
{
$lookup: {
from: "availableDays",
let: { days: "$availableDays" },
pipeline: [
{
$match: {
$expr: {
$in: ["$$availableDays", "$day"]
}
}
}
],
as: "availableDays"
}
},
{
$lookup: {
from: "users", // I dont know the collection name
let: { user_id: "$user" }
pipeline: [
{
$match: {
"gender": 'F',
$expr: {
$eq: ["$_id", "$$user_id"]
}
}
}
],
as: "users"
}
}
{ $sort: sort }
]);
return providerSchema
.aggregatePaginate(providerAggregate, paginateOptions)
.then(res => {
return res;
})
.catch(err => {
throw err;
});
};

Sequelize create not a function?

I am trying to use model.create and it's saying that model.create is not a function. I googled around and couldn't seem to find any rmesolutions. Please note that I'm using es6 imports / exports in node via babel.
model.js
'use strict';
export default (sequelize, DataTypes) => {
let attachments = sequelize.define('attachments', {
type: DataTypes.STRING,
category: DataTypes.STRING,
value: DataTypes.STRING,
}, {});
attachments.associate = (models) => {
};
return attachments;
};
controller
import attachments from '../../../models/attachments'
import to_attachments from '../../../models/to_attachments'
import AWSService from '../../utils/awsS3Api'
export async function createPhoto(ctx) {
...
try {
let attachment = await attachments.create({
type: 'image',
category: imageCategory,
value: data.location
});
..etc
What am I doing wrong?

If you declare model via function:
export default (sequelize, DataTypes) => {
...
};
Then you should import it with sequelize.import
const Attachment = sequelize.import('../../../models/attachments')
export async function createPhoto(ctx) {
const attachment = await Attachment.create({
type: 'image',
category: imageCategory,
value: data.location
});
}
P.S. Some recommendations
1) Name model variables with capital letter you deal with static classes.
const user = await User.findById(123)
const users = await User.findAll({where: {id: [1,2,3]}})
2) Singular not Users but User.