Develop Reference

Search query String Array Object in mongoose and use elemMatch is not working as expected

I am facing with an error which I am unable to resolve it.
I want to use query search to search if the data of 1 job matches with user data but I am stucked in some cases.
First case it is that my query search of the job looks like this I mean the data.
The problem it is that in the searchFilterSkills.searchSkillsOffer I have an array of objects and I want to match the name of each of them if one of them exist but I am unable to iterate through them because I get an array of Strings with .map() which cannot iterate on it.
After answer from Fabian if all elements matches it returns the elements and it matches but I want if 1 of the elements matches it will return the object.
I tried $in but did not work. Is there any query to use instead of $all
These are my data which I try to search.
"skillsOffer":[
{
"name":"Max",
"rate":0
},
{
"name":"Test",
"rate":0
},
{
"name":"Javascript",
"rate":0
}
],
"country":"DEU",
"postalCode":12345
And these are the user Data which he/she has.
"searchFilter" : {
"remote" : 0,
"data" : [
{
"region" : [
"1"
],
"country" : "DEU",
"searchActive" : false,
"postalCode" : "123",
"available" : {
"$date" : 1664955924380
}
}
]
},
"searchFilterSkills" : {
"searchSkillsOffer" : [
{
"name" : "Javascript",
"rate" : 100
},
{
"name" : "Test",
"rate" : 60
},
{
"name" : "Client",
"rate" : 0
}
],
}
At skillsOffer I want to search if only the name matches not the rate.
Then if remote is 1 then search the above query and without postalCode with remote or the above one and remote.
async searchUsers(req, res, next) {
const jobID = req.query.jobID;
let job = await Job.findById(jobID);
let postalCode = job.postalCode;
postalCode = postalCode.toString().slice(0, 1);
let postalCode2 = job.postalCode;
postalCode2 = postalCode2.toString().slice(0, 2);
let postalCode3 = job.postalCode;
postalCode3 = postalCode3.toString().slice(0, 3);
let postalCode4 = job.postalCode;
postalCode4 = postalCode4.toString().slice(0, 4);
let postalCode5 = job.postalCode;
postalCode5 = postalCode5.toString().slice(0, 0);
let userIds = job.skillsOffer.map(user => user.name).join(",");
let users = await User.find({
"searchFilter.data": {
$elemMatch: {
"$or": [
{
postalCode: postalCode,
},
{
postalCode: postalCode2,
},
{
postalCode: postalCode3,
},
{
postalCode: postalCode4,
},
{
postalCode: postalCode,
},
{
postalCode: postalCode5,
},
]
}
},
"searchFilter.data": {
$elemMatch: {
country: job.country
}
},
'searchFilterSkills.searchSkillsOffer': {
$elemMatch: {
name: {
$regex: new RegExp(`^${job.skillsOffer.map(jt => jt.name)}`, 'i'), but it does not return a thing here
},
},
},
});
if (job.remote.toString() === "1") {
users = await User.find({
"searchFilter.data": {
$elemMatch: {
"$or": [
{
postalCode: postalCode,
},
{
postalCode: postalCode2,
},
{
postalCode: postalCode3,
},
{
postalCode: postalCode4,
},
{
postalCode: postalCode,
},
{
postalCode: postalCode5,
},
]
}
},
"searchFilter.data": {
$elemMatch: {
country: job.country
}
},
"searchFilter.remote": job.remote,
});
}
if (!users) {
res.status(204).json({ error: "No Data" });
return;
}
return res.status(200).send({
user: users.map(t =>
t._id
)
});
},

I assume you want to match each name from skillsOffer array. This way you have to define an $elemMatch object for each name (basically mapping each name).
You can use the following (partial) query in your code in order to check whether all names are contained in your searchFilterSkills.searchSkillsOffer array.
{
'searchFilterSkills.searchSkillsOffer': {
$all: job.skillsOffer
.map((user) => user.name)
.map((name) => ({
$elemMatch: {
name: {
$regex: new RegExp(`^${name}$`, 'i'),
},
},
})),
},
}
If you would like to match any name, you should or the following code:
{
$or: job.skillsOffer
.map((user) => user.name)
.map((name) => ({
'searchFilterSkills.searchSkillsOffer': {
$elemMatch: {
name: {
$regex: new RegExp(`^${name}$`, 'i'),
},
},
},
})),
}

Sequelize search "Unknown column 'contact.name' in where clause

I have a service that is in charge of bringing the tickets with the last message of the users.
For this, the Contact, Queue, WhatsApp models were added to the include.
The problem is that when adding the Tags model, closely related to "Contact", the service stopped working and response with:
"Unknown column 'contact.name' in where clause
The only thing I added was the relationship with Tags, since it is new. Help me understand? It's like it no longer recognizes the column
interface Request {
searchParam?: string;
pageNumber?: string;
status?: string;
date?: string;
showAll?: string;
userId: string;
withUnreadMessages?: string;
queueIds: number[];
}
interface Response {
tickets: Ticket[];
count: number;
hasMore: boolean;
}
const ListTicketsService = async ({
searchParam = "",
pageNumber = "1",
queueIds,
status,
date,
showAll,
userId,
withUnreadMessages
}: Request): Promise<Response> => {
let whereCondition: Filterable["where"] = {
[Op.or]: [{ userId }, { status: "pending" }],
queueId: { [Op.or]: [queueIds, null] }
};
let includeCondition: Includeable[];
includeCondition = [
{
model: Contact,
as: "contact",
attributes: ["id", "name", "number", "profilePicUrl"],
include: [{
model: Tags,
as: "tags",
attributes: ["name"],
}]
},
{
model: Queue,
as: "queue",
attributes: ["id", "name", "color"]
},
{
model: Whatsapp,
as: "whatsapp",
attributes: ["name"]
},
];
if (showAll === "true") {
whereCondition = { queueId: { [Op.or]: [queueIds, null] } };
}
if (status) {
whereCondition = {
...whereCondition,
status
};
}
if (searchParam) {
const sanitizedSearchParam = searchParam.toLocaleLowerCase().trim();
includeCondition = [
...includeCondition,
{
model: Message,
as: "messages",
attributes: ["id", "body"],
where: {
body: where(
fn("LOWER", col("body")),
"LIKE",
`%${sanitizedSearchParam}%`
)
},
required: false,
duplicating: false
}
];
whereCondition = {
...whereCondition,
[Op.or]: [
{
"$contact.name$": where(
fn("LOWER", col("contact.name")),
"LIKE",
`%${sanitizedSearchParam}%`
)
},
{ "$contact.number$": { [Op.like]: `%${sanitizedSearchParam}%` } },
{
"$message.body$": where(
fn("LOWER", col("body")),
"LIKE",
`%${sanitizedSearchParam}%`
)
}
]
};
}
if (date) {
whereCondition = {
createdAt: {
[Op.between]: [+startOfDay(parseISO(date)), +endOfDay(parseISO(date))]
}
};
}
if (withUnreadMessages === "true") {
const user = await ShowUserService(userId);
const userQueueIds = user.queues.map(queue => queue.id);
whereCondition = {
[Op.or]: [{ userId }, { status: "pending" }],
queueId: { [Op.or]: [userQueueIds, null] },
unreadMessages: { [Op.gt]: 0 }
};
}
const limit = 40;
const offset = limit * (+pageNumber - 1);
const { count, rows: tickets } = await Ticket.findAndCountAll({
where: whereCondition,
include: includeCondition,
distinct: true,
limit,
offset,
order: [["updatedAt", "DESC"]], logging: console.log
});
const hasMore = count > offset + tickets.length;
return {
tickets,
count,
hasMore
};
};
export default ListTicketsService;
Another thing is, i don't know how but this is giving me only unique record. The problem is, 1 contact may have n Tags. So, its possible do this query?
Regards

How to sort data with dynamic fields in PouchDB?

I'm having this query to index first_name and sort data according to it and it's working fine
try {
Users.createIndex({
index: { fields: ['first_name'] }
}).then(function (response) {
console.log(response);
}).catch(function (err) {
console.log(err);
});
const users = (await Users.find({
limit, skip: limit * (page - 1),
selector: {first_name: {$gt: null}},
sort: [ { 'first_name' : 'asc'} ]
})).docs;
But when I try to use variables it triggers an error
Error: Cannot sort on field(s) "orderBy" when using the default index
Code
orderBy = (query.params !== undefined && query.params.orderBy !== undefined) ? query.params.orderBy.sortField : 'first_name',
sortOrder = (query.params !== undefined && query.params.orderBy !== undefined) ? query.params.orderBy.sortOrder : 'asc'
console.log('orderBy: ' + orderBy) // first_name
console.log('sortOrder: ' + sortOrder) // asc
try {
Users.createIndex({
index: { fields: [orderBy] }
}).then(function (response) {
console.log(response);
}).catch(function (err) {
console.log(err);
});
const users = (await Users.find({
limit, skip: limit * (page - 1),
selector: {orderBy: {$gt: null}},
sort: [ { orderBy : sortOrder } ]
})).docs;
How can I edit this to make it work with dynamic variable just like the static variable?

The variable orderBy is not going to be substituted by value in the following code
selector: {orderBy: {$gt: null}},
sort: [ { orderBy : sortOrder } ]
The code evaluates orderBy literally. To assign a dynamic key to an object, use the object indexer:
myObject[myVar] = myVal;
Therfore in your code, something like this should do.
const query = {
selector: {},
sort: []
};
// setup selector
query.selector[prop] = {
$gt: null
};
// setup sort
let sortParam = {};
sortParam[prop] = sortDirection;
query.sort.push(sortParam);
I added a pouchDB snippet illustrating that concept.
let db;
// init example db instance
async function initDb() {
db = new PouchDB('test', {
adapter: 'memory'
});
await db.bulkDocs(getDocsToInstall());
}
initDb().then(async() => {
await db.createIndex({
index: {
fields: ['first_name']
}
});
await doQuery("first_name", "desc");
});
async function doQuery(prop, sortDirection) {
const query = {
selector: {},
sort: []
};
// setup selector
query.selector[prop] = {
$gt: null
};
// setup sort
let sortParam = {};
sortParam[prop] = sortDirection;
query.sort.push(sortParam);
// log the query
console.log(JSON.stringify(query, undefined, 3));
// exec the query
const users = (await db.find(query)).docs;
users.forEach(d => console.log(d[prop]));
}
// canned test documents
function getDocsToInstall() {
return [{
first_name: "Jerry"
},
{
first_name: "Bobby"
},
{
first_name: "Phil"
},
{
first_name: "Donna"
},
{
first_name: "Ron"
},
{
first_name: "Mickey"
},
{
first_name: "Bill"
},
{
first_name: "Tom"
},
{
first_name: "Keith"
},
{
first_name: "Brent"
},
{
first_name: "Vince"
},
]
}
<script src="https://cdn.jsdelivr.net/npm/pouchdb#7.1.1/dist/pouchdb.min.js"></script>
<script src="https://github.com/pouchdb/pouchdb/releases/download/7.1.1/pouchdb.memory.min.js"></script>
<script src="https://github.com/pouchdb/pouchdb/releases/download/7.1.1/pouchdb.find.min.js"></script>

Implement feed with retweets in MongoDB

I want to implement retweet feature in my app. I use Mongoose and have User and Message models, and I store retweets as array of objects of type {userId, createdAt} where createdAt is time when retweet occurred. Message model has it's own createdAt field.
I need to create feed of original and retweeted messages merged together based on createdAt fields. I am stuck with merging, whether to do it in a single query or separate and do the merge in JavaScript. Can I do it all in Mongoose with a single query? If not how to find merge insertion points and index of the last message?
So far I just have fetching of original messages.
My Message model:
const messageSchema = new mongoose.Schema(
{
fileId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'File',
required: true,
},
userId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true,
},
likesIds: [{ type: mongoose.Schema.Types.ObjectId, ref: 'User' }],
reposts: [
{
reposterId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
createdAt: { type: Date, default: Date.now },
},
],
},
{
timestamps: true,
},
);
Edit: Now I have this but pagination is broken. I am trying to use newCreatedAt field for cursor, that doesn't seem to work. It returns empty array in second call when newCreatedAt is passed from the frontend.
messages: async (
parent,
{ cursor, limit = 100, username },
{ models },
) => {
const user = username
? await models.User.findOne({
username,
})
: null;
const options = {
...(cursor && {
newCreatedAt: {
$lt: new Date(fromCursorHash(cursor)),
},
}),
...(username && {
userId: mongoose.Types.ObjectId(user.id),
}),
};
console.log(options);
const aMessages = await models.Message.aggregate([
{
$addFields: {
newReposts: {
$concatArrays: [
[{ createdAt: '$createdAt', original: true }],
'$reposts',
],
},
},
},
{
$unwind: '$newReposts',
},
{
$addFields: {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original',
},
},
{ $match: options },
{
$sort: {
newCreatedAt: -1,
},
},
{
$limit: limit + 1,
},
]);
const messages = aMessages.map(m => {
m.id = m._id.toString();
return m;
});
//console.log(messages);
const hasNextPage = messages.length > limit;
const edges = hasNextPage ? messages.slice(0, -1) : messages;
return {
edges,
pageInfo: {
hasNextPage,
endCursor: toCursorHash(
edges[edges.length - 1].newCreatedAt.toString(),
),
},
};
},
Here are the queries. The working one:
Mongoose: messages.aggregate([{
'$match': {
createdAt: {
'$lt': 2020 - 02 - 02 T19: 48: 54.000 Z
}
}
}, {
'$sort': {
createdAt: -1
}
}, {
'$limit': 3
}], {})
And the non working one:
Mongoose: messages.aggregate([{
'$match': {
newCreatedAt: {
'$lt': 2020 - 02 - 02 T19: 51: 39.000 Z
}
}
}, {
'$addFields': {
newReposts: {
'$concatArrays': [
[{
createdAt: '$createdAt',
original: true
}], '$reposts'
]
}
}
}, {
'$unwind': '$newReposts'
}, {
'$addFields': {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original'
}
}, {
'$sort': {
newCreatedAt: -1
}
}, {
'$limit': 3
}], {})

This can be done in one query, although its a little hack-ish:
db.collection.aggregate([
{
$addFields: {
reposts: {
$concatArrays: [[{createdAt: "$createdAt", original: true}],"$reports"]
}
}
},
{
$unwind: "$reposts"
},
{
$addFields: {
createdAt: "$reposts.createdAt",
original: "$reposts.original"
}
},
{
$sort: {
createdAt: -1
}
}
]);
You can add any other logic you want to the query using the original field, documents with original: true are the original posts while the others are retweets.

Mongoose $slice and get orginal size array

I'm currently trying to get the total amount of items in my News object, and return a slice of the items as objects.
I found out how to use the $slice operator in my query, but I don't know how to get the original size of the array of items.
The code I'm currently using in NodeJS:
if (req.query.limit) {
limit = 5;
}
News.findOne({ connected: club._id }, {items: {$slice: limit}}).exec(function (err, news) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else if (!news || news.items.length === 0) {
res.jsonp([]);
} else {
const returnObj = { items: [], totalNumber: 0 };
const items = news.items.sort(function (a, b) {
return b.date - a.date
});
res.jsonp({
items: items,
totalNumber: news.items.length
});
}
});
The Mongo model:
var mongoose = require('mongoose'),
validator = require('validator'),
Schema = mongoose.Schema;
var NewsSchema = new Schema({
connected: {
type: Schema.Types.ObjectId,
required: 'Gelieve een club toe te wijzen.',
ref: 'Club'
},
items: [{
userFirstName: String,
action: String,
date: Date,
targetName: String
}],
created: {
type: Date,
default: Date.now
}
});
module.exports = mongoose.model('News', NewsSchema);
How would I do this efficiently?
Thanks!
EDIT: final code which works:
News.aggregate([{
$match: {
connected: club._id
}
}, {
$project: {
totalNumber: {
$size: '$items'
},
items: {
$slice: ['$items', limit]
}
}
}
]).exec(function (err, news) {
console.log(news);
if (!news || news[0].items.length === 0) {
res.jsonp([]);
} else {
res.jsonp(news[0]);
}
});

You cannot have both information at once using find and $slice.
The soluce you have :
Use aggregate to return the count and only the sliced values.
Like :
[{
$project: {
count: {
$size: "$params",
},
params: {
$slice: ["$params", 5],
},
},
}]
To help you out making aggregate, you can use the awesome mongodb-compass software and its aggregate utility tool.
Use a find without $slice, get the number of item there, and then slice in javascript the array before returning it.
EDIT :
[{
$sort: {
'items.date': -1,
},
}, {
$project: {
count: {
$size: "$items",
},
params: {
$slice: ["$items", 5],
},
},
}]