I have this function that updates a table accordingly. I have used the pipe (||) operator already to pick the new values or retain the old ones if nothing was supplied in the request body for that field but I realized this method may not be maintainable if the columns are much. Hence, is there a way one can just extract only the fields that are supplied into the request body and then inject them into their respective columns without using the pipe (||)?
static async updateProduct(req, res) {
const { id } = req.authData.payload;
const {
description, category, imageurl, quantity, unitPrice
} = req.body;
try {
const { rows } = await db.query(queryProductAdmin, [req.params.productId, id]);
if(!rows[0]) {
return res.status(404).json({
status: 'Fail',
message: 'The product is not available'
});
}
**/** Instead of using the pipe, how can something like this be achieved?
.update(req.body, { fields: Object.keys(req.body) })*/**
const values = [
description || rows[0].description,
category || rows[0].category,
imageurl || rows[0].imageurl,
quantity || rows[0].quantity,
unitPrice || rows[0].unitPrice,
req.params.productId,
id
];
const response = await db.query(updateProductQuery, values);
const updatedProduct = response.rows[0];
return res.status(200).json({
message: 'Update was successful',
updatedProduct
});
} catch(error) {
const { message } = error;
return res.status(500).json({
status: 'Fail',
message
});
}
N.B: The code is working fine. I just need ideas on how to update dynamically.
You could do something like:
const values = ['description', 'category', 'imageurl', 'unitPrice']
.map(v => req.body[v] || rows[0][v])
.push(req.params.productId, id)
To go a bit further you could get the table fields using postgres API and map those instead depending on whether or not you are showing all the fields in the query array or not
Related
I am developing the backend of an application using Node JS, Sequelize and Postgres database.
When the course is registered, the user must inform which organizations, companies and teachers will be linked to it.
The organization IDs are passed through an array to the backend, I am trying to do a check to make sure that the passed IDs exist.
What I've done so far is this:
const { organizations } = req.body;
const organizationsArray = organizations.map(async (organization) => {
const organizationExists = await Organization.findByPk(organization);
if (!organizationExists) {
return res
.status(400)
.json({ error: `Organization ${organization} does not exists!` });
}
return {
course_id: id,
organization_id: organization,
};
});
await CoursesOrganizations.bulkCreate(organizationsArray);
This link has the complete controller code, I believe it will facilitate understanding.
When !OrganizationExists is true, I am getting the return that the organization does not exist. The problem is when the organization exists, I am getting the following message error.
The Array.map() is returning an array of promises that you can resolve to an array using Promise.all(). Inside the map you should use throw new Error() to break out of the map - this error will be raised by Promise.all() and you can then catch it and return an error to the client (or swallow it, etc).
This is a corrected version of your pattern, resolving the Promise results.
const { organizations } = req.body;
try {
// use Promise.all to resolve the promises returned by the async callback function
const organizationsArray = await Promise.all(
// this will return an array of promises
organizations.map(async (organization) => {
const organizationExists = await Organization.findByPk(organization, {
attributes: ['id'], // we only need the ID
raw: true, // don't need Instances
});
if (!organizationExists) {
// don't send response inside the map, throw an Error to break out
throw new Error(`Organization ${organization} does not exists!`);
}
// it does exist so return/resolve the value for the promise
return {
course_id: id,
organization_id: organization,
};
})
);
// if we get here there were no errors, create the records
await CoursesOrganizations.bulkCreate(organizationsArray);
// return a success to the client
return res.json({ success: true });
} catch (err) {
// there was an error, return it to the client
return res.status(400).json({ error: err.message });
}
This is a refactored version that will be a bit faster by fetching all the Organizations in one query and then doing the checks/creating the Course inserts.
const { Op } = Sequelize;
const { organizations } = req.body;
try {
// get all Organization matches for the IDs
const organizationsArray = await Organization.findAll({
attributes: ['id'], // we only need the ID
where: {
id: {
[Op.in]: organizations, // WHERE id IN (organizations)
}
},
raw: true, // no need to create Instances
});
// create an array of the IDs we found
const foundIds = organizationsArray.map((org) => org.id);
// check to see if any of the IDs are missing from the results
if (foundIds.length !== organizations.length) {
// Use Array.reduce() to figure out which IDs are missing from the results
const missingIds = organizations.reduce((missingIds, orgId) => {
if (!foundIds.includes(orgId)){
missingIds.push(orgId);
}
return missingIds;
}, []); // initialized to empty array
throw new Error(`Unable to find Organization for: ${missingIds.join(', ')}`);
}
// now create an array of courses to create using the foundIds
const courses = foundIds.map((orgId) => {
return {
course_id: id,
organization_id: orgId,
};
});
// if we get here there were no errors, create the records
await CoursesOrganizations.bulkCreate(courses);
// return a success to the client
return res.json({ success: true });
} catch (err) {
// there was an error, return it to the client
return res.status(400).json({ error: err.message });
}
If you have an array of Ids and you want to check if they exist you should you use the (in) operator, this makes it so that you are hitting the DB only once and getting all the records in one hit (instead of getting them one by one in a loop), after you get these records you can check their lengths to determine if they all exist or not.
const { Op } = require("sequelize");
let foundOrgs = await Organization.findAll({
where: {
id: {
[Op.in]: organizationsArray,
}
}
});
i can not able to update nested data in my mongodb. here is my "update" module at back-end side.
exports.updateOne = (req, res) => {
if (!req.body) {
return res.status(400).send({
message: "Data to update can not be empty!"
});
}
const {id} = req.params;
console.log(req.body);
User.findByIdAndUpdate(id, req.body, { useFindAndModify: false, new: true}).populate('basic')
.then(data => {
if (!data) {
res.status(404).send({
message: `Cannot update User with id=${id}. Maybe User was not found!`
});
} else
res.send({ message: "User was dupdated successfully." , data});
})
.catch(err => {
res.status(500).send({
message:
err.message || "Error updating User with id=" + id
});
});
};
and my front-end side is;
onChangePosition(e) {
const position = e.target.value;
this.setState(prevState => ({
currentStaff: {
...prevState.currentStaff,
basic:
{
...prevState.currentStaff.basic,
position:position
}
}
}));
}
onChangeEmail(e) {
const emailBusiness = e.target.value;
this.setState(prevState => ({
currentStaff: {
...prevState.currentStaff,
emailBusiness:emailBusiness
}
}));
}
updateStaff() {
StaffDataService.updateOne(
this.state.currentStaff.id,
this.state.currentStaff
).then(response => {
console.log(response.data);
})
.catch(e => {
console.log(e);
})
}
i can change state properly, and my sending data "req.body" is what i want (it is an object). There is no problem.
as you see above, i can update "email" because it is on the main body of object, but can not update "position" (nested element) because it is inside of basic (populated data).
i tried different methods by mongoose, and tried "$set" command.
Can anyone solve this?
To update, the nested value/object in your document, you should use dot notations, so it depends from the req.body variable value.
req.body shouldn't be a Mongoose doc. In such case you mongoose.toObject.
Second thing is:
[update] Object should be: field_with_subdocument.key_value: updated_propery
like this:
/** Example doc */
{
_id: 1,
parent_field: {
baby_field: value
}
}
/** Inside async function */
...await Model.findByIdAndUpdate(id, { "parent_field.baby_field": value })
Also, take a look at [`{overwrite: true}`](https://mongoosejs.com/docs/api/model.html#model_Model.findByIdAndUpdate) option. It might be useful to you.
I faced the same issue, In my case, the defined mongoose schema for that model did not match the nested Object I was passing to the findByIdAndUpdate method. let me simplify it, here is the model
import { model, Schema } from 'mongooose';
const UserModel = model('user', new Schema({
profile: {
avatar: String,
bio: String,
}
}));
And here is the update query:
async function getDefaultProfile() {
const user = await UserModel.findById(process.env.DEFAULT_USER);
return user.profile;
}
const profile = await getDefaultProfile();
UserModel.findByIdAndUpdate('user id', {
$set: {
profile: profile
}
});
The important note was that my getDefaultProfile function returns a mongoose nested object, not a pure object. So in the returned object, I had $set, $get, etc function. So as you know this object is not what we define in the mongoose model, therefore the mongoose ignores it.
So I guess you have the same problem, or something close to my issue.
What should I do?
Run your project in debugging mode.
then check req.body or whatever that gives you the nested object (in my case getDefaultProfile).
Check it with your model, Are they equal?
And if that solution does not work for you, please try this solution, write a utility function:
export async function flatObjectAndSeparateThemByDot(
object: any,
): Promise<any> {
const res: any = {};
(function recurse(obj: any, current?: string) {
for (const key in obj) {
const value = obj[key];
// joined keys with dot
const newKey = current ? current + '.' + key : key;
if (value && typeof value === 'object') {
// it's a nested object, so do it again
recurse(value, newKey);
} else {
// it's not an object, so set the property
res[newKey] = value;
}
}
})(object);
return res;
}
then you can pass your nested object to this function and you will get something like this: { "profile.avatar": "lorem ipsum", "profile.bio": "bio temp" }. So to show you how this function works I will write a sample code:
const sampleProfile = {
profile: {
avatar: "asd",
bio: "sample"
}
}
const profile = await flatObjectAndSeparateThemByDot(sampleProfile);
await UserModel.findByIdAndUpdate('user id', {
$set: {
// other fields,
...profile
}
});
This is my first question here. I tried to save document in my collection, but it doesn't work. Response of function is exactly like I want, but it doesn't save in my db. In another controller (createRoom) foundUser.save() it works, but in this controller it doesn't. Thanks in advance!
I am using mongodb/mongooose and express.
const removeRoom = async (req,res,next) => {
const {roomId, userData} = req.body;
const { userId, token } = userData;
let foundUser;
let updatedRooms;
let indexOfNamespaces;
try {
foundUser = await User.findById(userId)
foundUser.namespaces.forEach((ns,i1)=>{
updatedRooms = ns.rooms.filter((room,i2) => {
if(room.id === roomId){
indexOfNamespaces = i1;
}
return room.id !== roomId
})
})
foundUser.namespaces[indexOfNamespaces].rooms = updatedRooms;
console.log(foundUser);
await foundUser.save();
} catch (err) {
console.log(err);
const error = new HttpError('Sth went wrong [removeRoom]', 500);
return next(error);
}
res.status(201).json({updatedNamespaces: foundUser.namespaces});
}
Mongoose does some optimizations where it will only actually save a field if it "changes". In this case you are modifyting an array, but the array is still the "same" array as in it still === (equals) the previous array. You need to use a new array to replace namespaces.
For example:
foundUser.namespaces = [
...foundUser.namespaces.slice(0, indexOfNamespaces),
{ ...foundUser.namespaces[indexOfNamespaces], rooms: updatedRooms },
...foundUser.namespaces.slice(indexOfNamespaces + 1)
]
Now, when you save Mongoose will see a "new" array that !== (does not equal) the previous array because it is a new instance and it will save it.
I am creating a Stripe payment system, and am having difficulty making this integrated code functional. The code was pulled from here. What I am trying to do is create a database reference to the specific user ID (uid) of the user, since the createStripeCharge waits for a write event to occur to something like /stripe_customers/XCXaweADowefj/charges/QC123XYZ. I'm not sure, but perhaps my issue is that I am incorrectly referencing the customer. If you have any ideas or feedback about how I can make this function correctly I greatly appreciate it!
The Trigger (this causes the Cloud Function to work):
payByStripe(amount,email,userId, token): firebase.database.Reference {
return firebase.database().ref(`/stripe_customers/${userId}/charges/`).push({
amount:amount,
email:email,
token:token
});
}
Cloud Function:
exports.createStripeCharge = functions.database.ref('/stripe_customers/{userId}/charges/{id}').onWrite(event => {
const val = event.data.val();
if (val === null || val.id || val.error) return null;
return admin.database().ref(`/stripe_customers/${event.params.userId}/customer_id`).once('value').then(snapshot => {
return snapshot.val();
}).then(customer => {
const amount = val.amount;
const idempotency_key = event.params.id;
let charge = {amount, currency, customer};
if (val.source !== null) charge.source = val.source;
return stripe.charges.create(charge, {idempotency_key});
}).then(response => {
return event.data.adminRef.set(response);
}, error => {
return event.data.adminRef.child('error').set(userFacingMessage(error)).then(() => {
return reportError(error, {user: event.params.userId});
});
}
);
});
Currently, the database is creating the stripe customers, but the charge doesn't go through, with error message "You have passed a blank string for 'customer'. You should remove the 'customer' parameter from your request or supply a non-blank value."
Update is I've been iterating on this. Tried the following:
firebase.database().ref(`/stripe_customers/${userId}/charges/`).push({
amount:amount,
email:email,
token:token,
customer: userId
});
&&
firebase.database().ref(`/stripe_customers/${userId}/charges/`).push({
amount:amount,
email:email,
token:token,
customer_id:{
customer: userId
}
});
And still getting "You have passed a blank string for 'customer'. You should remove the 'customer' parameter from your request or supply a non-blank value."
I'd like to perform a batch update using Knex.js
For example:
'UPDATE foo SET [theValues] WHERE idFoo = 1'
'UPDATE foo SET [theValues] WHERE idFoo = 2'
with values:
{ name: "FooName1", checked: true } // to `idFoo = 1`
{ name: "FooName2", checked: false } // to `idFoo = 2`
I was using node-mysql previously, which allowed multiple-statements. While using that I simply built a mulitple-statement query string and just send that through the wire in a single run.
I'm not sure how to achieve the same with Knex. I can see batchInsert as an API method I can use, but nothing as far as batchUpdate is concerned.
Note:
I can do an async iteration and update each row separately. That's bad cause it means there's gonna be lots of roundtrips from the server to the DB
I can use the raw() thing of Knex and probably do something similar to what I do with node-mysql. However that defeats the whole knex purpose of being a DB abstraction layer (It introduces strong DB coupling)
So I'd like to do this using something "knex-y".
Any ideas welcome.
I needed to perform a batch update inside a transaction (I didn't want to have partial updates in case something went wrong).
I've resolved it the next way:
// I wrap knex as 'connection'
return connection.transaction(trx => {
const queries = [];
users.forEach(user => {
const query = connection('users')
.where('id', user.id)
.update({
lastActivity: user.lastActivity,
points: user.points,
})
.transacting(trx); // This makes every update be in the same transaction
queries.push(query);
});
Promise.all(queries) // Once every query is written
.then(trx.commit) // We try to execute all of them
.catch(trx.rollback); // And rollback in case any of them goes wrong
});
Assuming you have a collection of valid keys/values for the given table:
// abstract transactional batch update
function batchUpdate(table, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(table)
.where('id', tuple.id)
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate('user', [...]);
Are you unfortunately subject to non-conventional column names? No worries, I got you fam:
function batchUpdate(options, collection) {
return knex.transaction(trx => {
const queries = collection.map(tuple =>
knex(options.table)
.where(options.column, tuple[options.column])
.update(tuple)
.transacting(trx)
);
return Promise.all(queries)
.then(trx.commit)
.catch(trx.rollback);
});
}
To call it
batchUpdate({ table: 'user', column: 'user_id' }, [...]);
Modern Syntax Version:
const batchUpdate = (options, collection) => {
const { table, column } = options;
const trx = await knex.transaction();
try {
await Promise.all(collection.map(tuple =>
knex(table)
.where(column, tuple[column])
.update(tuple)
.transacting(trx)
)
);
await trx.commit();
} catch (error) {
await trx.rollback();
}
}
You have a good idea of the pros and cons of each approach. I would recommend a raw query that bulk updates over several async updates. Yes you can run them in parallel, but your bottleneck becomes the time it takes for the db to run each update. Details can be found here.
Below is an example of an batch upsert using knex.raw. Assume that records is an array of objects (one obj for each row we want to update) whose values are the properties names line up with the columns in the database you want to update:
var knex = require('knex'),
_ = require('underscore');
function bulkUpdate (records) {
var updateQuery = [
'INSERT INTO mytable (primaryKeyCol, col2, colN) VALUES',
_.map(records, () => '(?)').join(','),
'ON DUPLICATE KEY UPDATE',
'col2 = VALUES(col2),',
'colN = VALUES(colN)'
].join(' '),
vals = [];
_(records).map(record => {
vals.push(_(record).values());
});
return knex.raw(updateQuery, vals);
}
This answer does a great job explaining the runtime relationship between the two approaches.
Edit:
It was requested that I show what records would look like in this example.
var records = [
{ primaryKeyCol: 123, col2: 'foo', colN: 'bar' },
{ // some other record, same props }
];
Please note that if your record has additional properties than the ones you specified in the query, you cannot do:
_(records).map(record => {
vals.push(_(record).values());
});
Because you will hand too many values to the query per record and knex will fail to match the property values of each record with the ? characters in the query. You instead will need to explicitly push the values on each record that you want to insert into an array like so:
// assume a record has additional property `type` that you dont want to
// insert into the database
// example: { primaryKeyCol: 123, col2: 'foo', colN: 'bar', type: 'baz' }
_(records).map(record => {
vals.push(record.primaryKeyCol);
vals.push(record.col2);
vals.push(record.colN);
});
There are less repetitive ways of doing the above explicit references, but this is just an example. Hope this helps!
The solution works great for me! I just include an ID parameter to make it dynamic across tables with custom ID tags. Chenhai, here's my snippet including a way to return a single array of ID values for the transaction:
function batchUpdate(table, id, collection) {
return knex.transaction((trx) => {
const queries = collection.map(async (tuple) => {
const [tupleId] = await knex(table)
.where(`${id}`, tuple[id])
.update(tuple)
.transacting(trx)
.returning(id);
return tupleId;
});
return Promise.all(queries).then(trx.commit).catch(trx.rollback);
});
}
You can use
response = await batchUpdate("table_name", "custom_table_id", [array of rows to update])
to get the returned array of IDs.
The update can be done in batches, i.e 1000 rows in a batch
And as long as it does it in batches, the bluebird map could be used.
For more information on bluebird map: http://bluebirdjs.com/docs/api/promise.map.html
const limit = 1000;
const totalRows = 50000;
const seq = count => Array(Math.ceil(count / limit)).keys();
map(seq(totalRows), page => updateTable(dbTable, page), { concurrency: 1 });
const updateTable = async (dbTable, page) => {
let offset = limit* page;
return knex(dbTable).pluck('id').limit(limit).offset(offset).then(ids => {
return knex(dbTable)
.whereIn('id', ids)
.update({ date: new Date() })
.then((rows) => {
console.log(`${page} - Updated rows of the table ${dbTable} from ${offset} to ${offset + batch}: `, rows);
})
.catch((err) => {
console.log({ err });
});
})
.catch((err) => {
console.log({ err });
});
};
Where pluck() is used to get ids in array form