Develop Reference

How can I use either req.query, or req.params, or req.* outside its scope without saving in database

I've been trying to build a helper function that will allow me to apply DRY pattern in order to stop repeating myself. My backend and the whole application is complete, however, I'd like to optimize my code, and my actual problem is that pretty much every express http method is similarly formatted the same way. The only time I've happened to come close to the solution is when I've omitted the req.params as arguments. The problem is that each method has its own req.params format. Here's how I was trying to solve:
I tried to use node-persist package to store req.params, and it only works after I change and resave the file, which makes sense. This happened as I first passed the params as the argument and tried to pass the persisted params value when I call the function. If there's a way to have the req.params stored somewhere locally first, I wouldn't be worried.
Second Option, I tried to use recursion and called the so-called function twice. I expected the first call to return an undefined params, and the second function call to return stored req.params, but unfortunately it wasn't the case.
I then decided to try using req.redirect where I've to access the database with req.params that came from req.query. Although this works, it still brings me back to the same problem as I'll keep redirecting everything
My problem, I want to have a helper function like the following:
export const storage = require('node-persist'); //package to persist data
Few of types used:
type AllHTTPMethods = "post" | "delete" | "all" | "get" | "put" | "patch" | "options" | "head";
type HTTPMethod = core.IRouterMatcher<core.Express, AllHTTPMethods>;
export async function onFetch(httpMethod: HTTPMethod | any, sql: string, path:string, params?: string){
httpMethod(path, async(req, res) => {
await storage.init({});
/**
Check if there is something already stored
*/
if (Object.keys(req.params).length === 0) {
await storage.setItem('params', req.params)
await storage.updateItem('params', req.params)
}
conn.query(sql, [params],
(err:any, data:any) => {
if (err) {
return new Error("Something went wrong\n"+err)
}
console.log("Successfully fetched");
console.log(data)
return res.json(data);
})
})
}
Here's how I invoked them:
//This one works because params aren't involved
async all() {
await onFetch(app.get.bind(app), "select * from products", "/products")
.then(() => console.log("Successfully fetched products"))
.catch(e => console.error(e))
}
//This one didn't work Because the function needs to called first before
//persisting
getProductById = async () => {
await storage.init()
const paramsObj = await storage.getItem("params"); //returns empty object {}
await onFetch(app.post.bind(app), "select * from products where id = ?", "/product/:id", paramsObj.id)
}
And the last trick I tried was to have req.params from client upfront, then redirect them to another router
Helper function to send req.params:
export function generateParams(
httpMethod: HTTPMethod,
path: string,
params: string,
) {
httpMethod(path, (req, res) => {
const paramsObject = JSON.stringify(req.params);
return res.redirect(`/params/api/?${params}=${paramsObject}`)
})
}
Calling:
generateParams(app.post.bind(app), "/product/:id", "product")
It works but it's still the same problem I was trying to avoid beforehand
app.get('/params/api', async (req, res)=> {
var product: string | string[] | any | undefined = req.query.product;
var id:any = JSON.parse(product).id
conn.query("select * from products where id = ?", [id], (err, data)=>{
if (err) {
return
}
res.json(data)
})
});
Thanks in advance

I created a helper function to handle the queries and params based on the current req.route.path, then return an array of string containing those queries
function setRequestData(request: any) {
var arr: any[] = [];
const query = request.query
const path = request.route.path
if (path === '/product/:id') {
arr = [...arr, request.params.id]
}
else if (path === '/add/user') {
arr = [...arr,
query.firstName,
query.lastName,
query.email,
query.phoneNumber,
query.passCode,
query.age,
]
}
console.log(arr)
return arr
}
Then I used it as the following:
export function onFetch(httpMethod: HTTPMethod, sql: string | mysql.QueryOptions, path:string){
try {
httpMethod(path, (req, res) => {
var queries:any[] = setRequestData(req)
conn.query(sql, queries, (err:any, data:any) => {
if (err) {
return new Error("Something went wrong\n"+err)
}
console.log("Successful request");
res.json(data);
})
})
} catch (error) {
console.error(error)
}
}
Now, I'd be just calling one line of code to communicate with mysql database no matter which method I intend to use:
var sql = `insert into Users(firstName, lastName, email, phoneNumber, passCode, age, joined) values(?, ?, ?, ?, ?, ?, current_timestamp())`;
onFetch(app.post.bind(app), sql, '/add/user')

Sequelize join on where condition returned from first table or return object values in an array derrived from foreach coming up empty

I've been trying to figure out this for a while now so any help would be very much appreciated.
I have one table called Interaction that searches with the client user's id and returns all interactions where they are the target user. Then I want to return the names of those users who initiated the interaction through the User table.
I tried using include to join the User table but I can't get the user's names using the where clause because it is based on a value returned in the first search of the Interaction table and don't know if I can search on a value that isn't the primary key or how?
The closest I've gotten is to use foreach and add the users to an array but I can't get the array to return in my response, because outside of the loop it is empty. I've tried suggestions I've found but can't figure out how to return the array outside of the foreach, if this is the best option. I am sure it is something really stupid on my behalf. TIA.
This is my attempt at include function:
getInvited: (req, res, next) => {
var user = {}
user = req.user;
let usrId = user[0]['facebookUserId'];
var userObjArray = [];
Interaction.findAll({
where: {
targetUserId: usrId,
status: 'invited',
},
include: [{
model: User,
attributes: [
'firstName'
],
where: {
facebookUserId: IwantToJoinOnInteraction.userId // replace with working code?
}]
}).then(function (users) {
res.send(users);
}).catch(next);
}
Or my attempt at foreach:
getInvited: (req, res, next) => {
var user = {}
user = req.user;
let usrId = user[0]['facebookUserId'];
var userObjArray = [];
Interaction.findAll({
where: {
targetUserId: usrId,
status: 'invited',
}
}).then(function (interactions) {
interactions.forEach((interaction) => {
User.findOne({
where: {
facebookUserId: interaction.userId // this is the where clause I don't know how to add in my first attempt with include
},
attributes: ['firstName', 'facebookUserId']
}).then(function (user) {
userObjArray.push(user['dataValues']);
console.log(userObjArray); // on the last loop it contains everything I need
})
})
res.status(200).send(userObjArray); // empty
}).catch(next);
},

You have to wait for all promises before sending the response. Your code runs async. With the forEach you are calling User.findOne async but you don't wait for all User.findOne to finish. A convenient way to make this work is Promise.all. You can pass an array of promises and the returned promise resolves to an array of all the resolved promises.
Promise.all(interactions.map(interaction => User.findOne(...)))
.then(users => {
res.status(200).send(users.map(user => user.dataValues))
})
You could write this much more easy to read woth async/await
getInvited: async (req, res, next) => {
...
const interactions = await Interaction.findAll(...)
const users = await Promise.all(interactions.map(interaction => User.findOne(...)))
res.status(200).send(users.map(user => user.dataValues))
}

Async await in Express middleware not working

I'm a bit of newbie to Node so be gentle. I'm creating an app for my wedding which takes an uploaded guestlist (in Excel file format) and turns it into a JSON array which I can then use to build profiles about each guest (dietary requirements, rsvp answer, etc).
So far I've got a form on the homepage which allows the user to upload the .xlsx file and when the form is submitted the user is redirected back to the homepage again.
I've created the following route:
router.post('/',
guestsController.upload,
guestsController.getGuestlist,
guestsController.displayGuestlist
);
and here's my guestsController:
const multer = require('multer');
const convertExcel = require('excel-as-json').processFile;
const storage = multer.diskStorage({ //multers disk storage settings
destination: function (req, file, cb) {
cb(null, './uploads/')
},
filename: function (req, file, cb) {
var datetimestamp = Date.now();
cb(null, file.fieldname + '-' + datetimestamp + '.' + file.originalname.split('.')[file.originalname.split('.').length -1])
}
});
exports.upload = multer({storage: storage}).single('file');
exports.getGuestlist = async (req, res, next) => {
try {
await convertExcel(req.file.path, null, null, (err, guestData) => {
req.guestlist = guestData.map((guestObj) => Object.values(guestObj)[0]);
});
console.log(req.guestlist);
next();
} catch (e){
res.json({error_code:1,err_desc:"Corrupted Excel file"});
next();
}
};
exports.displayGuestlist = (req, res) => {
console.log(req.guestlist);
};
At the moment because of the synchronous nature of the functions, displayGuestlist is returning undefined to the console because covertExcel has not finished grabbing the data. You can see I have tried to use the new async await syntax to resolve this but it hasn't fixed it unfortunately.
I have also tried putting the log in displayGuestlist in a timeout function which has proven that this is purely a timing issue.
Any helpers would be much appreciated.

It looks like convertExcel is not a Promise-returning function, but rather uses an old-school callback. await does not work with those, so it's instead awaiting Promise.resolve(undefined) since the function returns undefined, not a Promise. Thankfully, in Node 8 and later, there's a promisify utility to convert callback-style functions to Promise-returning functions so that await can be used.
const { promisify } = require('util');
const convertExcel = promisify(require('excel-as-json').processFile);
// ...
const guestData = await convertExcel(req.file.path, null, null);
req.guestlist = guestData.map((guestObj) => Object.values(guestObj)[0]);

You can encapsulate your code in a promise and await this promise to resolve.
exports.getGuestlist = async (req, res, next) => {
let promise = new Promise((resolve, reject) => {
convertExcel(req.file.path, null, null, (err, guestData) => {
if(err) reject(err);
else resolve(guestData);
});
});
try {
let guestData = await promise;
req.guestlist = guestData.map((guestObj) => Object.values(guestObj)[0]);
console.log(req.guestlist);
next();
} catch (e){
res.json({error_code:1,err_desc:"Corrupted Excel file"});
next();
}
};

Module exports async function undefined can't await

So, as far as I google it, I understood that this problem relevant to async / promise coding. I waste over 2Hr plus with this, but still receive no result. Probably because I'm bad in it. So my identifier.js code is right and works fine, it returns the exact data I want. app.js is still good too. So where is the problem? I can't export result value from identifier.js, because if I do it, I receive 'undefined':
const identifier = require("./db/ops/identifier");
trade_log.create({
Flag: req.body.Flag,
Instrument: req.body.Instrument,
Venue: req.body.Venue,
Price: req.body.Price,
Currency: req.body.Currency,
Quantity: req.body.Quantity,
Counterparty: req.body.Counterparty,
Identifier: identifier(req.body.Counterparty),
Commentary: req.body.Commentary,
but if I export it correcty (according to other guides), just like
let x = identifier(req.body.Counterparty).then(return value);
I receive an error at writing trade_log.create phase.
What is identifier.js? This module is a function that should request data from input form (req.body) via get and receive responce, return data & then in app.js it should be written in to MongoDB (writing works fine, already tested it)
app.js
const trade_log = require("./db/models/trade_log");
const identifier = require("./db/ops/identifier");
app.all('/log', function (req, res, next) {
let x = identifier(req.body.Counterparty.then();
trade_log.create({
Flag: req.body.Flag,
Instrument: req.body.Instrument,
Venue: req.body.Venue,
Price: req.body.Price,
Currency: req.body.Currency,
Quantity: req.body.Quantity,
Counterparty: req.body.Counterparty,
Identifier: x,
Commentary: req.body.Commentary,
},function (err, res) {
if (err) return console.log (req.body) + handleError(err);
console.log(res);
});
next();
});
identifier.js:
const identifier = (name) => {
request(['options, name'], { 'whatever' })
.then(response => {
/code that works fine
let charset = result;
return (charset);
module.exports = identifier;
I already tried in Identifier.js this export methods:
function name(param) {
//code here
}
module.exports = name();
and this:
module.exports = {
function: (name) => {
//code here
}
I also find relevant this SW answer: Asynchronous nodejs module exports but I still can't understand what am I doing wrong.
How should I correct my code accrding to new ES6 standard or should I use another module.exports method?

Testing express routes with mongoose query

I am trying to test my routes that have mongoose queries in. I keep getting back:
AssertionError: expected undefined to equal true
Below is the basic template of my test. At the moment I just want to confirm it calles the res.json.
The route returns all entries in the User model
route.js
const User = require('../../../models/users/users');
const listUsers = (req, res) => {
User.find((err, users) => {
if (err) res.send(err);
res.json(users);
})
};
module.exports = listUsers;
test.js
const expect = require('chai').expect;
const sinon = require('sinon');
const listUsers = require('../../../../src/routes/api/users/listUsers');
describe('listUsers', () => {
it('retrieves users', () => {
const req = {};
const res = {};
const spy = res.json = sinon.spy;
listUsers(req, res);
expect(spy.calledOnce).to.equal(true);
})
});

The find function takes two parameters. The first one is the criteria for the search and second is the callback function. Looks like you have missed the first parameter.
Since you want to find all users the criteria for you is - {}
So this will solve your problem -
User.find({}, (err, users) => {
if (err) res.send(err);
res.json(users);
})