Develop Reference

using async with mysql

So i got the following code. I want to check if Name already exists in my database. If not I want to insert it. And it basically works with my code. My only problem is, that the last bit of code where it checks whether "success" is true gets executed before it even checked the database. (when using a Name that doesn't exist in the database my console is like this:
Name is already taken <--- this piece of code should be executed last. Then that would not appear
No Duplicate, insert it
starting insert function
I've heard of async and await but I just could not figure it out. I could really need some help.
Here is my Code:
app.post("/api/createProject", (req, res) => {
var success = false;
// console.log(req.body);
const Name = req.body.Name;
const Status = req.body.Status;
const Deadline = req.body.Deadline;
const EstimatedHours = req.body.EstimatedHours;
const BudgetHours = req.body.BudgetHours;
const BudgetCost = req.body.BudgetCost;
const Description = req.body.Description;
// check if Projct Name is already taken
const SEARCH_NAME = `SELECT ID FROM projects WHERE Name = '${Name}'`;
db.query(SEARCH_NAME, (err, results) => {
if (err) {
return err;
} else {
if ((results.length < 1)) {
console.log("No Duplicate, insert it")
insertDatabase();
}
}
});
// will only get executed when no Errors got detected above (Name is not already taken)
function insertDatabase() {
console.log("starting insert function");
const CREATE_NEW_PROJECT = "INSERT INTO projects (Name, Status, Deadline, EstimatedHours, BudgetHours, BudgetCost, Description) VALUES (?,?,?,?,?,?,?)";
db.query(
CREATE_NEW_PROJECT,
[
Name,
Status,
Deadline,
EstimatedHours,
BudgetHours,
BudgetCost,
Description,
],
(err, result) => {
if (!err) {
success = true;
}
}
);
}
if (success == true){
return 0;
} else {
console.log("Name is already taken");
return "Name is already taken";
}
});

Your issue is to do with the asynchronous operation of NodeJS. What is happening in your code is all of your functions are getting "fired" because the code will execute sequentially, but it does not wait for the functions to return.
This is why your if / else statement is getting executed as the "else" - the query hasn't returned to set the "success" to true and therefore it hits the else statement because of its default value.
So we need to refactor your code a bit to make it suitable for async-await. The async-await will wait for a function to return before carrying on to the next statement. With the async-await command, the function it is calling must return a promise.
First we abstract your query into a function with a promise. A promise will run your query and only return when your query is complete:
async function Query (request) {
return new Promise ((resolve, reject) => {
const SEARCH_NAME = `SELECT ID FROM projects WHERE Name = '${request.Name}'`;
db.query(SEARCH_NAME, (err, results) => {
if (err) {
reject(err);
} else {
resolve(results)
}
});
})
}
The above function takes request, which would be your req.body as a parameter, carries out the query and returns the results.
We also need to refactor your insertDatabase function into async-await to see if our insert was successful or not (as this seems to be what you want to know):
async function insertDatabase(request) {
return new Promise((resolve, reject) => {
console.log("starting insert function");
const CREATE_NEW_PROJECT = "INSERT INTO projects (Name, Status, Deadline, EstimatedHours,
BudgetHours, BudgetCost, Description) VALUES (?,?,?,?,?,?,?)";
db.query(
CREATE_NEW_PROJECT,
[
request.Name,
request.Status,
request.Deadline,
request.EstimatedHours,
request.BudgetHours,
request.BudgetCost,
request.Description,
],
(err) => {
if (err) {
reject(false)
}
else {
resolve(true)
}
});
});
}
We then check the results for length less than 1:
var result = await Query(req.body);
if (result <1) {
console.log("No Duplicate, insert it")
success = await insertDatabase(req.body);
}
Putting it all together our code will look something like:
app.post("/api/createProject", async (req, res) => {
var success = false;
var result = await Query(req.body);
if (result <1) {
console.log("No Duplicate, insert it")
success = await insertDatabase(req.body);
if (success == true){
return 0;
} else {
console.log("Name is already taken");
return "Name is already taken";
}
}
}
async function Query (request) {
return new Promise ((resolve, reject) => {
const SEARCH_NAME = `SELECT ID FROM projects WHERE Name = '${request.Name}'`;
db.query(SEARCH_NAME, (err, results) => {
if (err) {
reject(err);
} else {
resolve(results)
}
});
})
}
async function insertDatabase(request) {
return new Promise((resolve, reject) => {
console.log("starting insert function");
const CREATE_NEW_PROJECT = "INSERT INTO projects (Name, Status, Deadline, EstimatedHours,
BudgetHours, BudgetCost, Description) VALUES (?,?,?,?,?,?,?)";
db.query(
CREATE_NEW_PROJECT,
[
request.Name,
request.Status,
request.Deadline,
request.EstimatedHours,
request.BudgetHours,
request.BudgetCost,
request.Description,
],
(err) => {
if (err) {
reject(false)
}
else {
resolve(true)
}
});
});
}
Notice our post callback has "async" in front of it. This is a required keyword in any function that contains the await command.
Additional:
I've left your code mostly how it was with a bit of refactoring, some extra code best practice notes for you:
Multiple return paths is a very bad idea as this leads to testing issues as well as later debugging nightmares, so I would suggest you refactor the return from the final if statement - maybe assign the variables in the if statement but return the variable at the end.
You shouldn't return different types from the same function. So either return 1 or 0 (true or false) on your result, or return two different sets of text. Do not return 0 for one case and text for another, this is very bad practice.
I've also short-cutted by passing your req.body into the functions which probably passes more data than needed. If you don't want to pass this, as a minimum you should encapsulate your assignments that go together into a JSON object:
var request = {
"Name" : req.body.Name,
"Status" : req.body.Status,
"Deadline" : req.body.Deadline,
"EstimatedHours" : req.body.EstimatedHours,
"BudgetHours" : req.body.BudgetHours,
"BudgetCost" : req.body.BudgetCost,
"Description" : req.body.Description
}
and then pass this variable request into the functions instead of req.body.

i'm also relatively new to this, but have an idea.
Try this:
make your function async
async function insertDatabase()
then await your query
await db.query()

Promises resolving to undefined with node.js mysql queries

I'm writing a route that takes in user-input data from a form and uses that data to execute a few separate queries on my database. That is all well-tested, and I know I can get the user input, query the database, and return the results. The part giving me trouble, however, is that I want to randomly select a few results out of all of the returned data, so I am pushing the results into one big array. But since the queries execute asynchronously, the only way to access the data is in the callback and of course if I tried to push the rows in the callback, I still can't access that array outside of the callback functions because that code will execute before the queries are even run. So I'm attempting to use JavaScript async/await syntax to do the queries, as follows:
module.exports = {
search: (req, res) => {
function saveResults(resultsObj) {
Object.keys(resultsObj).forEach(function (key) {
var row = resultsObj[key];
var song = row.song_name
resultsArray.push(song);
return resultsArray;
});
}
async function queryForSong(){
var songGenre;
getSongGenre = "The queries aren't important, I know they work as expected";
await mysqlConnection.query(getSongGenre, (err, rows) => {
if (err) throw err;
songGenre = row[0].song_genre;
songSearchQuery = "SELECT query";
mysqlConnection.query(songSearchQuery, (err, rows) => {
if (err) throw err;
return saveResults(rows);
});
});
}
async function queryForArtist() {
var artistID;
getArtistID = "SELECT query";
await mysqlConnection.query(getArtistID, (err, rows) => {
if (err) throw err;
artistID = rows[0].artist_id;
console.log(rows, artistID);
artistSearchQuery = "SELECT query";
mysqlConnection.query(artistSearchQuery, (err, rows) => {
if (err) throw err;
return saveResults(rows);
});
});
}
async function queryForGenre() {
genreSearchQuery = "SELECT query";
await mysqlConnection.query(genreSearchQuery, (err, rows) => {
if (err) throw err;
return saveResults(rows);
});
}
async function queryForDecade() {
decadeSearchQuery = "SELECT query";
await mysqlConnection.query(decadeSearchQuery, (err, rows) => {
if (err) throw err;
return saveResults(rows);
});
}
var resultsArray = [];
var artist = req.body.searchArtist;
var song = req.body.searchSong;
var genre = req.body.searchGenre;
var decade = req.body.searchDecade;
if (song) {
queryForSong().then((value) => console.log(value)); //these all log undefined
}
if (artist) {
queryForArtist().then((value) => console.log(value));
}
if (genre) {
queryForGenre().then((value) => console.log(value));
}
if (decade) {
queryForDecade().then((value) => console.log(value));
}
}
}
Then the idea is to save the resolved rows from the promises into one big array and randomize it from there. My only thought of why this isn't working is that it could be because the return statements are located inside the callback functions? I have also tried it using promises directly as this post explains, by writing all of my async functions like this instead:
function queryForGenre() {
genreSearchQuery = "SELECT query";
return new Promise(function (resolve, reject) {
mysqlConnection.query(genreSearchQuery, (err, rows) => {
if (err) reject(err);
resolve(saveResults(rows));
});
});
}
But either way, the console.log statements in the .then still print undefined, meaning my promises aren't resolving to the saveResults(rows) value that I give it. Do I need to make the saveResults function asynchronous as well? From my understanding of promises, this seems like it should work. What am I missing? Or is there a simpler way to do this?

Your functions queryForSong, queryForDecade, etc are using await, but the mysql driver query function accepts a callback and does not return a promise. Also, the return inside your callback is scoped within the callback, not scoped with respect to your outside async function. Instead wrap the function within a promise, resolve and reject where necessary. Also avoid using await unless absolutely necessary because using it blocks the event loop. I have given an example for how to wrap with promise for queryForSong
function queryForSong(){
return new Promise((resolve, reject) => {
let songGenre;
const getSongGenre = "The queries aren't important, I know they work as expected";
mysqlConnection.query(getSongGenre, (err, rows) => {
if (err) reject(err);
else {
songGenre = row[0].song_genre;
const songSearchQuery = "SELECT query";
mysqlConnection.query(songSearchQuery, (err, rows) => {
if (err) reject(err);
else
resolve(saveResults(rows));
});
}
});
});
}

So it turns out it was the saveResults function causing me issues. I don't totally understand why this would fix it, but instead of resolving to the value of the saveResults function, I instead tried just including the saveResults body in each function that returns a promise and that works. For example,
function queryForSong() {
var songResultsArray = [];
var songGenre;
getSongGenre = "SELECT query";
return new Promise(function (resolve, reject) {
mysqlConnection.query(getSongGenre, (err, rows) => {
if (err) reject(err);
var row = rows[0];
songGenre = row.song_genre;
songSearchQuery = "SELECT query #2";
mysqlConnection.query(songSearchQuery, (err, rows) => {
if (err) throw err;
//this is the body of the saveResults function
Object.keys(rows).forEach(function (key) {
var row = rows[key];
var song = row.song_name;
songResultsArray.push(song);
});
resolve(songResultsArray);
});
});
});
}

Function returning empty array, how to solve it? Javascript, mongoose

I have a javascript, mongoose function which should return an array. So basically what I have is a user schema, and a shopping list schema. User schema has a parameter which is authorizedLists, an array of shopping lists ids. What I want is, that by passing the user id as a parameter, I can show those lists the user is authorized in. This function right now returns an empty array, and what I expected was an array with the shopping list objects.
I know it is not working due to Javascript asynchronous way of reading code, but I´m really stucked here.
Also I´m relatively new to nodejs and mongoose so I may have made other mistakes but haven´t realized yet.
Code:
function getUserAuthLists(req,res){
const { userId } = req.params;
var list = [];
let i = 0;
User.findById( userId,async function(err, user){
if(err) return res.status(404).send(err);
console.log(user);
const array = user.authorizedLists;
console.log(array);
array.forEach( function(element){
console.log(element);
ShoppingList.findById(element , (error, result)=>{
if(error) {
console.log(error);
return res.status(404).send(error);
}
list.push(result);
console.log(list);
});
i++;
});
if(i === array.length) {
console.log(list);
return res.status(202).send(list);
}
});
}
Thanks in advance.

Refactor your function to use promises, collect all authorizedLists and pass it in one-go using the $in operator instead of repeatedly doing findById
async function getUserAuthLists(req, res){
try {
const { userId } = req.params;
let i = 0;
const user = await User.findById(userId).exec();
const list = await ShoppingList.find({
_id: {
$in: user.authorizedLists
}
}).exec();
return res.status(202).send(list);
} catch (err) {
return res.status(404).send(err);
}
}

Since ShoppingList.findById is an i/o call, control does not wait by default to finish this call and directly jump to list.push(result); with list empty values.
You need to use promise to make it wait until it gets results from i/o call and then you should send back response. That way list will be updated for each array item call.
Something like this-
async function getUserAuthLists(req, res) {
User.findById(userId, async function (err, user) {
if (err) return res.status(404).send(err);
console.log(user);
const array = user.authorizedLists;
console.log(array);
await ShoppingListFromDB(array)
.then((list) => {
console.log("list--", list);
return res.status(202).send(list);
})
.catch((error) => {
console.log(error);
return res.status(404).send(err);
})
});
}
function ShoppingListFromDB(array) {
return new Promise((resolve, reject) => {
let i = 0
var list = [];
for (const element of array) {
console.log("element ", element);
ShoppingList.findById(element, (error, result) => {
if (err) {
console.log(err)
return reject(error);
}
i++
list.push(result)
if (i == array.length) {
return resolve(list)
}
})
}
})
}
I have created a sample that simulated your requirement.
https://github.com/ajaysikdar/test_server
server
client

Sqlite .all() function returns an undefined promise. How can I use the result further?

I am just starting to work with JS and SQLite. And could not find any help for my specific question.
I want to keep a return to use it as a foreign key in a second table.
This is my function:
async function getIdbyName(table, row, name) {
let nameNeu = '"' + name + '"';
let sql =
"SELECT id as print FROM " +
table +
" WHERE " +
row +
" = " +
nameNeu +
" LIMIT 1;";
// await db.get(sql, (err, row) => {
// console.log(row.print);
// return row;
// });
return await db.get(sql);
}
getIdbyName("...", "...", "...")
.then(function (value) {
console.log("Success!", value); // <-- prints: Success! undefined
})
.catch(function (err) {
console.log("Caught an error!", err);
});
console.log(getIdbyName("r_Tag", "r_Tag", "test")); //<-- shows me a Promise
What do I have to do so that promise does not stay undefined outside of the function?
Rest of the code:
var sqlite3 = require("sqlite3").verbose();
let db = new sqlite3.Database("./assets/db/test.db", (err) => {
if (err) {
return console.error(err.message);
}
console.log("Connected to the SQlite database.");
});
My other function just creat some strings and I run a few times db.run(...) to add some tables.

To put it more plainly, your getIdByName function never returns anything. You need to return the value you get back from await db.get(...). Once you do that, when you call getIdByName, you should get your response from the database.
You should also know that your code is susceptible to SQL injection, which is a major security vulnerability. Instead of concatenating a string, you should use a prepared statement.
async function getIdbyName(table, row, name) {
return await db.get(sql);
}
Update: Promise Wrapper for SQLlite - Aug 1, 2020
Based on this blog post, it seems it's not possible to do native async/await using sqlite3. However, you can write a wrapper function around db.all to return a promise, which will allow you to use async/await. Note the use of ? in the SQL statement, which will be replaced by the values of the array in the second argument following the same order. For more help with parameterized queries, read the params bullet point in the documentation here.
const sqlite3 = require("sqlite3").verbose();
const db = new sqlite3.Database("./assets/db/test.db", (err) => {
if (err) {
return console.error(err.message);
}
console.log("Connected to the SQlite database.");
});
db.query = function (sql, params = []) {
const that = this;
return new Promise(function (resolve, reject) {
that.all(sql, params, function (error, result) {
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
};
async function getIdByName(table, name) {
// assemble sql statement
const sql = `
SELECT id
FROM ?
WHERE name = ?;
`;
return await db.query(sql, [table, name]);
}
// need async to call
(async () => {
const result = await getIdByName('books', 'my_name');
console.log(result);
})();

From where should I call module.exports to get a not null value?

Where should I call module.export, I assume, it's supposed to be a callback function.
But I'm confused as to where am I supposed to call the callback function.
I'm still confused with the solution, too complicated for me.
sql.connect(config, function(err) {
if (err)
console.log(err);
// create Request object
var request = new sql.Request();
// query to the database and get the records
request.query('select part_num,qty from CRM.CRM.Fishbowl_Inventory where not location = \'Shipping\'',
function(err, recordset) {
if (err)
console.log(err)
// send records as a response
var details = recordset;
});
});
module.exports = details;
Confusion:
Extremely sorry to bother you guys but I want to be sure that I'm doing no harm to our database by involving any database request through Javascript.
I'm testing directly with our production database, hence cautious
So as Max provided in his answer the following code
const connectToSql = require('./connectToSql');
connectToSql()
.then(details => {
console.log(details);
//Here I can do as much logic as I want
//And it won't affect my database or call multiple requests on my DB
})
.catch(err => {
console.log(err);
});
I can understand I'm asking super silly questions, very sorry about that.

You can't export the result of your function. You want to export a function that will return your value. Like this:
function connectToSql(config) {
return new Promise((resolve, reject) => {
sql.connect(config, function (err) {
if (err) {
console.log(err);
reject(err);
}
// create Request object
var request = new sql.Request();
// query to the database and get the records
request.query('select part_num,qty from CRM.CRM.Fishbowl_Inventory where not location = \'Shipping\'',
function (requestErr, recordset) {
if (err) {
console.log(requestErr);
reject(requestErr);
}
resolve(recordset);
});
});
});
}
module.exports = connectToSql;
Because your function is async, I returned a promise that will return your result. Also, your second error from your query is named the same as your first error from the connection. That would cause problems.
Example of how to use this:
const connectToSql = require('./connectToSql');
connectToSql()
.then(details => {
console.log(details);
})
.catch(err => {
console.log(err);
});