Develop Reference

Solved Why do MySQL statements work normally in Node REPL when typed line by line, but not in a function? [duplicate]

In the code
var stuff_i_want = '';
stuff_i_want = get_info(parm);
And the function get_info:
get_info(data){
var sql = "SELECT a from b where info = data"
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
console.log(stuff_i_want); // Yep. Value assigned..
}
in the larger scope
stuff_i_want = null
What am i missing regarding returning mysql data and assigning it to a variable?
============ New code per Alex suggestion
var parent_id = '';
get_info(data, cb){
var sql = "SELECT a from b where info = data"
connection.query(sql, function(err, results){
if (err){
throw err;
}
return cb(results[0].objid); // Scope is larger than function
}
==== New Code in Use
get_data(parent_recording, function(result){
parent_id = result;
console.log("Parent ID: " + parent_id); // Data is delivered
});
However
console.log("Parent ID: " + parent_id);
In the scope outside the function parent_id is null

You're going to need to get your head around asynchronous calls and callbacks with javascript, this isn't C#, PHP, etc...
Here's an example using your code:
function get_info(data, callback){
var sql = "SELECT a from b where info = data";
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
return callback(results[0].objid);
})
}
//usage
var stuff_i_want = '';
get_info(parm, function(result){
stuff_i_want = result;
//rest of your code goes in here
});
When you call get_info this, in turn, calls connection.query, which takes a callback (that's what function(err, results) is
The scope is then passed to this callback, and so on.
Welcome to javascript callback hell...
It's easy when you get the hang of it, just takes a bit of getting used to, coming from something like C#

I guess what you really want to do here is returning a Promise object with the results. This way you can deal with the async operation of retrieving data from the DBMS: when you have the results, you make use of the Promise resolve function to somehow "return the value" / "resolve the promise".
Here's an example:
getEmployeeNames = function(){
return new Promise(function(resolve, reject){
connection.query(
"SELECT Name, Surname FROM Employee",
function(err, rows){
if(rows === undefined){
reject(new Error("Error rows is undefined"));
}else{
resolve(rows);
}
}
)}
)}
On the caller side, you use the then function to manage fulfillment, and the catch function to manage rejection.
Here's an example that makes use of the code above:
getEmployeeNames()
.then(function(results){
render(results)
})
.catch(function(err){
console.log("Promise rejection error: "+err);
})
At this point you can set up the view for your results (which are indeed returned as an array of objects):
render = function(results){ for (var i in results) console.log(results[i].Name) }
Edit
I'm adding a basic example on how to return HTML content with the results, which is a more typical scenario for Node. Just use the then function of the promise to set the HTTP response, and open your browser at http://localhost:3001
require('http').createServer( function(req, res){
if(req.method == 'GET'){
if(req.url == '/'){
res.setHeader('Content-type', 'text/html');
getEmployeeNames()
.then(function(results){
html = "<h2>"+results.length+" employees found</h2>"
html += "<ul>"
for (var i in results) html += "<li>" + results[i].Name + " " +results[i].Surname + "</li>";
html += "</ul>"
res.end(html);
})
.catch(function(err){
console.log("Promise rejection error: "+err);
res.end("<h1>ERROR</h1>")
})
}
}
}).listen(3001)

Five years later, I understand asynchronous operations much better.
Also with the new syntax of async/await in ES6 I refactored this particular piece of code:
const mysql = require('mysql2') // built-in promise functionality
const DB = process.env.DATABASE
const conn = mysql.createConnection(DB)
async function getInfo(data){
var sql = "SELECT a from b where info = data"
const results = await conn.promise().query(sql)
return results[0]
}
module.exports = {
getInfo
}
Then, where ever I need this data, I would wrap it in an async function, invoke getInfo(data) and use the results as needed.

This was a situation where I was inserting new records to a child table and needed the prent record key, based only on a name.
This was a good example of understanding the asynchronous nature of node.
I needed to wrap the all the code affecting the child records inside the call to find the parent record id.
I was approaching this from a sequential (PHP, JAVA) perspective, which was all wrong.

Easier if you send in a promise to be resolved
e.g
function get_info(data, promise){
var sql = "SELECT a from b where info = data";
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
promise.resolve(results[0].objid);
}
}
This way Node.js will stay fast because it's busy doing other things while your promise is waiting to be resolved

I've been working on this goal since few weeks, without any result, and I finally found a way to assign in a variable the result of any mysql query using await/async and promises.
You don't need to understand promises in order to use it, eh, I don't know how to use promises neither anyway
I'm doing it using a Model class for my database like this :
class DB {
constructor(db) {
this.db = db;
}
async getUsers() {
let query = "SELECT * FROM asimov_users";
return this.doQuery(query)
}
async getUserById(array) {
let query = "SELECT * FROM asimov_users WHERE id = ?";
return this.doQueryParams(query, array);
}
// CORE FUNCTIONS DON'T TOUCH
async doQuery(queryToDo) {
let pro = new Promise((resolve,reject) => {
let query = queryToDo;
this.db.query(query, function (err, result) {
if (err) throw err; // GESTION D'ERREURS
resolve(result);
});
})
return pro.then((val) => {
return val;
})
}
async doQueryParams(queryToDo, array) {
let pro = new Promise((resolve,reject) => {
let query = queryToDo;
this.db.query(query, array, function (err, result) {
if (err) throw err; // GESTION D'ERREURS
resolve(result);
});
})
return pro.then((val) => {
return val;
})
}
}
Then, you need to instantiate your class by passing in parameter to constructor the connection variable given by mysql. After this, all you need to do is calling one of your class methods with an await before. With this, you can chain queries without worrying of scopes.
Example :
connection.connect(function(err) {
if (err) throw err;
let DBModel = new DB(connection);
(async function() {
let oneUser = await DBModel.getUserById([1]);
let allUsers = await DBModel.getUsers();
res.render("index.ejs", {oneUser : oneUser, allUsers : allUsers});
})();
});
Notes :
if you need to do another query, you just have to write a new method in your class and calling it in your code with an await inside an async function, just copy/paste a method and modify it
there are two "core functions" in the class, doQuery and doQueryParams, the first one only takes a string as a parameter which basically is your mysql query. The second one is used for parameters in your query, it takes an array of values.
it's relevant to notice that the return value of your methods will always be an array of objects, it means that you'll have to do var[0] if you do a query which returns only one row. In case of multiple rows, just loop on it.

Cannot await for sqlite3.Database.get() function completion in Node.js

I'm struggling with some basic async/await problem in node.js using node-sqlite3.
My objective is to select some value from SQLite DB, check it for some condition and take some actions in case the condition is met. Here's the code:
const sqlite3 = require('sqlite3').verbose();
main();
async function main() {
let ordersDb = await createDbConnection('./ProcessedOrders.db');
var orderProcessed = await orderAlreadyProcessed(ordersDb, "555");
console.log("orderProcessed = " + orderProcessed);
if (!orderProcessed) {
console.log("So condition is met!");
}
}
async function orderAlreadyProcessed(ordersDb, orderNumberStr) {
console.log('starting orderAlreadyProcessed function'); //DEBUG
var result;
var query = 'select count(SoldOrderNumber) as "recsCount" from ProcessedSoldOrders where SoldOrderNumber = ?;';
await ordersDb.get(query
,[orderNumberStr]
,(err, row) => {
console.log('Row with count = ' + row); //DEBUG
console.log('row.recsCount = ' + row.recsCount); //DEBUG
result = typeof row !== 'undefined' && row.recsCount > 0;
});
console.log('Returning ' + result); //DEBUG
return result;
}
async function createDbConnection(dbFileName) {
let db = new sqlite3.Database(dbFileName, (err) => {
if (err) {
console.log(err.message);
}
});
return db;
}
But what I get is code executing further, not awaiting for Database.get() method at all! As a result, here's what I see printing in console:
starting orderAlreadyProcessed function
Returning undefined
orderProcessed = undefined
So IF condition met!
Row with count = [object Object]
row.recsCount = 1
As we can see, we return from orderAlreadyProcessed too early with return value = 'undefined'. So condition is met, actions taken, and only then Database.get() returns. But if it was properly awaited, condition would not be met.
How can I make it await for result value?

Since you want to use async/await, and the node-sqlite3 (sqlite3) library does not support the Promise API, you need to use the node-sqlite (sqlite) library, which is a wrapper over sqlite3 and adds support for the Promise API. Then, your code will look something like this:
const sqlite3 = require('sqlite3');
const { open } = require('sqlite');
async function main() {
try {
sqlite3.verbose();
const ordersDb = await createDbConnection('./ProcessedOrders.db');
const orderProcessed = await orderAlreadyProcessed(ordersDb, "555");
console.log("orderProcessed = " + orderProcessed);
if (!orderProcessed) {
console.log("So condition is met!");
}
} catch (error) {
console.error(error);
}
}
async function orderAlreadyProcessed(ordersDb, orderNumberStr) {
try {
console.log('Starting orderAlreadyProcessed function');
const query = 'SELECT COUNT(SoldOrderNumber) as `recsCount` from ProcessedSoldOrders where SoldOrderNumber = ?;'
const row = await ordersDb.get(query, [orderNumberStr]);
console.log('Row with count =', row);
console.log('row.recsCount =', row.recsCount);
const result = typeof row !== 'undefined' && row.recsCount > 0;
console.log('Returning ' + result);
return result;
} catch (error) {
console.error(error);
throw error;
}
}
function createDbConnection(filename) {
return open({
filename,
driver: sqlite3.Database
});
}
main();
I specifically did not remove your console.log and other parts of the code so as not to confuse the original logic of your program.

If we don't to use another library
then we can return a new Promise function & use await, as below:
Note: Below has example for INSERT/run, instead of SELECT/get, but promise/await works same
const sqlite3 = require("sqlite3").verbose();
let db;
db = new sqlite3.Database('./Chinook.db');
function insert() {
return new Promise((resolve, reject) => { // return new Promise here <---
const userId = uuid4();
let sql = `INSERT INTO Users(id) VALUES (?)`; // INSERT <----
let params = [userId];
return db.run(sql, params, function (err, res) { // .run <----
if (err) {
console.error("DB Error: Insert failed: ", err.message);
return reject(err.message);
}
return resolve("done");
});
});
}
let result = await insert(); // now await works fine <------
res.send({ result });

Async functions in for loops javascript

I am not experienced with async functions and I would like to perform a request in a for loop. Here's my code:
app.route('/friendlist').post((req, res) => {
var body = req.body;
var list = "";
con.query(`SELECT * FROM player_friends WHERE main_user_id = '${body.player_id}'`, (err, row, fields) => {
if (err) throw err;
async function queryOutUserData(data) {
var rows = await new Promise((resolve, reject) => {
con.query(`SELECT * FROM players WHERE player_id = '${data.player_id}'`, (error, player, field) => {
if (error) {
console.log(error);
return reject(error);
}
resolve(player);
});
});
rows.then(message => {
return message
});
}
for (var i = 0; i <= row.length; i++) {
console.log(row[i].main_user_id);
var result = await queryOutUserData(row[i]);
list = list + ";" + result[0].player_id + ":" + result[0].player_username;
}
console.log(list);
return list;
});
});
Actually here's the full problem: I did some debugging and apparently value i in for loop increases before the promise is resolved. Also as I mentioned I am not familiar with async functions, could you provide me a descriptive resource about how promises and async functions work?
Thanks
NOTE: For better indentation, here's the code: https://hastebin.com/fovayucodi.js

Instead of using async/await I suggest doing everything in one query using WHERE IN rather than one query per player. See if the following fits your needs:
app.route('/friendlist').post((req,res) => {
var body = req.body;
var list = "";
con.query(`SELECT * FROM player_friends WHERE main_user_id = '${body.player_id}'`, (err, row, fields) => {
if (err) throw err;
const playerIds = row.map(player => player.player_id);
con.query(`SELECT * FROM players WHERE player_id IN ${playerIds}`, (error, players, field) => {
for (let player of players) {
list += `;${player.player_id}:${player.player_username}`;
}
});
console.log(list);
return list;
});
});

If you await a promise, it evaluates to the result of that promise, so rows is not a promise, it's the result. So this:
rows.then(message => {return message});
Doesn't make much sense, just do:
return message;
Also, you have an await inside of a regular function, thats a syntax error.
Additionally return list; doesn't do much (if that is express), you might want to return res.json({ list });.
: I did some debugging and apparently value i in for loop increases before the promise is resolved.
I doubt that you can debug code if you can't actually run it because of the syntax errors.

try to use for-of instead just a for.
something like this:
Async Function:
async function test() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(true);
return
}, 1000)
})
}
Here another function using for and waiting for the finish of loop
async function myFunction() {
const data = [1,2,3,4,5,6,7,8]
for(let i of data) {
const value = await test();
console.log(value)
}
console.log("finish");
}

How to handle Asynchronous nature of function calls in Java Script

I am creating a function that will return me the result of a DB query. The problem is when I am calling the main function, the query function within the main function(getSessionCode) is not getting executed first.
I tried the Promise manager approach as well as the callbacks but none of those seems to be working.
Code:
dbConnectionObject = require('./DatabaseHandler.js');
dbConnect = dbConnectionObject.getdatabaseConnection();
var sessioncode;
function getSessionCode(emailaddress){
var queryBuild = "select <column name> from <table_name> where email="+ "\'"+ emailaddress +"\'";
var responseArray;
dbConnect.query(queryBuild,(err, res) =>{
if(err)
{
console.log(err);
}
else if(res){
responseArray=res.rows;`enter code here`
dbConnect.end();
console.log(responseArray);
return responseArray;
}
});
}
sessioncode = getSessionCode('gautam.pruthi#abc.com');
console.log(sessioncode);

Because dbConnect.query() use callback pattern to handle query result, this callback pattern block you to make query result as your function return value directly.
To resolve such problem, Promise came out to resolve callback pyramid.
1) change getSessionCode() to return a Promise
2) consume Promise eventual value with then() or await pattern
dbConnectionObject = require('./DatabaseHandler.js');
dbConnect = dbConnectionObject.getdatabaseConnection();
var sessioncode;
function getSessionCode(emailaddress) {
var queryBuild = "select <column name> from <table_name> where email=" + "\'" + emailaddress + "\'";
var responseArray;
return new Promise(function(resove, reject){
dbConnect.query(queryBuild, (err, res) => {
if (err) {
console.log(err);
reject(err);
} else if (res) {
responseArray = res.rows;
`enter code here`
dbConnect.end();
console.log(responseArray);
resove(responseArray);
}
});
});
}
sessioncode = getSessionCode('gautam.pruthi#abc.com');
sessioncode.then((session)=>{
console.log(session);
})
sessioncode = await getSessionCode('gautam.pruthi#abc.com');
console.log(sessioncode);

Push into an array from foreach and make it available outside foreach

I stuck by looping through an array that receive values from a promise and push values into a new array which is available outside the foreach.
What i have:
app.post('/submit', function (req, res) {
uploadPics(req, res, function (err) {
if (err instanceof multer.MulterError) {
res.send(JSON.stringify({UploadResult: err.message}));
console.log(err.message + ' ' +'Redirect /home');
} else if (err) {
console.log(err);
} else {
res.send(JSON.stringify({UploadResult: 'Success'}));
var filesarray = req.files;
var picinfos = [];
filesarray.forEach(function(file){
GetFileMetaInfo.filemetainfo(file.path).then(function (metadata){
//Stuck here! Can push values into an array (picinfos) but only available in the foreach. not outside..
})
})
//I need picinfos array here....
}
})
})
How i receive my metadata:
var exif = require('exif-parser');
var fs = require('fs');
exports.filemetainfo = function (filepath) {
return new Promise((resolve) => {
var file = filepath;
var buffer = fs.readFileSync(file);
var parser = exif.create(buffer);
var result = parser.parse();
resolve (result);
}).then(function (metadata){
if (metadata.tags.CreateDate !== undefined){
date = new Date (metadata.tags.CreateDate*1000);
datevalues = [
date.getFullYear(),
date.getMonth()+1,
date.getDate(),
date.getHours(),
date.getMinutes(),
date.getSeconds(),
];
CreateDate = date.getFullYear()+'-'+(date.getMonth()+1)+'-'+date.getDate();
CreateTime = date.getHours()+':'+date.getMinutes()+':'+date.getSeconds();
console.log("CrDate:" +CreateDate, "CrTime:" +CreateTime );
} else {
console.log("No Metadata Creation Infos found in " +filepath);
CreateDate = "";
CretaeTime = "";
}
if (metadata.tags.GPSLatitude !== undefined){
GPSLat = metadata.tags.GPSLatitude;
GPSLon = metadata.tags.GPSLongitude;
console.log("GPSLat:" + GPSLat , "GPSLon:" +GPSLon);
}
else {
console.log("No Metadata GPS Infos found in " +filepath)
GPSLat = "";
GPSLon = "";
}
return MetaData = {
GPSLat: GPSLat ,
GPSLon: GPSLon,
CreateDate: CreateDate,
CreateTime: CreateTime,
}
})
}
May i ask someone to give a hand. How can i make my array available outside the foreach. thank you very much!

The reason you're getting empty array at the end of forEach is because, GetFileMetaInfo.filemetainfo() returns a promise and forEach won't wait for async actions.
You could use async/await with for...of loop to get your desired result.
app.post('/submit', function (req, res) {
uploadPics(req, res, async function (err) { // note async here
if (err instanceof multer.MulterError) {
res.send(JSON.stringify({UploadResult: err.message}));
console.log(err.message + ' ' +'Redirect /home');
} else if (err) {
console.log(err);
} else {
res.send(JSON.stringify({UploadResult: 'Success'}));
var filesarray = req.files;
var picinfos = [];
for(let file of filesarray) {
const metadata = await GetFileMetaInfo.filemetainfo(file.path);
// push metadata into your array here
picinfos.push(metadata);
}
// You will have picinfos here
}
})
})

Although the question is already answered by Dinesh Pandiyan there are still some adjustments that can be made. The following code in his answer runs sequential, meaning that every async request is made after the previously returned result is resolved.
for(let file of filesarray) {
const metadata = await GetFileMetaInfo.filemetainfo(file.path);
// ^- pauses the execution of the current running code
// push metadata into your array here
picinfos.push(metadata);
}
async call #1 ╌╌await╌╌> async call #2 ╌╌await╌╌> async call #3 ╌╌await╌╌> result
You could make the code concurrent by first executing all async statements and then wait until all results are resolved. This can be done by simply changing the following:
// execute all the async functions first, reducing the wait time
for(let file of filesarray) {
const metadata = GetFileMetaInfo.filemetainfo(file.path);
// ^- remove the await
// push metadata into your array here
picinfos.push(metadata);
}
// wait for all results to be resolved
picinfos = await Promise.all(picinfos);
// ^- instead await here
async call #1 ╌╌┐
async call #2 ╌╌┼╌╌await all╌╌> result
async call #3 ╌╌┘
The above could be further simplified by simply using an Array.map() in combination with the already shown Promise.all().
var filesarray = req.files;
var picinfos = await Promise.all(filesarray.map(file => {
return GetFileMetaInfo.filemetainfo(file.path);
}));
// picinfos should be present
Or if you want to avoid working with async/await:
var filesarray = req.files;
Promise.all(filesarray.map(file => {
return GetFileMetaInfo.filemetainfo(file.path);
})).then(picinfos => {
// picinfos should be present
});