Develop Reference

Solved Why do MySQL statements work normally in Node REPL when typed line by line, but not in a function? [duplicate]

In the code
var stuff_i_want = '';
stuff_i_want = get_info(parm);
And the function get_info:
get_info(data){
var sql = "SELECT a from b where info = data"
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
console.log(stuff_i_want); // Yep. Value assigned..
}
in the larger scope
stuff_i_want = null
What am i missing regarding returning mysql data and assigning it to a variable?
============ New code per Alex suggestion
var parent_id = '';
get_info(data, cb){
var sql = "SELECT a from b where info = data"
connection.query(sql, function(err, results){
if (err){
throw err;
}
return cb(results[0].objid); // Scope is larger than function
}
==== New Code in Use
get_data(parent_recording, function(result){
parent_id = result;
console.log("Parent ID: " + parent_id); // Data is delivered
});
However
console.log("Parent ID: " + parent_id);
In the scope outside the function parent_id is null

You're going to need to get your head around asynchronous calls and callbacks with javascript, this isn't C#, PHP, etc...
Here's an example using your code:
function get_info(data, callback){
var sql = "SELECT a from b where info = data";
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
return callback(results[0].objid);
})
}
//usage
var stuff_i_want = '';
get_info(parm, function(result){
stuff_i_want = result;
//rest of your code goes in here
});
When you call get_info this, in turn, calls connection.query, which takes a callback (that's what function(err, results) is
The scope is then passed to this callback, and so on.
Welcome to javascript callback hell...
It's easy when you get the hang of it, just takes a bit of getting used to, coming from something like C#

I guess what you really want to do here is returning a Promise object with the results. This way you can deal with the async operation of retrieving data from the DBMS: when you have the results, you make use of the Promise resolve function to somehow "return the value" / "resolve the promise".
Here's an example:
getEmployeeNames = function(){
return new Promise(function(resolve, reject){
connection.query(
"SELECT Name, Surname FROM Employee",
function(err, rows){
if(rows === undefined){
reject(new Error("Error rows is undefined"));
}else{
resolve(rows);
}
}
)}
)}
On the caller side, you use the then function to manage fulfillment, and the catch function to manage rejection.
Here's an example that makes use of the code above:
getEmployeeNames()
.then(function(results){
render(results)
})
.catch(function(err){
console.log("Promise rejection error: "+err);
})
At this point you can set up the view for your results (which are indeed returned as an array of objects):
render = function(results){ for (var i in results) console.log(results[i].Name) }
Edit
I'm adding a basic example on how to return HTML content with the results, which is a more typical scenario for Node. Just use the then function of the promise to set the HTTP response, and open your browser at http://localhost:3001
require('http').createServer( function(req, res){
if(req.method == 'GET'){
if(req.url == '/'){
res.setHeader('Content-type', 'text/html');
getEmployeeNames()
.then(function(results){
html = "<h2>"+results.length+" employees found</h2>"
html += "<ul>"
for (var i in results) html += "<li>" + results[i].Name + " " +results[i].Surname + "</li>";
html += "</ul>"
res.end(html);
})
.catch(function(err){
console.log("Promise rejection error: "+err);
res.end("<h1>ERROR</h1>")
})
}
}
}).listen(3001)

Five years later, I understand asynchronous operations much better.
Also with the new syntax of async/await in ES6 I refactored this particular piece of code:
const mysql = require('mysql2') // built-in promise functionality
const DB = process.env.DATABASE
const conn = mysql.createConnection(DB)
async function getInfo(data){
var sql = "SELECT a from b where info = data"
const results = await conn.promise().query(sql)
return results[0]
}
module.exports = {
getInfo
}
Then, where ever I need this data, I would wrap it in an async function, invoke getInfo(data) and use the results as needed.

This was a situation where I was inserting new records to a child table and needed the prent record key, based only on a name.
This was a good example of understanding the asynchronous nature of node.
I needed to wrap the all the code affecting the child records inside the call to find the parent record id.
I was approaching this from a sequential (PHP, JAVA) perspective, which was all wrong.

Easier if you send in a promise to be resolved
e.g
function get_info(data, promise){
var sql = "SELECT a from b where info = data";
connection.query(sql, function(err, results){
if (err){
throw err;
}
console.log(results[0].objid); // good
stuff_i_want = results[0].objid; // Scope is larger than function
promise.resolve(results[0].objid);
}
}
This way Node.js will stay fast because it's busy doing other things while your promise is waiting to be resolved

I've been working on this goal since few weeks, without any result, and I finally found a way to assign in a variable the result of any mysql query using await/async and promises.
You don't need to understand promises in order to use it, eh, I don't know how to use promises neither anyway
I'm doing it using a Model class for my database like this :
class DB {
constructor(db) {
this.db = db;
}
async getUsers() {
let query = "SELECT * FROM asimov_users";
return this.doQuery(query)
}
async getUserById(array) {
let query = "SELECT * FROM asimov_users WHERE id = ?";
return this.doQueryParams(query, array);
}
// CORE FUNCTIONS DON'T TOUCH
async doQuery(queryToDo) {
let pro = new Promise((resolve,reject) => {
let query = queryToDo;
this.db.query(query, function (err, result) {
if (err) throw err; // GESTION D'ERREURS
resolve(result);
});
})
return pro.then((val) => {
return val;
})
}
async doQueryParams(queryToDo, array) {
let pro = new Promise((resolve,reject) => {
let query = queryToDo;
this.db.query(query, array, function (err, result) {
if (err) throw err; // GESTION D'ERREURS
resolve(result);
});
})
return pro.then((val) => {
return val;
})
}
}
Then, you need to instantiate your class by passing in parameter to constructor the connection variable given by mysql. After this, all you need to do is calling one of your class methods with an await before. With this, you can chain queries without worrying of scopes.
Example :
connection.connect(function(err) {
if (err) throw err;
let DBModel = new DB(connection);
(async function() {
let oneUser = await DBModel.getUserById([1]);
let allUsers = await DBModel.getUsers();
res.render("index.ejs", {oneUser : oneUser, allUsers : allUsers});
})();
});
Notes :
if you need to do another query, you just have to write a new method in your class and calling it in your code with an await inside an async function, just copy/paste a method and modify it
there are two "core functions" in the class, doQuery and doQueryParams, the first one only takes a string as a parameter which basically is your mysql query. The second one is used for parameters in your query, it takes an array of values.
it's relevant to notice that the return value of your methods will always be an array of objects, it means that you'll have to do var[0] if you do a query which returns only one row. In case of multiple rows, just loop on it.

Waiting for a forEach to finish before return from my promise / function

I am using Firebase Cloud Firestore, however, I think this may be more of a JavaScript asynchronous vs synchronous promise return issue.
I am doing a query to get IDs from one collection, then I am looping over the results of that query to lookup individual records from another collection based on that ID.
Then I want to store each found record into an array and then return the entire array.
results.length is always 0 because return results fires before the forEach completes. If I print results.length from inside the forEach it has data.
How can I wait until the forEach is done before returning from the outer promise and the outer function itself?
getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
var results = [];
return dbPromise.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
docRef.get().then(function(doc) {
if (doc.exists) {
results.push(doc);
}
})
});
console.log(results.length);
return results;
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}

The trick here is to populate results with promises rather than the result. You can then call Promise.all() on that array of promises and get the results you want. Of course, you can't check if doc.exists before pushing the promise so you will need to deal with that once Promise.all() resolves. For example:
function getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
// return the main promise
return dbPromise.then(function(querySnapshot) {
var results = [];
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
// push promise from get into results
results.push(docRef.get())
});
// dbPromise.then() resolves to a single promise that resolves
// once all results have resolved
return Promise.all(results)
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
getFacultyFavoritesFirebase
.then(results => {
// use results array here and check for .exists
}

If you have multiple items of work to perform at the same time that come from a loop, you can collect all the promises from all the items of work, and wait for them all to finish with Promise.all(). The general form of a possible solution looks like this:
const promises = [] // collect all promises here
items.forEach(item => {
const promise = item.doWork()
promises.push(promise)
})
Promise.all(promises).then(results => {
// continue processing here
// results[0] is the result of the first promise in the promises array
})
You can adapt this to something that suits your own specific form.

Use for of instead of forEach. Like this:
for (const item of array) {
//do something
}
console.log("finished");
"finished" will be logged after finishing the loop.

Well I know, the thread is old, but the problem is still the same. And because I did run into the same issue and non of the answers did work for me, I want share my solution.
I think it will help someone out there. And maybe it will help me, if I run into the same problem again. ;-)
So the solution is super easy. firebase implements "map" but not direct on snaposhot, but on snapshot.docs.map.
In combination with Promieses.all it works just fine.
const promise = snapshot.docs.map(async (tenant) => {
return CheckTenant(tenant.id).catch(error =>
reject(error),
);
});
Promise.all(promise).then(result => {
// do somothing with the result});

Pass variables between Promises and functions

I have a problem.
I have to do two different SOAP calls to retrieve two list of vouchers and then use these lists to do a check on them and to do some job.
I put the two calls in different Promise functions because I want start the job on the lists after the call returned its result.
This is the first Promise call:
let vouchers = function(voucherTypeList){
return new Promise(function(resolve,reject){
const categoryId = "1000";
let args = {
"tns:CategoryId": categoryId
};
var header = {
"tns:Authenticate": {
"tns:UserName": soapVoucherWsdlUsername,
"tns:Password": soapVoucherWsdlPassword
}
};
// let voucherTypeList;
voucherClient.addSoapHeader(header);
voucherClient.GetVouchers(args, function(err, result) {
console.log("DENTRO GET VOUCHERS");
if (err) {
console.log(err);
writeResponse(res, '200', err);
} else {
//++++++++++++++++++++++
//voucherTypeList is what I want to return to the main function
voucherTypeList = mapGetVoucherTypeListResponse(result);
//++++++++++++++++++++++
}
resolve("done 1");
});
});
}
This is the second Promise call:
let issuedVouchers = function(accountId) {
return new Promise(function (resolve, reject) {
const categoryId = "1000";
let args = {
"tns:CategoryId": categoryId,
"tns:CheckRedeem": true,
"tns:IncludeRedeemed": false,
"tns:CardId": accountId
};
var header = {
"tns:Authenticate": {
"tns:UserName": soapVoucherWsdlUsername,
"tns:Password": soapVoucherWsdlPassword
}
};
let issuedVoucherList;
voucherClient.addSoapHeader(header);
voucherClient.GetVouchers(args, function (err, result) {
console.log("DENTRO GET ISSUED VOUCHERS");
if (err) {
console.log(err);
writeResponse(res, '200', err);
} else {
//++++++++++++++++++++++
//issuedTypeList is what I want to return to the main function
issuedTypeList = mapGetVoucherTypeListResponse(result);
//++++++++++++++++++++++
}
resolve("done 2");
});
});
}
And this is the main function, with the Promise flow:
function getAvailableVoucherTypes(req, res) {
var accountId = req.params.accountId;
vouchers(voucherTypeList).
then(issuedVouchers(accountId)).
then(function() {
//here I want to use voucherTypeList and issuedTypeList
//and do some jobs on them
console.log("OK");
});
}
How can I do this? I tried many solutions, but I'm not able to see voucherTypeList and issuedTypeList in the main function.

The then callbacks are getting the value of what you pass to the resolve function in your promises. You are currently passing arbitrary strings, which is useless... But for the demonstration, let's keep those and just log their values in your main script:
function getAvailableVoucherTypes(req, res) {
var accountId = req.params.accountId;
vouchers(voucherTypeList).
then(function(result){
console.log(result); //done 1
return issuedVouchers(accountId);
}).
then(function(result) {
console.log(result); //done 2
//here I want to use voucherTypeList and issuedTypeList
//and do some jobs on them
console.log("OK");
});
}
I'll let you play with your promises to pass the right variables...
Now, it seems that your 2 calls do not need to be sequential, so let's make them parallel, it's gonna be slightly easier for us too.
function getAvailableVoucherTypes(req, res) {
var accountId = req.params.accountId;
var promises = [vouchers(),issuedVouchers(accountId)]
Promise.all(promises).then(function(results){
//In Promise.all, the results of each promise are passed as array
//the order is the same as the order of the promises array.
var voucherTypeList = results[0];
var issuedTypeList = results[1];
});
}
BONUS: I do not want to complicate this task too much before you grasp it correctly. So I won't add more code. But note that you should use reject too, instead of handling your errors in every promise, you should reject them when things go wrong. Just reject(err) and add a second callback to your main script's then to handle any error that may happen. If you keep resolving your promises that did not work, you will not be passing the elements you are expecting and you'll need to add checks over every step.
Let's modify the GetVouchers callback to fit what I suggest.
voucherClient.GetVouchers(args, function (err, result) {
console.log("DENTRO GET ISSUED VOUCHERS");
if (err) {
reject(err);
} else {
resolve(mapGetVoucherTypeListResponse(result));
}
});
Once it is done on both your promises, we can change your main script to handle the error accordingly.
Promise.all(promises).then(function(results){
//Handle success like above.
},function(err){
//Handle error.
console.log(err.stack || err);
writeResponse(res, '200', err);
});

JavaScript Promises - Creating an array of promises to be executed together

I'm working with Node.js on async calls to noSQL DynamoDB. I first query to see which 'buddy list(s)' the account belongs. That may return from zero to 'n' new Primary Keys that will contain lists of all of the members of each of those buddy lists. Think of them as clubs to which a person belongs... you may have none or many; each club has several members or even one.
So far (and I am working with Promises for the first time here... though I have used callbacks on prior JA projects) I'm OK with where I am, but I know that I am assembling the array of promises incorrectly. That is, I can see in the console that the .then function executes before both promises resolve. For the record, I do expect that... it seems reasonable that any .then may be satisfied with a single promise resolving.
Anyways, Can someone offer up some tips as to how to structure? I'd like to end up with pseudo:
getBuddyLists
.then(getAllTheBuddiesInTheLists)
.then(function(){//doSomething});
my Node.js code:
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient({region:'us-east-1'});
var buddyGroups = [];
exports.handler = function(event, context, callback) {
var params = {
TableName: 'USER_INFORMATION',
Key: {
"DEVICE_ID": event.DEVICE_ID
}
};
var getBuddyList = new Promise(function(resolve, reject){
docClient.get(params, function(err, data){
if(err){
reject(err);
}
else{
var dataJSON = JSON.parse(JSON.stringify(data));
dataJSON.Item.my_buddies.values.forEach(function(value){
buddyGroups.push(value);
});
console.log('Got buddy groups: ' + buddyGroups);
resolve(buddyGroups);
}
});
});
getBuddyList.then(function(capturedData){ // capturedData => an array of primary keys in another noSQL document
console.log('groups: ' + capturedData);
var myPromises = [];
capturedData.forEach(function(value){
var reqParams = {
TableName: "buddy_list",
Key: {"BUDDY_ID": value}
};
myPromises.push(new Promise (function(resolve, reject){
docClient.get(reqParams, function(err, data){
if(err){
//console.log(err, data);
reject(err);
}else{
var returnedJSON = JSON.parse(JSON.stringify(data));
console.log(returnedJSON);
resolve(returnedJSON);
}
});
}));
});
//
Promise.all(myPromises); // ADDED IN EDIT <<<<<<<<<<<<<<<<<<<<<<
// how to make sure all of myPromises are resolved?
//
}).then(function(){
console.log("done");
})
.catch(function(err){
console.log("error message:" + error);
});
};
EDIT: Added location of Promise.all(myPromises);

Just to clarify how you would use Promise.all, even though you accepted an answer, you edited the question, but still are not using it correctly
Also, the discussion of .map vs .forEach - this is how you would use .map
getBuddyList.then(function(capturedData){ // capturedData => an array of primary keys in another noSQL document
console.log('groups: ' + capturedData);
// changes to use .map rather than .forEach
// also, you need to RETURN a promise - your edited code did not
return Promise.all(capturedData.map(function(value){
return new Promise (function(resolve, reject){
var reqParams = {
TableName: "buddy_list",
Key: {"BUDDY_ID": value}
};
docClient.get(reqParams, function(err, data){
if(err){
//console.log(err, data);
reject(err);
}else{
var returnedJSON = JSON.parse(JSON.stringify(data));
console.log(returnedJSON);
resolve(returnedJSON);
}
});
});
}));
}).then(function(){
console.log("done");
})
.catch(function(err){
console.log("error message:" + error);
});

You're looking for Promise.all(myPromises), which returns a single promise of an array of the results of the promises.

Asynchronous function call inside for loop

I am having a problem trying to make my for loop behave synchronously when I am making an asynchronous call in each iteration of the loop.
Here is my code:
pipeline: function(userContext, options, done){
var orderData = [];
options.data.forEach(function(store, index){
store.orders.forEach(function(order){
shopify.getPipeline(userContext, {'order':order,'storeId':index}, function(result){
var snapshotId = "30775bf1bb854c5d84c9c2af37bc8fb0";
var resourceToQuery = config.structrUrls.getUrl("ConfigurationSnapshot") + '/' + snapshotId ;
var requestOptions = {
method: "GET",
timeout: 8000
};
requestify.request(resourceToQuery, requestOptions)
.then(function(snapshotResult){
result.snapshots = snapshotResult.getBody().result;
result.snapshots.configurationPayload = JSON.parse(snapshotResult.getBody().result.configurationPayload);
orderData.push(result);
})
.catch(function(err){
console.log (err);
done(err);
});
});
});
});
done(null, orderData);
}
I understand the problem here, but do not know how to remedy it. Let me explain the function:
options.data contains an array of stores, and each store contains an array of orders. For each order, I am calling shopify.getPipeline() for pipeline data (this is a synchronous operation), and in the callback I make a requestify request (a node module used for making http requests) for snapshot data, which I want to append to the result before pushing it onto my "orderData" array. When this all completes, I am calling "done", a callback function, with my orderData. As you can see, since requestify calls are asynchronous, done is called before any data is added to the orderData array.
I think I need to use some kind of promise in order to guarantee the result before calling done, but I have been unsuccessful in implementing a promise into this function. In the documentation for q, it seems like the function I would want to use is promise.all(), which 'Returns a promise that is fulfilled with an array containing the fulfillment value of each promise, or is rejected with the same rejection reason as the first promise to be rejected'. I'm failing to see how to translate my forEach loop into an array of promises. I was also looking at the async parallel function and ran into the same problem regarding the array of functions.
Any help is greatly appreciated. Thanks!

To construct an array of Promises for use in Promise.all, you can map across the array of stores, and again across the array of orders, returning a Promise for each order which will be resolved or rejected based on the result of requestify.request. Merging the resulting nested array gives you a single array of promises which can then be passed to Promise.all.
Using your example:
pipeline: function(userContext, options, done){
var nestedPromises = options.data.map.forEach(function(store, index){
return store.orders.map(function(order){
return new Promise(function(resolve, reject){
shopify.getPipeline(userContext, {'order':order,'storeId':index}, function(result){
var snapshotId = "30775bf1bb854c5d84c9c2af37bc8fb0";
var resourceToQuery = config.structrUrls.getUrl("ConfigurationSnapshot") + '/' + snapshotId ;
var requestOptions = {
method: "GET",
timeout: 8000
};
requestify.request(resourceToQuery, requestOptions)
.then(function(snapshotResult){
result.snapshots = snapshotResult.getBody().result;
result.snapshots.configurationPayload = JSON.parse(snapshotResult.getBody().result.configurationPayload);
resolve(result);
})
.catch(function(err){
reject(err);
});
});
});
});
});
// Flatten nested array.
var promises = Array.prototype.concat.apply([], nestedPromises);
Promise.all(promises).then(function(orderData){
done(null, orderData);
}).catch(function(err){
done(err);
});
}