I'm trying to build a translation engine using node.js. I have a Python/R background so I'm not getting the hang of these so called callbacks . . .
The input is a sentence :
var sentence = "I want to translate this"
When I hit the "Translate" button, it should trigger the translation.
Now this is the database query, I call a node.js backend at /translate
function query(string){
var query =
db.call.get(query, function(result){
if(result.length>0){
console.log(result[0].translation);
} else {
// not found in database
console.log(string);
}
});
}
So then it goes like this:
var wordList = sentence.split(" ");
for(i=0; i<wordList.length; i++){
// i call the database to return me the translation for each word
query(wordList[i]);
}
But then, the output in the console log comes like this:
output[0]: "translate", "want", "to", "I"
How can I make it come back in order? I understand there is some async and callback stuff going on, but I do believe that the guys who coded node are smart and that there is surely a way to solve this pretty easily.
Thanks
You need to take into account the fact that your query function might take a different time to return depending on the string argument you pass it. Right now, in your example code, if the query for "want" returns before the query for "I", then your translations will be outputted in the same order that the queries return (i.e. translation of "want", translation of "I").
Callbacks can help you get around this problem, because they are a function that will only get called when your query function returns. Then it's up to you to define what your program should do with the result. For instance, your program can output the results in the same order that the original array was in. This could be achieved using an index as suggested by Guffa, or using the async library that we will discuss later on.
So, one thing you could try is to pass query a callback like this:
function query(string, callback){
db.call.get(query, function(result){
if(result.length>0){
callback(null, result[0].translation);
} else {
// not found in database
callback(null, string);
}
});
}
The canonical way to use callbacks in Node.js, is to give it two parameters, so it looks like: callback(err, result). If there is no error, you can call callback(null, result), while when there is an error, you can call callback(err) or callback (err, result), depending on what you want to report.
Once your query function takes a callback, you are able to know when it did return a value, and you can use the async.map function (from the async library), like so:
var async = require('async');
var sentence = "I want to translate this";
var words = sentence.split(' ');
async.map(words, query, function (err, translations) {
console.log(translations);
}
What this function does, is:
run in parallel query on all the items in the words array
wait for all the callbacks from all the query functions to return
then call async.map's own callback (here the third argument:
function (err, translations) {console.log(translations);})
The only way to make them come back in order would be to chain the calls, but then you take away the point of having asynchronous calls in the first place.
Send along the index for the word also, that way you can put the results together in the right order:
var wordList = sentence.split(" ");
var resultList = [];
var resultCount = 0;
for (i = 0; i < wordList.length; i++){
// i call the database to return me the translation for each word
query(wordList[i], i);
}
function query(string, index){
var query = db.call.get(query, function(result){
if (result.length > 0){
resultList[index] = result[0].translation;
} else {
// not found in database
resultList[index] = string;
}
resultCount++;
if (resultCount == wordList.length) {
console.log(resultList);
}
});
}
Related
I'm trying to make a MySQL query to filter data from a table. Effectively what I want to do is:
SELECT data FROM table WHERE column IN ?
The filter is coming from checkboxes in a form on a webpage, so I can pass an array or object fairly easily, but it'll be a varying number of parameters for the IN each time, so I can't us multiple ?. I tried making a for loop to make multiple queries concatenate the arrays that the queries returned, but I ran into scope issues with that. I also tried passing an array directly to the query, but that throws a syntax error. I'm sure there's a straightforward answer to this but I'm not sure how to do it.
Edit: source code added:
Here's where I'm at:
const filterShowQuery = `SELECT sl_Show.showId, sl_Band.BandName,
sl_Show.date, sl_Venue.venueName,
sl_Show.length, sl_Show.attendance, sl_Show.encore FROM sl_Show
JOIN sl_Band on sl_Show.BandID = sl_Band.BandId
JOIN sl_Venue on sl_Show.VenueId = sl_Venue.VenueId
WHERE sl_Band.BandName IN (?)
ORDER BY sl_Band.BandName;`;
Trying to get an array into the ? in WHERE sl_Band.BandName IN
const getShows = (req, res,next) =>{
var {bands, venues} = req.body;
var i = 0; //left over from previous attempt
var data = [];
for (b in bands){
mysql.pool.query(filterShowQuery, bands[b], (err, result) => {
if(err){
console.log('filter band error');
next(err);
return;
}
data = data.concat(result);
console.log(data); //data concatenates property and increases through for loop
})
// same action to be performed with venues once solved
// for (v in venues){
// conditions[i] = venues[v];
// i++;
console.log(data); //data is empty when logging from here or using in res
res.json({rows:data});
}
}
SECURITY WARNING!
I must to say: NEVER, NEVER PASS DATA DIRECTLY TO YOUR SQL!
If you don't know why, just google for SQL Injection. There are lots of examples on how it is done, how easily it can be done, and how to protect your application from this sort of attack.
You should always parametrize your queries. But in the very rare case which you really need to insert data concatenating a string into your sql, validate it before.
(E.g.) If it's a number, than use a Regex or some helper method to check if the value you are inserting into your SQL String is really and only a number and nothing else.
But aside that, you did not provide any source code, so it's really hard to give any help before you do that.
I'm making discord bot in which I have loop to loop through my JSON data. That works fine however I want to paste that data to Hastebin.
const emails = require("../../json/emails.json");
const hastebin = require("hastebin-gen");
for (var obj in emails) {
if (emails.hasOwnProperty(obj)) {
for (var prop in emails[obj]) {
if (emails[obj].hasOwnProperty(prop)) {
var results = emails[obj]["email"]
}
}
}
var haste = await hastebin(results, { extension: "txt" });
}
console.log(haste)
JSON
{
"test": {
"email": "test#gmail.com"
},
"Test1": {
"email": "mhm#gmail.com"
}
}
As you can see I have 2 emails in JSON so it creates 2 hastebin links and in them is always only 1 email.
EDIT
Didn't really say what is the problem. I want to log only 1 Hastebin url with all the emails inside. Not 2 Hastebin urls and each one has only 1 email.
You need to push all promises to an array and then use Promise.allSettled to wait for all of them to be fulfilled or rejected:
const emails = require("../../json/emails.json");
const hastebin = require("hastebin-gen");
const promises = Object.values(emails).map(({ email }) => hastebin(email, { extension: "txt" }));
const results = await Promise.allSettled(promises);
console.log(results);
According to your comments, if you want to generate a single Hastebin, then you should move the hastebin call after the for loop and pass it all the results concatenated:
const emails = require("../../json/emails.json");
const hastebin = require("hastebin-gen");
const content = Object.values(emails).map(({ email }) => email).join('\n');
const results = await hastebin(content, { extension: "txt" });
console.log(results);
If I understand correctly you want to put all emails into just one file, rather than creating a link for each. The way to correcting this is to understand why it is doing it this way.
Let's first look at where you are creating the link (your hasebin() call), this is inside the for loop. You need to move this outside of your loop, but you want something within your loop still to create an array of the emails you wish to add to hastebin.
Based on your title, you understand this - it sounds like you have a problem with javascripts asynchronous nature, in that your call outside of the loop would not wait for the loop to finish, thus it would not yet have any data within the array.
To achieve this in Javascript we use something called a "callback".
See this example:
function functionOne(callbackFunction) {
console.log("1");
callbackFunction();
}
function onComplete() {
console.log("2");
}
functionOne(onComplete);
The code above would call functionOne with a parameter referencing the onComplete method. functionOne gets ran (1 is printed to console), and then onComplete is called via callbackFunction() and 2 is printed to the console.
So, what does this mean for you example? Your loop should run and then have a callback which is where you create your hastebin, this will ensure your loop has finished constructing/fetching the data prior to trying to upload it.
I need these values in a row, but I don't want them reply if its already in div.
My try is here. I try to build an array and put these values from firebase to here, but still I can't see in html.
var fruits = [];
var fetchPostsleft = function(postsRef, sectionElement,fruits) {
postsRef .orderByChild('timeStamp').on('child_added', function(data) {
console.log(data.val());
var author = data.val().senderName;
var containerElement2 = sectionElement.getElementsByClassName('posts-containerleft')[0];
fruits.push(data.val().senderName);
console.log(fruits.length);
});
};
fetchPostsleft(topUserPostsRef, topUserPostsSectionleft,fruits);
var fLen = fruits.length;
console.log(fruits.length);
for (var i = 0; i < fLen; i++) {
// text += "<li>" + fruits[i] + "</li>";
topUserPostsSectionleft.getElementsByClassName('posts-containerleft')[0].insertBefore( createheaders(fruits[i], ""),
topUserPostsSectionleft.getElementsByClassName('posts-containerleft')[0].firstChild);
}
The data is loaded from Firebase asynchronously. This means that by the time your looping over the array, it hasn't been populated yet. It's easiest to see this by replacing most of the code with a few log statements:
console.log("Before query");
postsRef.orderByChild('timeStamp').on('child_added', function(data) {
console.log("In child_added");
});
console.log("After query");
If you run this snippet, the logging will be:
Before query
After query
In child_added
This is probably not the order you expected, since it's different from the order the log statements are in your code. This is because the data is loaded from Firebase asynchronously and the rest of your code continues while it's loading.
It's slightly easier to see if you turn the callback into a separate function:
function onChildAdded(data) {
console.log("In child_added");
});
console.log("Before query");
postsRef.orderByChild('timeStamp').on('child_added', onChildAdded);
console.log("After query");
Now you can more easily see that the first few lines just declare the onChildAdded function. They don't run it yet. And we're just passing that function in to the query, so that the query can call onChildAdded whenever it gets the data.
This is the most common pitfall of web programming with remote servers. But since most of the modern web is based on such asynchronous APIs, you will have to learn it.
One way I've found that works is to reframe your problem. Your current code is based on "first fetch the posts, then add them to the HTML". But in asynchronous programming it's better to think "start fetching the posts. When we get them, add them to the HTML". This translates into the following code:
function fetchPostsleft(postsRef, sectionElement) {
postsRef.orderByChild('timeStamp').on('child_added', function(data) {
var author = data.val().senderName;
topUserPostsSectionleft.getElementsByClassName('posts-containerleft')[0].insertBefore(createheaders(author, ""),
});
};
fetchPostsleft(topUserPostsRef, topUserPostsSectionleft);
Now all the code that needs the new data is inside the callback, so it's only run when the snapshot is actually available.
I am using NodeJS postgresql client to fetch some data loop through it and give an output. I'm using ExpressJS together with postgresql client.
This is my code
var main_data = some array of data
var user_info = {}
for (var key in result.rows) {
var user = main_data[key].user
client.query('SELECT name,age,address FROM users WHERE user_id = $1 LIMIT 1;' , [user], function(queryErr, result){
var return_data = result.rows[0]
user_info.name = return_data.name
user_info.gender = return_data.gender
user_info.address = return_data.address
main_data[key].user_info = user_info
})
}
done()
res.json(main_data)
return
However when I run this code I don't get a proper output. The userinfo is not pushed to the main_data. It just outputs main_data variable just as the beginning of the code.
It is not simple as the suggested duplicate
Because the function sits inside a for loop so I can't just make the response call once the function is done. I have to wait till the whole loop is finished which may be 1000+ of loops to finish before I can make the response call.
So tell me how can I achieve it.
What am I doing wrong and how can fix it?
Thank you very much
I would use async.js for this, myself.
var rows = // your result.rows stuff.
function doTheQuery(item, callback){
// your query logic here, calling the callback with the results
}
async.series(rows, doTheQuery, function(err){
// handle any errors
}
As an exercise to teach myself more about node js I started making a basic CRUD REST server for SimpleDB (sdb) using the aws-sdk.
Everything was running smoothly until I got to a function for reading the domains. The aws-sdk has two functions for this purpose: listDomains and domainMetadata. listDomains returns an array of sdb domain names. domainMetadata will return additional statistics about a domain, but will only return them for one domain at a time. It does not include the domain name in the results.
My script is running listDomains and returning an array in the JSON response just fine. I would like to make my api readDomains function more ambitious though and have it return the metadata for all of the domains in the same single api call. After all, running a handful of domainMetadata calls at the same time is where node's async io should shine.
The problem is I can't figure out how to run a variable number of calls, use the same callback for all of them, match the results of each domainMetadata call to it's domainName (since it's async and they're not guaranteed to return in the order they were requested) and tell when all of the metadata requests have finished so that I can send my final response. Put into code my problem areas are:
domain.receiveDomainList = function(err, data){
var domainList = [];
for(var i=0; i<data.DomainNames.length; i++){
sdb.domainMetaData({"DomainName":data.DomainNames[i]},domain.receiveMetadata);
// alternatively: domainList.push({"DomainName":data.DomainNames[i]});
}
// alternatively:
// async.map(domainList, sdb.domainMetadata, domain.receiveMetadata)
console.log(domainList);
}
domain.receiveMetadata = function (err, data){
// I figure I can stash the results one at a time in an array in the
// parent scope but...
// How can I tell when all of the results have been received?
// Since the domainname used for the original call is not returned with
// the results how do I tell what result matches what request?
}
Based on my reading of async's readme the map function should at least match the metadata responses with the requests through some black magic, but it causes node to bomb out in the aws sync library with an error of " has no method 'makeRequest'".
Is there any way to have it all: requests run in parallel, requests matched with responses and knowing when I've received everything?
Using .bind() you can set the context or this values as well as provide leading default arguments to the bound function.
The sample code below is purely to show how you might use .bind() to add additional context to your response callbacks.
In the code below, .bind is used to:
set a domainResults object as the context for the receiveMetaData callback
pass the current domain name as an argument to the callback
The domainResults object is used to:
keep track of the number of names received in the first request
keep track of the completedCount (incremented on each callback from the metaData request)
keep track of both error and success responses in list
provide a complete callback
Completely untested code for illustrative purposes only:
domain.receiveDomainList = function(err, data) {
// Assuming err is falsey
var domainResults = {
nameCount: data.DomainNames.length,
completeCount: 0,
list: {},
complete: function() {
console.log(this.list);
}
};
for (var i = 0; i < data.DomainNames.length; i++) {
sdb.domainMetaData({ "DomainName": data.DomainNames[i] },
domain.receiveMetadata.bind(domainResults, data.DomainNames[i]));
}
}
domain.receiveMetadata = function(domainName, err, data) {
// Because of .bind, this === domainResults
this.completeCount++;
this.list[domainName] = data || {error: err};
if(this.completeCount === this.nameCount) {
this.complete();
}
}