I'm struggling trying to have a javascript asynchrone sequence, although I have seen a lot of tutorials online.
Maybe you could help me :-)
I have an array of 6 functions (F1..F6)
Those function needs to be ran sequentially, forwarding a shared parameter from one function to the other.
And inside F3, I wanted to call another little function multiple times (currently using an array.map) ==> let's call them F3a, F3b, F3c
F3a, F3b, F3c needs to be sequential as well ; at least F4 must start AFTER all F3 are completed - and still those multilevel function are sharing the same global parameter.
I'm not sure I understand the full concept of Promises but I bet I need to learn more on them. :-)
The expected scheme I have in mind:
F1---|
******F2------|
***************F3a-|
********************F3b-|
*************************F3c-|
******************************F4------|
******************************F5------|
******************************F6------|
It might looks easy for all of you, but as a non developer, it's a bit harder for me to have the code working.
I have something partially working (using async and some Promises) and I guess (not sure) it is on the following sequencing; so not as expected :
F1---|
******F2------|
***************F3a-|
***************F3b-|
***************F3c-|
********************F4------|
********************F5------|
********************F6------|
I can't really show you the current code ; it's a total mess and I'd prefer to explain you the idea rather than getting the solution from you.
Should I use a nodejs library or should I code it using native js? Any library suggestion?
Any thought?
Thanks a lot.
You can do something like this without any external library
const sharedVariable = {}
async function doTask(data) {
const data1 = await f1(data, sharedVariable);
const data2 = await f2(data1, sharedVariable);
const data3 = await f3(data2, sharedVariable);
const data4 = await f4(data3, sharedVariable);
const data5 = await f5(data4, sharedVariable);
const data6 = await f6(data5, sharedVariable);
return data6;
}
async function f3(data) {
const data1 = await f3a(data, sharedVariable);
const data2 = await f3b(data1, sharedVariable);
const data3 = await f3c(data2, sharedVariable);
return data3;
}
doTask();
I'm not sure I understand the full concept of Promises but I bet I need to learn more on them. :-)
Yes, go ahead! What you describe is really easy to do using promises with async/await syntax:
async function f3(arg) {
await f3a();
await f3b();
await f3c();
return res;
}
async function main(val) {
await f6(await f5(await f4(await f3(await f2(await f1(val))))));
}
You can also easily use an actual loop if you have the functions in an array:
async function main(val) {
for (const f of fs)
val = await f(val);
}
d3-queue does a pretty good job for me. I also used it to create multiple nested queues. The API's easy to understand.
in your case it can look like this:
// all functions f1-f6 should have the same signarure
// function fN(sharedPArameter, done) ...
// done - is a callback you should call when done
function F3(sharedParameter, done) {
let nestedQ = d3.queue(1);
nestedQ.defer(f3a, sharedParameter);
nestedQ.defer(f3b, sharedParameter);
nestedQ.defer(f3c, sharedParameter);
nestedQ.awaitAll(error => {
done(error);
});
}
let q = d3.queue(1); // put 1 for sequential execution
q.defer(f1, sharedParameter);
q.defer(f2, sharedParameter);
q.defer(f3, sharedParameter);
q.defer(f4, sharedParameter);
q.awaitAll((error, result) => {
// when everything's done
});
Related
Heres my code, and I find myself having to do this ALOT for parsing json, or making functions.
// Adder Functions... hate repeating myself
async function addPetImagesToPet(petId, fileName) {
const pet = await db.Pet.findById(petId);
await pet.petImages.push({'fileName':fileName});
pet.updated = Date.now();
await pet.save();
//console.log(pet);
}
async function addPropertyImagesToProperty(propertyId, fileName) {
const property = await db.Property.findById(propertyId);
await property.propertyImages.push({'fileName':fileName});
property.updated = Date.now();
await property.save();
//console.log(property);
}
async function addAccountImagesToAccount(accountId, fileName) {
const account = await db.Account.findById(accountId);
await account.accountImages.push({'fileName':fileName});
account.updated = Date.now();
await account.save();
//console.log(account);
}
//how can I do `await db.${my_type/Object_Name!!! <--this is what im asking about}.findById(this obviously can be a var np)
I find myself doing this repeat alot for most of my services, im trying to expand this further for most of my services front end and back end. if i figured out how to do this I could literally have one "CRUD" service for a majority of my use cases for each object type.
Comes alot and has been driving me crazy for parsing JSON objects, making the selector on the object be dynamic...
I use ${} because thats how I dynamically build strings, but cant do it or make it work for functions, or naming, I cant for instance make a string variable and use that name as a name for function, or its methods, but thats exactly what I want to do. Thank you again for anyone able to help.
Thanks in advance.
You could do something like there where you create a hashMap with the individual values.
const hashMap = {
'pet': {
dbName:"Pet",
key: 'petImages'
},
'account': {
dbName: "Account",
key: 'accountImages'
},
'property':{
dbName: "Property",
key: 'propertyImages'
}
}
async function addImagesToAccount(accountId, fileName, type) { // type will be the key for the object above.
const {dbName, key} = hashMap[type];
const account = await db[dbName].findById(accountId);
await account[key].push({'fileName':fileName});
account.updated = Date.now();
await account.save();
}
You could use something like this:
async function addImages(type, id, fileName) {
const entity = await db[type].findById(id);
entity.updated = Date.now();
const attribute = `${type.toLowerCase()}Images`;
entity[attribute].push({ fileName })
await entity.save();
}
async function main() {
await addImages('Property', 2, 'text-02');
await addImages('Account', 3, 'text-03');
await addImages('Pet', 3, 'text-03');
}
Also I think like the Factory Pattern could help you to code something reusable :).
Ideally, if you wanna use more variables, it is to use and array maybe. It's like a good practice to keep a functions with no more than three variables. But that's a personal perference.
Well i would use a single line to do that, no need to wrap it in a function
await db.Pet.updateOne({ _id: petId }, { $push: : { petImages : {'fileName':fileName}}, updated: Date.now() });
Thank you everyone for the comments and help.
My final code looks like this, sure I could get it shorter.
async function addImagesToObject(objId, objType, fileName) {
const entity = await db[`${objType}`].findById(objId);
const attribute = `${objType.toLowerCase()}Images`; //<-- defining selector
await entity[attribute].push({'fileName':fileName});
entity.updated = Date.now();
await entity.save();
}
I am new to JavaScript and have been trying to read up a lot on why this is not working. Here is my code. I have also read a number of articles here on stack overflow but still feeling dense
Also if my title does not make sense, please suggest an edit
listRef.listAll()
.then(response => {
let files = []
response.items.forEach(item => {
var text
getText(item.name).then(res=>{text = res});
const id = {uid: guid()}
const url = item.getDownloadURL().then(url => {return url} )
const gsurl = `gs://archivewebsite.appspot.com/${folder}/${item.name}`
files.push({...item, name:item.name, url, gsurl, id:id.uid, text})
});
this.files = files;
})
.catch(error => console.log(error));
async function getText(docID) {
var docRef = firestore.collection("recipes").doc(docID);
let doc = await docRef.get()
if (doc.exists){
return doc.data().text
}
}
that code "works" in that it logs the response to the console but the text variable is a pending promise object.
I understand that async functions return a promise so when I call getText I need to use .then - what I am struggling with and have refactored this code a few times is this:
how can I assign the value of doc.data().text to a variable to be used later in other words, how can var text be an actual string and not a promise object pending
Also for my own learning on javascript inside the async function if I replace
if (doc.exists){
return doc.data().text
}
with
if (doc.exists){
return Promise.resolve(doc.data().text)
}
I get the same result in console.log - is this expected? is return simply short hand for the handler to resolve the promise?
I have also refactored this code to be non async and I get the same result where my var text is basically a pending promise and never the resolved data
Thanks for your help - also any articles to help explain this to me would be great! I have been going through courses on udemy but little confused by this right now
Actually you are assigning the complete promise to the variable text
Replace
var text = getText(item.name).then(res=>console.log(res))
by
var text = await getText(item.name);
OR
var text
getText(item.name).then(res=>{text = res});
Of course text is going to be a Promise. Promise.then() always returns a Promise.
Consider this code:
function doA(n) {
// do something here...
console.log("A" + n);
}
asnyc function doB(n) {
// do something here...
console.log("B" + n);
}
doA(1);
doA(2);
doB(3); // async
doA(4);
doB(5); // async
doB(6); // async
doA(7);
What do you expect the output to be?
1, 2, 4, and 7 will always be in order, because they are executed synchronously.
3 will not ever print before 1 and 2. Likewise, 5 and 6 will not ever print before 1, 2, and 4.
However, 3, 5, and 6 can be printed in any order, because async functions do not guarantee execution order once created.
Also, 4 can print before 3. Likewise, 7 can print before 5 and 6.
Basically, think of async functions as a parallel task that runs independently (although not really; single thread JS only simulates this behavior). It can return (fulfill/reject) at any moment. For this reason, you cannot just simply assign a return value of an async function to a variable using synchronous code - the value is not guaranteed to be (and probably is not) available at the moment of synchronous execution.
Therefore you need to put all the code that requires the value of text to be set into the callback block of the Promise, something like this:
getText(item.name).then((text) => {
// put everything that uses text here
});
This can of course lead to the infamous "callback hell", where you have layers inside layers of async callback. See http://callbackhell.com for details and mitigation techniques.
async/await is just one of the newer ways to do the same thing: MDN has an excellent article here: https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await
OK I worked with someone at work and found a solution - it was related to this post
https://stackoverflow.com/a/37576787/5991792
I was using async function inside a for each loop
the refactored code is here
async function buildFiles(){
let items = await listRef.listAll()
let files = []
for (const item of item.items){
const text = await getText(item.name)
const url = await item.getDownloadURL()
const gsurl = `gs://archivewebsite.appspot.com/${folder}/${sermon.name}`
files.push({...item, name:item.name, url, gsurl, text})
}
return files
}
async function getText(docID) {
var docRef = firestore.collection("recipies").doc(docID);
let doc = await docRef.get()
if (doc.exists){return await doc.data().text}}
buildFiles().then(res=>this.files = res)
Thanks also to #cyqsimon and #Nav Kumar V
I have 3 node different projects(services). This is structure.
BIG SERVICE
-service_1
-start.js
-service_2
-start.js
-service_3
-start.js
-big_start.js
I want,that when I run big_start.js node file,first work service_1->start.js if everything is OK, second work service_2->start.js,if everything is OK,third work service_3-> start.js.
If you are able to use async/await then it should be relatively easy to model them in series:
async awaitChain() {
try {
const result1 = await startService1();
const result2 = await startService2();
...
const result3 = await startService3();
...
catch (err) {
// handle error
}
// all is well each has been initialized in series
return await bigStart();
}
This pattern could be abstracted and generalized to execute any number of async functions serially, and bluebird has an each primitive that should do exactly this.
I have some javascript code which does the following:
Read a .txt file and fill up an array of objects
Loop through these itens
Loop through an array of links inside each of these itens and make a request using nightmarejs
Write the result in Sql Server
My code is like this:
const Nightmare = require('nightmare');
const fs = require('fs');
const async = require('async');
const sql = require('mssql');
var links = recuperarLinks();
function recuperarLinks(){
//Read the txt file and return an array
}
const bigFunction = () => {
var aparelho = '';
async.eachSeries(links, async function (link) {
console.log('Zip Code: ' + link.zipCode);
async.eachSeries(link.links, async function(url){
console.log('URL: ' + url);
try {
await nightmare.goto(link2)
.evaluate(function () {
//return some elements
})
.end()
.then(function (result) {
//ajust the result
dadosAjustados.forEach(function (obj) {
//save the data
saveDatabase(obj, link.cep);
});
});
} catch (e) {
console.error(e);
}
}
}, function(err){
console.log('Erro: ');
console.log(err);
})
}, function (erro) {
if (erro) {
console.log('Erro: ');
console.log(erro);
}
});
}
async function salvarBanco(dados, cep){
const pool = new sql.ConnectionPool({
user: 'sa',
password: 'xxx',
server: 'xxx',
database: 'xxx'
});
pool.connect().then(function(){
const request = new sql.Request(pool);
const insert = "some insert"
request.query(insert).then(function(recordset){
console.log('Dado inserido');
pool.close();
}).catch(function(err){
console.log(err);
pool.close();
})
}).catch(function(err){
console.log(err);
});
}
lerArquivo();
It works fine, but i'm finding this async loop inside another async loop like a hack of some sort.
My outputs are something like this:
Fetching Data from cep 1
Fetching Data from url 1
Fetching Data from cep 2
Fetching Data from url 2
Fetching Data from cep 3
Fetching Data from url 3
Then it starts making the requests. Is there a better (and possibly a correct way) of doing this?
If you want to serialize your calls to nightmare.goto() and you want to simplify your code which is what you seem to be trying to do with await, then you can avoid mixing the callback-based async library with promises and accomplish your goal by only using promises like this:
async function bigFunction() {
var aparelho = '';
for (let link of links) {
for (let url of link.links) {
try {
let result = await nightmare.goto(url).evaluate(function () {
//return some elements
}).end();
//ajust the result
await Promise.all(dadosAjustados.map(obj => saveDatabase(obj, link.cep)));
} catch (e) {
// log error and continue processing
console.error(e);
}
}
}
}
Asynchronous loop inside an asynchronous loop: a good idea?
It's perfectly fine and necessary sometimes to nest loops that involve asynchronous operations. But, the loops have to be designed carefully to both work appropriately and to be clean, readable and maintainable code. Your bigFunction() does not seem to be either to me with your mix of async coding styles.
It works fine, but i'm finding this async loop inside another async loop like a hack of some sort.
If I were teaching a junior programmer or doing a code review for code from any level of developer, I would never allow code that mixes promises and the callback-based async library. It's just mixing two completely different programming styles for both control flow and error handling and all you get is a very hard to understand mess. Pick one model or the other. Don't mix. Personally, it seems to me that the future of the language is Promises for both async control flow and error propagation so that's what I would use.
Note: This appears to be pseudo-code because some references used in this code are not used or defined such as result and dadosAjustados. So, you will have to adapt this concept to your real code. For the future, we can offer you much more complete answers and often make suggested improvements that you're not even aware of if you include your real code, not an abbreviated pseudo-code.
Given an array of promises, what's the idiomatic way to get the results in ES7?
Here's what I want to do:
async function getImports() {
let imports = [System.import('./package1.js'), System.import('./package2.js')];
let promises = await* imports;
let results = [];
await promises.forEach(val => val.then(data => results.push(data))); //seems hacky
console.log(results); // array of 2 resolved imports
}
The result is correct, but I'm still doing a forEach and a then to turn the resolved promises into results. This just doesn't seem right to me. Is there a cleaner way?
As mentioned in the issue you filed, the core issue is that await* is no longer a thing and has been removed. Unfortunately, it was not properly throwing a syntax error in Babel 6 and was essentially being treated like a normal await.
You'll need to explicitly
let [p1, p2] = await Promise.all([
System.import('./package1.js'), System.import('./package2.js')]);
I cannot believe it actually works, forEach does return undefined which you cannot await. If you need a loop, use map to get an array of (promised) results.
In your case, you seem to be looking for a simple
async function getImports() {
let promises = [System.import('./package1.js'), System.import('./package2.js')];
let results = await Promise.all(promises)
console.log(results);
}
One way to do it this ....
async function abc() {
let p1 = getReviews();
let p2 = getMenu();
let [reviews, menu] = await results(p1, p2);
}
function results(...rest) {
return Promise.all(rest).catch(err => console.log(err));
}