I am trying out a Chrome extension.
How do I get a replaceAll's callback function to use await ?
window.onload = async () =>
{
const elem = document.getElementById('#myID');
const elem_modified = elem.replaceAll(regex, await myFunction);
document.getElementById('#myID').innerHTML = elem_modified;
}
myFunction :
const myFunction = async (str) =>
{
const items = [...str.matchAll(regex)][0];
// Do some manipulation
let amount = number * value;
const amount_modified = await calculate(amount);
console.log("amount_modified = " + amount_modified); // This shows correctly
return items[0] + ' (' + amount_modified + ')';
}
The result is like
over [object Promise] to many people
You don't, replaceAll — like lots of functions that support callbacks — doesn't expect a promise as a return value from the callback. In the vast majority of cases, you can't usefully pass a promise to code that doesn't expect one. (map is one exception to that rule.)
You haven't provided a runnable example or an example of your regular expression, but the general shape of the solution here will be:
Split the string into parts (the segments you want to pass through your async function and the segments between them).
Start the calculation for each of the segments you want to process and get a promise (from your async function) of the result; for the segments in-between, just create a promise fulfilled to the original segment value.
Wait for all of those to complete by using Promise.all, awaiting the promise it provides. (That's assuming it's okay to start processing all the segments at once, rather than waiting for the processing of one to end before starting processing the next.)
Assemble a new string from the updated segments and the segments that used to be between them.
Here's a rough sketch that isolates the numbers in a string and builds a new string with the numbers doubled. This version doesn't, of course, need to be async at all, but you can substitute a truly async operation for calculate:
async function calculate(str) {
await new Promise((resolve) => setTimeout(resolve, 10)); // Just so we do actually do something async
return str ? String(Number(str) * 2) : str;
}
async function example(str) {
const rex1 = /(\d+)/; // Note the capture group, so `split` retains the delimiter between matches
const rex2 = /^\d+$/; // Same as above, but full string match and no capture group
// #1
const parts = str.split(rex1);
// #2 and #3 (the `await`)
const updated = await Promise.all(parts.map(async (part) => {
if (rex2.test(part)) {
return await calculate(part);
}
return part;
}));
// #4
return updated.join("");
}
example("Testing 1 2 3 testing")
.then((result) => console.log(result))
.catch((error) => console.error(error));
Side note: The DOM event system won't pay any attention to the promise the async function you're using as an event handler returns, so it's important not to allow any errors (promise rejections) to terminate that handler, since those rejections will be unhandled.
Related
I was testing node-fetch and when I was using async / await, a question came up : I have to make my function async if I use await in it BUT since my function is async I need to await it and make the parent function async. And so on... Context :
async functionA()
{
var result = await functionB();
}
async functionB()
{
//do things
var result = await functionC();
//do things
var blob;
//do things
return blop;
}
async functionC()
{
//here is the real async call
var result = await fetch(....);
var json = await result.json();
return json;
}
}
How can I make this async / await chaining stop ? I have one method in my program using fetch and it will make me transform all my others methods to async method. Is this really how all others program developped with fetch look like ? Perhaps I didn't understand something with async/ await.
Thanks for your time :)
Call the asynchronous function once, at the beginning of your script. After its response comes back, call the rest of your script with the result synchronously. Something along the lines of:
async function functionC() {
//here is the real async call
const response = await fetch(....);
const result = await result.json();
return result;
}
functionC()
.then((results) => {
// do stuff with results
functionA(results);
functionB(results); // if needed
// etc
})
.catch(handleErrors); // don't forget this part
This way, only one part of your script - near the entry point - needs to handle the asynchronous logic. Implementing this may well require some refactoring, but it's worth it over the alternative of making everything async.
(also note that .json resolves to a plain object or array, probably - it doesn't resolve to a string in JSON format, it parses the Response stream as if it was JSON and resolves to the result, which is no longer a JSON-formatted string, but a plain value, usually an object or array)
Call an async method from a non-async method.
Short answer:
int GetData(DateTime x)
{
Task<int> taskGetData = Task.Run( () => GetDataAsync(x));
taskGetData.Wait();
return taskGetData.Result;
}
async Task<int> TaskGetDataAsync(DateTime x)
{
...
}
Longer answer: Whenever GetDataAsync is awaiting (for instance wait until data is written to a file, or fetched from a database), your procedure is free to do something else.
int GetData(DateTime x)
{
Task<int> taskGetData = Task.Run( () => GetDataAsync(x));
DoSomethingElse();
taskGetData.Wait();
int taskResult = taskGetData.Result;
return taskResult;
}
This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 2 years ago.
I'm building a trading bot that needs to get stock names from separate files. But even I have used async function and await in my code, that doesn't work.
My index file init method.
const init = async () => {
const symbols = await getDownTrendingStock();
console.log("All stocks that are down: " + symbols);
const doOrder = async () => {
//do stuff
}
doOrder();
}
my getDownTrendeingStock file
const downStocks = []
function getDownTrendingStock () {
for(i = 0; i < data.USDTPairs.length; i++){
const USDTPair = data.USDTPairs[i] + "USDT";
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if(prevDay.priceChangePercent < -2){
downStocks.push(symbol)
}
});
}
return downStocks;
}
I have tried to also use async in for loop because the getDownTrendinStock function returns an empty array before for loop is finished. I didn't find the right way to do that because I was confused with all async, promise and callback stuff. What is the right statement to use in this situation?
Output:
All stocks that are down:
Wanted output:
All stocks that are down: [BTCUSDT, TRXUSDT, ATOMUSDT...]
I think the main issue in the code you posted is that you are mixing callbacks and promises.
This is what's happening in your getDownTrendingStock function:
You start iterating over the data.USDTPairs array, picking the first element
You call binance.prevDay. This does nothing yet, because its an asynchronous function that takes a bit of time. Notably, no data is added to downStocks yet.
You continue doing 1-2, still, no data is added
You return downStocks, which is still empty.
Your function is complete, you print the empty array
Now, at some point, the nodejs event loop continues and starts working on those asynchronous tasks you created earlier by calling binance.prevDay. Internally, it probably calls an API, which takes time; once that call is completed, it calls the function you provided, which pushes data to the downStocks array.
In summary, you didn't wait for the async code to complete. You can achieve this in multiple ways.
One is to wrap this in a promise and then await that promise:
const result= await new Promise((resolve, reject) => {
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if (error) {
reject(error);
} else {
resolve({prevDay, symbol});
}
});
});
if(result.prevDay.priceChangePercent < -2){
downStocks.push(result.symbol)
}
Note that you can probably also use promisify for this. Also, this means that you will wait for one request to finish before starting the next, which may slow down your code considerably, depending on how many calls you need; you may want to look into Promise.all as well.
Generally speaking, I use two technics:
const asyncFunc = () => {smthAsync};
const arrayToProcess = [];
// 1
const result = await arrayToProcess.reduce((acc, value) => acc.then(() => asyncFunc(value)), Promise.resolve(someInitialValue));
// 2
// here will be eslint errors
for(let i = 0 i < arrayToProcess.length; i+=1) {
const processResult = await asyncFunc(value);
// do with processResult what you want
};
I am new to JavaScript and have been trying to read up a lot on why this is not working. Here is my code. I have also read a number of articles here on stack overflow but still feeling dense
Also if my title does not make sense, please suggest an edit
listRef.listAll()
.then(response => {
let files = []
response.items.forEach(item => {
var text
getText(item.name).then(res=>{text = res});
const id = {uid: guid()}
const url = item.getDownloadURL().then(url => {return url} )
const gsurl = `gs://archivewebsite.appspot.com/${folder}/${item.name}`
files.push({...item, name:item.name, url, gsurl, id:id.uid, text})
});
this.files = files;
})
.catch(error => console.log(error));
async function getText(docID) {
var docRef = firestore.collection("recipes").doc(docID);
let doc = await docRef.get()
if (doc.exists){
return doc.data().text
}
}
that code "works" in that it logs the response to the console but the text variable is a pending promise object.
I understand that async functions return a promise so when I call getText I need to use .then - what I am struggling with and have refactored this code a few times is this:
how can I assign the value of doc.data().text to a variable to be used later in other words, how can var text be an actual string and not a promise object pending
Also for my own learning on javascript inside the async function if I replace
if (doc.exists){
return doc.data().text
}
with
if (doc.exists){
return Promise.resolve(doc.data().text)
}
I get the same result in console.log - is this expected? is return simply short hand for the handler to resolve the promise?
I have also refactored this code to be non async and I get the same result where my var text is basically a pending promise and never the resolved data
Thanks for your help - also any articles to help explain this to me would be great! I have been going through courses on udemy but little confused by this right now
Actually you are assigning the complete promise to the variable text
Replace
var text = getText(item.name).then(res=>console.log(res))
by
var text = await getText(item.name);
OR
var text
getText(item.name).then(res=>{text = res});
Of course text is going to be a Promise. Promise.then() always returns a Promise.
Consider this code:
function doA(n) {
// do something here...
console.log("A" + n);
}
asnyc function doB(n) {
// do something here...
console.log("B" + n);
}
doA(1);
doA(2);
doB(3); // async
doA(4);
doB(5); // async
doB(6); // async
doA(7);
What do you expect the output to be?
1, 2, 4, and 7 will always be in order, because they are executed synchronously.
3 will not ever print before 1 and 2. Likewise, 5 and 6 will not ever print before 1, 2, and 4.
However, 3, 5, and 6 can be printed in any order, because async functions do not guarantee execution order once created.
Also, 4 can print before 3. Likewise, 7 can print before 5 and 6.
Basically, think of async functions as a parallel task that runs independently (although not really; single thread JS only simulates this behavior). It can return (fulfill/reject) at any moment. For this reason, you cannot just simply assign a return value of an async function to a variable using synchronous code - the value is not guaranteed to be (and probably is not) available at the moment of synchronous execution.
Therefore you need to put all the code that requires the value of text to be set into the callback block of the Promise, something like this:
getText(item.name).then((text) => {
// put everything that uses text here
});
This can of course lead to the infamous "callback hell", where you have layers inside layers of async callback. See http://callbackhell.com for details and mitigation techniques.
async/await is just one of the newer ways to do the same thing: MDN has an excellent article here: https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await
OK I worked with someone at work and found a solution - it was related to this post
https://stackoverflow.com/a/37576787/5991792
I was using async function inside a for each loop
the refactored code is here
async function buildFiles(){
let items = await listRef.listAll()
let files = []
for (const item of item.items){
const text = await getText(item.name)
const url = await item.getDownloadURL()
const gsurl = `gs://archivewebsite.appspot.com/${folder}/${sermon.name}`
files.push({...item, name:item.name, url, gsurl, text})
}
return files
}
async function getText(docID) {
var docRef = firestore.collection("recipies").doc(docID);
let doc = await docRef.get()
if (doc.exists){return await doc.data().text}}
buildFiles().then(res=>this.files = res)
Thanks also to #cyqsimon and #Nav Kumar V
I am struggling to understand how to return a single array from a function that calls another function several times. Currently, the console.log in the code below outputs a growing array each time the scrapingfunction runs.
The final last time that the scrapingfunction runs is actually what I want, but I want to find a way to return a single array at the end of the hello function so that I can drop each object into my database. I'm guessing this is just not me understanding javascript well enough yet.
const hello = async () => {
//[launch puppeteer and navigate to page I want to scrape]
await scrapingfunction(page)
//[navigate to the next page I want to scrape]
await scrapingfunction(page)
//[navigate to the next page I want to scrape]
await scrapingfunction(page)
}
const scrapingfunction = async (page) => {
const html = await page.content()
const $ = cheerio.load(html)
const data = $('.element').map((index, element)=>{
const dateElement = $(element).find('.sub-selement')
const date = dateElement.text()
return {date}
}).get()
console.log(data)
}
hello();
The problem you've encountered is that Promises (which is what async/await uses "under the covers") cannot return values outside the Promise chain.
Think of it this way.
You ask me to write a StackOverflow article for you and immediately demand the result of that task, without waiting for me to finish it.
When you set me the task, I haven't yet completed it, so I cannot provide a response.
You will need to restructure your request to return values from your awaits which can then be operated upon by the surrounding async function, such as:
# Assume doubleValue() takes some unknown time to return a result, like
# waiting for the result of an HTTP query.
const doubleValue = async (val) => return val * 2
const run = async () => {
const result = []
result.push(await doubleValue(2))
result.push(await doubleValue(4))
result.push(await doubleValue(8))
console.log(result)
}
which will print [4, 8, 16] to the console.
You might think you could return the result from run() and print it to the console as in:
const run = async () => {
const result = []
result.push(await doubleValue(2))
result.push(await doubleValue(4))
result.push(await doubleValue(8))
return result
}
console.log(run())
But since Node has no idea when run() has everything it needs to create a result, console.log will not print anything. (That's not expressly true since an async function returns a Promise, but the explanation works for this example.)
The rule is that you can await the result of other functions from within a function marked as async, but you cannot return any useful result to its surrounding context.
Since an async function does return a Promise, you could:
run().then(result => console.log(result))
But note that the result never leaves the Promise chain.
Is there a pattern for making a stream iterable using ES6 generators?
See 'MakeStreamIterable' below.
import {createReadStream} from 'fs'
let fileName = 'largeFile.txt'
let readStream = createReadStream(fileName, {
encoding: 'utf8',
bufferSize: 1024
})
let myIterableAsyncStream = MakeStreamIterable(readStream)
for (let data of myIterableAsyncStream) {
let str = data.toString('utf8')
console.log(str)
}
I'm not interested in co or bluebird's coroutine or blocking with deasync.
The gold is MakeStreamIterable should be a valid function.
Is there a pattern for making a stream iterable using ES6 generators?
No, this cannot be achieved because generators are synchronous. They must know what they are yielding and when. Iteration of an asynchronous data source can only currently be achieved by using some kind of callback-based implementation. So, there is no way to make MakeStreamIterable 'a valid function' if what you mean by this is 'a valid function whose result can be given to a for-of loop'.
Streams are Asynchronous
A stream represents a potentially infinite amount of data received asynchronously over a potentially infinite amount of time. If we take a look at the definition of an iterator on MDN we can define in more detail what it is about a stream that makes it 'uniterable':
An object is an iterator when it knows how to access items from a collection one at a time, while keeping track of its current position within that sequence. In JavaScript an iterator is an object that provides a next() method which returns the next item in the sequence. This method returns an object with two properties: done and value.
(Emphasis is my own.)
Let's pick out the properties of an iterable from this definition. The object must...
know how to access items from a collection one at a time;
be able to keep track of its current position within the sequence of data;
and provide a method, next, that retrieves an object with a property that holds the next value in the sequence or notifies that iteration is done.
A stream doesn't meet any of the above criteria because...
it is not in control of when it receives data and cannot 'look into the future' to find the next value;
it cannot know when or if it has received all data, only when the stream has closed;
and it does not implement the iterable protocol and so does not expose a next method which a for-of can utilise.
______
Faking It(eration)
We can't actually iterate the data received from a stream (definitely not using a for-of), however we can build an interface that pretends to by using Promises (yay!) and abstracting away the stream's event handlers inside a closure.
// MakeStreamIterable.js
export default function MakeStreamIterable (stream) {
let collection = []
let index = 0
let callback
let resolve, reject
stream
.on('error', err => reject && reject(err))
.on('end', () => resolve && resolve(collection))
.on('data', data => {
collection.push(data)
try {
callback && callback(data, index++)
} catch (err) {
this.end()
reject(err)
}
})
function each (cb) {
if(callback) {
return promise
}
callback = (typeof cb === 'function') ? cb : null
if (callback && !!collection) {
collection.forEach(callback)
index = collection.length
}
return promise
}
promise = new Promise((res, rej) => {
resolve = res
reject = rej
})
promise.each = each
return promise
}
And we can use it like this:
import {MakeStreamIterable} from './MakeStreamIterable'
let myIterableAsyncStream = MakeStreamIterable(readStream)
myIterableAsyncStream
.each((data, i) => {
let str = data.toString('utf8')
console.log(i, str)
})
.then(() => console.log('completed'))
.catch(err => console.log(err))
Things to note about this implementation:
It is not necessary to call each immediately on the 'iterable stream'.
When each is called, all values received prior to its call are passed to the callback one-by-one forEach-style. Afterwards all subsequent data are passed immediately to the callback.
The function returns a Promise which resolves the complete collection of data when the stream ends, meaning we actually don't have to call each at all if the method of iteration provided by each isn't satisfactory.
I have fostered the false semantics of calling this an iterator and am therefore an awful human being. Please report me to the relevant authority.
Soon you are going to be able to use Async Iterators and Generators. In node 9.8 you can use it by running with --harmony command line option.
async function* streamAsyncIterator(stream) {
// Get a lock on the stream
const reader = stream.getReader();
try {
while (true) {
// Read from the stream
const {done, value} = await reader.read();
// Exit if we're done
if (done) return;
// Else yield the chunk
yield value;
}
}
finally {
reader.releaseLock();
}
}
async function example() {
const response = await fetch(url);
for await (const chunk of streamAsyncIterator(response.body)) {
// …
}
}
Thanks to Jake Archibald for the examples above.
2020 Update:
It looks like streams will be "natively" iterable in the future - just waiting on browsers to implement it:
https://streams.spec.whatwg.org/#rs-asynciterator
https://github.com/whatwg/streams/issues/778
https://bugs.chromium.org/p/chromium/issues/detail?id=929585
https://bugzilla.mozilla.org/show_bug.cgi?id=1525852
https://bugs.webkit.org/show_bug.cgi?id=194379
for await (const chunk of stream) {
...
}