Develop Reference

Why is async function slowing down my image loading process?

I have written an image-loading function like this:
export function loadImage(url: string): Promise<HTMLImageElement> {
return new Promise((resolve, reject) => {
if (image_map.has(url)) {
resolve(image_map.get(url));
} else {
let image: HTMLImageElement = new Image();
if (typeof window.static_url != 'undefined' && url[0] == '/') {
image.src = `${window.static_url}${url}`;
} else {
image.src = url;
}
image_map.set(url, image);
image.crossOrigin = 'Anonymous'; // Prevent canvas getImageData CORS issue
image.onload = function () {
resolve(image);
};
image.onerror = reject;
}
});
}
I have about 200 images to load, and prior to asking this question, I have been writing code like this:
for (let url of image_list) {
await loadImage(url);
}
And it took me ages (16s or more) to load all these images.
Today I decided to remove the await from my code, and miracle happened: the image loading process finished within 2s. How is that even possible? I thought javascript is single-threaded, and I had expected that, by removing await, the image loading should simply take place after the rest of my code, instead of taking place before, but the total time for them to load should not differ so greatly.

Your current code loads the images serially (one after the other).
You can instead execute them in parallel using promise all.
// 1. simultaneously kick off load image on each url
const promises = image_list.map(loadImage);
// 2. await them all
await Promise.all(promises);
// 3. Images have loaded.
Or more succinctly.
await Promise.all(image_list.map(loadImage));

Best way to to use async/promise/callbacks in for loop [duplicate]

This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 2 years ago.
I'm building a trading bot that needs to get stock names from separate files. But even I have used async function and await in my code, that doesn't work.
My index file init method.
const init = async () => {
const symbols = await getDownTrendingStock();
console.log("All stocks that are down: " + symbols);
const doOrder = async () => {
//do stuff
}
doOrder();
}
my getDownTrendeingStock file
const downStocks = []
function getDownTrendingStock () {
for(i = 0; i < data.USDTPairs.length; i++){
const USDTPair = data.USDTPairs[i] + "USDT";
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if(prevDay.priceChangePercent < -2){
downStocks.push(symbol)
}
});
}
return downStocks;
}
I have tried to also use async in for loop because the getDownTrendinStock function returns an empty array before for loop is finished. I didn't find the right way to do that because I was confused with all async, promise and callback stuff. What is the right statement to use in this situation?
Output:
All stocks that are down:
Wanted output:
All stocks that are down: [BTCUSDT, TRXUSDT, ATOMUSDT...]

I think the main issue in the code you posted is that you are mixing callbacks and promises.
This is what's happening in your getDownTrendingStock function:
You start iterating over the data.USDTPairs array, picking the first element
You call binance.prevDay. This does nothing yet, because its an asynchronous function that takes a bit of time. Notably, no data is added to downStocks yet.
You continue doing 1-2, still, no data is added
You return downStocks, which is still empty.
Your function is complete, you print the empty array
Now, at some point, the nodejs event loop continues and starts working on those asynchronous tasks you created earlier by calling binance.prevDay. Internally, it probably calls an API, which takes time; once that call is completed, it calls the function you provided, which pushes data to the downStocks array.
In summary, you didn't wait for the async code to complete. You can achieve this in multiple ways.
One is to wrap this in a promise and then await that promise:
const result= await new Promise((resolve, reject) => {
binance.prevDay(USDTPair, (error, prevDay, symbol) => {
if (error) {
reject(error);
} else {
resolve({prevDay, symbol});
}
});
});
if(result.prevDay.priceChangePercent < -2){
downStocks.push(result.symbol)
}
Note that you can probably also use promisify for this. Also, this means that you will wait for one request to finish before starting the next, which may slow down your code considerably, depending on how many calls you need; you may want to look into Promise.all as well.

Generally speaking, I use two technics:
const asyncFunc = () => {smthAsync};
const arrayToProcess = [];
// 1
const result = await arrayToProcess.reduce((acc, value) => acc.then(() => asyncFunc(value)), Promise.resolve(someInitialValue));
// 2
// here will be eslint errors
for(let i = 0 i < arrayToProcess.length; i+=1) {
const processResult = await asyncFunc(value);
// do with processResult what you want
};

await loop vs Promise.all [duplicate]

This question already has answers here:
Any difference between await Promise.all() and multiple await?
(6 answers)
Closed 4 years ago.
Having a set of async operations on db to do, I'm wondering what's the difference performance-wise of doing a "blocking" await loop versus a Promise.all.
let insert = (id,value) => {
return new Promise(function (resolve, reject) {
connnection.query(`insert into items (id,value) VALUES (${id},"${value}")`, function (err, result) {
if (err) return reject(err)
return resolve(result);
});
});
};
Promise.all solution (it needs a for loop to builds the array of promises..)
let inserts = [];
for (let i = 0; i < SIZE; i++) inserts.push(insert(i,"..string.."))
Promise.all(inserts).then(values => {
console.log("promise all ends");
});
await loop solution
let inserts = [];
(async function loop() {
for (let i = 0; i < SIZE; i++) {
await insert(i, "..string..")
}
console.log("await loop ends");
})
Edit: thanks for the anwsers, but I would dig into this a little more.
await is not really blocking, we all know that, it's blocking in its own code block. An await loop sequentially fire requests, so if in the middle 1 requests takes longer, the other ones waits for it.
Well this is similar to Promise.all: if a 1 req takes longer, the callback is not executed until ALL the responses are returned.

Your example of using Promise.all will create all promises first before waiting for them to resolve. This means that your requests will fire concurrently and the callback given to Promise.all(...).then(thisCallback) will only fire if all requests were successful.
Note: promise returned from Promise.all will reject as soon as one of the promises in the given array rejects.
const SIZE = 5;
const insert = i => new Promise(resolve => {
console.log(`started inserting ${i}`);
setTimeout(() => {
console.log(`inserted ${i}`);
resolve();
}, 300);
});
// your code
let inserts = [];
for (let i = 0; i < SIZE; i++) inserts.push(insert(i, "..string.."))
Promise.all(inserts).then(values => {
console.log("promise all ends");
});
// requests are made concurrently
// output
// started inserting 0
// started inserting 1
// started inserting 2
// ...
// started inserting 4
// inserted 0
// inserted 1
// ...
// promise all ends
Note: It might be cleaner to use .map instead of a loop for this scenario:
Promise.all(
Array.from(Array(SIZE)).map((_, i) => insert(i,"..string.."))
).then(values => {
console.log("promise all ends");
});
Your example of using await on the other hand, waits for each promise to resolve before continuing and firing of the next one:
const SIZE = 5;
const insert = i => new Promise(resolve => {
console.log(`started inserting ${i}`);
setTimeout(() => {
console.log(`inserted ${i}`);
resolve();
}, 300);
});
let inserts = [];
(async function loop() {
for (let i = 0; i < SIZE; i++) {
await insert(i, "..string..")
}
console.log("await loop ends");
})()
// no request is made until the previous one is finished
// output
// started inserting 0
// inserted 0
// started inserting 1
// ...
// started inserting 4
// inserted 4
// await loop ends
The implications for performance in the above cases are directly correlated to their different behavior.
If "efficient" for your use case means to finish up the requests as soon as possible, then the first example wins because the requests will be happening around the same time, independently, whereas in the second example they will happen in a serial fashion.
In terms of complexity, the time complexity for your first example is equal to O(longestRequestTime) because the requests will happen essentially in parallel and thus the request taking the longest will drive the worst-case scenario.
On the other hand, the await example has O(sumOfAllRequestTimes) because no matter how long individual requests take, each one has to wait for the previous one to finish and thus the total time will always include all of them.
To put things in numbers, ignoring all other potential delays due to the environment and application in which the code is ran, for 1000 requests, each taking 1s, the Promise.all example would still take ~1s while the await example would take ~1000s.
Maybe a picture would help:
Note: Promise.all won't actually run the requests exactly in parallel and the performance in general will greatly depend on the exact environment in which the code is running and the state of it (for instance the event loop) but this is a good approximation.

The major difference between the two approaches is that
The await version issues server requests sequentially in the loop. If one of them errors without being caught, no more requests are issued. If request errors are trapped using try/catch blocks, you can identify which request failed and perhaps code in some some form of recovery or even retry the operation.
The Promise.all version will make server requests in or near parallel fashion, limited by browser restrictions on the maximum number of concurrent requests permitted. If one of the requests fails the Promise.all returned promise fails immediately. If any requests were successful and returned data, you lose the data returned. In addition if any request fails, no outstanding requests are cancelled - they were initiated in user code (the insert function) when creating the array of promises.
As mentioned in another answer, await is non blocking and returns to the event loop until its operand promise is settled. Both the Promise.all and await while looping versions allow responding to other events while requests are in progress.

Each has different advantages, it's up to us which one we need to solve our problem.
await loop
for(let i = 0;i < SIZE; i++){
await promiseCall();
}
It will call all promises in parallel if any promise rejected it won't have any effect on other promises.
In ES2018 it has simplified for certain situation like if you want to call the second iteration only if the first iteration got finished, refer the following ex.
async function printFiles () {
const files = await getFilePaths()
for await (const file of fs.readFile(file, 'utf8')) {
console.log(contents)
}
}
Promise.all()
var p1 = Promise.resolve(32);
var p2 = 123;
var p3 = new Promise((resolve, reject) => {
setTimeout(() => {
resolve("foo");
}, 100);
});
Promise.all([p1, p2, p3]).then(values => {
console.log(values); // [32, 123, "foo"]
});
This will execute every promise sequentially and finally return combined revolved values array.
If any one of these promise get rejected it will return value of that rejected promise only. follow following ex,
var p1 = Promise.resolve(32);
var p2 = Promise.resolve(123);
var p3 = new Promise((resolve, reject) => {
setTimeout(() => {
resolve("foo");
}, 100);
});
Promise.all([p1, p2, p3]).then(values => {
console.log(values); // 123
});

Node.js line by line with async operation

I am attempting to read a stream line by line and for each line do some async processing. The issue I'm having is how to determine when the operations for all lines are complete. I thought I could use the readline "close" event but that seems to be triggered long before the async operations I've started on the "line" event complete.
Here's the event handlers in question:
logReader.on("line", inputLine => {
self._processLogLine(inputLine, () => {
if (self.streamFinished) {
completionCallback(err);
}
});
});
logReader.on("close", () => {
self.streamFinished = true;
});
The completionCallback method should be called when all line processing on the stream is done.
I've thought about adding counters for line operations started/completed and calling the completionCallback when they match but that seems awkward.

Here is one way you could do it basically keeping track of each lines success. I don't have enough of your code to test it...
const lineProcessing = [];
const doCompletion = () => {
if (self.streamFinished && !lineProcessing.includes(false)) {
// Not sure where 'err' comes from
completionCallback(err);
}
}
logReader.on("line", inputLine => {
lineProcessing.push(false);
let processingIndex = lineProcessing.length - 1;
self._processLogLine(inputLine, () => {
lineProcessing[processingIndex] = true;
doCompletion();
});
});
logReader.on("close", () => {
self.streamFinished = true;
});

It's a little difficult to tell for sure what needs to happen without more context. But here's a quick and somewhat dirty fix that should be adaptable to your situation. It shows how you can accomplish this with promises:
let promises = []
logReader.on("line", inputLine => {
promises.push(new Promise(resolve => {
self._processLogLine(inputLine, resolve)
})
})
logReader.on("close", () => {
Promise.all(promises)
.then(completionCallback)
})
Essentially, just create an array of promises. When you know that all promises have been added to the array, call Promise.all() on it.
If you use this approach, you'd probably want to add error checking (rejecting and catching).

Look into generators. They give you a way to do a "for" loop...and when you're done, you just return from the generator function.
So you can have a generator to read the file, and a function that uses the generator, then when the loop ends, call your callback.

JS Promise - instantly retrieve some data from a function that returns a Promise

Can anyone recommend a pattern for instantly retrieving data from a function that returns a Promise?
My (simplified) example is an AJAX preloader:
loadPage("index.html").then(displayPage);
If this is downloading a large page, I want to be able to check what's happening and perhaps cancel the process with an XHR abort() at a later stage.
My loadPage function used to (before Promises) return an id that let me do this later:
var loadPageId = loadPage("index.html",displayPage);
...
doSomething(loadPageId);
cancelLoadPage(loadPageId);
In my new Promise based version, I'd imagine that cancelLoadPage() would reject() the original loadPage() Promise.
I've considered a few options all of which I don't like. Is there a generally accepted method to achieve this?

Okay, let's address your bounty note first.
[Hopefully I'll be able to grant the points to someone who says more than "Don't use promises"... ]
Sorry, but the answer here is: "Don't use promises". ES6 Promises have three possible states (to you as a user): Pending, Resolved and Rejected (names may be slightly off).
There is no way for you to see "inside" of a promise to see what has been done and what hasn't - at least not with native ES6 promises. There was some limited work (in other frameworks) done on promise notifications, but those did not make it into the ES6 specification, so it would be unwise of you to use this even if you found an implementation for it.
A promise is meant to represent an asynchronous operation at some point in the future; standalone, it isn't fit for this purpose. What you want is probably more akin to an event publisher - and even that is asynchronous, not synchronous.
There is no safe way for you to synchronously get some value out of an asynchronous call, especially not in JavaScript. One of the main reasons for this is that a good API will, if it can be asynchronous, will always be asynchronous.
Consider the following example:
const promiseValue = Promise.resolve(5)
promiseValue.then((value) => console.log(value))
console.log('test')
Now, let's assume that this promise (because we know the value ahead of time) is resolved synchronously. What do you expect to see? You'd expect to see:
> 5
> test
However, what actually happens is this:
> test
> 5
This is because even though Promise.resolve() is a synchronous call that resolves an already-resolved Promise, then() will always be asynchronous; this is one of the guarantees of the specification and it is a very good guarantee because it makes code a lot easier to reason about - just imagine what would happen if you tried to mix synchronous and asynchronous promises.
This applies to all asynchronous calls, by the way: any action in JavaScript that could potentially be asynchronous will be asynchronous. As a result, there is no way for you do any kind of synchronous introspection in any API that JavaScript provides.
That's not to say you couldn't make some kind of wrapper around a request object, like this:
function makeRequest(url) {
const requestObject = new XMLHttpRequest()
const result = {
}
result.done = new Promise((resolve, reject) => {
requestObject.onreadystatechange = function() {
..
}
})
requestObject.open(url)
requestObject.send()
return requestObject
}
But this gets very messy, very quickly, and you still need to use some kind of asynchronous callback for this to work. This all falls down when you try and use Fetch. Also note that Promise cancellation is not currently a part of the spec. See here for more info on that particular bit.
TL:DR: synchronous introspection is not possible on any asynchronous operation in JavaScript and a Promise is not the way to go if you were to even attempt it. There is no way for you to synchronously display information about a request that is on-going, for example. In other languages, attempting to do this would require either blocking or a race condition.

Well. If using angular you can make use of the timeout parameter used by the $http service if you need to cancel and ongoing HTTP request.
Example in typescript:
interface ReturnObject {
cancelPromise: ng.IPromise;
httpPromise: ng.IHttpPromise;
}
#Service("moduleName", "aService")
class AService() {
constructor(private $http: ng.IHttpService
private $q: ng.IQService) { ; }
doSomethingAsynch(): ReturnObject {
var cancelPromise = this.$q.defer();
var httpPromise = this.$http.get("/blah", { timeout: cancelPromise.promise });
return { cancelPromise: cancelPromise, httpPromise: httpPromise };
}
}
#Controller("moduleName", "aController")
class AController {
constructor(aService: AService) {
var o = aService.doSomethingAsynch();
var timeout = setTimeout(() => {
o.cancelPromise.resolve();
}, 30 * 1000);
o.httpPromise.then((response) => {
clearTimeout(timeout);
// do code
}, (errorResponse) => {
// do code
});
}
}
Since this approach already returns an object with two promises the stretch to include any synchronous operation return data in that object is not far.
If you can describe what type of data you would want to return synchronously from such a method it would help to identify a pattern. Why can it not be another method that is called prior to or during your asynchronous operation?

You can kinda do this, but AFAIK it will require hacky workarounds. Note that exporting the resolve and reject methods is generally considered a promise anti-pattern (i.e. sign you shouldn't be using promises). See the bottom for something using setTimeout that may give you what you want without workarounds.
let xhrRequest = (path, data, method, success, fail) => {
const xhr = new XMLHttpRequest();
// could alternately be structured as polymorphic fns, YMMV
switch (method) {
case 'GET':
xhr.open('GET', path);
xhr.onload = () => {
if (xhr.status < 400 && xhr.status >= 200) {
success(xhr.responseText);
return null;
} else {
fail(new Error(`Server responded with a status of ${xhr.status}`));
return null;
}
};
xhr.onerror = () => {
fail(networkError);
return null;
}
xhr.send();
return null;
}
return xhr;
case 'POST':
// etc.
return xhr;
// and so on...
};
// can work with any function that can take success and fail callbacks
class CancellablePromise {
constructor (fn, ...params) {
this.promise = new Promise((res, rej) => {
this.resolve = res;
this.reject = rej;
fn(...params, this.resolve, this.reject);
return null;
});
}
};
let p = new CancellablePromise(xhrRequest, 'index.html', null, 'GET');
p.promise.then(loadPage).catch(handleError);
// times out after 2 seconds
setTimeout(() => { p.reject(new Error('timeout')) }, 2000);
// for an alternative version that simply tells the user when things
// are taking longer than expected, NOTE this can be done with vanilla
// promises:
let timeoutHandle = setTimeout(() => {
// don't use alert for real, but you get the idea
alert('Sorry its taking so long to load the page.');
}, 2000);
p.promise.then(() => clearTimeout(timeoutHandle));

Promises are beautiful. I don't think there is any reason that you can not handle this with promises. There are three ways that i can think of.
The simplest way to handle this is within the executer. If you would like to cancel the promise (like for instance because of timeout) you just define a timeout flag in the executer and turn it on with a setTimeout(_ => timeout = true, 5000) instruction and resolve or reject only if timeout is false. ie (!timeout && resolve(res) or !timeout && reject(err)) This way your promise indefinitely remains unresolved in case of a timeout and your onfulfillment and onreject functions at the then stage never gets called.
The second is very similar to the first but instead of keeping a flag you just invoke reject at the timeout with proper error description. And handle the rest at the then or catch stage.
However if you would like to carry the id of your asych operation to the sync world then you can also do it as follows;
In this case you have to promisify the async function yourself. Lets take an example. We have an async function to return the double of a number. This is the function
function doubleAsync(data,cb){
setTimeout(_ => cb(false, data*2),1000);
}
We would like to use promises. So normally we need a promisifier function which will take our async function and return another function which when run, takes our data and returns a promise. Right..? So here is the promisifier function;
function promisify(fun){
return (data) => new Promise((resolve,reject) => fun(data, (err,res) => err ? reject(err) : resolve(res)));
}
Lets se how they work together;
function promisify(fun){
return (data) => new Promise((resolve,reject) => fun(data, (err,res) => err ? reject(err) : resolve(res)));
}
function doubleAsync(data,cb){
setTimeout(_ => cb(false, data*2),1000);
}
var doubleWithPromise = promisify(doubleAsync);
doubleWithPromise(100).then(v => console.log("The asynchronously obtained result is: " + v));
So now you see our doubleWithPromise(data) function returns a promise and we chain a then stage to it and access the returned value.
But what you need is not only a promise but also the id of your asynch function. This is very simple. Your promisified function should return an object with two properties; a promise and an id. Lets see...
This time our async function will return a result randomly in 0-5 secs. We will obtain it's result.id synchronously along with the result.promise and use this id to cancel the promise if it fails to resolve within 2.5 secs. Any figure on console log Resolves in 2501 msecs or above will result nothing to happen and the promise is practically canceled.
function promisify(fun){
return function(data){
var result = {id:null, promise:null}; // template return object
result.promise = new Promise((resolve,reject) => result.id = fun(data, (err,res) => err ? reject(err) : resolve(res)));
return result;
};
}
function doubleAsync(data,cb){
var dur = ~~(Math.random()*5000); // return the double of the data within 0-5 seconds.
console.log("Resolve in " + dur + " msecs");
return setTimeout(_ => cb(false, data*2),dur);
}
var doubleWithPromise = promisify(doubleAsync),
promiseDataSet = doubleWithPromise(100);
setTimeout(_ => clearTimeout(promiseDataSet.id),2500); // give 2.5 seconds to the promise to resolve or cancel it.
promiseDataSet.promise
.then(v => console.log("The asynchronously obtained result is: " + v));

You can use fetch(), Response.body.getReader(), where when .read() is called returns a ReadableStream having a cancel method, which returns a Promise upon cancelling read of the stream.
// 58977 bytes of text, 59175 total bytes
var url = "https://gist.githubusercontent.com/anonymous/"
+ "2250b78a2ddc80a4de817bbf414b1704/raw/"
+ "4dc10dacc26045f5c48f6d74440213584202f2d2/lorem.txt";
var n = 10000;
var clicked = false;
var button = document.querySelector("button");
button.addEventListener("click", () => {clicked = true});
fetch(url)
.then(response => response.body.getReader())
.then(reader => {
var len = 0;
reader.read().then(function processData(result) {
if (result.done) {
// do stuff when `reader` is `closed`
return reader.closed.then(function() {
return "stream complete"
});
};
if (!clicked) {
len += result.value.byteLength;
}
// cancel stream if `button` clicked or
// to bytes processed is greater than 10000
if (clicked || len > n) {
return reader.cancel().then(function() {
return "read aborted at " + len + " bytes"
})
}
console.log("len:", len, "result value:", result.value);
return reader.read().then(processData)
})
.then(function(msg) {
alert(msg)
})
.catch(function(err) {
console.log("err", err)
})
});
<button>click to abort stream</button>

The method I am currently using is as follows:
var optionalReturnsObject = {};
functionThatReturnsPromise(dataToSend, optionalReturnsObject ).then(doStuffOnAsyncComplete);
console.log("Some instant data has been returned here:", optionalReturnsObject );
For me, the advantage of this is that another member of my team can use this in a simple way:
functionThatReturnsPromise(data).then(...);
And not need to worry about the returns object. An advanced user can see from the definitions what is going on.