Develop Reference

JavaScript - Attach context to async function call?

The bounty expires in 19 hours. Answers to this question are eligible for a +100 reputation bounty.
David Callanan wants to draw more attention to this question.
Synchronous function call context
In JavaScript, it's easy to associate some context with a synchronous function call by using a stack in a global scope.
// Context management
let contextStack = [];
let context;
const withContext = (ctx, func) {
contextStack.push(ctx);
context = ctx;
try {
return func();
} finally {
context = contextStack.pop();
}
};
// Example
const foo = (message) => {
console.log(message);
console.log(context);
};
const bar = () => {
withContext("calling from bar", () => foo("hello"));
};
This allows us to write context-specific code without having to pass around a context object everywhere and have every function we use depend on this context object.
This is possible in JavaScript because of the guarantee of sequential code execution, that is, these synchronous functions are run to completion before any other code can modify the global state.
Generator function call context
We can achieve something similar with generator functions. Generator functions give us an opportunity to take control just before conceptual execution of the generator function resumes. This means that even if execution is suspended for a few seconds (that is, the function is not run to completion before any other code runs), we can still ensure that there is an accurate context attached to its execution.
const iterWithContext = function* (ctx, generator) {
// not a perfect implementation
let iter = generator();
let reply;
while (true) {
const { done, value } = withContext(ctx, () => iter.next(reply));
if (done) {
return;
}
reply = yield value;
}
};
Question: Async function call context?
It would also be very useful to attach some context to the execution of an async function.
const timeout = (ms) => new Promise(res => setTimeout(res, ms));
const foo = async () => {
await timeout(1000);
console.log(context);
};
const bar = async () => {
await asyncWithContext("calling from bar", foo);
};
The problem is, to the best of my knowledge, there is no way of intercepting the moment before an async function resumes execution, or the moment after the async function suspends execution, in order to provide this context.
Is there any way of achieving this?
My best option right now is to not use async functions, but to use generator functions that behave like async functions. But this is not very practical as it requires the entire codebase to be written like this.
Background / Motivation
Using context like this is incredibly valuable because the context is available deep down the call-stack. This is especially useful if a library needs to call an external handler such that if the handler calls back to the library, the library will have the appropriate context. For example, I'd imagine React hooks and Solid.js extensively use context in this way under-the-hood. If not done this way, the programmer would have to pass a context object around everywhere and use it when calling back to the library, which is both messy and error-prone. Context is a way to neatly "curry" or abstract away a context object from function calls, based on where we are in the call stack. Whether it is good practice or not is debatable, but I think we can agree that it's something library authors have chosen to do. I would like to extend the use of context to asynchronous functions, which are supposed to conceptually behave like synchronous functions when it comes to the execution flow.

As far as I know ECMA has no specification for "contexts" (regardless if it's a normal or async function).
Therefore the solution you posted for normal functions is already a hack.
As per ECMA standard, there is no JavaScript based API to hook await in order to do a generator like trick. So you have to rely on (environment based) hacks.
These hacks may highly depend on the environment you're using.
JavaScript Only (requires async stack traces)
A solution which is purly based async stack traces is the following one.
Since nearly every JavaScript interpreter is based on V8 this works on nearly every use case.
const kContextIdFunctionPrefix = "__context_id__";
const kContextIdRegex = new RegExp(`${kContextIdFunctionPrefix}([0-9]+)`);
let contextIdOffset = 0;
function runWithContextId(target, ...args) {
const contextId = ++contextIdOffset;
let proxy;
eval(`proxy = async function ${kContextIdFunctionPrefix}${contextId}(target, ...args){ return await target.call(this, ...args); }`);
return proxy.call(this, target, ...args);
}
function getContextId() {
const stack = new Error().stack.split("\n");
for(const frame of stack) {
const match = frame.match(kContextIdRegex);
if(!match) {
continue;
}
const id = parseInt(match[1]);
if(isNaN(id)) {
console.warn(`Context id regex matched, but failed to parse context id from ${match[1]}`);
continue;
}
return id;
}
console.log(new Error().stack)
throw new Error("getContextId() called without providing a context (runWithContextId(...))");
}
A simple demo:
async function main() {
const target = async () => {
const contextId = getContextId();
console.log(`Context Id: ${contextId}`);
await new Promise(resolve => setTimeout(resolve, 1000));
console.log(`Context Id (After await): ${getContextId()} (before: ${contextId})`);
return contextId;
};
const contextIdA = runWithContextId(target);
const contextIdB = runWithContextId(target);
// Note: We're first awaiting the second call!
console.log(`Invoke #2 context id: ${await contextIdB}`);
console.log(`Invoke #1 context id: ${await contextIdA}`);
}
main();
This solution leverages stack traces in order to identify a context id. Traversing the (sync and async) stack trace and using dynamically generated functions with special names allows to pass a special value (a number in this instance).
NodeJS (AsyncLocalStorage)
NodeJS offers a way for Asynchronous context tracking:
https://nodejs.org/api/async_context.html#class-asynclocalstorage
It should be possible to build an async context by using AsyncLocalStorage.
Using a transpiler
You might want to use a transpiler (like babel or typescript) which convert async functions to generator functions on the fly.
Using a transpiler allows you to even write a plugin for implementing async contexts based on generator functions.

Undefined value for input fill

Follow my previous article Built method time is not a function, I managed to successfully implement the functions with an appropriate wait time by following a combination of #ggorlen's comment and #Konrad Linkowski answer, additionally, this article puppeteer: wait N seconds before continuing to the next line that #ggorlen answered in, this comment especially helped: -
Something else? Run an evaluate block and add your own code to wait for a DOM mutation or poll with setInterval or requestAnimationFrame and effectively reimplement waitForFunction as fits your needs.
Instead I incorporated waitForSelector, produces the following script:
const puppeteer = require('puppeteer')
const EXTENSION = '/Users/usr/Library/Application Support/Google/Chrome/Profile 1/Extensions/gidnphnamcemailggkemcgclnjeeokaa/1.14.4_0'
class Agent {
constructor(extension) {
this._extension = extension
}
async runBrowser() {
const browser = await puppeteer.launch({
headless:false,
devtools:true,
args:[`--disable-extensions-except=${this._extension}`,
`--load-extension=${this._extension}`,
'--enable-automation']
})
return browser
}
async getPage(twitch) {
const page = await (await this.runBrowser()).newPage()
await page.goto('chrome-extension://gidnphnamcemailggkemcgclnjeeokaa/popup.html')
const nextEvent = await page.evaluate(async () => {
document.getElementById('launch-trace').click()
})
const waitSelector = await page.waitForSelector('.popup-body')
const finalEvent = (twitch) => new Promise(async (twitch) => page.evaluate(async (twitch) => {
const input = document.getElementById('user-trace-id')
input.focus()
input.value = twitch
}))
await finalEvent(twitch)
}
}
const test = new Agent(EXTENSION)
test.getPage('test')
However, my webpage produces undefined rather than test, I am a little confused by the parameters twich and k, and how to properly assert the parameter twitch so its entered inside the function finalEvent.
Alternatively, I have also tried wrapping finalEvent into a Promise so I can assert the parameter twitch into it as a function, but this does not fill any value:
const finalEvent = (val) => new Promise(async () => await page.evaluate(async () => {
const nextTime = () => new Promise(async () => setInterval(async () => {
const input = document.getElementById('user-trace-id')
input.focus()
input.value = val
}, 3000))
//await nextTime(k)
}))
await finalEvent(twitch)

There are a few issues here. First,
const page = await (await this.runBrowser()).newPage()
hangs the browser handle and leaks memory which keeps the process alive. Always close the browser when you finish using it:
const browser = await this.runBrowser();
const page = await browser.newPage();
// ... do your work ...
await browser.close();
Here, though, Puppeteer can throw, again leaking the browser and preventing your app from cleanly exiting, so I suggest adding a try/catch block with a finally block that closes the browser.
Generally speaking, try to get the logic working first, then do a refactor to break code into functions and classes. Writing abstractions and thinking about design while you're still battling bugs and logical problems winds up making both tasks harder.
Secondly, there's no need to async a function if you never use await in it, as in:
const nextEvent = await page.evaluate(async () => {
document.getElementById('launch-trace').click()
})
Here, nextEvent is undefined because evaluate()'s callback returned nothing. Luckily, you didn't attempt to use it. You also have const waitSelector = await page.waitForSelector('.popup-body') which does return the element, but it goes unused. I suggest enabling eslint no-unused-vars, because these unused variables make a confusing situation worse and often indicate typos and bugs.
On to the main problem,
const finalEvent = (twitch) => new Promise(async (twitch) => page.evaluate(async (twitch) => {
const input = document.getElementById('user-trace-id')
input.focus()
input.value = twitch
}))
await finalEvent(twitch)
There are a number of misunderstandings here.
The first is the age-old Puppeteer gotcha, confusing which code executes in the browser process and which code executes in the Node process. Everything in an evaluate() callback (or any of its family, $eval, evaluateHandle, etc) executes in the browser, so Node variables that look like they should be in scope won't be. You have to pass and return serializable data or element handles to and from these callbacks. In this case, twitch isn't in scope of the evaluate callback. See the canonical How can I pass a variable into an evaluate function? for details.
The second misunderstanding is technically cosmetic in that you can make the code work with it, but it's a serious code smell that indicates significant confusion and should be fixed. See What is the explicit promise construction antipattern and how do I avoid it? for details, but the gist is that when you're working with a promise-based API like Puppeteer, you should never need to use new Promise(). Puppeteer's methods already return promises, so it's superfluous at best to wrap more promises on top of the them, and at worst, introduces bugs and messes up error handling.
A third issue is that the first parameter to new Promise((resolve, reject) => {}) is always a resolve function, so twitch is a confusing mislabel. Luckily, it won't matter as we'll be dispensing with the new Promise idiom when using Puppeteer 99.9% of the time.
So let's fix the code, keeping these points in mind:
await page.evaluate(twitch => {
const input = document.getElementById('user-trace-id');
input.focus();
input.value = twitch;
},
twitch
);
Note that I'm not assigning the return value to anything because there's nothing being returned by the evaluate() callback.
"Selecting, then doing something with the selected element" is such a common pattern that Puppeteer provides a handy method to shorten the above code:
await page.$eval("#user-trace-id", (input, twitch) => {
input.focus();
input.value = twitch;
},
twitch
);
Now, I can't run or reproduce your code as I don't have your extension, and I'm not sure what goal you're trying to achieve, but even the above code looks potentially problematic.
Usually, you want to use Puppeteer's page.type() method rather than a raw DOM input.value = ..., which doesn't fire any event handlers that might be attached to the input. Many inputs won't register such a change, and it's an untrusted event.
Also, it's weird that you'd have to .focus() on the input before setting its value. Usually focus is irrelevant to setting a value property, and the value will be set either way.
So there may be more work to do, but hopefully this will point you in the right direction by resolving the first layer of immediate issues at hand. If you're still stuck, I suggest taking a step back and providing context in your next question of what you're really trying to accomplish here in order to avoid an xy problem. There's a strong chance that there's a fundamentally better approach than this.

Javascript: Need help understanding async/await with mongooseODM

const deleteTaskCount = async(id) => {
const task = Task.findByIdAndDelete(id);
const count = await Task.countDocument({completed:false})
return count;
}
deleteTaskCount("1234").then((count)=>{
console.log(count);
})
I understood that with await in const task = Task.findByIdAndDelete(id); it deletes the user with the id.But without the await keyword(like in the above function) the function still happens, that is it deletes the user but in a non-blocking way after some time.When i ran the above code the count is showing properly but the const task = Task.findByIdAndDelete(id); is not deleting the user.

You should use await for the Task.findByIdAndDelete(id) as well. When you use await, NodeJS will go do some other things until that resolves, and it will only then continue to execute the function (it will not block your code at all).
const deleteTaskCount = async(id) => {
const task = await Task.findByIdAndDelete(id);
const count = await Task.countDocument({completed:false})
return count;
}
deleteTaskCount("1234").then((count)=>{
console.log(count);
})

I don't know much about the mongoose API but I suspect mongoose.query behaves similar to mongo cursors and many other asynchronous database drivers/ORMs. Queries often don't execute unless awaited or chained upon.
In any case, without awaiting Task.findByIdAndDelete, there's no guarantee in concurrency between the two queries. If you want the result of countDocument to reflect the changes in findByIdAndDelete, you must wait on the client-side for that query to complete and then run countDocument.
You would achieve this by adding the await keyword before Task.findByIdAndDelete.
I'd presonally recommend using the promises/thenables as opposed to await/async.
Task.findByIdAndDelete("1234")
.then(task => Task.countDocument({completed:false}))
.then(count => console.log(count));
Imho, it's cleaner, less error prone, more functional, less confusing. YMMV

My Async won't stop for my Await - What does AWAIT wait for then? [duplicate]

This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 2 years ago.
I am getting into Node, and being thrown into the world of async programming.
I thoroughly read this article on Mozilla's site (along with many other links teaching about the async and await keywords):
https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await
I have a project I am starting that will require numerous database calls, one after another. Some queries will rely upon the previous one's results. That being said, I decided I better practice and write some testing code.
Here is my testing code:
var users, items;
const getData = async function(){
// This is a DB call. Will take
// a little bit of time to complete.
setTimeout(() => {
var queryResults = "A list of items";
items = queryResults;
}, 1000);
return items;
}
const getUsers = async function(){
setTimeout(() => {
var queryResults = "A list of items";
users = queryResults;
}, 1000);
return users;
}
const init = async function(){
var itemSuccess = await getData();
var userSuccess = await getUsers();
console.log(`Here is your items: ${items}, and here are your users: ${users}`);
}
init();
My code failed with:
Here is your items: undefined, and here are your users: undefined
So naturally I headed back to Mozilla. I re-read the docs, and tried a few changes. Still, no success. Ultimately I ended up back at my original code above.
I cannot see why this is failing. I am simulating the time a query would take by using a timeout. If async does what it says, both functions should return their value, and then my console log should print out the correct results.
Does anyone see where I am going wrong?
Thank you.

Show us your REAL code with the actual database calls in it and we can help you much more specifically, including showing you working code. Please resist the temptation to post questions with pseudo-code. Show real code and we can help you so much faster and better if you show us the real code.
await ONLY does something useful when you await a promise that resolves or rejects when something you are interested in completes or errors.
It has no magic to somehow know when a a setTimeout() or any other asynchronous operation is done doing whatever it does.
So, when you do this:
var itemSuccess = await getData();
And, getData() looks like this:
const getData = async function(){
// This is a DB call. Will take
// a little bit of time to complete.
setTimeout(() => {
var queryResults = "A list of items";
items = queryResults;
}, 1000);
return items;
}
All you end up doing is:
var itemSuccess = await undefined
because your getData() function doesn't return a promise and, in fact, it doesn't even return the items because it returns BEFORE your timer so items (which doesn't appear to be declared properly here) has not yet received its value.
So, I would suggest you start back with the basics and go read what await really does. Make sure you understand how it interacts with a promise since it's only useful when used with a promise. And, then learn how to make your asynchronous operations return promises that are connected to when your asynchronous operations complete. Then, and only then, can you make proper use of await.
In a well organized world, you make sure that all your asynchronous operations are already returned promises and then you can async/await and .then() and .catch() to control the flow of all your asynchronous operations. In many cases, this involves learning the promise interface for whatever you're using. For example, modern versions of node.js have an entire promise interface for the file system in fs.promises.
In a few cases, there may not exist a promise interface for some library you're using and you may have to wrap the asynchronous operation in a promise to make your own promise interface. util.promisify() is built-into node.js to help you do that.
If you show the ACTUAL code, include which database you're using and the actual database call you're trying to make, then someone here can probably help you with the proper way to use the promise interface on the database. This pseudo code prevents us from helping you that specifically. Without your actual code and knowing the actual DB calls you're making in a specific DB, it's hard for us to advise much more than go use the promise interface to your database and learn how to use those promises properly. Happy to help more specifically, but we need to see you REAL code to do that.
And, lastly in Javascript if you're assigning a value to something in a higher scope from within any asynchronous callback, there's a 99.99% chance you're doing things wrong. The only place you can actually use that value is inside the callback itself or in a function that you call from there and pass the value to. So, when you do items = queryResults, where items is declared in a couple scopes above you, then alarm bells should go off that this is not going to work for you. It's not how you program with asynchronous code in Javascript, not just for style reasons, but because it doesn't work.

Use:
function timeout(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
instead of
setTimeout(() => {
}, 1000);

await waits for a Promise to resolve.
// normal function that returns a promise
const getData = function(){
// This is a DB call. Will take
// a little bit of time to complete.
return new Promise(resolve => {
setTimeout(() => {
var queryResults = "A list of data";
resolve(queryResults);
}, 1000);
});
}
const getUsers = function(){
return new Promise(resolve => {
setTimeout(() => {
var queryResults = "A list of users";
resolve(queryResults);
}, 1000);
});
}
const init = async function(){
var itemSuccess = await getData();
var userSuccess = await getUsers();
console.log(`Here is your items: ${itemSuccess}, and here are your users: ${userSuccess}`);
}
init();
An async function does 2 things.
it makes it possible to use the await keyword to wait for promises.
it makes the function automatically return a promise itself that wraps whatever the function returns
async function foo() {
return 123;
}
const v = foo();
console.log(v instanceof Promise);
v.then(value => console.log(value));
A more normal database query might look something like this
async function dbQuery(queryUrl) {
const res = await fetch(queryUrl); // fetch returns a promise that resolves to a Response
return res.json(); // res.json returns a promise that resolves to the body of response parsed as json.
}
async function main() {
const films = await dbQuery('https://greggman.github.io/doodles/test/data/films.json');
console.log(JSON.stringify(result, null, 2));
}
main();

Move object/variable away from async function

I understand that this is a basic question, but I can't figure it out myself, how to export my variable "X" (which is actually a JSON object) out of "for" cycle. I have tried a various ways, but in my case function return not the JSON.object itself, but a "promise.pending".
I guess that someone more expirienced with this will help me out. My code:
for (let i = 0; i < server.length; i++) {
const fetch = require("node-fetch");
const url = ''+(server[i].name)+'';
const getData = async url => {
try {
const response = await fetch(url);
return await response.json();
} catch (error) {
console.log(error);
}
};
getData(url).then(function(result) { //promise.pending w/o .then
let x = result; //here is real JSON that I want to export
});
}
console.log(x); // -element is not exported :(

Here's some cleaner ES6 code you may wish to try:
const fetch = require("node-fetch");
Promise.all(
server.map((srv) => {
const url = String(srv.name);
return fetch(url)
.then((response) => response.json())
.catch((err) => console.log(err));
})
)
.then((results) => {
console.log(results);
})
.catch((err) => {
console.log('total failure!');
console.log(err);
});
How does it work?
Using Array.map, it transforms the list of servers into a list of promises which are executed in parallel. Each promise does two things:
fetch the URL
extract JSON response
If either step fails, that one promise rejects, which will then cause the whole series to reject immediately.
Why do I think this is better than the accepted answer? In a word, it's cleaner. It doesn't mix explicit promises with async/await, which can make asynchronous logic muddier than necessary. It doesn't import the fetch library on every loop iteration. It converts the server URL to a string explicitly, rather than relying on implicit coercion. It doesn't create unnecessary variables, and it avoids the needless for loop.
Whether you accept it or not, I offer it up as another view on the same problem, solved in what I think is a maximally elegant and clear way.
Why is this so hard? Why is async work so counterintuitive?
Doing async work requires being comfortable with something known as "continuation passing style." An asynchronous task is, by definition, non-blocking -- program execution does not wait for the task to complete before moving to the next statement. But we often do async work because subsequent statements require data that is not yet available. Thus, we have the callback function, then the Promise, and now async/await. The first two solve the problem with a mechanism that allows you to provide "packages" of work to do once an asynchronous task is complete -- "continuations," where execution will resume once some condition obtains. There is absolutely no difference between a boring node-style callback function and the .then of a Promise: both accept functions, and both will execute those functions at specific times and with specific data. The key job of the callback function is to act as a receptacle for data about the asynchronous task.
This pattern complicates not only basic variable scoping, which was your main concern, but also the issue of how best to express complicated workflows, which are often a mix of blocking and non-blocking statements. If doing async work requires providing lots of "continuations" in the form of functions, then we know that doing this work will be a constant battle against the proliferation of a million little functions, a million things needing names that must be unique and clear. This is a problem that cannot be solved with a library. It requires adapting one's style to the changed terrain.
The less your feet touch the ground, the better. :)

Javascript builds on the concept of promises. When you ask getData to to do its work, what is says is that, "OK, this is going to take some time, but I promise that I'll let you know after the work is done. So please have faith on my promise, I'll let you know once the work is complete", and it immediately gives you a promise to you.
That's what you see as promise.pending. It's pending because it is not completed yet. Now you should register a certain task (or function) with that promise for getData to call when he completes the work.
function doSomething(){
var promiseArray = [];
for (let i = 0; i < server.length; i++) {
const fetch = require("node-fetch");
const url = ''+(server[i].name)+'';
const getData = async url => {
try {
const response = await fetch(url);
return await response.json();
} catch (error) {
console.log(error);
}
};
promiseArray.push(getData(url)); // keeping track of all promises
}
return Promise.all(promiseArray); //see, I'm not registering anything to promise, I'm passing it to the consumer
}
function successCallback(result) {
console.log("It succeeded with " + result);
}
function failureCallback(error) {
console.log("It failed with " + error);
}
let promise = doSomething(); // do something is the function that does all the logic in that for loop and getData
promise.then(successCallback, failureCallback);