I have some server API methods I call during startup of my application at different places, which can't be refactored in terms of where they are called, let's say getSettings() and getSchedule(). Both methods are based on a login(): Promise<Account> method as settings as well as the schedule are user-based.
Now I have solved the beginning of getSettings() and getSchedule() like the following:
class UserFunctions {
private fetch_login: Promise<AccountItem> = undefined;
async getSchedule(): Promise<any> {
var account = getAccount();
if (!account) {
// No account loaded
let isLogginIn = this.fetch_login;
if (!this.fetch_login) {
// Not logging in from any parallel method
this.fetch_login = login();
}
account = await this.fetch_login;
this.fetch_login = undefined;
}
// Now get schedule...
}
}
The idea behind is that the login() function is only called once no matter how often it could be called. That's why I keep a reference on the Promise to await it multiple times. This works, but I noticed that sometimes when login() is done getSettings() gets earlier the okay to continue work and getSchedule() stays a few seconds until it continues executing. Sometimes it's the other way around and sometimes both methods return in the same time.
Here I have a print of the output:
06-05 16:46:08.126 27376 27397 I ReactNativeJS: Logging in back
06-05 16:46:08.690 27376 27397 I ReactNativeJS: Logged in back
06-05 16:46:08.696 27376 27397 I ReactNativeJS: Schedule downloaded
06-05 16:46:09.274 27376 27397 I ReactNativeJS: Logged in back
Do you have any idea how the code can be improved that once login() is done, both methods continue working?
I think you've got the right idea, but the logic needs a little tweaking:
var account = getAccount();
if (!account) {
// No account loaded
if (!this.fetch_login) {
// Not logging in from any parallel method
this.fetch_login = login();
}
account = await this.fetch_login;
}
Basically the idea is that you'll set fetch_login to have the value of the promise the first time it gets called. After that, you can await that same promise as many times as you need to.
Two things:
You're calling fetch_login. You should just be referring to it:
account = await this.fetch_login; // <== No ()
Remove this line:
this.fetch_login = undefined;
E.g.:
async getSchedule(): Promise<any> {
var account = getAccount();
if (!account) {
// No account loaded
if (!this.fetch_login) {
// Not logging in from any parallel method
this.fetch_login = login();
}
account = await this.fetch_login;
}
// Now get schedule...
}
You keep the promise resulting from the login attempt, so you can await it each time. (Nothing about awaiting a promise prevents your doing so again.)
It seems like this should be handled at the level of getAccount and login, though, rather than within UserFunctions. getAccount and login appear to be global (within your app or module or whatever). I would probably have them manage this, with getAccount returning a promise:
var accountPromise = null;
function getAccount() {
if (!accountPromise) {
accountPromise = login();
}
return accountPromise;
}
then
async getSchedule(): Promise<any> {
var account = await getAccount();
// Now get schedule...
}
Example:
var accountPromise = null;
function getAccount() {
if (!accountPromise) {
accountPromise = login();
}
return accountPromise;
}
function login() {
return new Promise(resolve => {
console.log("Logging in...");
setTimeout(() => {
console.log("Logged in");
resolve({}); // {} = the account
}, 800);
});
}
class UserFunctions {
/*private*/ fetch_login/*: Promise<AccountItem>*/ = undefined;
async getSchedule()/*: Promise<any>*/ {
var account = await getAccount();
// Now get schedule...
console.log("Got account, getting schedule");
}
}
// The first time it's used
const u1 = new UserFunctions();
u1.getSchedule(); // Logs in and gets schedule
// The second time it's used
setTimeout(() => {
const u2 = new UserFunctions();
u2.getSchedule(); // Gets schedule (already logged in)
}, 2000);
Related
I'm working on a library and I'd like to prevent users from calling a specific function in order to prevent infinite loops.
Usually I'd go about doing it like this:
let preventFooCalls = false;
function fireUserCallbacks() {
preventFooCalls = true;
// Fire callbacks of the user here...
preventFooCalls = false;
}
function foo() {
if (preventFooCalls) throw Error();
// Run the content of foo() ...
// It will probably call fireUserCallbacks() at some point
}
However, if fireUserCallbacks is async, this method is not possible. It might be called multiple times, and with async user callbacks, preventFooCalls is not guaranteed to have the correct value. For instance:
let preventFooCalls = false;
async function fireUserCallbacks() {
preventFooCalls = true;
// Fire callbacks of the user here one of which being:
await new Promise(r => setTimeout(r, 1000));
preventFooCalls = false;
}
// Then when doing:
fireUserCallbacks();
foo(); // This will throw even though it's being called from outside fireUserCallbacks()
How can I detect if code is running from within a specific promise?
The only thing I can think of is new Error().stack, but oof that sounds like a terrible way to do it.
Some context
The reason why I want this is because I'm working on a part of a library that takes care of loading assets. Some of these assets might contain other assets with the possibility of infinite recursion. In order to handle recursion I have another function that I want users to call instead. Therefore I want to warn users when they call foo() from within one of the fireUserCallbacks() callbacks. While this will only be an issue when assets actually contain infinite loops, I'd rather block the usage of foo() completely to prevent unexpected hangs due to infinite loops.
edit: Here's a somewhat more sophisticated example of my actual code. I would share my actual code but that is really way too long for this format, tbh this example is already getting a bit too complex.
class AssetManager {
constructor() {
this.registeredAssetTypes = new Map();
this.availableAssets = new Map();
}
registerAssetType(typeId, assetTypeConstructor) {
this.registeredAssetTypes.set(typeId, assetTypeConstructor);
}
fillAvailableAssets(assetDatas) {
for (const assetData of assetDatas) {
const constructor = this.registeredAssetTypes.get(assetData.type);
const asset = new constructor(assetData.id, assetData.data);
this.availableAssets.set(assetData.id, asset);
}
}
async loadAsset(assetId, recursionTracker = null) {
// I have some extra code here that makes sure this function will only have
// a single running instance, but this example is getting way too long already
const asset = this.availableAssets.get(assetId);
let isRootRecursionTracker = false;
if (!recursionTracker) {
isRootRecursionTracker = true;
recursionTracker = new RecursionTracker(assetId);
}
const assetData = await asset.generateAsset(recursionTracker);
if (isRootRecursionTracker) {
// If this call was made from outside any `generateAsset` implementation,
// we will wait for all assets to be loaded and put on the right place.
await recursionTracker.waitForAll();
// Finally we will give the recursionTracker the created root asset,
// in case any of the sub assets reference the root asset.
// Note that circular references in any of the sub assets (i.e. not
// containing the root asset anywhere in the chain) are already taken care of.
if (recursionTracker.rootLoadingAsset) {
recursionTracker.rootLoadingAsset.setLoadedAssetData(assetData);
}
}
return assetData;
}
}
const assetManager = new AssetManager();
class RecursionTracker {
constructor(rootAssetId) {
this.rootAssetId = rootAssetId;
this.rootLoadingAsset = null;
this.loadingAssets = new Map();
}
loadAsset(assetId, cb){
let loadingAsset = this.loadingAssets.get(assetId);
if (!loadingAsset) {
loadingAsset = new LoadingAsset(assetId);
this.loadingAssets.set(assetId, loadingAsset);
if (assetId != this.rootAssetId) {
loadingAsset.startLoading(this);
} else {
this.rootLoadingAsset = loadingAsset;
}
}
loadingAsset.onLoad(cb);
}
async waitForAll() {
const promises = [];
for (const loadingAsset of this.loadingAssets.values()) {
promises.push(loadingAsset.waitForLoad());
}
await Promise.all(promises);
}
}
class LoadingAsset {
constructor(assetId) {
this.assetId = assetId;
this.onLoadCbs = new Set();
this.loadedAssetData = null;
}
async startLoading(recursionTracker) {
const loadedAssetData = await assetManager.loadAsset(this.assetId, recursionTracker);
this.setLoadedAssetData(loadedAssetData);
}
onLoad(cb) {
if (this.loadedAssetData) {
cb(this.loadedAssetData)
} else {
this.onLoadCbs.add(cb);
}
}
setLoadedAssetData(assetData) {
this.loadedAssetData = assetData;
this.onLoadCbs.forEach(cb => cb(assetData));
}
async waitForLoad() {
await new Promise(r => this.onLoad(r));
}
}
class AssetTypeInterface {
constructor(id, rawAssetData) {
this.id = id;
this.rawAssetData = rawAssetData;
}
async generateAsset(recursionTracker) {}
}
class AssetTypeFoo extends AssetTypeInterface {
async generateAsset(recursionTracker) {
// This is here just to simulate network traffic, an indexeddb lookup, or any other async operation:
await new Promise(r => setTimeout(r, 200));
const subAssets = [];
for (const subAssetId of this.rawAssetData.subAssets) {
// This won't work, as it will start waiting for itself to finish:
// const subAsset = await assetManager.loadAsset(subAssetId);
// subAssets.push(subAsset);
// So instead we will create a dummy asset:
const dummyAsset = {}
const insertionIndex = subAssets.length;
subAssets[insertionIndex] = dummyAsset;
// and load the asset with a callback rather than waiting for a promise
recursionTracker.loadAsset(subAssetId, (loadedAsset) => {
// since this will be called outside the `generateAsset` function, this won't hang
subAssets[insertionIndex] = loadedAsset;
});
}
return {
foo: this.id,
subAssets,
}
}
}
assetManager.registerAssetType("foo", AssetTypeFoo);
class AssetTypeBar extends AssetTypeInterface {
async generateAsset(recursionTracker) {
// This is here just to simulate network traffic, an indexeddb lookup, or any other async operation:
await new Promise(r => setTimeout(r, 200));
// We'll just return a simple object for this one.
// No recursion here...
return {
bar: this.id,
};
}
}
assetManager.registerAssetType("bar", AssetTypeBar);
// This is all the raw asset data as stored on the users disk.
// These are not instances of the assets yet, so no circular references yet.
// The assets only reference other assets by their "id"
assetManager.fillAvailableAssets([
{
id: "mainAsset",
type: "foo",
data: {
subAssets: ["subAsset1", "subAsset2"]
}
},
{
id: "subAsset1",
type: "bar",
data: {},
},
{
id: "subAsset2",
type: "foo",
data: {
subAssets: ["mainAsset"]
}
}
]);
// This sets the loading of the "mainAsset" in motion. It recursively loads
// all referenced assets and finally puts the loaded assets in the right place,
// completing the circle.
(async () => {
const asset = await assetManager.loadAsset("mainAsset");
console.log(asset);
})();
Maintain a queue and a set. The queue contains pending requests. The set contains pending requests, requests in progress, and successfully completed requests. (Each item would include the request itself; the request's status: pending, processing, complete; and possibly a retry counter.)
When request is made, check if it is in the set. If it is in the set, it was already requested and will be processed, is being processed, or was processed successfully and is already available. If not in the set, add it to both the set and the queue, then trigger queue processing. If queue processing is already running, the trigger is ignored. If not, queue processing starts.
Queue processing pulls requests off the queue, one by one, and processes them. If a request fails, it can either be put back onto the queue for repeat attempts (a counter can be included in the item to limit retries) or it can be removed from the set so it can be requested again later. Queue processing ends when the queue is empty.
This avoids recursion and unnecessary repeat requests.
Arriving with a theory question :)
I have a front that sends (axios) N requests in a Promise.all() with a map function. This works fine. Each time one of the promises is good, I have a little table that gets updated with each request's answer until I get the full table and the array of the answers at the end. ✅
The problem comes when I want to read, at the same time, the logs of the server
So my objective is to run another axios request to my express.js server that will run each 2 seconds to retrieve the logs of the last 2 seconds, this way I could show the logs of what is happening with each answer in real time.
Any ideas of how doing this two tasks in parallel?
In the front I'm using react and the promise.All has this is structure:
setIsLoading(true); // setting a flag to know this is running
const doAllTheTable = await Promise.all(
tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable) // my axios.get request
return updateTableLine(answer) // the functions that update the good line
})
);
//all promises are good now
setIsLoading(false)
So, basically I want to have another loop that runs each 2 seconds while "isLoading" is true to update another part of my front and show the logs meanwhile. But I need both things to happen at the same time!
Thank you for your ideas :)
Rather than awaiting your Promise.all immediately, store a reference to the promise so you can start checking the logs:
const doAllTheTablePromise = Promise.all(
tableData.map(async lineOfMyTable => {
const answer = await doMyRequest(lineOfMyTable); // my axios.get request
return updateTableLine(answer); // the functions that update the good line
});
);
let cancelled = false;
(async () => {
while (!cancelled) {
// Check your logs..
await new Promise(r => setTimeout(r, 2000)); // 2 second delay
}
})();
await doAllTheTablePromise;
cancelled = true;
Once your doAllTheTablePromise has resolved, you can stop checking the logs.
Must be many ways to write this. Here's one involving a token provided by the caller of two async processes, foo() and bar(), for communication between them.
async function foo(tableData, token) {
try {
await Promise.all(tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable);
return updateTableLine(answer);
}));
token.setIsLoading = false; // lower flag when all requests are complete
} catch(error) {
token.setIsLoading = false; // lower flag if synchronous or asynchronous error occurs
}
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to foo() and bar() as a means of communication between them.
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(foo(tableData, loadingToken), bar(loadingToken));
}
EDIT:
Maybe better written like this, with the caller looking after lowering the flag:
async function foo(tableData) {
return Promise.all(tableData.map(async (lineOfMyTable) => {
return updateTableLine(await doMyRequest(lineOfMyTable));
}));
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to bar().
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(
foo(tableData).finally(() => { loadingToken.setIsLoading = false }),
bar(loadingToken)
);
}
This perplexes me. I'm six months into a firebase project and have been using Javascript for firebase-functions. I've learned a lot along the way by adding transactions, promises, batch writes and neat tricks. However, it seems like complete luck for a function to execute correctly. More often than not, the functions do execute correctly, but there are strange periods when bursts of consecutive function calls where functions half complete with no errors in the logs.
For example. I have a function for when a new user joins my app. It does a little bit of server data construction and also notifies the two admins that a new user has joined. Last night I did a test run with two new users and got no notification, but their user profiles constructed correctly on the server database. I checked the function logs and there were no errors.
Am I not handling Promises in the correct way? If a firebase function hangs, does it mess up the next few function calls?
exports.onNewUser = functions.firestore
.document('/users/{userId}')
.onCreate(async (snapshot, context) => {
user = snapshot.data().username;
//Notification payload
const payload = {
notification: {
title: `New user!`,
body: `${user} has joined [AppName]`
}
};
var promises = [];
//Check if usename unique
var passed = true;
promises.push(db.runTransaction(async t => {
const docRef = db.collection('users').doc('index');
const doc = await t.get(docRef);
var newIndex = doc.data().usernames;
if (newIndex[user.toUpperCase()] == true) {
t.delete(snapshot.ref);
passed = false;
return null;
} else {
newIndex[user.toUpperCase()] = true;
t.set(docRef, { 'usernames': newIndex });
}
}));
if (!passed) return Promise.all(promises);
//add new user to Algolia database
const algoliasearch = require('algoliasearch');
const algoliaClient = algoliasearch(functions.config().algolia.appid, functions.config().algolia.apikey);
const collectionIndex = algoliaClient.initIndex(collectionIndexName);
await saveDocumentInAlgolia(snapshot, collectionIndex);
//Notify Admins
db.collection('notificationTokens')
.doc(admin1)
.get().then((doc) => {
if (doc.exists && doc.data().notificationToken != null)
promises.push(pushNotification(doc.data().notificationToken, payload));
});
db.collection('notificationTokens')
.doc(admin2)
.get().then((doc) => {
if (doc.exists && doc.data().notificationToken != null)
promises.push(pushNotification(doc.data().notificationToken, payload));
});
return Promise.all(promises);
});
Just change
return Promise.all(promises);
to
return await Promise.all(promises);
You have to wait till the promises resolve before you return the function, as that would stop the instance of the cloud function.
I’ve nodes program which I need to run two function in the beginning of the program
And later on access the function results, currently with await each function at a time this works,
However in order to save a time and not waiting to GetService and GetProcess as I need the data later on in the project
It takes about 4 seconds to get this data and I want to run it on the background as I don’t need the results immediately,
How I can do it in node js, If I run promise.all It would wait until the getService and getProcess and then go to rest of the program.
an example
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdress(this.process);
}
im actually running yeoman generator
https://yeoman.io/authoring/
https://yeoman.io/authoring/user-interactions.html
export default class myGenerator extends Generator {
//here I want run those function in background to save time as the prompt to the user takes some time (lets say user have many questions...)
async initializing() {
let service = await GetServices();
this.process = await GetProcess();
}
async prompting() {
const answers = await this.prompt([
{
type: "input",
name: "name",
message: "Your project name",
default: this.appname // Default to current folder name
},
{
type: "confirm",
name: "list",
choises: this.process //here I need to data from the function running in background
}
]);
}
Let's assume that getServices() may take 3 seconds and getProcess() may take 4 seconds, so if you run these both functions at the same time you will be returned in total 4 seconds with the return values from both promises.
You can execute the code while this process is running in the background there will be a callback when the promises resolved, your late functions will be called at this stage.
Check the below simple example;
let service;
let process;
function main() {
// Both functions will execute in background
Promise.all([getServices(), getProcess()]).then((val) => {
service = val[0];
process = val[1];
console.log(service, process);
// Aafter completed this code will be called
// let users = GetUser(service);
// let users = GetAdress(process);
console.log('I am called after all promises completed.')
});
// Current example.
// let service = await GetServices();
// this.process = await GetProcess();
/* Code blocks.. */
console.log('Code will execute without delay...')
}
function getServices() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("service is returned")
}, 3000);
});
}
function getProcess() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("process is returned")
}, 4000);
});
}
main();
You can start the asynchronous operation but not await it yet:
function suppressUnhandledRejections(p) {
p.catch(() => {});
return p;
}
async function main() {
// We have to suppress unhandled rejections on these promises. If they become
// rejected before we await them later, we'd get a warning otherwise.
const servicePromise = suppressUnhandledRejections(GetServices());
this.processPromise = suppressUnhandledRejections(GetProcess());
// Do other stuff
const service = await servicePromise;
const process = await this.processPromise;
}
Also consider using Promise.all() which returns a promise for the completion of all promises passed to it.
async function main() {
const [ services, process, somethingElse ] = await Promise.all([
GetServices(),
GetProcess(),
SomeOtherAsyncOperation(),
]);
// Use the results.
}
To do what who you need, you have to understand the event loop.
Nodejs is designed to work in a single thread unlike languages like go, however nodejs handle proccess on different threads. so you can use nextTick () to add a new event to the main thread and it will be executed at the end of the whole block.
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//Let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdr(this.process);
}
function someFunction(){
// do something...
}
main();
process.nextTick(someFunction());// happens after all main () processes are terminated...
I have a function createSession for getting users with returned session object. createSession either returns users or throws string exception. My code is as follow, if session is successfull (which means I have users) continue code.
However I do not know when users are available because other resources also asks for it. So, I want a modify my code that it will try getting users in a timeout, lest say try it every 1 second until timeout 30 seconds reached.
async function callTest() {
const capabilities = {
type: {
chrome: 2
}
}
var session = await working.createSession(capabilities)
if(!session) return
callTest()
// remaining code goes here
}
I though to use setTimout in a promise but the problem is I can not use await inside a normal function. Can someone help me fixing my code ?
var checkAuth = function(capabilities) {
return new Promise(function(resolve) {
var id = setInterval(function() {
var session = await working.createSession(capabilities)
if (session) {
clearInterval(id);
resolve(id);
}
});
}, 10);
});
}
You can try to use it like this and it should work well
var checkAuth = function(capabilities) {
return new Promise(function(resolve) {
setInterval(async function() {
var session = await working.createSession(capabilities)
if (session) {
clearInterval(id);
resolve(id);
}
}, 10);
});
}
You should add async keyword to function definition in setInterval, because await can only work in async functions.
Update: also move ,10 parameter from Promise definition to setInterval one