const productIds = ['abc', 'def', 'ghi', 'jkl', 'mno', 'pqr', ...]
const generateBearerToken = async () => {
await //api calling
} // will return bearerToken
const getSubmissions = async () => {
await // api calling
}
let sellerId = null
const getPublisherId = async (productId) => {
//generating bearer token using generateBearerToken()
await GenerateBearerToken()
//calling API here and setting the value of sellerId
const response = await axios.get(url, { header })
//From this response, I am setting the value of sellerId, then calling getSubmission()
sellerId = response.data.sellerId
await getSubmission()
}
productsIds.map((productId) => {
await getPublisherId(productId)
})
The sellerId which I am getting from getPublisherId, I am using that value in the header for calling getSubmissions. This value (sellerId) is different for different product Ids. But when I am calling this above map function, the sellerId of one is getting passed in other calling getSubmissions, that should be not the case. The sellerId should be passed to that particular getSubmissions only. How to avoid this collision?
This shouldn't be global:
let sellerId = null
Every asynchronous operation is modifying that value, and any other operation will use whatever the value is at that time, not whatever it was when that operation may have been invoked.
Don't use global variables to pass values from one function to another. Just pass the value to the function:
const getSubmissions = async (sellerId) => {
//...
};
and:
const getPublisherId = async (productId) => {
await GenerateBearerToken();
const response = await axios.get(url, { header });
await getSubmissions(response.data.sellerId);
};
As an aside, the code suggests that you're making the same mistake elsewhere and this same thing may need to be corrected in other places.
avoid globals
you are using .map as a "for each". Map is made to transform data.
At a minimum you want:
/* let sellerId = null // < Remove global */
const getPublisherId = async (productId) => {
[...]
const sellerId = response.data.sellerId
await getSubmission()
return sellerId
}
const allSellerIds = productsIds.map((productId) => {
await getPublisherId(productId)
})
However you will probably want to look up Promise.allSettled and others to avoid each publisher from being fetched one at a time sequentially.
To make my code cleaner I want to use fetched API data in a few different functions, instead of one big. Even though I 've did manage to reffer to that data in other functions, the problem is the API im a fetching throws different, randomized results every time it is called. And so the output from userData() does not equal that from userData2(), even though my intention is different and I'd like the result variable contents to be the same between functions.
const getData = () =>
fetch("https://opentdb.com/api.php?amount=10").then((response) =>
response.json()
);
const useData = async () => {
const result = await getData();
console.log(result);
};
const useData2 = async () => {
const result = await getData();
console.log(result);
};
Your getData() function returns a promise. One fun fact about promises is that while they can only resolve once, that resolved value can be accessed and used as many times as you want.
const dataPromise = getData();
const useData = async () => {
const result = await dataPromise;
console.log(result);
};
const useData2 = async () => {
const result = await dataPromise;
console.log(result);
};
Using await resolves the promise value, the equivalent of...
dataPromise.then((result) => {
console.log(result);
});
// or `dataPromise.then(console.log)` if you like brevity
I like to point this out about the fetch-api... you should always check the Response.ok property
const getData = async () => {
const res = await fetch("https://opentdb.com/api.php?amount=10");
if (!res.ok) {
throw new Error(`${res.status}: ${await res.text()}`);
}
return res.json();
};
I would like to load data asynchronously from the backend and have some arguments applied into a thunk so that the memoized function is available for import into other areas of my code. Because await at the top level is not currently allowed, I can't wait for the value to be passed in to the exported function.
For example, I have this function:
const aFunction = (someData) => (props) => {
// do stuff
}
const someDataFromApi = await getData(); // <-- will not work: top level await not allowed
const appliedAFunction = aFunction(someDataFromApi);
export { appliedAFunction, aFunction};
Then I would have the applied function be readily available for import everywhere (or of course the non-applied version available as well):
import {appliedAFunction, aFunction} from './aFunction'
const cProcess = () => {
const props = { a: 'foo' };
appliedAFunction(props)
}
const dProcess = async () => j
const data = await getData();
const props = {b: 'bar'};
aFunction(data)(props)
}
Is there an accepted/recommended way to asynchronously apply arguments so its available for export/import?
This is the solution I came up with which allows me to export functions which have been passed promised values. Its a different answer than to await at the top level, but it works just the same allowing asynchronously retrieved values to be passed and stored so that its available when the exported function is called.
This function allows me to preload promises as an argument, which means I can just export the function before the promise has resolved.
const passToThunkPromiseAsArg = (func: Function) => (
promise: Promise<any>
) => {
return (...args) => {
return promise
.then((resolvedValue) => {
return func(resolvedValue)(...args);
})
.catch((e) => {
// you might want to resolve errors differently
return func(null)(...args);
});
};
};
Example
Before I export, I pass my function a promise of a value instead of a value:
// ...
const promiseOfValue: Promise<any> = extremelySlowCalculation();
const appliedAFunction = passToThunkPromiseAsArg(aFunction)(promiseOfValue);
export default {aFunction, appliedAFunction}
Then in other files I can call and await or Promise.then the appliedAFunction:
import {appliedAFunction} from './aFunction'
const cProcess = async () => {
const props = { a: 'foo' };
const b = await appliedAFunction(props)
}
By the time cProcess calls appliedAFunction, the promise has very likely resolved, so in most cases there is no actual delay when b gets the resolved value. In the rare case it hasn't been resolved, the function will just await the value.
I have this function:
export const changeNotes = (userId, note, isStay)
I want to use to change a note, and sometimes I want to use it to change the value of isStay.
const handleOnChange = async e => {
await changeNotes(user.id,null ,true) // await changeNotes(user.id,10 ,null)
}
How do I send different cases?
You can pass an object as an argument to your changeNote function, and then use function argument destructuring to use properties from it. This way, order doesn't matter and only the values you send will be used (anything you don't pass will be undefined).
export const changeNotes = ({ userId, note, isStay }) => {
if (note) { // undefined
}
if (isStay) { // true
// do something
}
}
const handleOnChange = async e => {
await changeNotes({
userId: user.id,
isStay: true
})
}
I'm trying to pass a variable into a page.evaluate() function in Puppeteer, but when I use the following very simplified example, the variable evalVar is undefined.
I can't find any examples to build on, so I need help passing that variable into the page.evaluate() function so I can use it inside.
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
const evalVar = 'WHUT??';
try {
await page.goto('https://www.google.com.au');
await page.waitForSelector('#fbar');
const links = await page.evaluate((evalVar) => {
console.log('evalVar:', evalVar); // appears undefined
const urls = [];
hrefs = document.querySelectorAll('#fbar #fsl a');
hrefs.forEach(function(el) {
urls.push(el.href);
});
return urls;
})
console.log('links:', links);
} catch (err) {
console.log('ERR:', err.message);
} finally {
// browser.close();
}
})();
You have to pass the variable as an argument to the pageFunction like this:
const links = await page.evaluate((evalVar) => {
console.log(evalVar); // 2. should be defined now
…
}, evalVar); // 1. pass variable as an argument
You can pass in multiple variables by passing more arguments to page.evaluate():
await page.evaluate((a, b c) => { console.log(a, b, c) }, a, b, c)
The arguments must either be serializable as JSON or JSHandles of in-browser objects: https://pptr.dev/#?show=api-pageevaluatepagefunction-args
I encourage you to stick on this style, because it's more convenient and readable.
let name = 'jack';
let age = 33;
let location = 'Berlin/Germany';
await page.evaluate(({name, age, location}) => {
console.log(name);
console.log(age);
console.log(location);
},{name, age, location});
Single Variable:
You can pass one variable to page.evaluate() using the following syntax:
await page.evaluate(example => { /* ... */ }, example);
Note: You do not need to enclose the variable in (), unless you are going to be passing multiple variables.
Multiple Variables:
You can pass multiple variables to page.evaluate() using the following syntax:
await page.evaluate((example_1, example_2) => { /* ... */ }, example_1, example_2);
Note: Enclosing your variables within {} is not necessary.
It took me quite a while to figure out that console.log() in evaluate() can't show in node console.
Ref: https://github.com/GoogleChrome/puppeteer/issues/1944
everything that is run inside the page.evaluate function is done in the context of the browser page. The script is running in the browser not in node.js so if you log it will show in the browsers console which if you are running headless you will not see. You also can't set a node breakpoint inside the function.
Hope this can help.
For pass a function, there are two ways you can do it.
// 1. Defined in evaluationContext
await page.evaluate(() => {
window.yourFunc = function() {...};
});
const links = await page.evaluate(() => {
const func = window.yourFunc;
func();
});
// 2. Transform function to serializable(string). (Function can not be serialized)
const yourFunc = function() {...};
const obj = {
func: yourFunc.toString()
};
const otherObj = {
foo: 'bar'
};
const links = await page.evaluate((obj, aObj) => {
const funStr = obj.func;
const func = new Function(`return ${funStr}.apply(null, arguments)`)
func();
const foo = aObj.foo; // bar, for object
window.foo = foo;
debugger;
}, obj, otherObj);
You can add devtools: true to the launch options for test
I have a typescript example that could help someone new in typescript.
const hyperlinks: string [] = await page.evaluate((url: string, regex: RegExp, querySelect: string) => {
.........
}, url, regex, querySelect);
Slightly different version from #wolf answer above. Make code much more reusable between different context.
// util functions
export const pipe = (...fns) => initialVal => fns.reduce((acc, fn) => fn(acc), initialVal)
export const pluck = key => obj => obj[key] || null
export const map = fn => item => fn(item)
// these variables will be cast to string, look below at fn.toString()
const updatedAt = await page.evaluate(
([selector, util]) => {
let { pipe, map, pluck } = util
pipe = new Function(`return ${pipe}`)()
map = new Function(`return ${map}`)()
pluck = new Function(`return ${pluck}`)()
return pipe(
s => document.querySelector(s),
pluck('textContent'),
map(text => text.trim()),
map(date => Date.parse(date)),
map(timeStamp => Promise.resolve(timeStamp))
)(selector)
},
[
'#table-announcements tbody td:nth-child(2) .d-none',
{ pipe: pipe.toString(), map: map.toString(), pluck: pluck.toString() },
]
)
Also not that functions inside pipe cant used something like this
// incorrect, which is i don't know why
pipe(document.querySelector)
// should be
pipe(s => document.querySelector(s))