I'm building chatbot using Dialogflow. I wanted to integrate it with Google Calendar, I followed the Google official tutorial on youtube. My code looks as follows:
function makeAppointment (agent) {
// Calculate appointment start and end datetimes (end = +1hr from start)
//console.log("Parameters", agent.parameters.date);
const appointment_type = agent.parameters.AppointmentType;
const dateTimeStart = new Date(Date.parse(agent.parameters.date.split('T')[0] + 'T' + agent.parameters.time.split('T')[1].split('-')[0] + timeZoneOffset));
const dateTimeEnd = new Date(new Date(dateTimeStart).setHours(dateTimeStart.getHours() + 1));
const appointmentTimeString = dateTimeStart.toLocaleString(
'en-US',
{ month: 'long', day: 'numeric', hour: 'numeric', timeZone: timeZone }
);
// Check the availibility of the time, and make an appointment if there is time on the calendar
var result = undefined;
var not_needed = createCalendarEvent(dateTimeStart, dateTimeEnd, appointment_type).then(() => {
agent.add(Ok, let me see if we can fit you in. ${appointmentTimeString} is fine!.);
result = 1;
}).catch(() => {
agent.add(I'm sorry, there are no slots available for ${appointmentTimeString}.);
result = 1
});
while(result == undefined) continue;
return not_needed;
}
function createCalendarEvent (dateTimeStart, dateTimeEnd, appointment_type) {
return new Promise((resolve, reject) => {
calendar.events.list({
auth: serviceAccountAuth, // List events for time period
calendarId: calendarId,
timeMin: dateTimeStart.toISOString(),
timeMax: dateTimeEnd.toISOString()
}, (err, calendarResponse) => {
// Check if there is a event already on the Calendar
if (err || calendarResponse.data.items.length > 0) {
reject(err || new Error('Requested time conflicts with another appointment'));
} else {
// Create event for the requested time period
calendar.events.insert({ auth: serviceAccountAuth,
calendarId: calendarId,
resource: {summary: appointment_type +' Appointment', description: appointment_type,
start: {dateTime: dateTimeStart},
end: {dateTime: dateTimeEnd}}
}, (err, event) => {
err ? reject(err) : resolve(event);
}
);
}
});
});
}
There is nothing in the response so I created an infinite while loop which is waiting for the promise to get resolved or rejected. Since than cloud function is getting timed out after 60 seconds because the while loop never breaks. Why is the callback passed to calendar.events.lists never called?
Thank you
The Dialogflow library is aware of promises. In fact, it is so aware of them that you must return a Promise if you are doing any asynchronous operations in your Handler function. It will wait for the Promise to be resolved before it sends anything back.
Since createCalendarEvent() returns a Promise already, you can return this Promise from makeAppointment().
You don't need the while loop - in fact, that is probably a big part of what isn't working. Since node is single-threaded, the thread never leaves the while loop to process the Promise, so result will never actually be set to 1.
Additionally, you probably don't need to wrap the calls to calendar.events.list() and calendar.events.insert() inside a new Promise(). If you don't provide a callback function for these, they will return a Promise, and you can handle them with standard then/catch blocks or await (if you're using a sufficiently modern version of node).
Related
I want to limit the call of function not more than once in 3s.
but hope it can be invoked at first and delay all others after the wait.
My use case is, limit the sending requests and then batch all on in the waiting to third party services. but not for the first call, it's a pain to wait for the first call.
so I made a throttle with lodash like this
const maxThrottle = _.throttle(() => console.log(new Date()), 3000, { trailing: true, leading: true })
and the calls are:
console.log('start: ', new Date())
maxThrottle()
await setTimeout(100)
maxThrottle()
await setTimeout(3000)
maxThrottle()
what I expected:
start: 2022-09-04T06:58:01.099Z
2022-09-04T06:58:01.100Z
2022-09-04T06:58:04.104Z
2022-09-04T06:58:07.104Z
actual:
start: 2022-09-04T06:58:01.099Z
2022-09-04T06:58:01.100Z
2022-09-04T06:58:04.104Z
2022-09-04T06:58:04.214Z // it is violating the speed of service which I want to avoid.
how can I do it with throttle? is it possible?
I can achieve the behavior with this code:
function limitSpeed(fn: () => void, wait: number) {
let calledAt: number | null
let promise: Promise<void> | null
return () => {
const timeLeft = calledAt ? wait - (Date.now() - calledAt) : 0
if (promise)
return
if (timeLeft <= 0) {
calledAt = Date.now()
fn()
} else {
promise = setTimeout(timeLeft).then(() => {
calledAt = Date.now()
fn()
promise = null
})
}
}
}
const maxThrottle = limitSpeed(() => console.log(new Date()), 3000)
console.log('start: ', new Date())
maxThrottle()
await setTimeout(100)
maxThrottle()
await setTimeout(3000)
maxThrottle()
while(true) {
maxThrottle()
await setImmediate()
}
result:
start: 2022-09-04T07:22:13.621Z
2022-09-04T07:22:13.622Z
2022-09-04T07:22:16.630Z
2022-09-04T07:22:19.629Z
2022-09-04T07:22:22.628Z
2022-09-04T07:22:25.627Z
2022-09-04T07:22:28.626Z
2022-09-04T07:22:31.625Z
2022-09-04T07:22:34.624Z
2022-09-04T07:22:37.623Z
2022-09-04T07:22:40.622Z
2022-09-04T07:22:43.621Z
2022-09-04T07:22:46.621Z
2022-09-04T07:22:49.620Z
2022-09-04T07:22:52.619Z
2022-09-04T07:22:55.618Z
2022-09-04T07:22:58.617Z
2022-09-04T07:23:01.616Z
2022-09-04T07:23:04.615Z
2022-09-04T07:23:07.614Z
2022-09-04T07:23:10.613Z
2022-09-04T07:23:13.613Z
2022-09-04T07:23:16.613Z
2022-09-04T07:23:19.613Z
2022-09-04T07:23:22.613Z
2022-09-04T07:23:25.613Z
2022-09-04T07:23:28.613Z
2022-09-04T07:23:31.613Z
2022-09-04T07:23:34.613Z
2022-09-04T07:23:37.613Z
2022-09-04T07:23:40.613Z
It seems it is a bug of lodash. It is working perfect with underscore.
After replacement of underscore, the problem has gone.
I guess lodash doesn't handle the case of throttle after the trailing.
It will trigger immediately without wait with leading=true.
As of Sep 2022, lodash is no longer maintaining for two years. So I prefer switching to underscore if you have the same issue.
I am building a backend to handle pulling data from a third party API.
There are three large steps to this, which are:
Delete the existing db data (before any new data is inserted)
Get a new dataset from the API
Insert that data.
Each of these three steps must happen for a variety of datasets - i.e. clients, appointments, products etc.
To handle this, I have three Promise.all functions, and each of these are being passed individual async functions for handling the deleting, getting, and finally inserting of the data. I have this code working just for clients so far.
What I'm now trying to do is limit the API calls, as the API I am pulling data from can only accept up to 200 calls per minute. To quickly test the rate limiting functionality in code I have set it to a max of 5 api calls per 10 seconds, so I can see if it's working properly.
This is the code I have so far - note I have replaced the name of the system in the code with 'System'. I have not included all code as there's a lot of data that is being iterated through further down.
let patientsCombinedData = [];
let overallAPICallCount = 0;
let maxAPICallsPerMinute = 5;
let startTime, endTime, timeDiff, secondsElapsed;
const queryString = `UPDATE System SET ${migration_status_column} = 'In Progress' WHERE uid = '${uid}'`;
migrationDB.query(queryString, (err, res) => {
async function deleteSystemData() {
async function deleteSystemPatients() {
return (result = await migrationDB.query("DELETE FROM System_patients WHERE id_System_account = ($1) AND migration_type = ($2)", [
System_account_id,
migrationType,
]));
}
await Promise.all([deleteSystemPatients()]).then(() => {
startTime = new Date(); // Initialise timer before kicking off API calls
async function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function getSystemAPIData() {
async function getSystemPatients() {
endTime = new Date();
timeDiff = endTime - startTime;
timeDiff /= 1000;
secondsElapsed = Math.round(timeDiff);
if (secondsElapsed < 10) {
if (overallAPICallCount > maxAPICallsPerMinute) {
// Here I want to sleep for one second, then check again as the timer may have passed 10 seconds
getSystemPatients();
} else {
// Proceed with calls
dataInstance = await axios.get(`${patientsPage}`, {
headers: {
Authorization: completeBase64String,
Accept: "application/json",
"User-Agent": "TEST_API (email#email.com)",
},
});
dataInstance.data.patients.forEach((data) => {
patientsCombinedData.push(data);
});
overallAPICallCount++;
console.log(`Count is: ${overallAPICallCount}. Seconds are: ${secondsElapsed}. URL is: ${dataInstance.data.links.self}`);
if (dataInstance.data.links.next) {
patientsPage = dataInstance.data.links.next;
await getSystemPatients();
} else {
console.log("Finished Getting Clients.");
return;
}
}
} else {
console.log(`Timer reset! Now proceed with API calls`);
startTime = new Date();
overallAPICallCount = 0;
getSystemPatients();
}
}
await Promise.all([getSystemPatients()]).then((response) => {
async function insertSystemData() {
async function insertClinkoPatients() {
const SystemPatients = patientsCombinedData;
Just under where it says ' if (secondsElapsed < 10) ' is where I want to check the code every second to see if the timer has passed 10 seconds, in which case the timer and the count will be reset, so I can then start counting again over the next 10 seconds. Currently the recursive function is running so often that an error displayed related to the call stack.
I have tried to add a variety of async timer functions here but every time the function is returned it causes the parent promise to finish executing.
Hope that makes sense
I ended up using the Bottleneck library, which made it very easy to implement rate limiting.
const Bottleneck = require("bottleneck/es5");
const limiter = new Bottleneck({
minTime: 350
});
await limiter.schedule(() => getSystemPatients());
I am adding some epoch time to my date variable and then converting it to iso to put it inside db. but my code is not geting correct time.
my code is not stoping to execute => var releaseDate = new Date(epochDate).toISOString();
how should I make it happen
my code output is
epoch and now =
Mon Dec 21 2020 02:16:56 GMT+0530 (IST)604800000
1618003233700
2020-12-20T20:46:56.000Z
date should be today + 7 days but it didn't change.
async function schedule(data, epochDate) {
const endpoint = config.main+"/schedule"
const opt = {
headers: {
"id": 0
}
}
//add 7 days in epoch time milliseconds
epochDate += 604800000;
const now = Date.now();
console.log("epoch and now =")
console.log(epochDate)
console.log(now)
if(epochDate <= now){
epochDate = now + 604800000;
}
var releaseDate = new Date(epochDate).toISOString();
console.log(releaseDate);
const payload = {
Id: data,
date: new Date(epochDate).toISOString()
}
return await axios.post(endpoint, payload, opt)
.then((result)=> result.data)
.catch((error) => {
console.log('error in axios post catch %j', error.response.data)
throw error
})
}
You have declared function schedule() as async meaning that it must return a promise that will be fulfilled. At the end of your function, you are returning the promise made by axios but you are also attaching then and catch functions to that promise which is likely producing very confusing results. If I was doing it, I would declare schedule() as you have but make its implementation specifically a promise. Within that promise, you would invoke axios() and, in its then and/or catch, you would call either the accept handler or the reject handler.
I need to wait for mapping function to finish before I send the data to the console. I know it has something to do with Promise. I've been trying for hours and I couldn't get it to work even after reading so much about promises and async functions...
async function inactiveMemberWarner() {
var msg = "```javascript\nI have sent warnings to members that have been inactive for 2 weeks.\n\n"
var inactiveMembers = '';
var count = 0;
var guildMembers = client.guilds.find(g => g.name === mainGuild).members;
const keyPromises = await guildMembers.map(async (member) => {
if (isMod(member)) {
connection.query(`SELECT * from users WHERE userID='${member.id}'`, (err, data) => {
if (data[0]) {
if (!data[0].warnedForInactivity && moment().isSameOrAfter(moment(data[0].lastMSGDate).add('2', 'week'))) {
count++;
var updateWarning = {warnedForInactivity: 1}
connection.query(`UPDATE users SET ? WHERE userID='${data[0].userID}'`, updateWarning);
member.send(`**[*]** WARNING: You've been inactive on \`\`${mainGuild}\`\` for 2 weeks. Members that have been inactive for at least a month will be kicked.`);
inactiveMembers += `${count}. ${member.user.tag}\n`;
return inactiveMembers;
}
}
});
}
});
await Promise.all(keyPromises).then(inactiveMembersData => console.log(inactiveMembers)); // RETURNS AN EMPTY STRING
setTimeout(() => console.log(inactiveMembers), 5000); // RETURNS THE INACTIVE MEMBERS AFTER WAITING FOR 5 SECONDS (PRMITIVE WAY)
}
inactiveMemberWarner();
Thank you in advance!
You're close, but not quite there.
First, some notes:
await can be used on any value, but it is entirely pointless to use it on anything that isn't a Promise. Your guildMembers.map(...); returns an array, not a Promise.
Mixing await and .then(...) works, but is kinda messy. You're already using await - why bother dealing with callbacks?
Using guildMembers.map(async ...) like this will ensure that all the requests are fired more or less instantaneously, and they could finish in any order. This is fine, but it is kind of a race condition and results in a more or less random order of results.
This is not a good approach even just conceptually! Any time you ever have to loop queries, try and investigate ways to do it in only one query. SQL is quite powerful.
The reason your current code doesn't work is because your connection.query function escapes the async control flow. What I mean by this is that the whole point of using async/await and Promises is basically to keep track of the callbacks locally, and to make use of promise chaining to dynamically add callbacks. If you call an async function which returns a Promise, you can now carry that Promise anywhere else in your code and attach a success handler to it dynamically: either with .then() or with the sugar await.
But the connection.query function doesn't return a Promise, it just has you pass another naked callback - this one is not being tracked by a Promise! The Promise doesn't have a reference to that callback, it can't know when that callback is getting called, and thus your async/await control flow is escaped and your promises resolve long before the queries have ran.
You can resolve this by making a new Promise in the async function:
async function inactiveMemberWarner() {
var msg = "```javascript\nI have sent warnings to members that have been inactive for 2 weeks.\n\n"
var inactiveMembers = '';
var count = 0;
var guildMembers = client.guilds.find(g => g.name === mainGuild).members;
const keyPromises = guildMembers.map(async (member) => {
if (isMod(member)) {
return new Promise((resolve, reject) => {
connection.query(`SELECT * from users WHERE userID='${member.id}'`, (err, data) => {
if (err) reject(err); //make errors bubble up so they can be handled
if (data[0]) {
if (!data[0].warnedForInactivity && moment().isSameOrAfter(moment(data[0].lastMSGDate).add('2', 'week'))) {
count++;
var updateWarning = {warnedForInactivity: 1}
connection.query(`UPDATE users SET ? WHERE userID='${data[0].userID}'`, updateWarning);
member.send(`**[*]** WARNING: You've been inactive on \`\`${mainGuild}\`\` for 2 weeks. Members that have been inactive for at least a month will be kicked.`);
resolve(`${count}. ${member.user.tag}\n`;);
}
} else resolve(""); //make sure to always resolve or the promise may hang
});
});
}
});
let inactiveMembersData = await Promise.all(keyPromises); // Returns an array of inactive member snippets.
inactiveMembers = inactiveMembersData.join(""); //join array of snippets into one string
}
inactiveMemberWarner();
This will work, but there is a much much much better way. SQL supports the IN operator, which allows you to have conditions like WHERE userID IN (list_of_ids). In other words, you can do this in one query. You can even specify more conditions, such as warnedForInactivity = 0 and lastMSGDate BETWEEN (NOW() - INTERVAL 14 DAY) AND NOW(). This way you can offload all of your current processing logic onto the SQL server - something that you should try to do virtually every single time you can. It would simplify this code a lot too. I won't go any further as it's out of scope for this question but feel free to ask another if you can't figure it out.
I can't test this, but this is what normally works for me is when wanting to wait on smt:
async function inactiveMemberWarner() {
new Promise(function(cb,rj){
var msg = "```javascript\nI have sent warnings to members that have been inactive for 2 weeks.\n\n"
var inactiveMembers = '';
var count = 0;
var guildMembers = client.guilds.find(g => g.name === mainGuild).members;
const keyPromises = await guildMembers.map(async (member) => {
if (isMod(member)) {
connection.query(`SELECT * from users WHERE userID='${member.id}'`, (err, data) => {
if (data[0]) {
if (!data[0].warnedForInactivity && moment().isSameOrAfter(moment(data[0].lastMSGDate).add('2', 'week'))) {
count++;
var updateWarning = {warnedForInactivity: 1}
connection.query(`UPDATE users SET ? WHERE userID='${data[0].userID}'`, updateWarning);
member.send(`**[*]** WARNING: You've been inactive on \`\`${mainGuild}\`\` for 2 weeks. Members that have been inactive for at least a month will be kicked.`);
inactiveMembers += `${count}. ${member.user.tag}\n`;
cb(inactiveMembers);
}
}
});
}
});
cb('No Members');
}).then(inactiveMembersData => console.log(inactiveMembers)); // SHOULD RETURNS THE INACTIVE MEMBERS
}
inactiveMemberWarner();
I want to test how much requests i can do and get their total time elapsed. My Promise function
async execQuery(response, query) {
let request = new SQL.Request();
return new Promise((resolve, reject) => {
request.query(query, (error, result) => {
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
}
And my api
app.get('/api/bookings/:uid', (req, res) => {
let st = new stopwatch();
let id = req.params.uid;
let query = `SELECT * FROM booking.TransactionDetails WHERE UID='${id}'`;
for (let i = 0; i < 10000; i++) {
st.start();
db.execQuery(res, query);
}
});
I can't stop the for loop since its async but I also don't know how can I stop executing other calls after the one which first rejects so i can get the counter and the elapsed time of all successful promises. How can i achieve that?
You can easily create a composable wrapper for this, or a subclass:
Inheritance:
class TimedPromise extends Promise {
constructor(executor) {
this.startTime = performance.now(); // or Date.now
super(executor);
let end = () => this.endTime = performance.now();
this.then(end, end); // replace with finally when available
}
get time() {
return this.startTime - this.endTime; // time in milliseconds it took
}
}
Then you can use methods like:
TimedPromise.all(promises);
TimedPromise.race(promises);
var foo = new TimedPromise(resolve => setTimeout(resolve, 100);
let res = await foo;
console.log(foo.time); // how long foo took
Plus then chaining would work, async functions won't (since they always return native promises).
Composition:
function time(promise) {
var startTime = performance.now(), endTime;
let end = () => endTime = performance.now();
promise.then(end, end); // replace with finally when appropriate.
return () => startTime - endTime;
}
Then usage is:
var foo = new Promise(resolve => setTimeout(resolve, 100);
var timed = time(foo);
await foo;
console.log(timed()); // how long foo took
This has the advantage of working everywhere, but the disadvantage of manually having to time every promise. I prefer this approach for its explicitness and arguably nicer design.
As a caveat, since a rejection handler is attached, you have to be 100% sure you're adding your own .catch or then handler since otherwise the error will not log to the console.
Wouldn't this work in your promise ?
new Promise((resolve, reject) => {
var time = Date.now();
request.query(query, (error, result) => {
if (error) {
reject(error);
} else {
resolve(result);
}
});
}).then(function(r){
//code
}).catch(function(e){
console.log('it took : ', Date.now() - time);
});
Or put the .then and .catch after your db.execQuery() call
You made 2 comments that would indicate you want to stop all on going queries when a promise fails but fail to mention what SQL is and if request.query is something that you can cancel.
In your for loop you already ran all the request.query statements, if you want to run only one query and then the other you have to do request.query(query).then(-=>request.query(query)).then... but it'll take longer because you don't start them all at once.
Here is code that would tell you how long all the queries took but I think you should tell us what SQL is so we could figure out how to set connection pooling and caching (probably the biggest performance gainer).
//removed the async, this function does not await anything
// so there is no need for async
//removed initializing request, you can re use the one created in
// the run function, that may shave some time off total runtime
// but not sure if request can share connections (in that case)
// it's better to create a couple and pass them along as their
// connection becomes available (connection pooling)
const execQuery = (response, query, request) =>
new Promise(
(resolve, reject) =>
request.query(
query
,(error, result) =>
(error)
? reject(error)
: resolve(result)
)
);
// save failed queries and resolve them with Fail object
const Fail = function(detail){this.detail=detail;};
// let request = new SQL.Request();
const run = (numberOfTimes) => {
const start = new Date().getTime();
const request = new SQL.Request();
Promise.all(
(x=>{
for (let i = 0; i < numberOfTimes; i++) {
let query = `SELECT * FROM booking.TransactionDetails WHERE UID='${i}'`;
db.execQuery(res, query, request)
.then(
x=>[x,query]
,err=>[err,query]
)
}
})()//IIFE creating array of promises
)
.then(
results => {
const totalRuntime = new Date().getTime()-start;
const failed = results.filter(r=>(r&&r.constructor)===Fail);
console.log(`Total runtime in ms:${totalRuntime}
Failed:${failed.length}
Succeeded:${results.length-failed.length}`);
}
)
};
//start the whole thing with:
run(10000);