I'm building a command line app that will query a DynamoDB table, collect email addresses for each item on the table that has not already had an email sent to it, map over that array of items to create a customized message object for each one, pass that array into SendGrid's sgMail.send() method to send an email for each message inside, and then create a Communication object for each one on a second DynamoDB table. I followed the steps outlined in option three of this Twilio blog, and the code is working.
However, this method does not seem to allow error handling on a message by message basis when you pass in an array to .send(). Right now I first run sgMail.send(messages) in a try/catch block, and then create all of my Communication objects separately for each item only if the .send() call is completely successful. I'd prefer to be able to create an individual Communication object each time an email is successfully sent, rather than hoping that the try/catch block hits no errors and then wholesale creating the Communication objects after the fact.
My first thought was to iterate through the array returned from the DynamoDB query, and for each one first send an email, then once successful, create a Communication object. That method didn't work (the Communication was being created but the email was never sent, even though I was using await on all async calls). Once that didn't work, I found the blog linked above and used that method.
My worry is that if a random batch of the messages in the array fails and triggers the exception, but another chunk of emails was successfully sent before the error was caught, then I'll have a portion of emails sent out without corresponding Communication objects for them since the creation of those objects happens entirely independently from the email being sent.
I've combed through SendGrid's documentation as well as numerous Google searches, but haven't been able to find a helpful answer. Here's the current working code:
// First wait to send emails
console.log("Sending emails...")
try {
const messages = items.map((item) => {
return {
to: item.details.email,
from: SendGrid domain here,
subject: Email subject here,
html: Email content here
})
// Send email to each address passed in.
await sgMail.send(messages).then(() => {
console.log('emails sent successfully!');
}).catch(error => {
console.log(error);
});
} catch (error) {
throw error
}
// Then create communications
console.log("Creating communication objects...")
try {
for (let i = 0; i < items.length; i++) {
const params = {
TableName: process.env.COMMUNICATION_TABLE,
Item: {
communication_id: uuidv1(),
item_id: items[i].item_id,
date: new Date().toLocaleDateString()
}
};
try {
await docClient.put(params).promise();
} catch (error) {
throw error
}
}
} catch (error) {
throw error
}
// On success
console.log("Process complete!")
Twilio SendGrid developer evangelist here.
When you send an array of mail objects to sgMail.send, the #sendgrid/mail package actually loops through the array and makes an API call for each message. You are right though, that one may fail to send and you will be left in the state where some emails have sent and others failed or were cancelled.
So, you can loop through the array yourself and handle the successes/failures yourself too.
Try something like this:
console.log("Sending emails...");
items.forEach(async (item) => {
const mail = {
to: item.details.email,
from: "SendGrid domain here",
subject: "Email subject here",
html: "Email content here",
};
try {
await sgMail.send(mail);
console.log(`Sent mail for ${item.item_id} successfully, now saving Communication object`);
const params = {
TableName: process.env.COMMUNICATION_TABLE,
Item: {
communication_id: uuidv1(),
item_id: item.item_id,
date: new Date().toLocaleDateString(),
},
};
try {
await docClient.put(params).promise();
console.log(`Communications object saved successfully for ${item.item_id}.`);
} catch (error) {
console.log(
"There was a problem saving the Communications object: ",
error
);
}
} catch (error) {
console.error("Email could not be sent: ", error);
}
});
// On success
console.log("Process complete!");
Related
I'm trying to set up an email verification flow in my project, but I can't seem to get it right.
How my flow works now is the user enters their credentials (email and password), which are used to create a new firebase user. Then, once that promise is resolved, it sends an email verification link to the new user that was created. The code looks like this:
async createUser(email: string, password: string) {
try {
console.log("Creating user...");
const userCredentials = await createUserWithEmailAndPassword(
auth,
email,
password
);
console.log("Successfully created user");
const { user } = userCredentials;
console.log("Sending email verification link...");
await this.verifyEmail(user);
console.log("EMAIL VERIFICATION LINK SUCCESSFULLY SENT");
return user;
} catch (err) {
throw err;
}
}
async verifyEmail(user: User) {
try {
sendEmailVerification(user);
} catch (err) {
throw err;
}
}
The link is sent through fine, but once I press on it, I'm redirected to a page that says this:
Strangely, the user's email is verified after this, in spite of the error message displayed. Any idea why this is happening?
Update:
I managed to figure it out. The email provider I'm using is my university's, and it seems to be preventing the verification link from working properly. I did try with my personal email to see if that was the case, but I wasn't seeing the verification link appearing there. I eventually realized that it was because it was being stored in the spam folder. It's working on other email providers, though, ideally, I'd want it to work on my university's email provider (the emails that users sign up with are supposed to be exclusively student emails). Any ideas how I could resolve this?
I eventually figured out that the issue was with my email provider. I was using my student email, which the university provides, and I imagine they've placed rigorous measures in place to secure them as much as possible. I have no idea what was preventing it from working, but I managed to figure out a workaround.
In brief, I changed the action URL in the template (which can be found in the console for your Firebase project in the Authentication section, under the Templates tab) to a route on my website titled /authenticate. I created a module to handle email verification. Included in it is a function that parses the URL, extracting the mode (email verification, password reset, etc.), actionCode (this is the important one. It stores the id that Firebase decodes to determine if it's valid), continueURL (optional), and lang (optional).
export const parseUrl = (queryString: string) => {
const urlParams = new URLSearchParams(window.location.search);
const mode = urlParams.get("mode");
const actionCode = urlParams.get("oobCode");
const continueUrl = urlParams.get("continueUrl");
const lang = urlParams.get("lang") ?? "en";
return { mode, actionCode, continueUrl, lang };
};
I created another method that handles the email verification by applying the actionCode from the URL using Firebase's applyActionCode.
export const handleVerifyEmail = async (
actionCode: string,
continueUrl?: string,
lang?: string
) => {
try {
await applyActionCode(auth, actionCode);
return { alreadyVerified: false };
} catch (err) {
if (err instanceof FirebaseError) {
switch (err.code) {
case "auth/invalid-action-code": {
return { alreadyVerified: true };
}
}
}
throw err;
}
};
The auth/invalid-action-code error seems to be thrown when the user is already verified. I don't throw an error for it, because I handle this differently to other errors.
Once the user presses the verification link, they're redirected to the /authenticate page on my website. This page then handles the email verification by parsing the query appended to the route. The URL looks something like this http://localhost:3000/authenticate?mode=verifyEmail&oobCode=FLVl85S-ZI13_am0uwWeb4Jy8DUWC3E6kIiwN2LLFpUAAAGDUJHSwA&apiKey=AIzaSyA_V9nKEZeoTOECWaD7UXuzqCzcptmmHQI&lang=en
Of course, in production, the root path would be the name of the website instead of localhost. I have my development environment running on port 3000.
Once the user lands on the authentication page, I handle the email verification in a useEffect() hook (Note: I'm using Next.js, so if you're using a different framework you might have to handle changing the URL differently):
useEffect(() => {
verifyEmail();
async function verifyEmail() {
const { actionCode } = parseUrl(window.location.search);
if (!actionCode) return;
router.replace("/authenticate", undefined, { shallow: true });
setLoadingState(LoadingState.LOADING);
try {
const response = await handleVerifyEmail(actionCode!);
if (response.alreadyVerified) {
setEmailAlreadyVerified(true);
onEmailAlreadyVerified();
return;
}
setLoadingState(LoadingState.SUCCESS);
onSuccess();
} catch (err) {
console.error(err);
onFailure();
setLoadingState(LoadingState.ERROR);
}
}
}, []);
It first checks if there is an action code in the URL, in case a user tries to access the page manually.
The onSuccess, onFailure, and onEmailAlreadyVerified callbacks just display toasts. loadingState and emailAlreadyVerified are used to conditionally render different responses to the user.
I want to make an a discord bot that sends a scraped message.
I expected it to send the message but it gave me an error:
throw new DiscordAPIError(request.path, data, request.method, res.status);
DiscordAPIError: Cannot send an empty message
I tried to use message.channel.send(); but it doesn't seem to work.
Code:
let data = await page.evaluate(() => {
let Name = document.querySelector('div[id="title"]').innerText;
let Description = document.querySelector('div[id="content"]').innerText;
return {
Name,
Description
}
});
console.log(data);
message.channel.send(data);
debugger;
await browser.close();
The problem is that you shouldn't send the dictionary directly. While message.channel.send only accepts StringResolvable or APIMessage, data as a dictionary is neither. For more information, see the documentation.
Instead, you can convert data to a string first. The following is one of the solutions.
// Convert using JSON.stringify
message.channel.send(JSON.stringify(data));
Full code example (I tried it on https://example.com and thus there are different queries):
let data = await page.evaluate(() => {
let Name = document.querySelector('h1').innerText;
let Description = document.querySelector('p').innerText;
return {
Name,
Description
}
});
console.log(data);
message.channel.send(JSON.stringify(data));
Message sent by the bot without errors being thrown:
{"Name":"Example Domain","Description":"This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission."}
If you expect a different message, just make sure that the argument you are passing to message.channel.send is acceptable or errors might be thrown.
The Nodejs functions return an error from try/catch scope, such as the one below if the user doesn't exist of if a database is not reachable:
router.delete('/delete/:email', async (req, res) => {
var email = req.params.email;
try {
let result = await User.remove({"email": email});
res.status(204).send(email);
} catch (err) {
res.status(400).send(err);
}
});
I can also return the Error from Nodejs server by myself:
return res.status(400).send(new Error(`The user with email ${email} doesn't exist.`));
The first problem is that I can't find the error message that is embedded somewhere deep in the body the returned Error object. It is stored in one of its 100+ attributes. Where should I look for it so I could display in on a screen for the end user to read it?
Then, the err object generated by the try/catch scope has a set of different attributes comparing to the Error object created with new Error("Here is my error message"). Is there a way to normalize the returned Errors so they all have the same or similar attributes?
You don't need to return the whole error object from the server, and arguably shouldn't since error messages can expose internals about your code and infrastructure.
One way you could handle this is to format and return an error message from the server yourself. Assuming you're using express this would look something like:
return res.status(400).json({ message: `The user with email ${email} doesn't exist.` });
Alternatively you could use an error handling middleware like strong-error-handler found here: https://github.com/strongloop/strong-error-handler which automatically builds a json formatted message that's easier to parse, but keep in mind that the content of the message differs depending on whether you set debug mode to true or no.
If you want to develop a secure web application with nice error handling, i will suggest you the following structure.
Step 1. At front end divide your api calls in four main operations for e.g. inset,update,query and filter.
now whenever your page loads and you want to show some data fetched from server then your api call must be like 'https://domainname.tld/server/query' and send some payload with this api call according to need of your data requirements to be fetched.
At backend probably at Server.js handle like this :
app.all("/server/query", function (req, res) {
try {
console.log(a);
// some database or io blocking process
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
function writeCustomError(message) {
var errorObject = {};
errorObject.message = message;
errorObject.code = 10001; // as you want
errorObject.status = "failed";
return errorObject;
}
in try block you can also handle logical errors using same function i.e writeCustomError
So if you use this approach you can also implement end-to-end encryption and send only eP('encrypted payload') and eK('encryption Key'),by doing this end users and bad end users even can not evaluate your serve API calls.
If you are thinking how will you route different paths at server then simplest solution is send uri in payload from client to server for e.g
User wants to reset password :-
then
call api like this
https://domain.tld/server/execute and send Json object in payload like this {uri:"reset-password",old:"",new:""}.
at backend
use
app.all("/server/execute", function (req, res) {
try {
// decrypt payload
req.url = payload.uri;
next();
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
app.all("/reset-password", function (req, res) {
try {
// reset logic
} catch (error) {
// error handling
var err = writeCustomError(error.message || error.errmsg || error.stack);
res.status(417).json(err).end();
}
});
so in this way only developer know where password reset logic and how it can called and what parameters are required.
I will also suggest you to create different router files for express like QueryRouter,InsertRouter etc.
Also try to implement end-to-end encryption.Any query regarding post,kindly comment it.
I have a Firebase Cloud Function that does the following:
const firestore = require('firebase-admin')
const functions = require('firebase-functions')
const regex = require('../../utils/regex')
exports = module.exports = functions.https.onCall((data, context) => {
if (!context.auth) {
throw new functions.https.HttpsError('failed-precondition', 'The function must be called while authenticated.')
}
if (!data['displayName']) {
throw new functions.https.HttpsError('failed-precondition', 'An display name must be provided.')
}
if (!regex.noSpecialCharactersExceptSpace.test(data['displayName'])) {
throw new functions.https.HttpsError('input-validation-failed', 'Your display name cannot have special characters in it.')
}
return firestore.firestore().collection('profiles').doc(context.auth.uid)
.update({
displayName: data['displayName']
})
.then(() => {
console.info('Successful public profile update user='+ context.auth.uid)
return { text: 'Your profile has successfully been updated' };
})
.catch((error) => {
console.error('Error updating public profile user=' + context.auth.uid + ' error=' + error)
throw new functions.https.HttpsError('failed-precondition', 'An error happened, our team will look into it.')
})
})
From my front-end, when I call this function, it can take up to 20-30 seconds for it to complete and return a status code. This delay really disrupts the user experience.
What's the best way to improve the response time?
Testing
Direct calls to Firestore from the UI resolve incredibly quickly. Thus, it seems unlikely that the problem is DNS from our side.
Other API calls to Cloud Function from the UI, like "Invite a Friend", resolve quickly, and are not affected by this fault.
Calls to other Cloud Functions, which do not return a Promise, but do things like send emails with Postmark, are not affected by this fault and also resolve incredibly quickly. Thus, it seems that the problem isn't the location of the Firebase project (us-central1). Although changing the location hasn't been tested.
The fault only occurs on this one function. The one copied and pasted above.
The only difference between the affected function, and our other functions, is that the one with the fault returns a Promise for a Firestore operation.
The situation is that I want to store some information in a DynamoDB database, and then send a notification only if I am sure that this information has been stored successfully. Here's what my code looks like:
const putObj = Bluebird.promisify(docClient.put, { context: docClient })
// ... a promise chain collecting information...
const params = {
TableName: 'my_table',
Item: {
name: itemName
}
}
return putObj(params)
.then((data) => {
if (Ramda.equals(data, {})) {
// ... send notification here
.catch((err) => {
throw err
})
I assume that if there is an error storing the data, this will be caught in the catch block and the notification will not be sent. However, I'm seeing in some cases that notifications are being sent despite the fact that the respective information has not been stored in the database. This only happens infrequently - in almost all cases the code functions as desired.
Any ideas about why/how this could be happening, and what I can do to stop it. DynamoDB doesn't return any useful information into the .then - it is only an empty object in case of success, and I'm already checking this before sending the notification.
Are you using 'eventually consistent' or 'strongly consistent' reads?
When you read data from a DynamoDB table, the response might not
reflect the results of a recently completed write operation. The
response might include some stale data. If you repeat your read
request after a short time, the response should return the latest
data.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html
You may need to retry your read, or else change the read consistency value.