Develop Reference

Async Lambda Function: Returning promise or sending responseURL does not terminate CloudFormation custom resource invocation

I have a lambda function invoked as a custom resource via a CloudFormation template. It Creates/Deletes AWS Connect instances. The API calls work fine but I cannot seem to terminate the custom resource invocation, so the last CF block remains CREATE_IN_PROGRESS. No matter what I return from the async function it just won't terminate the CF execution with a success.
I'm able to use a non-async handler successfully as in https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-custom-resources-lambda-lookup-amiids.html but I need to make multiple API calls and await completions, hence the need for async handler.
Below is the code in it's simplest form, though I've tried just about everything, including using callback and context (ie exports.handler = async function(event, context, callback) {...}), both of which should be unnecessary with an async handler. I've tried using cfn-response to directly send a response which seems to be ignored with async handlers. I've tried returning directly the promises with and without the await before them, tried returning variables containing various responseStatus and responseData, nothing seems to work.
Transform: 'AWS::Serverless-2016-10-31'
Parameters:
IdentityManagementType:
Description: The type of identity management for your Amazon Connect users.
Type: String
AllowedValues: ["SAML", "CONNECT_MANAGED", "EXISTING_DIRECTORY"]
Default: "SAML"
InboundCallsEnabled:
Description: Whether your contact center handles incoming contacts.
Type: String
AllowedValues: [true, false]
Default: true
InstanceAlias:
Description: The name for your instance.
Type: String
MaxLength: 62
OutboundCallsEnabled:
Description: Whether your contact center allows outbound calls.
Type: String
AllowedValues: [true, false]
Default: true
DirectoryId:
Description: Optional. The identifier for the directory, if using this type of Identity Management.
Type: String
ClientToken:
Description: Optional. The idempotency token. Used for concurrent deployments
Type: String
MaxLength: 500
Region:
Description: Region to place the AWS Connect Instance
Type: String
Default: us-east-1
#Handler for optional values
Conditions:
HasClientToken: !Not
- !Equals
- ""
- !Ref ClientToken
HasDirectoryId: !Not
- !Equals
- ""
- !Ref DirectoryId
Resources:
CreateConnectInstance:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub "${AWS::StackName}-AWSConnectInstance"
Handler: index.handler
Runtime: nodejs12.x
Description: Invoke a function to create an AWS Connect instance.
MemorySize: 128
Timeout: 30
Role: !GetAtt LambdaExecutionRole.Arn
Layers:
- !Sub "arn:aws:lambda:us-east-1:${AWS::AccountId}:layer:node_sdk:1"
Environment:
Variables:
IdentityManagementType:
Ref: IdentityManagementType
InboundCallsEnabled:
Ref: InboundCallsEnabled
InstanceAlias:
Ref: InstanceAlias
OutboundCallsEnabled:
Ref: OutboundCallsEnabled
Region:
Ref: Region
#Optional Values
ClientToken: !If
- HasClientToken
- !Ref ClientToken
- !Ref "AWS::NoValue"
DirectoryId: !If
- HasClientToken
- !Ref ClientToken
- !Ref "AWS::NoValue"
InlineCode: |
var aws = require("aws-sdk");
exports.handler = async function(event) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
var connect = new aws.Connect({region: event.ResourceProperties.Region});
var isInboundCallsEnabled = (process.env.InboundCallsEnabled == 'true');
var isOutboundCallsEnabled = (process.env.OutboundCallsEnabled == 'true');
var createInstanceParams = {
InboundCallsEnabled: isInboundCallsEnabled,
OutboundCallsEnabled: isOutboundCallsEnabled,
IdentityManagementType: process.env.IdentityManagementType,
ClientToken: process.env.ClientToken,
DirectoryId: process.env.DirectoryId,
InstanceAlias: process.env.InstanceAlias
};
// Create AWS Connect instance using specified parameters
if (event.RequestType == "Create") {
return await connect.createInstance(createInstanceParams).promise();
// I can store this in a variable and read the contents fine, but...
// returning the promise does not terminate execution
}
};
InvokeCreateConnectInstance:
Type: Custom::CreateConnectInstance
Properties:
ServiceToken: !GetAtt CreateConnectInstance.Arn
Region: !Ref "AWS::Region"
The documentaiton at https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html explicitly states that you should be able to return await apiCall.promise() directly from any async function, exactly what I'm trying to do, such as
const s3 = new AWS.S3()
exports.handler = async function(event) {
return s3.listBuckets().promise()
}
Why can't I return from my async function? Again the API calls are working, the Connect instances are created and deleted (though I've omitted the delete code for brevity), but CF just hangs hours and hours until eventually saying "Custom Resource failed to stabilize in expected time"
Here's the inline code by itself for readability:
exports.handler = async function(event) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
var connect = new aws.Connect({region: event.ResourceProperties.Region});
var isInboundCallsEnabled = (process.env.InboundCallsEnabled == 'true');
var isOutboundCallsEnabled = (process.env.OutboundCallsEnabled == 'true');
var createInstanceParams = {
InboundCallsEnabled: isInboundCallsEnabled,
OutboundCallsEnabled: isOutboundCallsEnabled,
IdentityManagementType: process.env.IdentityManagementType,
ClientToken: process.env.ClientToken,
DirectoryId: process.env.DirectoryId,
InstanceAlias: process.env.InstanceAlias
};
// Create AWS Connect instance using specified parameters
if (event.RequestType == "Create") {
return await connect.createInstance(createInstanceParams).promise();
// I can store this in a variable and read the contents fine, but...
// returning the promise does not terminate CF execution
}
};
UPDATE: I've implemented the sendResponse method exactly as shown in the AMI lookup example (the first link) and am sending exactly the correct structure for the response, it even includes the newly created connect instance ID in the data field:
{
"Status": "SUCCESS",
"Reason": "See the details in CloudWatch Log Stream: 2020/12/23/[$LATEST]6fef3553870b4fba90479a37b4360cee",
"PhysicalResourceId": "2020/12/23/[$LATEST]6fef3553870b4fba90479a37b4360cee",
"StackId": "arn:aws:cloudformation:us-east-1:642608065726:stack/cr12/1105a290-4534-11eb-a6de-0a8534d05dcd",
"RequestId": "2f7c3d9e-941f-402c-b739-d2d965288cfe",
"LogicalResourceId": "InvokeCreateConnectInstance",
"Data": {
"InstanceId": "2ca7aa49-9b20-4feb-8073-5f23d63e4cbc"
}
}
And STILL the custom resource will just not close in CloudFormation. I just don't understand why this is happening when I am returning the above to the event.responseURL. It's like specifying an async handler completely breaks the custom resource handler and prevents it from closing.
UPDATE: When I manually CURL the above response directly to the event.responseUrl the CF resource registers a success! WTF... I'm sending the exact same response as the lambda function is sending, and it accepts it from the CURL but not from my lambda function.
UPDATE: latest code including sendResponse, etc
var aws = require("aws-sdk");
exports.handler = async function(event, context, callback) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
var connect = new aws.Connect({region: event.ResourceProperties.Region});
var isInboundCallsEnabled = (process.env.InboundCallsEnabled == 'true');
var isOutboundCallsEnabled = (process.env.OutboundCallsEnabled == 'true');
var createInstanceParams = {
InboundCallsEnabled: isInboundCallsEnabled,
OutboundCallsEnabled: isOutboundCallsEnabled,
IdentityManagementType: process.env.IdentityManagementType,
ClientToken: process.env.ClientToken,
DirectoryId: process.env.DirectoryId,
InstanceAlias: process.env.InstanceAlias
};
var responseStatus;
var responseData = {};
// Create Connect instance
if (event.RequestType == "Create") {
try {
var createInstanceRequest = await connect.createInstance(createInstanceParams).promise();
responseStatus = "SUCCESS";
responseData = {"InstanceId": createInstanceRequest.Id};
} catch (err) {
responseStatus = "FAILED";
responseData = {Error: "CreateInstance failed"};
console.log(responseData.Error + ":\n", err);
}
sendResponse(event, context, responseStatus, responseData);
return;
}
// Look up the ID and call deleteInstance.
if (event.RequestType == "Delete") {
var instanceId;
var listInstanceRequest = await connect.listInstances({}).promise();
listInstanceRequest.InstanceSummaryList.forEach(instance => {
if (instance.InstanceAlias == createInstanceParams.InstanceAlias) {
instanceId = instance.Id;
}
});
if (instanceId !== undefined) {
try {
var deleteInstanceRequest = await connect.deleteInstance({"InstanceId": instanceId}).promise();
responseStatus = "SUCCESS";
responseData = {"InstanceId": instanceId};
} catch (err) {
responseStatus = "FAILED";
responseData = {Error: "DeleteInstance call failed"};
console.log(responseData.Error + ":\n", err);
}
} else {
responseStatus = "FAILED";
responseData = {Error: "DeleteInstance failed; no match found"};
console.log(responseData.Error);
}
sendResponse(event, context, responseStatus, responseData);
return;
}
};
// Send response to the pre-signed S3 URL
function sendResponse(event, context, responseStatus, responseData) {
var responseBody = JSON.stringify({
Status: responseStatus,
Reason: "CloudWatch Log Stream: " + context.logStreamName,
PhysicalResourceId: context.logStreamName,
StackId: event.StackId,
RequestId: event.RequestId,
LogicalResourceId: event.LogicalResourceId,
Data: responseData
});
console.log("RESPONSE BODY:\n", responseBody);
var https = require("https");
var url = require("url");
var parsedUrl = url.parse(event.ResponseURL);
var options = {
hostname: parsedUrl.hostname,
port: 443,
path: parsedUrl.path,
method: "PUT",
headers: {
"content-type": "",
"content-length": responseBody.length
}
};
console.log("SENDING RESPONSE...\n");
var request = https.request(options, function(response) {
console.log("STATUS: " + response.statusCode);
console.log("HEADERS: " + JSON.stringify(response.headers));
// Tell AWS Lambda that the function execution is done
context.done();
});
request.on("error", function(error) {
console.log("sendResponse Error:" + error);
// Tell AWS Lambda that the function execution is done
context.done();
});
// write data to request body
request.write(responseBody);
request.end();
}
Been at this for two days now :(
PS in the logs the "RESPONSE BODY" is shown as expected like I copied above, and log shows the "SENDING RESPONSE" but does not get to the the "STATUS: " and "HEADERS: " portion of the request.https() call, which makes me think something with async interferes with this call... IDK

This one was really tricky but finally have everything figured out. I had to make the sendResponse function asynchronous by adding a promise to it, awaiting that promise and returning it. This allowed me to ultimately call "return await sendResponse(event, context, responseStatus, responseData);" and finally everything is working, both create and delete operations are successful and the CloudFormation custom resource completes as expected. Phew. Posting code here in hopes that others will benefit from it.
var aws = require("aws-sdk");
exports.handler = async function(event, context, callback) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
var connect = new aws.Connect({region: event.ResourceProperties.Region});
var isInboundCallsEnabled = (process.env.InboundCallsEnabled == 'true');
var isOutboundCallsEnabled = (process.env.OutboundCallsEnabled == 'true');
var createInstanceParams = {
InboundCallsEnabled: isInboundCallsEnabled,
OutboundCallsEnabled: isOutboundCallsEnabled,
IdentityManagementType: process.env.IdentityManagementType,
ClientToken: process.env.ClientToken,
DirectoryId: process.env.DirectoryId,
InstanceAlias: process.env.InstanceAlias
};
var responseStatus;
var responseData = {};
if (event.RequestType == "Create") {
try {
var createInstanceRequest = await connect.createInstance(createInstanceParams).promise();
responseStatus = "SUCCESS";
responseData = {"InstanceId": createInstanceRequest.Id};
} catch (err) {
responseStatus = "FAILED";
responseData = {Error: "CreateInstance failed"};
console.log(responseData.Error + ":\n", err);
}
return await sendResponse(event, context, responseStatus, responseData);
}
if (event.RequestType == "Delete") {
var instanceId;
var listInstanceRequest = await connect.listInstances({}).promise();
listInstanceRequest.InstanceSummaryList.forEach(instance => {
if (instance.InstanceAlias == createInstanceParams.InstanceAlias) {
instanceId = instance.Id;
}
});
if (instanceId !== undefined) {
try {
var deleteInstanceRequest = await connect.deleteInstance({"InstanceId": instanceId}).promise();
responseStatus = "SUCCESS";
responseData = {"InstanceId": instanceId};
} catch (err) {
responseStatus = "FAILED";
responseData = {Error: "DeleteInstance call failed"};
console.log(responseData.Error + ":\n", err);
}
} else {
responseStatus = "FAILED";
responseData = {Error: "DeleteInstance failed; no match found"};
console.log(responseData.Error);
}
return await sendResponse(event, context, responseStatus, responseData);
}
};
async function sendResponse(event, context, responseStatus, responseData) {
let responsePromise = new Promise((resolve, reject) => {
var responseBody = JSON.stringify({
Status: responseStatus,
Reason: "CloudWatch Log Stream: " + context.logStreamName,
PhysicalResourceId: context.logStreamName,
StackId: event.StackId,
RequestId: event.RequestId,
LogicalResourceId: event.LogicalResourceId,
Data: responseData
});
console.log("RESPONSE BODY:\n", responseBody);
var https = require("https");
var url = require("url");
var parsedUrl = url.parse(event.ResponseURL);
var options = {
hostname: parsedUrl.hostname,
port: 443,
path: parsedUrl.path,
method: "PUT",
headers: {
"content-type": "",
"content-length": responseBody.length
}
};
console.log("SENDING RESPONSE...\n");
var request = https.request(options, function(response) {
console.log("STATUS: " + response.statusCode);
console.log("HEADERS: " + JSON.stringify(response.headers));
resolve(JSON.parse(responseBody));
context.done();
});
request.on("error", function(error) {
console.log("sendResponse Error:" + error);
reject(error);
context.done();
});
request.write(responseBody);
request.end();
});
return await responsePromise;
}

This answer is a variant on the OP's answer for those using the "ZipFile" option in the "Code" property of an AWS::Lambda::Function resource in CloudFormation. The advantage of the ZipFile approach is that in addition permitting Lambda code inlined into the CF template, it also automatically bundles a "cfn-response.js" function very similar to the "async function sendResponse" in the OP's answer. With the insight gained from the OP's answer regarding a promised response (thank you, I was stuck and perplexed), this is how I incorporated the cfn-response function as an await-able Promise to signal CF after my asynchronous AWS API calls (omitted for brevity) were complete:
CreateSnapshotFunction:
Type: AWS::Lambda::Function
Properties:
Runtime: nodejs12.x
Handler: index.handler
Timeout: 900 # 15 mins
Code:
ZipFile: !Sub |
const resp = require('cfn-response');
const aws = require('aws-sdk');
const cf = new aws.CloudFormation({apiVersion: '2010-05-15'});
const rds = new aws.RDS({apiVersion: '2014-10-31'});
exports.handler = async function(evt, ctx) {
if (evt.RequestType == "Create") {
try {
// Query the given CF stack, determine its database
// identifier, create a snapshot of the database,
// and await an "available" status for the snapshot
let stack = await getStack(stackNameSrc);
let srcSnap = await createSnapshot(stack);
let pollFn = () => describeSnapshot(srcSnap.DBSnapshot.DBSnapshotIdentifier);
let continueFn = snap => snap.DBSnapshots[0].Status !== 'available';
await poll(pollFn, continueFn, 10, 89); // timeout after 14 min, 50 sec
// Send response to CF
await send(evt, ctx, resp.SUCCESS, {
SnapshotId: srcSnap.DBSnapshot.DBSnapshotIdentifier,
UpgradeRequired: upgradeRequired
});
} catch(err) {
await send(evt, ctx, resp.FAILED, { ErrorMessage: err } );
}
} else {
// Send success to CF for delete and update requests
await send(evt, ctx, resp.SUCCESS, {});
}
};
function send(evt, ctx, status, data) {
return new Promise(() => { resp.send(evt, ctx, status, data) });
}

How to send a html email in AWS Lambda node js returning a well formed response for AWS api gateway

I need to create an api that sends html emails through amazon SES. I created properly my ses credentials and I want to create an AWS lambda in javascript (nodejs). Due the lack of good AWS documentation for sending an email using javascript, I need to know how to create a lambda that sends an email and responds with a proper message to AWS api gateway.

The following was code written in javascript. Works perfect in AWS lambda and works great when is invoked from AWS Api gateway (no malformed message errors):
var aws = require("aws-sdk");
var ses = new aws.SES({
accessKeyId: "Your SES Access Key",
secretAccesskey: "Your Secret key",
region: "us-east-1" // CHANGE with the region where you configured SES
});
exports.handler = function(event, context, callback) {
var requestPath = JSON.parse(JSON.stringify(event.pathParameters));
var requestBody = JSON.parse(event.body);
var responseBody = {};
var response = {};
if (
requestBody &&
requestBody.emailFrom &&
requestBody.subject &&
requestBody.htmlBody
) {
var emailTo = requestPath.emailto;
var emailFrom = requestBody.emailFrom;
var subject = requestBody.subject;
var htmlBody = requestBody.htmlBody;
} else {
responseBody = {
result: "fail",
resultCode: 400,
description:
"Incorrect Parameters. Mandatory: emailFrom, subject and bodyHTML"
};
response = {
statusCode: 400,
headers: {
"Access-Control-Allow-Origin": "*"
},
body: JSON.stringify(responseBody),
isBase64Encoded: false
};
callback(null, response);
}
var emailParams = {
Destination: {
ToAddresses: [emailTo]
},
Message: {
Body: {
Html: {
Data: htmlBody
}
},
Subject: {
Data: subject
}
},
Source: emailFrom
};
var email = ses.sendEmail(emailParams, function(err, data) {
var resultCode = 200;
if (err) {
var responseBody = {
result: "FAIL",
resultCode: 500,
description: "Error sending email: " + err
};
resultCode = 500;
} else {
var responseBody = {
result: "OK",
resultCode: 200,
description: "Email sent successfully"
};
}
response = {
statusCode: resultCode,
headers: {
"Access-Control-Allow-Origin": "*"
},
body: JSON.stringify(responseBody),
isBase64Encoded: false
};
callback(null, response);
});
};

aws upload object to S3 bucket and pass details of data to lambda

Working my way through tutorials for AWS...So ive created an S3 bucket which when a file is dropped into it calls my lambda 'testHelloWorld' which sends an email...this all works fine (see below)
'use strict';
console.log('Loading function');
var aws = require('aws-sdk');
var ses = new aws.SES({
region: 'us-west-2'
});
exports.handler = function(event, context) {
console.log("Incoming: ", event);
// var output = querystring.parse(event);
var eParams = {
Destination: {
ToAddresses: ["johnb#hotmail.com"]
},
Message: {
Body: {
Text: {
Data: "Hey! What is up?"
}
},
Subject: {
Data: "Email Subject!!!"
}
},
Source: "johnb#hotmail.com"
};
console.log('===SENDING EMAIL===');
var email = ses.sendEmail(eParams, function(err, data){
if(err) console.log(err);
else {
console.log("===EMAIL SENT===");
console.log(data);
console.log("EMAIL CODE END");
console.log('EMAIL: ', email);
context.succeed(event);
}
});
};
but I want to extend the email to include data on the file that was uploaded to the bucket. I have found How to trigger my Lambda Function once the file is uploaded to s3 bucket which gives a node.js code snippet which should capture the data. I have tried to import this into my existing lambda
'use strict';
console.log('Loading function');
var aws = require('aws-sdk');
var ses = new aws.SES({
region: 'us-west-2'
});
var s3 = new aws.S3({ apiVersion: '2006-03-01', accessKeyId: process.env.ACCESS_KEY, secretAccessKey: process.env.SECRET_KEY, region: process.env.LAMBDA_REGION });
exports.handler = function(event, context, exit){
console.log("Incoming: ", event);
// var output = querystring.parse(event);
// Get the object from the event and show its content type
// const bucket = event.Records[0].s3.bucket.name;
// const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
const params = {
Bucket: 'bucketName',
Key: 'keyName',
Source : 'SourceName',
Destination : 'DestinationName',
Message : 'MessageName'
};
s3.getObject(function(err, data){
if (err) {
console.log('ERROR ' + err);
// exit(err);
} else {
// the data has the content of the uploaded file
var eParams = {
Destination: {
ToAddresses: ["johnboy#hotmail.com"]
},
Message: {
Body: {
Text: {
Data: data
}
},
Subject: {
Data: "Email Subject!!!"
}
},
Source: "johnboy#hotmail.com"
};
}
});
console.log('===SENDING EMAIL===');
var email = ses.sendEmail(eParams, function(err, data){
if(err) console.log(err);
else {
console.log("===EMAIL SENT===");
console.log(data);
console.log("EMAIL CODE END");
console.log('EMAIL: ', email);
context.succeed(event);
}
});
};
but this is failing on the params
message: 'There were 3 validation errors:
* MissingRequiredParameter: Missing required key \'Source\' in params
* MissingRequiredParameter: Missing required key \'Destination\' in params
* MissingRequiredParameter: Missing required key \'Message\' in params',
code: 'MultipleValidationErrors',
errors:
These source, destination and message are listed in the params, are they not correctly formatted and it isnt picking them up?
I cant find much online....any help appreciated
UPDATE
Ok iv got it working without failing...if i use the test function in the lambda with the following code...
'use strict';
console.log('Loading function');
var aws = require('aws-sdk');
var ses = new aws.SES({
region: 'us-west-2'
});
var s3 = new aws.S3({ apiVersion: '2006-03-01', accessKeyId: process.env.ACCESS_KEY, secretAccessKey: process.env.SECRET_KEY, region: process.env.LAMBDA_REGION });
exports.handler = function(event, context) {
console.log("Incoming: ", event);
// var output = querystring.parse(event);
var testData = null;
// Get the object from the event and show its content type
// const bucket = event.Records[0].s3.bucket.name;
// const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
const params = {
Bucket: 'bucket',
Key: 'key',
};
s3.getObject(params, function(err, data){
if (err) {
console.log('ERROR ' + err);
exit(err);
} else {
testData = data;
}
});
var eParams = {
Destination: {
ToAddresses: ["jim#him.com"]
},
Message: {
Body: {
Text: { Data: 'testData2' + testData}
},
Subject: {
Data: "Email Subject!!!"
}
},
Source: "jim#him.com"
};
console.log('===SENDING EMAIL===');
var email = ses.sendEmail(eParams, function(err, data){
if(err) console.log(err);
else {
console.log("===EMAIL SENT===");
console.log(data);
console.log("EMAIL CODE END");
console.log('EMAIL: ', email);
context.succeed(event);
}
});
};
I get the email with the body- testData2null
So I tried uploading an image through the s3 bucket and I still get the email with the body testData2null
is there anyway to debug this further or does anyone kno who it is saying null. I never actually tested the code from the other post which passes the data over to the email I just assumed it would work. Does anyone else know who to obtain the data from the upload please? thanks

You are declaring the var eParams within the callback of s3.getObject, but then you run the ses.sendMail outside of the callback. I think that's why!
You also need to move the ses.sendEmail to inside the callback of s3.getObject if you want to send the data from your object inside the email.
Try this:
s3.getObject(function(err, objectData) {
if (err) {
console.log('Could not fetch object data: ', err);
} else {
console.log('Data was successfully fetched from object');
var eParams = {
Destination: {
ToAddresses: ["johnboy#hotmail.com"]
},
Message: {
Body: {
Text: {
Data: objectData
}
},
Subject: {
Data: "Email Subject!!!"
}
},
Source: "johnboy#hotmail.com"
};
console.log('===SENDING EMAIL===');
var email = ses.sendEmail(eParams, function(err, emailResult) {
if (err) console.log('Error while sending email', err);
else {
console.log("===EMAIL SENT===");
console.log(objectData);
console.log("EMAIL CODE END");
console.log('EMAIL: ', emailResult);
context.succeed(event);
}
});
}
});

You need to read on how Nodejs works. It is event based and depends on callbacks and promises. You should do -
s3.getObject(params, function(err, data){
//This is your callback for s3 API call. DO stuff here
if (err) {
console.log('ERROR ' + err);
exit(err);
} else {
testData = data;
// Got your data. Send the mail here
}
});
I have added my comments in code above. Since Nodejs is single threaded it will make S3 api call and go ahead. When it is sending mail s3 api call is not complete so data is null. It is better to use promises here.
Anyway read up on callback and promises in nodejs and how it works. But hope this answers your logical error.

Async - Can't set headers after they are sent

I am new to node and async...
I am getting an error saying I can't set headers after they sent when I am sending a response back to api-ai
Any idea why?
Below is the code for function - getUserFirstName(userId, name, callback):
var name = "";
function getUserFirstName(userId, name, callback) {
console.log('withParams function called');
request({
method: 'GET',
uri: "https://graph.facebook.com/v2.6/" + userId + "?fields=first_name,last_name,profile_pic,locale,timezone,gender&access_token=" + FB_PAGE_ACCESS_TOKEN
},
function (error, response) {
if (error) {
console.error('Error while userInfoRequest: ', error);
} else {
if(!typeof response.body != 'object'){
var body = JSON.parse(response.body);
name = body.first_name;
callback(null,name);
}else{
name = response.body.first_name;
callback(null,name);
}
}
});
}
Here is the code being executed:
app.post('/webhook/', (req, res) => {
var data = JSONbig.parse(req.body);
var action = data.result.action;
var facebook_message = [];
if(action == "input.welcome"){
var userId = data.originalRequest.data.sender.id;
async.series([
function(callback) {
getUserFirstName(userId, name, callback);
}
], function(err,results) {
if (results != undefined){ // results = "John"
facebook_message = [{
"text":"Heyyyoo. Welcome!"
}]
}else{
facebook_message = [{
"text":"Hey " + results +"! Welcome!" // Hey John! Welcome!
}]
}
res.json({ // line 308 - error here!
speech: "Greetings",
displayText: "Greetings",
"data": {
"facebook": facebook_message
},
source: "webhook"
});
});
}
// BUNCH OF LONG AND MESSY CODES OVER HERE...
return res.status(200).json({
status: "ok"
});
Error
Error: Cant set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:356:11)
at ServerResponse.header (/app/node_modules/express/lib/response.js:719:10)
at ServerResponse.send (/app/mode_modules/express/lib/response.js:164:12)
at ServerRespose.json (/app/mode_modules/express/lib/response.js:250:15)
at /app/src/app.js: 308:15
at /app/node_modules/async/dist/async.js:3694:9
at /app/node_modules/async/dist/async.js:356:16
at replenish (/app/node_modules/async/dist/async.js:877.25)
at iterateeCallback (/app/node_modules/async/dist/async.js:867:17)
at /app/node_modules/async/dist/async.js:840:16

Remove the following:
return res.status(200).json({
status: "ok"
});

Upload images to twitter API from node.js

I'm attempting to post an image onto the twitter api, v1.1
I've tried just about all the example out there, and nothing seems to be able to post it.
include Posting images to twitter in Node.js using Oauth
I'm using the oauth library mentioned there, and I also had jsOauth, which I thought I'd give a shot according to https://gist.github.com/lukaszkorecki/1038408
Nothing has worked, and at this point I'm starting to lose hope on whether I can even do this.
function postStatusWithMedia(status, file) {
var err = new Object();
if(fs.existsSync(file) === false) {
err.message = "File not found :(";
parseTwitterError(err);
} else {
var oauth = OAuth(options = {
"consumerKey": consumer_key,
"consumerSecret": consumer_secret,
"accessTokenKey": access_token,
"accessTokenSecret": access_token_secret
});
callbacks = {
onSuccess : function() {
console.log('upload worked!')
},
onFailure : function() {
console.log('upload failed!');
console.dir(arguments);
}
},
uploadData = {
'status' : status,
'media' : Base64.encode(fs.readFileSync(file))
};
oauth.post('https://api.twitter.com/1.1/statuses/update_with_media.json',uploadData, callbacks.onSuccess, callbacks.onFailure);
return false;
}
}
If it can't be done, can you please explain why?
Otherwise, anything that could lead me to the right direction would be great.

var fs = require('fs');
var request = require('request');
var FormData = require('form-data');
var utf8 = require('utf8');
// Encode in UTF-8
status = utf8.encode(status);
var form = new FormData();
form.append('status', status)
form.append('media[]', fs.createReadStream(file));
// Twitter OAuth
form.getLength(function(err, length){
if (err) {
return requestCallback(err);
}
var oauth = {
consumer_key: consumer_key,
consumer_secret: consumer_secret,
token: access_token,
token_secret: access_token_secret
};
var r = request.post({url:"https://api.twitter.com/1.1/statuses/update_with_media.json", oauth:oauth, host: "api.twitter.com", protocol: "https:"}, requestCallback);
r._form = form;
r.setHeader('content-length', length);
});
function requestCallback(err, res, body) {
if(err) {
throw err;
} else {
console.log("Tweet and Image uploaded successfully!");
}
}
I ended up using request and node-form-data to manually construct a multipart/form-data request and send it with the status request, utf8 was for encoding the status into UTF-8, not doing so caused issues with '<3' and other characters.

I have not tested these code.Its from my colleague.sure the code is working.
Perhaps this will help.
//twitter_update_with_media.js
(function() {
var fs, path, request, twitter_update_with_media;
fs = require('fs');
path = require('path');
request = require('request');
twitter_update_with_media = (function() {
function twitter_update_with_media(auth_settings) {
this.auth_settings = auth_settings;
this.api_url = 'https://api.twitter.com/1.1/statuses/update_with_media.json';
}
twitter_update_with_media.prototype.post = function(status, imageUrl, callback) {
var form, r;
r = request.post(this.api_url, {
oauth: this.auth_settings
}, callback);
form = r.form();
form.append('status', status);
return form.append('media[]', request(imageUrl));
};
return twitter_update_with_media;
})();
module.exports = twitter_update_with_media;
}).call(this);
next file
//upload_to_twitter.js
var tuwm = new twitter_update_with_media({
consumer_key: TWITTER_OAUTH_KEY,
consumer_secret: TWITTER_OAUTH_SECRET,
token: access[0],
token_secret: access[1]
});
media_picture.picture = imageURL;
if (media_picture.picture) {
console.log('with media upload');
request.head(media_picture.picture,
function (error, response, body) {
if (!error && response.statusCode == 200) {
var image_size = response.headers['content-length'];
if (image_size > 2000000) { // 2mb max upload limit
console.log('greater than 2mb');
sendMessageWithoutImage(err, req, res, next, twit, wallpost, access);
} else {
console.log('less than 2mb');
console.log('twitter text', content);
tuwm.post(content, media_picture.picture, function(err, response) {
if (err) {
console.log('error', err);
return next(err);
}
error_parse = JSON.parse(response.body);
console.log('with media response', response.body);
if (error_parse.errors) {
console.log('have errors', error_parse);
res.json({
status: 500,
info: error_parse.errors[0].code + ' ' + error_parse.errors[0].message
});
} else {
res.json({
status: 200,
info: "OK",
id: response.id
});
}
});
}
}
});