I have an automation that takes a Webhook from a service and posts it to Slack. The webhook goes to an API Gateway URL, authorizes through a Lambda authorizer function, and then goes to the Lambda function. This process is outlined here.
The verification is in the header["authentication"] field, which is validated by the authorizer. I scoped that text in the authorizer, and it passes it to the Lambda proxy as authenticationToken. If validated, a certificate is created and the webhook is passed to Lambda. This has all been working for a few months.
This year, as many companies tend to do, the service with the webhook is deprecating this auth method, and changing their authentication to a "Challenge-Response Check" style. So, the webhook sends a validation webhook, and the authorizer needs to make a hashed token out of several headers, including timestamp, and then the authorizer passes that hashed token back to the webhook, before sending the real webhook payload. Here is a node/express sample of how to do this:
app.post('/webhook', (req, res) => {
var response
console.log(req.body)
console.log(req.headers)
// construct the message string
const message = `v0:${req.headers['x-zm-request-timestamp']}:${JSON.stringify(req.body)}`
const hashForVerify = crypto.createHmac('sha256', process.env.ZOOM_WEBHOOK_SECRET_TOKEN).update(message).digest('hex')
// hash the message string with your Webhook Secret Token and prepend the version semantic
const signature = `v0=${hashForVerify}`
// you validating the request came from Zoom https://marketplace.zoom.us/docs/api-reference/webhook-reference#notification-structure
if (req.headers['x-zm-signature'] === signature) {
// Zoom validating you control the webhook endpoint https://marketplace.zoom.us/docs/api-reference/webhook-reference#validate-webhook-endpoint
if(req.body.event === 'endpoint.url_validation') {
const hashForValidate = crypto.createHmac('sha256', process.env.ZOOM_WEBHOOK_SECRET_TOKEN).update(req.body.payload.plainToken).digest('hex')
response = {
message: {
plainToken: req.body.payload.plainToken,
encryptedToken: hashForValidate
},
status: 200
}
....
Is this methodology supported by the authorizer/Lambda method? The APIG authorizer would need to send several headers to the Lambda authorizer function, vs just the one. I see a new feature that is changing the authorizer Type to Request (from Token), but I was not able to pass the headers to the auth function that way, after re-creating and re-deploying the APIG.
Or, should I get rid of the authorizer all together, and just do my authentication on the actual Lambda?
Or, should I start to use Lambda's newer URL feature, and get rid of APIG all together?
What is the best workflow in this use case?
Any advice or links appreciated.
Related
Context
I'm following Google's RTDNs guide on enabling Real-Time Developer Notifications. I've successfully created the topic and subscription and have received the push notifications sent to the API that I have created. I would now like to authenticate and validate these messages. For that, I'm following this guide on Authentication and Authorization. Their developer documentation here and here has a seemingly useful example.
The Issue
After following the resources outlined above, I get the following error:
Error: No pem found for envelope: {"typ":"JWT","alg":"HS256"}
Relevant Code
const authClient = new OAuth2Client();
// ...
app.post('/pubsub/authenticated-push', jsonBodyParser, async (req, res) => {
// Verify that the push request originates from Cloud Pub/Sub.
try {
// Get the Cloud Pub/Sub-generated JWT in the "Authorization" header.
const bearer = req.header('Authorization');
const [, token] = bearer.match(/Bearer (.*)/);
// Verify and decode the JWT.
// Note: For high volume push requests, it would save some network
// overhead if you verify the tokens offline by decoding them using
// Google's Public Cert; caching already seen tokens works best when
// a large volume of messages have prompted a single push server to
// handle them, in which case they would all share the same token for
// a limited time window.
// verifyIdToken is failing here with the `No pem found for envelope` error
const ticket = await authClient.verifyIdToken({
idToken: token,
audience: 'example.com',
});
// ...
} catch (e) {
res.status(400).send('Invalid token');
return;
}
res.status(200).send();
});
The Questions
From this, I'm assuming I need to have some public key.
Where do I get said public key?
Where do I put said public key so that the google client is initialized with it?
How can I generate an example JWT to test my endpoint?
Edits
I was able to find the source of this error in their code here:
if (!Object.prototype.hasOwnProperty.call(certs, envelope.kid)) {
// If this is not present, then there's no reason to attempt verification
throw new Error('No pem found for envelope: ' + JSON.stringify(envelope));
}
However, I've verified that the kid attribute does indeed exist in the decoded object:
{"alg":"RS256","kid":"7d680d8c70d44e947133cbd499ebc1a61c3d5abc","typ":"JWT"}
Turns out the kid was invalid and therefore threw the No pem found for envelope error. Once a valid kid was supplied, the error no longer persisted.
I have an AWS Lambda function written in python that is initiated by a Zapier trigger that I set up. As I pass some input parameters to the function in the Zapier trigger, I can access to the input parameters in my python code by using variables such as event[parameter1]. It perfectly works.
I'm trying to access the same Lambda function in Airtable Scripting environment. In order to do it, I set up an API Gateway trigger for the Lambda function, but I can't figure out how to pass input parameters in the vanilla JS environment. Below is the code that I have, which gives me "Internal Server Error".
Your help would be definitely appreciated!
const awsUrl = "https://random-id.execute-api.us-west-2.amazonaws.com/default/lambda-function";
let event = {
"queryStringParameters": {
"gdrive_folder_id": consFolderId,
"invitee_email": email
}
};
let response = await fetch(awsUrl, {
method: "POST",
body: JSON.stringify(event),
headers: {
"Content-Type": "application/json",
}
});
console.log(await response.json());
[Edited] Plus, here's the code of the Lambda function and the latest cloudwatch log after a successful execution invoked by Zapier. It's a simple code that automates Google Drive folder sharing based on 2 inputs. (Folder ID + email address) Please bear with me for the poor code quality!
from __future__ import print_function
from googleapiclient.discovery import build
from google.oauth2 import service_account
SCOPES = ['https://www.googleapis.com/auth/drive']
SERVICE_ACCOUNT_FILE = 'service.json'
def lambda_handler(event, context):
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
# 2-legged OAuth from Google service account
creds = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
drive_service = build('drive', 'v3', credentials=creds)
# change multiple permissions with batch requests
folder_id = event['gdrive_folder_id']
email_address = event['invitee_email']
def callback(request_id, response, exception):
if exception:
# Handle error
print(exception)
else:
print("Permission Id: {}".format(response.get('id')))
batch = drive_service.new_batch_http_request(callback=callback)
user_permission = {
'type': 'user',
'role': 'writer',
'emailAddress': email_address
}
batch.add(drive_service.permissions().create(
fileId=folder_id,
body=user_permission,
fields='id',
))
batch.execute()
I'm not a Python expert and I don't know how you've setup your API Gateway integration with Lambda but I believe your code can have two issues:
1.) Internal Server Error as a response from the API Gateway endpoint also often refers to a problem in the integration between the API Gateway and your Lambda function. In this case here I can not see where you are returning a valid response back to the API Gateway. In your example the return value of batch.execute() is probably returned, right? However, by default the API Gateway expects an object that contains a statusCode and body and optionally headers. You can have a look at the AWS Lambda handler documentation for Python and their examples. Also this documentation page might be of interest for you.
2.) In your function you are accessing the event data like event['gdrive_folder_id']. However, I can not see that you are parsing the event data somewhere. Are you using a custom integration between your API Gateway? Because in case of a proxy integration the API Gateway sends an object that has a body field and from there you'd need to read the HTTP body. See examples on this documentation page.
Here are some more thing you can check on your own:
Have you also checked what you get when you just print the event data? Also, is the batch.execute() waiting for the batch processing or does it return anything? If so, what does it return?
One note here: You haven't told us anything about the integration between your API Gateway and your Lambda function. Since you can do some mapping between the API Gateway and AWS Lambda, it could be possible that you are converting the request and response outside of the Lambda function and hence, my suggestions above are wrong. Let me know if this is true or not and we can further investigate it.
What Is Happening
My Webhook from shopify is not passing the details to my SuiteScript 2.0 Suitelet in NetSuite.
What do I want to happen
I want shopify to send the JSON object to my netsuite Suitelet so I can process the order in NetSuite.
Details
I am trying to make a connection between shopify and Netsuite using Shopify's webhooks.
I have set up a webhook as follows
The URL for my webhook is;
https://XXXXXXX-sb1.extforms.netsuite.com/app/site/hosting/scriptlet.nl?script=XXX&deploy=XX&compid=XXXXXXX_SB1&h=XXXXXXXXXXXXXXXXXXX&caller=ecommerce&key=XXXX-XXXX-XXXX-XXXX
This link calls a Suitelet which when I personally paste the link in the URL is works. However when I click "Send Test Notification" I do not see any evidencethat the Suitelet has executed. The first line of the suitelet is;
log.debug("Running");
I have changed the Webhooks URL to instead go to RequestBin and sure enough the webhook works.
WHAT HAVE I TRED
I have removed the extra query string parameters "caller" and "key"
from the URL. Does not solve the problem.
I have confirmed the Webhook works when changing the URL to RequestBin.
One frustrating limitation with public Suitelets is that they require the User-Agent header to claim to be a browser. (See for example SuiteAnswer #38695).
I had the same issue as you with a BigCommerce webhook, and what I ended up doing was proxy the webhook through a simple Google Cloud Function that modified the user agent.
const request = require('request');
exports.webhook = (req, res) => {
request.post(
{
url: process.env.NETSUITE_SUITELET_URL,
body: req.body,
json: true,
headers: {
'User-Agent': 'Mozilla/5',
Authorization: req.headers['authorization'],
},
},
function(error, response, body) {
res.send(body);
}
);
};
I am using AWS API gateway for API's and cognito UserPool's for security. After Authenticating the user we will get tokens and I am using that token to authorise my API.
Now, I am trying to enable signout to cognito authorised users using javascript. Used the below code.
if (cognitoUser != null) {
cognitoUser.globalSignOut({
onFailure: e => console.log(e),
onSuccess: r =>
console.log('Logout success: ' + r)
})}
I am getting response as success but still I am able to access my API with the previous tokens.Please suggest me how to inactivate all the tokens issued to that cognito user.
The id token, which API Gateway uses to authenticate API calls, stays valid for a while.
I would test for the access token. It should expire right after you call global sign out.
The key word is should above. Please see this issue. It’s an on-going struggle to get AWS to implement an immediate revocation. Here’s a relevant quote:
I worked with AWS Cognito team to get this taken care and got released as a fix through CLI as following.
aws cognito-identity update-identity-pool --identity-pool-id --identity-pool-name --allow-unauthenticated-identities --cognito-identity-providers ProviderName=,ClientId=,ServerSideTokenCheck=<true|false>
By setting the ServerSideTokenCheck to true on a Cognito Identity
Pool, that Identity Pool will check with Cognito User Pools to make
sure that the user has not been globally signed out or deleted before
the Identity Pool provides an OIDC token or AWS credentials for the
user. Now we are running into another issue of this Token being cached
in API Gateway for 10mins which would let that OID token still be
active for 10mins even though the User has globally signed out.
Here's what I mean by test for the accessToken (I have had success with method #2):
1.) you could develop a custom authorizer for API Gateway;
2.) you could perform a check at the start of your lambda functions or on your servers, using:
const AWS = require('aws-sdk');
const awsConfig = require('./awsConfig');
const cognito = new AWS.CognitoIdentityServiceProvider(awsConfig);
// accessToken provided from API Gateway
new Promise((resolve, reject) => {
cognito.getUser({ accessToken }, (errorCallback, response) => {
if (errorCallback) {
reject(errorCallback);
} else {
resolve(response);
}
});
});
The errorCallback and response do not matter. If you get an error, the token is invalid. If you don’t, it’s valid.
I'm trying to load an album from Google Photos via javascript but I don't understand how the api works, I started reading Google Photos API but no luck. Is there a code reference that I can follow to get a list of the photos of my album?
I found this but doesn't work
<script>
var scopeApi = ['https://www.googleapis.com/auth/photoslibrary', 'https://www.googleapis.com/auth/photoslibrary.readonly', 'https://www.googleapis.com/auth/photoslibrary.readonly.appcreateddata'];
function onAuthPhotoApiLoad() {
window.gapi.auth.authorize(
{
'apiKey': 'MY_API_KEY',
'client_id': "MY_CLIEND_ID",
'scope': scopeApi,
'immediate': false
},
handlePhotoApiAuthResult);
}
function handlePhotoApiAuthResult(authResult) {
if (authResult && !authResult.error) {
oauthToken = authResult.access_token;
GetAllPhotoGoogleApi();
}
}
function GetAllPhotoGoogleApi() {
gapi.client.request({
'path': 'https://photoslibrary.googleapis.com/v1/albums',
'method': 'POST'
}).then(function (response) {
console.log(response);
}, function (reason) {
console.log(reason);
});
}
onAuthPhotoApiLoad();
While in the process of developing a Photos synching script, I spent a few days researching and testing the Oauth 2.0 documentation. It's a lot to take in, but hopefully this Cliff-notes version is helpful:
App Setup You first need to get an application configuration through the developer console at console.developers.google.com/ and make sure that the Photos data is shared.
You'll get a JSON file that looks like this
{"installed":{
"client_id":"xxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com",
"project_id":"xxxx-xxxxxxxx-123456",
"auth_uri":"https://accounts.google.com/o/oauth2/auth",
"token_uri":"https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs",
"client_secret":"xxxxxxxxxxxxxxxxxxxxxxxx",
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]
}}
Request Authorization Code - You then need to write code that uses those values to get an authorization token - basically a string that indicates the user has allowed your application access to their data.
Send a request to the auth_uri endpoint with these values in the querystring:
scope - a space-delimited list of scopes from developers.google.com/photos that says you want your user to grant access to these features
redirect_uri - a URL you own that can capture an incoming querystring
client_id - from your developer config in step 1
state - 32 random bytes, base64 encoded and made URL-friendly by replacing "+","/","=" with "-","_","" respectively
code_challenge - a SHA256 hash of another 32 random bytes, base64 encoded and made URL-friendly
code_challenge_method - "S256" (no quotes)
Authorization round trip Sending this composed URI to a user's browser will allow them to choose a Google account and show which scopes are being requested. Once that form is submitted, it will redirect to your redirect_uri with querystring (Method = GET) values:
code - the authorization code you can use to request an access token
state - a string you can use to validate against your hash
Get an access_token Finally you exchange the authorization code for an OAuth AccessToken that you'll put in the HTTP header of all the API requests. The request goes to the token_uri from step 1 and has these request body (Method = POST) parameters:
code - you got from the redirect querystring in Step 3
redirect_uri - same as above, but this may not be used
client_id - from configuration
code_verifier - code_challenge before it was hashed
client_secret - from configuration
scope - can be empty here
grant_type - "authorization_code" (no quotes)
Use the access tokens The response from that request will have an access_token and a refresh_token. You can use the short-lived access_token immediately in your API request's HTTP header. Store the long-lived refresh_token so you can get a new access_token without authorizing again.
That's the gist of it. You can look at my Powershell script for an example of the authorization and authentication flows which work even though the rest is a little buggy and incomplete. Paging through albums is getting a 401 error sometimes.