Develop Reference

can a sns.Topic.fromTopicArn be used to run a CodePipeline

I'm new to aws and my task is to rebuild the app (trigger the codepipeline) when we receive an sns message.
looking for something similar to the code below but not on a schedule instead using sns but i dont think i can use an sns event:
// A pipeline being used as a target for a CloudWatch event rule.
import * as targets from '#aws-cdk/aws-events-targets';
import * as events from '#aws-cdk/aws-events';
// kick off the pipeline every day
const rule = new events.Rule(this, 'Daily', {
schedule: events.Schedule.rate(Duration.days(1)),
});
declare const pipeline: codepipeline.Pipeline;
rule.addTarget(new targets.CodePipeline(pipeline));
these are the code fragments i collected but i dont think i can do what i want to do using a lambda function either.
const consumerTopic = sns.Topic.fromTopicArn(
this,
"myTopicId",
"arn:aws:sns:*******");
const fn = new Function(this, 'aFunction', {
runtime: Runtime.NODEJS_16_X,
handler: 'snsHandler.handler',
code: Code.fromAsset(__dirname),
});
consumerTopic.addSubscription(new LambdaSubscription(fn))

You can trigger a pipeline when you receive a sns message like this:
// create sns topic
const topic = new sns.Topic(this, 'MyTopic');
// or subscribe to an existing sns topic
const consumerTopic = sns.Topic.fromTopicArn(
this,
"myTopicId",
"arn:aws:sns:***YOUR ARN***");
const fn = new Function(this, 'aFunction', {
description: "lambda function to trigger a code pipeline build from a sns message",
runtime: Runtime.NODEJS_16_X,
handler: 'snsHandler.handler',
code: Code.fromAsset(join(__dirname, '../lambda')),
});
consumerTopic.addSubscription(new LambdaSubscription(fn))
// 👇 create a policy statement
const policy = new iam.PolicyStatement({
actions: ['codepipeline:StartPipelineExecution'],
resources: ['*'],
});
// 👇 add the policy to the Function's role
fn.role?.attachInlinePolicy(
new iam.Policy(this, 'run-pipeline-policy', {
statements: [policy],
}),
);
// ***** set a message:
// aws sns publish \
// --subject "Just testing 🚀" \
// --message "Hello world 🐊" \
// --topic-arn "***YOUR ARN***"
and the lambda function:
import { SNSMessage } from 'aws-lambda';
import {AWSError} from "aws-sdk";
import {StartPipelineExecutionOutput} from "aws-sdk/clients/codepipeline";
const AWS = require('aws-sdk');
exports.handler = async function(event:SNSMessage) {
const codePipeline = new AWS.CodePipeline({region: "ap-southeast-2" });
const params = {
name: '***NAME OF CODE PIPELINE**', /* required */
// clientRequestToken: 'STRING_VALUE'
};
return new Promise ((resolve, reject) => {
codePipeline.startPipelineExecution(params, function(err:AWSError, data:StartPipelineExecutionOutput) {
if (err) {
console.log(err, err.stack);
reject(err)
} // an error occurred
else {
console.log(data);
resolve(data)
} // successful response
});
})
};
one more example here:
AccessDeniedException while starting pipeline

How to fetch Amazon Cognito Identity ID (user_identity_id) for the user from the lambda function?

In the Amplify documentation, under the Storage/File access levels section there is a paragraph that states:
Files are stored under private/{user_identity_id}/ where the user_identity_id corresponds to the unique Amazon Cognito Identity ID for that user.
How to fetch user_identity_id from the lambda function?
Request to the lambda is authorized, the event.requestContext.authorizer.claims object is available, I can see the user data, but not the user_identity_id.
EDIT: Now I see that there is a field event.requestContext.identity.cognitoIdentityId, but the value is null. Still need to find the way to fetch it.

Ok, so there's no right way to map Cognito identity ID and Cognito user. There is a lengthy discussion here where a couple of workarounds can be found. For now, I'm going to use this solution where, instead of identity_id, you can specify a custom attribute (most likely a sub) as a folder name.
EDIT: There is another solution that might help (found somewhere on the internet, and I verified that it works)
const AWS = require('aws-sdk')
const cognitoIdentity = new AWS.CognitoIdentity();
function getCognitoIdentityId(jwtToken) {
const params = getCognitoIdentityIdParams(jwtToken);
return cognitoIdentity
.getId(params)
.promise()
.then(data => {
if (data.IdentityId) {
return data.IdentityId;
}
throw new Error('Invalid authorization token.');
});
}
function getCognitoIdentityIdParams(jwtToken) {
const loginsKey = `cognito-idp.${process.env.REGION}.amazonaws.com/${process.env.USERPOOLID}`;
return {
IdentityPoolId: `${process.env.IDENTITY_POOL_ID}`,
Logins: {
[loginsKey]: jwtToken,
},
};
}

If the user accesses the lambda through graphql via the AppSync service then the identity is stored event.identity.owner
Here is some typescript code I use to pull the user_identity_id from the event. However, the user doesn't always call the lambda direct sp the user_identity can also be based in if from an authorized IAM role.
export function ownerFromEvent(event: any = {}): string {
if (
event.identity.userArn &&
event.identity.userArn.split(":")[5].startsWith("assumed-role")
) {
// This is a request from a function over IAM.
return event.arguments.input.asData.owner;
} else {
return event.identity.owner;
}
}

For anyone else still struggling with this, I was finally able to use the aws-sdk for JavaScript v3 to obtain a Cognito User's IdentityId & Credentials in a Lambda Function invoked via API-Gateway with a Cognito User Pool Authorizer from the Cognito User's identity jwtToken passed into the Authorization header of the request.
Here is the code used in my JavaScript Lambda Function:
const IDENTITY_POOL_ID = "us-west-2:7y812k8a-1w26-8dk4-84iw-2kdi849sku72"
const USER_POOL_ID = "cognito-idp.us-west-2.amazonaws.com/us-west-2_an976DxVk"
const { CognitoIdentityClient } = require("#aws-sdk/client-cognito-identity");
const { fromCognitoIdentityPool } = require("#aws-sdk/credential-provider-cognito-identity");
exports.handler = async (event,context) => {
const cognitoidentity = new CognitoIdentityClient({
credentials: fromCognitoIdentityPool({
client: new CognitoIdentityClient(),
identityPoolId: IDENTITY_POOL_ID,
logins: {
[USER_POOL_ID]:event.headers.Authorization
}
}),
});
var credentials = await cognitoidentity.config.credentials()
console.log(credentials)
// {
// identityId: 'us-west-2:d393294b-ff23-43t6-d8s5-59876321457d',
// accessKeyId: 'ALALA2RZ7KTS7STD3VXLM',
// secretAccessKey: '/AldkSdt67saAddb6vddRIrs32adQCAo99XM6',
// sessionToken: 'IQoJb3JpZ2luX2VjEJj//////////...', // sessionToken cut for brevity
// expiration: 2022-07-17T08:58:10.000Z
// }
var identity_ID = credentials.identityId
console.log(identity_ID)
// us-west-2:d393294b-ff23-43t6-d8s5-59876321457d
const response = {
statusCode: 200,
headers: {
"Access-Control-Allow-Headers": "*",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods" : "OPTIONS,POST,GET,PUT"
},
body:JSON.stringify(identity_ID)
};
return response;
}
After a Cognito User has signed in to my application, I can use the Auth directive of aws-amplify and fetch() in my React-Native app to invoke the lambda function shown above by sending a request to my API-Gateway trigger (authenticated with a Cognito User Pool Authorizer) by calling the following code:
import { Auth } from 'aws-amplify';
var APIGatewayEndpointURL = 'https://5lstgsolr2.execute-api.us-west-2.amazonaws.com/default/-'
var response = {}
async function getIdentityId () {
var session = await Auth.currentSession()
var IdToken = await session.getIdToken()
var jwtToken = await IdToken.getJwtToken()
var payload = {}
await fetch(APIGatewayEndpointURL, {method:"POST", body:JSON.stringify(payload), headers:{Authorization:jwtToken}})
.then(async(result) => {
response = await result.json()
console.log(response)
})
}
More info on how to Authenticate using aws-amplify can be found here https://docs.amplify.aws/ui/auth/authenticator/q/framework/react-native/#using-withauthenticator-hoc

Key element does not match the schema with DocumentClient

I have a Lambda that I have created using the AWS Toolkit for Visual Studio Code. I have a small lambda.js file that exports the handler for API Gateway events.
const App = require('./app');
const AWS = require('aws-sdk');
exports.handler = async (event, context) => {
let config = {
aws_region: 'us-east-1',
//endpoint: 'http://localhost:8000',
};
let dynamo = new AWS.DynamoDB.DocumentClient();
let app = new App(dynamo, config, event);
try {
let response = await app.run();
return response;
} catch(err) {
return err;
}
};
The app.js file represents the logic of my Lambda. This is broken up to improve testing.
const AWS = require('aws-sdk');
class App {
constructor(datastore, configuration, event) {
this.datastore = datastore;
this.httpEvent = event;
this.configuration = configuration;
}
async run() {
AWS.config.update({
region: this.configuration.aws_region,
endpoint: this.configuration.endpoint,
});
let params = {
TableName: 'test',
Key: {
'year': 2015,
'title': 'The Big New Movie',
},
};
const getRequest = this.datastore.get(params);
// EXCEPTION THROWN HERE
var result = await getRequest.promise();
let body = {
location: this.configuration.aws_region,
url: this.configuration.endpoint,
data: result
};
return {
'statusCode': 200,
'body': JSON.stringify(body)
};
}
}
module.exports = App;
I can write the following mocha test and get it to pass.
'use strict';
const AWS = require('aws-sdk');
const chai = require('chai');
const sinon = require('sinon');
const App = require('../app');
const expect = chai.expect;
var event, context;
const dependencies = {
// sinon.stub() prevents calls to DynamoDB and allows for faking of methods.
dynamo: sinon.stub(new AWS.DynamoDB.DocumentClient()),
configuration: {
aws_region: 'us-west-2',
endpoint: 'http://localhost:5000',
},
};
describe('Tests handler', function () {
// Reset test doubles for isolating individual test cases
this.afterEach(sinon.reset);
it('verifies successful response', async () => {
dependencies.dynamo.get.returns({ promise: sinon.fake.resolves('foo bar')});
let app = new App(dependencies.dynamo, dependencies.configuration, event);
const result = await app.run()
expect(result).to.be.an('object');
expect(result.statusCode).to.equal(200);
expect(result.body).to.be.an('string');
console.log(result.body);
let response = JSON.parse(result.body);
expect(response).to.be.an('object');
expect(response.location).to.be.equal(dependencies.configuration.aws_region);
expect(response.url).to.be.equal(dependencies.configuration.endpoint);
expect(response.data).to.be.equal('foo bar');
});
});
However, when I run the Lambda locally using the Debug Locally via the Code Lens option in VS Code the results from the AWS.DynamoDB.DocumentClient.get call throws an exception.
{
"message":"The provided key element does not match the schema",
"code":"ValidationException",
...
"statusCode":400,
"retryable":false,
"retryDelay":8.354173589804192
}
I have a table created in the us-east-1 region where the non-test code is configured to go to. I've confirmed the http endpoint being hit by the DocumentClient as being dynamodb.us-east-1.amazonaws.com. The table name is correct and I have a hash key called year and a sort key called title. Why would this not find the key correctly? I pulled this example from the AWS documentation, created a table to mirror what the key was and have not had any luck.

The issue is that the Key year was provided a number value while the table was expecting it to be a string.
Wrapping the 2015 in quotes let the Document know that this was a string and it could match to the schema in Dynamo.
let params = {
TableName: 'test',
Key: {
'year': '2015',
'title': 'The Big New Movie',
},
};

Google cloud dialogflow intent detection nodejs example not working

I am trying to implement a very simple dialogflow agent integration with nodejs.
Here is what I did so far
I followed the code from Intent detection
I added the service account private key file .json to my server.
I added the environment variable GOOGLE_APPLICATION_CREDENTIALS with the path to my .json private key file.
Here is the code I am trying to run right now:
require('dotenv').config()
const projectId = 'gg-chatbot-216808';
const sessionId = 'quickstart-session-id';
const query = 'hello';
const languageCode = 'en-US';
// Instantiate a DialogFlow client.
const dialogflow = require('dialogflow');
const sessionClient = new dialogflow.SessionsClient();
// Define session path
const sessionPath = sessionClient.sessionPath(projectId, sessionId);
// The text query request.
const request = {
session: sessionPath,
queryInput: {
text: {
text: query,
languageCode: languageCode,
},
},
};
// This prints the private key path correctly.
console.log(process.env.GOOGLE_APPLICATION_CREDENTIALS);
// Send request and log result
sessionClient
.detectIntent(request)
.then(responses => {
console.log('Detected intent');
const result = responses[0].queryResult;
console.log(` Query: ${result.queryText}`);
console.log(` Response: ${result.fulfillmentText}`);
if (result.intent) {
console.log(` Intent: ${result.intent.displayName}`);
} else {
console.log(` No intent matched.`);
}
})
.catch(err => {
console.error('ERROR:', err);
});
Then I get this error in the console when I run this file
Auth error:Error: invalid_user: Robot is disabled.
ERROR: { Error: 14 UNAVAILABLE: Getting metadata from plugin failed with error: invalid_user: Robot is disabled.
at Object.exports.createStatusError (/var/www/html/google_auth/node_modules/grpc/src/common.js:87:15)
at Object.onReceiveStatus (/var/www/html/google_auth/node_modules/grpc/src/client_interceptors.js:1188:28)
at InterceptingListener._callNext (/var/www/html/google_auth/node_modules/grpc/src/client_interceptors.js:564:42)
at InterceptingListener.onReceiveStatus (/var/www/html/google_auth/node_modules/grpc/src/client_interceptors.js:614:8)
at callback (/var/www/html/google_auth/node_modules/grpc/src/client_interceptors.js:841:24)
code: 14,
metadata: Metadata { _internal_repr: {} },
details: 'Getting metadata from plugin failed with error: invalid_user: Robot is disabled.' }

i also faced a similar issue for my angular bot.
What i did was, instead of using using the google_credentials from the json file, i created an object with private_key,client_email {these values can be taken from the service account private key file .json}, and passed the object while setting up the session client.
var config = {
credentials: {
private_key: "YOUR_PRIVATE_KEY",
client_email: "YOUR_CLIENT_EMAIL"
}
}
const sessionClient = new dialogflow.SessionsClient(config);
note: do copy the full private_key string from .json. It will start as "-----BEGIN PRIVATE KEY-----\n......" .
Also, in GCP go to the project->IAM then try setting role for the service as DIALOGLOW API ADMIN. Check if this works.

If this has not been resolved yet , the solution is to provide "fileKey" inside sessionClient.
const sessionClient = new dialogflow.SessionsClient({
fileKey:" path of your credentials.json file"
});
or
let filePath = process.env.GOOGLE_APPLICATION_CREDENTIALS ="Location of credentials file".
const sessionClient = new dialogflow.SessionsClient({
fileKey:filePath
});
this will even work if there is no system env variable is set as GOOGLE_APPLICATION_CREDENTIALS.
Hope this is helpful.

How to authenticate and send contract method using web3.js 1.0

I am confused about how I should be executing a contract's method using the web3 1.0 library.
This code works (so long as I manually unlock the account first):
var contract = new web3.eth.Contract(contractJson, contractAddress);
contract.methods
.transfer("0x0e0479bC23a96F6d701D003c5F004Bb0f28e773C", 1000)
.send({
from: "0x2EBd0A4729129b45b23aAd4656b98026cf67650A"
})
.on('confirmation', (confirmationNumber, receipt) => {
io.emit('confirmation', confirmationNumber);
});
I get this error (if I don't unlock manually first):
Returned error: authentication needed: password or unlock
The above code is an API endpoint in node.js, so I want it to unlock or authenticate programmatically.
There is no method in web3.js 1.0 to unlock the account.
I also don't think this is necessary (at least that's what I am confused about). Since I am managing accounts, I know what the private key is.
I am thinking the transaction needs to be signed with the private key?? Is this correct? Is this effectively the same thing as "unlocking the account"?
I tried doing this:
var contract = new web3.eth.Contract(contractJson, contractAddress);
var tx = {
from: "...{fromAddress -- address that has the private key below}",
to: "...",
value: ...
};
var signed = web3.eth.accounts.signTransaction(tx,
"...{privateKey}");
console.log(signed);
var promise = web3.eth.sendSignedTransaction(signed);
I get this error:
Returned error: The method net_version does not exist/is not available
What is the easiest way to authenticate and submit a transaction?
Ideally, I want to use the first approach in my code sample, as it is the cleanest.

This code allows me to sign a transaction server-side (node.js) using the privateKey from the account I created (using web3.eth.accounts.create()), and send the signed transaction to the network without having to unlock the account.
I am using Geth 1.7.1
var contract = new web3.eth.Contract(contractJson, contractAddress);
var transfer = contract.methods.transfer("0x...", 490);
var encodedABI = transfer.encodeABI();
var tx = {
from: "0x...",
to: contractAddress,
gas: 2000000,
data: encodedABI
};
web3.eth.accounts.signTransaction(tx, privateKey).then(signed => {
var tran = web3.eth.sendSignedTransaction(signed.rawTransaction);
tran.on('confirmation', (confirmationNumber, receipt) => {
console.log('confirmation: ' + confirmationNumber);
});
tran.on('transactionHash', hash => {
console.log('hash');
console.log(hash);
});
tran.on('receipt', receipt => {
console.log('reciept');
console.log(receipt);
});
tran.on('error', console.error);
});

A way to be able to call your contract methods without having to sign the transaction explicitly is this (web3js 1.0.0):
const privateKey = 'e0f3440344e4814d0dea8a65c1b9c488bab4295571c72fb879f5c29c8c861937';
const account = web3.eth.accounts.privateKeyToAccount('0x' + privateKey);
web3.eth.accounts.wallet.add(account);
web3.eth.defaultAccount = account.address;
// ...
contract = new web3.eth.Contract(JSON_INTERFACE, address);
contract.methods.myMethod(myParam1, myParam2)
.send({
from: this.web3.eth.defaultAccount,
gas: myConfig.gas,
gasPrice: myConfig.gasPrice
})

Here's a complete example of how to sign a transaction without a local wallet account. Especially useful if you are using infura for the transaction. This was written for
'use strict';
const Web3 = require('web3');
const wsAddress = 'wss://rinkeby.infura.io/ws';
const contractJson = '(taken from solc or remix online compiler)';
const privateKey = '0xOOOX';
const contractAddress = '0xOOOX';
const walletAddress = '0xOOOX';
const webSocketProvider = new Web3.providers.WebsocketProvider(wsAddress);
const web3 = new Web3(new Web3.providers.WebsocketProvider(webSocketProvider));
const contract = new web3.eth.Contract(
JSON.parse(contractJson),
contractAddress
);
// change this to whatever contract method you are trying to call, E.G. SimpleStore("Hello World")
const query = contract.methods.SimpleStore('Hello World');
const encodedABI = query.encodeABI();
const tx = {
from: walletAddress,
to: contractAddress,
gas: 2000000,
data: encodedABI,
};
const account = web3.eth.accounts.privateKeyToAccount(privateKey);
console.log(account);
web3.eth.getBalance(walletAddress).then(console.log);
web3.eth.accounts.signTransaction(tx, privateKey).then(signed => {
const tran = web3.eth
.sendSignedTransaction(signed.rawTransaction)
.on('confirmation', (confirmationNumber, receipt) => {
console.log('=> confirmation: ' + confirmationNumber);
})
.on('transactionHash', hash => {
console.log('=> hash');
console.log(hash);
})
.on('receipt', receipt => {
console.log('=> reciept');
console.log(receipt);
})
.on('error', console.error);
});
Using
"web3": "1.0.0-beta.30"

This is my implementation using "#truffle/hdwallet-provider": "^2.0.3", "web3": "^1.6.1",
function getWeb3Provider() {
return new HDWalletProvider({
privateKeys: [NFT_MINTER_ACCOUNT_PRIVATE_KEY],
providerOrUrl: BSC_RPC_ENDPOINT,
});
}
const web3 = new Web3(BSC_RPC_ENDPOINT);
const contract = new web3.eth.Contract(
jsonContractABI as unknown as AbiItem[],
NFT_CONTRACT_ADDRESS
);
contract.setProvider(getWeb3Provider());
then in send methods
contract.methods.safeMint(receiverAddress, itemUri).send({
from: NFT_MINTER_ACCOUNT,
});
in call methods
contract.methods.balanceOf(address).call();