Is there any way to terminate the EC-2 instance after a particular time like 2 hours from the time of creation.
I am using NodeJS to perform the operations on AWS EC-2.
Is there any parameter I have to pass when I create the instance or I need to create a separate function for this method like some built-in method for auto-terminating the EC-2 instance.
Thanks in advance!
You can achieve this using AWS Lambda anonymous functions. You can have Lambda create your EC2 instance, and then have it delete it 2 hours later through use of a 'step function'. More info here: AWS Step Functions
The easiest method would be to have the instance terminate itself.
Upon startup, it can run a simple script like:
sleep 7200
sudo shutdown -h
To have the instance terminate on the sudo shutdown command, set the Shutdown Behavior (when the instance is launched) to Terminate.
This script could be triggered from a User Data script when the instance is started. Run the script in the background, such as:
./shutdown_script &
The benefit of doing it this way is that the instance is responsible for turning itself off. Any other option would require a timer to be set somewhere (eg Amazon CloudWatch Events), which starts to get more tricky.
Related
I have a Azure Function (node.js) and a list of exact times (7:30, 8:05, etc.) in a database table. I would like to trigger the Azure Function at exact times using the database table.
Now my problems are
does timer trigger only take a single cron schedule?
can I maybe use environment variables to trigger at time1 (e.g. 7:30) then when it is done, change the environment variable to time2 (e.g. 8:05) in the code? So that it would run again at time2 (8:05)?
Can a different function (HTTP triggered) run my original function (timer triggered) and also change the environment variable?
You could also use Durable Functions for this: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-timers#usage-for-delay
There you can dynamically load the times from your database and create a new timer for the next execution. Once that one ran, create the next timer.
Basically you'll end up with an eternal orchestrator (which is ok): https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-eternal-orchestrations#periodic-work-example
does timer trigger only take a single cron schedule?
Yes, timer trigger could have only one cron expression
About dynamically cron expression, you could refer to this Configuration, the ScheduleExpression alows the setting in app settings. You could set with "schedule": "%TriggerSchedule%".And define TriggerSchedule in your appsettings. Then modify your appsettings dynamically.
The other way is use the Kudu API to modify the function.json.
PUT https://{functionAppName}.scm.azurewebsites.net/api/vfs/{pathToFunction.json}, Headers: If-Match: "*", Body: new function.json content
Then sync the function trigger.
POST https://{functionAppName}.scm.azurewebsites.net/api/functions/synctriggers
Can a different function (HTTP triggered) run my original function
(timer triggered) and also change the environment variable?
You could invoke a HTTP trigger function in a Timer function, however the Azure Functions runtime configuration file is not writable. But as it runs App Service, you can manage those settings programmatically via PowerShell, REST api, or through the CLI.
Keep in mind that changes to those settings will trigger a site restart
I'm currently using AWS's Javascript SDK to launch custom EC2 instances and so far so good.
But now, I need these instances to be able to run some tasks when they are created, for example, clone a repo from Github, install a software stack and configure some services.
This is meant to emulate a similar behaviour I have for local virtual machine deployment. In this case, I run some provisioning scripts with Ansible that get the job done.
For my use case, which would be the best option amongst AWS's different services to achieve this using AWS's Javascript SDK?
Is there anyway I could maybe have a template script to which I passed along some runtime obtained variables to execute some tasks in the instance I just created? I read about user-data but I can't figure out how that wraps with AWS's SDK. Also, it doesn't seem to be customisable.
At the end of the day, I think I need a way to use the SDK to do this:
"On the newly created instance, run this script that is stored in such place, replacing these
placeholder values in the script with these I'm giving you now"
Any hints?
As Mark B. stated, UserData is the way to go for executing commands on instance launch. As you tagged the question with javascript here's an example on passing this in the ec2.runInstances command:
let AWS = require('aws-sdk')
let ec2 = new AWS.EC2({region: 'YOUR_REGION'})
// Example commands to create a folder, a file and delete it
let commands = [
'#!/usr/bin/env bash',
'mkdir /home/ubuntu/test',
'touch /home/ubuntu/test/examplefile',
'rm -rf /home/ubuntu/test'
];
let params = {
...YOUR PARAMS HERE...
UserData: new Buffer(commands.join("\n")).toString('base64')
}
// You need to encode it with Base64 for it to be executed by the userdata interpreter
ec2.runInstances(params).promise().then(res => { console.log(res); })
When you launch the new instances you can provide the user-data at that time, in the same AWS SDK/API call. That's the best place to put any server initialization code.
The only other way to kick off a script on the instance via the SDK is via the SSM service's Run Command feature. But that requires the instance to already have the AWS SSM agent installed. This is great for remote server administration, but user-data is more appropriate for initializing an instance on first boot.
I'm trying to listen for data changes in my firebase using firebase's package for Node. I'm using the on() method which is supposed to listen for changes non-stop (as opposed to once() method that only listens to the first occurrence of a specific event ) My listener.js file on the server is exactly like this:
var Firebase=require('firebase');
var Ref= new Firebase('https://mydatabase.firebaseio.com/users/');
Ref.on('child_changed',function(childsnapshot,prevchildname){
Ref.child(childsnapshot.key()).push("I hear you!");
} ) ;
But it only works the for the first occurrence and throws a fatal memory error after a second occurrence.
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
I'm very new to server side programming and don't know what to do. I must be missing something important. Should I set up special server settings with node first? or maybe make a daemon that runs a script with once() method every second or so ?
I'm pretty sure you're creating an endless loop here:
You push a value to https://mydatabase.firebaseio.com/users/
the on('child_changed' event fires in your script
your script pushes a new child under the value
so we go back to step 2 and repeat
It will happen quite rapidly too, since Firebase clients fire local events straight away.
It looks like you're trying to create a chat bot. Which means you more likely want to create sibling messages:
var Firebase=require('firebase');
var ref= new Firebase('https://mydatabase.firebaseio.com/users/');
ref.on('child_changed',function(childsnapshot,prevchildname){
ref.push("I hear you!");
}) ;
Note that it is pretty inefficient to use StackOverflow to debug code. Since you seem to be on Windows, I recommend installing Visual Studio and its node tools. They have a great debugger that allows you to step through the code. Setting a breakpoint in your callback (so in the line with ref.push), will quickly show you what is going wrong.
I have seen the potential of NodeJS recently and try this for a small task to use .
about the following:
i want to create a web service , which allows you to start and stop multiple client instances (multiple nodejs tasks who should run, until i send a stop request to a instance):
so a http server modul waits for start/stop requests (with params) (maybe with express?) like
[start]
http://taskservice/start?user=abc&pass=123
the service should create a instance of a task like:
var task = new MyTask();
task.login(user, pass);
task.on("loginsuccess", function() {
task.startwork();
};
[stop identified by user]
http://taskservice/stop?user=abc
The problem is, I do not know how this can be best implemented . It must always be able to create any number of tasks and the tasks must run until I say stop . (the tasks require very little performance)
Does anyone have an idea or a small code example ?
I am not sure if this is the proper way to do it but I think you can do it with forever. Check this link; http://www.codingdefined.com/2014/08/use-of-forever-in-nodejs.html
You can stop and start child nodejs processes through your root script.
I am building out a data analysis tool in Meteor and am running into issues with the Meteor server shutdown and restart. Server-side, I am pinging several external APIs on a setInterval, trimming the responses down to just new data that I haven't already captured, running a batch of calculations on that new data, and finally storing the computed results in Mongo. For every chunk of new data that I receive from the external APIs, there are about 15 different functions/computations that I need to run, and each of the 15 outputs are being stored in Mongo inside separate documents. Client-side, users can subscribe to any one out of the 15 documents, enabling them to view the data in the manner in which they please.
new data is captured from the API as {A} and {A} is stored in Mongo
|
begin chain
|
function1 -> transforms {A} into {B} and stores {B} in Mongo
|
function2 -> transforms {A} into {C} and stores {C} in Mongo
|
...
|
function15 -> transforms {A} into {P} and stores {P} in Mongo
|
end chain
The problem, is that when I shut down Meteor, or deploy new code to the server (which will automatically restart the server), the loop that iterates through these functions is being interrupted. Say, functions 1-7 ran successfully, and then Meteor restarted, causing functions 8-15 to never run (or even worse, for function 8 to be interrupted while 9-15 never ran). This causes my documents to no longer be in sync with {A} data that was stored before the loop began.
How can this risk be mitigated? Is it possible, to tell Meteor to gracefully shutdown / wait until this process has completed? Thanks!
I've never heard of a graceful shutdown, and even if there were one, I'm not sure I'd trust it...
What I'd do is assign a complete flag to {A} that is triggered when function15 is finished. If the functions are async (or just expensive, but I doubt that's your case...), then create a record in {B}:{P} when {A} is created with the _id of {A} and a complete flag.
Regardless, run the functions on a query of batches where complete is false.
There is currently no official way of a graceful shutdown. So you would have to come up with some other way of making sure your data isn't stored in a inconsistent state.
The easiest way that comes up in my mind would be to disable meteor automatic restarts with meteor --once.
Then add a shutdown mode to your application.
In shutdown mode your application should not pick up new tasks only finish what it is currently working on. This would be easy to do if you use meteor-synced-cron which has a stop method which doesn't kill currently running functions.
Make sure that a task finishing never leaves a document in a inconsistent state.
So you can have multiple tasks working on a document just make sure that when task1 finishes the document will always be in a state to be picked up by task2.