I am playing with the Node.JS library provided by Amazon to manage the EC2 and Autosclaing group. I have written a program to delete an instance so that the autoscaling group will create a new instance. But before I delete any additional instances, I need to make sure that the new one is generated by amazon and is running. For that I need to pause my program and keep checking till I get a positive response. So this is how it works ---
(I usually have 3-4 instances and have to replace with 3-4 instances of new type.)
So my program is --
updateServer(){
Which has the code to retrieve the instances from the server.
foreach(instance in the list of instances){
replace(); (with new one.)
}
}
replace() {
delete the old one.
while(new instance is not generated) {
check();
}
}
check(){
return the status.
}
Now the question is I need this to happen in sequence. How can I pause the program?
Because in my case, the foreach loop executes and the program ends and it does not enter that check loop. Can you please give me ideas.
Thanks
Links -- http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/frames.html
The problem you are describing here is one one 'polling' and while talking of 'pauses' is normally an anti-pattern in node's asynchronous programming model, polling is possibly the one case where it has valid application.
However for reasons of code maintainability - in this case the ability to add other tasks later (such as other checks), you should also handle the polling asynchronously.
Here are some different approaches that should solve your problem.
1. Don't handle the problem in node.js at all - invoke the node application from the host's crontab and design the program to exit. This isn't very node-ish though but its certainly robust solution.
2. Use npm to install the node timer module [https://github.com/markussieber/timer]. Using timer you would pass check() function as an argument so it would call it back periodically. This is more slippery but scales in that you can have lots of checks running which for a scaleable EC2 deployment is what is probably called for.
timer = require('timer'); //import the time module
function check(){ //check function now polls
//add an if(something==true) statement here to allow you to exit if necessary
console.log("Checking stuff");
timer.timer(200, check); //calls this function repeatedly every 200ms
}
check(); //starts the code polling
3. Having said that, the functionality you are trying to implement sounds to me like it is the same as that provide by Amazon Autoscaling [http://aws.amazon.com/autoscaling/] and Amazon Elastic Beanstalk [http://aws.amazon.com/elasticbeanstalk/]. You might also have some luck with those.
Happy node hacking :)
nodejs isn't made to pause, really. you might be looking for something like an eventEmitter, so you can call a function when you emit an event. http://nodejs.org/api/events.html
You should never pause in node. Take a look at async (https://github.com/caolan/async). What you need is probably async.forEachSeries(), which allows you to do things in series, but asyncronously.
In your case it would start up a server and when that is started a callback is called that makes the series continue on the next server etc.
Here's some sample code from node-levelup https://github.com/rvagg/node-levelup/blob/master/test/benchmarks/index.js#L60-L84
Related
I know that I can use setInterval to send a request to server every 1 minute to see if something new added to the database so I can fetch it and update the UI according to this
but for what I've read from different article I got confused some say it's bad practice and other use it
SO I'm asking here to see if there's better approach to accomplish this or setInterval will handle the mission without any issues
Well If you had to know if something has changed I would think about using sockets is a better choice. Of course if your server is ready to implement this kind of mechanism.
If you don't have an opportunity to change logic on the server (third-party API or something) setTimeout on practice most common solutions.
But with this kind of solutions you have some caveats:
you should handle by yourself when to clear your timeout; if you don't then you'll have a memory leaks at least;
something would never change, big amount of data - this kind of optimizations you need to implement with caching strategies;
not sure about security - servers can block your app because of DDOS in the case when a lot of your users work with this service;
setTimeout by itself is ok. Depends on your situation if there is better or more appropriate solutions.
You may configure a simple cronjob.
Whether you use it from your VPS or shared hosting (most of them have cron installed), whether you may use a free online service like cron-job.org.
No problem with setInterval and setTimeout but don't forget JS is mono-thread and chosing the one that corresponds to our needs, given that the difference is important between them.
This interesting point not from me :
The setInterval thing is less of a performance tip but more of a
concurrency best practice. setInterval has the potential to execute
code prematurely to when you're expecting.
E.g.,
function doSomething () {
// I take 500 ms to run
}
doSomething();
setInterval(doSomething, 150);
This bit of code will run then next iteration doSomething immediately
after it's finished executing the current iteration. The interval
timer never stops! This has the potential to freeze your runtime
environment by continuously running its code and not allowing the
system to do other things.
function doSomething () {
// I take 500 ms to run
setTimeout(doSomething, 150);
}
doSomething();
Even though this looks essentially the same the difference is that
doSomething will always wait 150 ms after running before requeuing the
next iteration for execution. This allows the system to run other
operations and prevents your runtime from being locked up.
https://www.reddit.com/r/javascript/comments/51srsf/is_it_still_true_that_we_shouldnt_use_setinterval/
I'm running a C tool compiled to wasm using emscripten. The tool works on very large files. When running this tool normally on the CLI, often operations stream the results and terminate the program early once enough data has been returned. For example you might run:
./tool <input-file> | head -n 100
The tool would terminate after it detects stdout has been closed by head, effectively only reading a small portion of the input.
The problem is that stdout with emscripten appears to be asynchronous (by overriding Module.print), so the tool runs to completion every time. Is there a way to make it block on stdout so I can only read as much as I need and then terminate the tool?
You can redirect the output to a file and then put the task in the background. Meanwhile monitor the log file. When it reaches 100 lines kill the child pid.
Some like this should work:
rm -f /tmp/log
touch /tmp/log
./tool input_file > /tmp/log 2>&1 &
pid=$!
while sleep 1
do
ret=`cat /tmp/log | wc -l`
if [ "$ret" -ge 100 ]
then
kill $pid
exit 0
fi
done
I put in the touch to create an empty log file. This will avoid a race condition where we cat it before it is created by the child process. wc -l will return the number of lines. Change the sleep value to whatever is appropriate for your test time.
You need to implement a way to tell your tool to stop. There are many ways to do this. Two that come to mind:
Have it take an extra argument indicating the number of lines of output after which it should stop, then call it with this argument. This is the simplest approach and easiest to implement. The main drawback is that you need to know the max number of lines ahead of time, so that you can include it in your call arguments, and tool must be able to accurately know how to translate that into when to stop-- but if that is the case, which it sounds like, then just do this and you're done.
But I suppose if your tool did not know how to count lines-- perhaps it's just outputting blobs, or perhaps you have some downstream filter that is only counting some lines towards your maximum and in any case, your tool needs some other function to tell it when to stop, then this would not work, in which case, read on...
Use a callback. Create and export another function e.g. tool_stop(). In your Module.print override function, at the appropriate time, call tool_stop(). In your C code, create some flag, let's call it stop_processing, that is visible to your tool command and also visible to your function that is processing the input. In your processing loop (e.g. before each fread call), your tool command checks this flag and if it is set, it stops processing (when I say "visible", that could mean you make it a global variable, if you'll never have more than one concurrent invocation running, or make it part of some context data that is allocated via some init call and passed whenever a process() or stop() call is made, and then deallocated via a destroy() call. The latter approach is generally cleaner, more scalable and more maintainable, though is a bit more work since you have to add init + destroy, and add a context pointer to each of your function definitions and calls)
Im building a node.js script/app/process (new to node and its definitions) that should run at a certain hour (from a conf file) every day.
Im using the Node Schedule package to schedule the main function to run at that appointed time everyday. But the thing is this package does an "in-process scheduling", which means my script must keep running in the background for the scheduling to work.
So basically my code should look like this at the end:
global consts initializing and dependencies (requires)
job scheduling
defining all functions necessary
logic to make script run permanently.
This node.js app runs inside a docker on an ec2 machine in AWS.
How can I accomplish the task of running it permanently? I need a programatic solution, something (a package, a design pattern) I can embed inside my code.
Might be important to note that inside my code I have only a few lines of "requires" and const initializations, and then 1 main function that invokes everything else, if it helps somehow. Also, because im using that Node Scheduler I mentioned, restarting the script when it ends (for example through docker if its possible) is a bad idea, because the scheduling job would be lost and the script itself does nothing except initializations and scheduling the main function to the desired time, so it would restart possibly every few milliseconds, which I guess is a bad practice.
Well, someone in the comment section pointed me to my mistake... I saw the following paragraph on node-scheduler and didn't even test if it works without tryin to find a way to "run forever":
"Note that Node Schedule is designed for in-process scheduling, i.e. scheduled jobs will only fire as long as your script is running, and the schedule will disappear when execution completes. If you need to schedule jobs that will persist even when your script isn't running, consider using actual cron."
But apparently they meant I should keep the process running, effectively means the scheduler will terminate if I close the terminal. Here is a topic with a question and answer about how to prevent that from happening, if someone needs a reference:
How to make a node.js application run permanently?
Thanks to all who tried to help :)
.:Disclaimer:.
Please note that I am trying to achieve something with node.js which might go against its design principles. However as node.js goes against the norm of "javascript is for client side", I am also trying something different here. Please bear with me.
.:Background:.
I have a requirement where Java Scripts need to be narrative (read from beginning to end) for simplistic scripts for simplistic users. I will also offer an Async scripting ability for more advanced users.
I understand the event driven methodology of node.js, and have a solution working based on Async callbacks. Now I am trying to simplify this for our more basic scripting requirements.
.:The Question:.
Is there a way to run a script (using its own sandbox) where execution can be paused while a result is being delivered.
E.g.
var util = require('util'),
vm = require('vm'),
user = require('user');
vm.runInThisContext('user.WaitForProceed(); console.log('Waiting over');');
console.log('Finished!');
User is my own module that will do the waiting. Basically I want it to sit there and block on that line in the vm object until it has received a result back. After which it will continue onto the console.log line.
The output of this example unimportant as it is also achievable through callbacks. The narrative nature of the example script is more important for this solution.
J
There is no way to pause the execution in Node. At least they tell us so :)
There are some libraries which support an advanced flow control, like Invoke, maybe this could help, but I understands, that's not what you asked for :)
Also you could implement a busy-loop using nextTick()
Or you could implement a blocking call in C(++) and provide it as a library. I never did this.
One last way is to readFileSync() to a namedpipe which closes on a certian event.
As you already mentioned, it's against the language principes, therefor these solutions are all hacky.
If you really want to sleep the execution of the Node process, you can. However, as you state, it seems that you're fully aware of the implications.
Here is an NPM module to do this:
https://github.com/ErikDubbelboer/node-sleep
You can use it like so:
var sleep = require('sleep');
sleep.sleep(1); //sleep for 1 sec
sleep.usleep(2000000); //sleep for 2 sec
I partially write this for future visitors that arrive by Google search: You should not use the aforementioned technique. If you decide that you must, be aware that this will block your Node process and it won't be able to do any additional work until the sleep period is over. Additionally, you will violate every expectation of any user who is aware that it's Node.js programs, as Node.js programs are suppose to be non-blocking.
Good luck.
If you realy want to pause execution while waiting for result you may try to work with node-sync. It build on node-fibers. Your application needs to be executed with node-fibers script instead of node. node-sync adds sync method to Function.prototype that allows to run it syncroniously.
Also you need to wrap your call in the fiber (thread) so as not to block the event-loop.
var Sync = require('sync');
var someAsyncFunction = function(a, b, callback) {
setTimeout(function() {
var result = a + b;
callback(undefined, result);
}, 1000);
};
// Run in a fiber
Sync(function(){
// This code runs in separate fiber and does not block the event loop
// First argument is 'this' context
var result = someAsyncFunction.sync(null, 2, 3);
// Waiting one second
console.log(result); // Will output 5
});
// Event loop here
Please be careful with it. You need to understand that is not the node way.
Quite often see in JavaScript libraries code like this:
setTimeout(function() {
...
}, 0);
I would like to know why use such a wrapper code.
Very simplified:
Browsers are single threaded and this single thread (The UI thread) is shared between the rendering engine and the js engine.
If the thing you want to do takes alot of time (we talking cycles here but still) it could halt (paus) the rendering (flow and paint).
In browsers there also exists "The bucket" where all events are first put in wait for the UI thread to be done with whatever it´s doing. As soon as the thread is done it looks in the bucket and picks the task first in line.
Using setTimeout you create a new task in the bucket after the delay and let the thread deal with it as soon as it´s available for more work.
A story:
After 0 ms delay create a new task of the function
and put it in the bucket. At that exact moment the UI thread is busy
doing something else, and there is another tasks in the bucket
already. After 6ms the thread is available and gets the task infront
of yours, good, you´re next. But what? That was one huge thing! It has
been like foreeeeeever (30ms)!!
At last, now the thread is done with that and comes and gets your
task.
Most browsers have a minimum delay that is more then 0 so putting 0 as delay means: Put this task in the basket ASAP. But telling the UA to put it in the bucket ASAP is no guarantee it will execute at that moment. The bucket is like the post office, it could be that there is a long queue of other tasks. Post offices are also single threaded with only one person helping all the task... sorry customers with their tasks. Your task has to get in the line as everyone else.
If the browser doesn´t implement its own ticker, it uses the tick cycles of the OS. Older browsers had minimum delays between 10-15ms. HTML5 specifies that if delay is less then 4ms the UA should increase it to 4ms. This is said to be consistent across browsers released in 2010 and onward.
See How JavaScript Timers Work by John Resig for more detail.
Edit: Also see What the heck is the event loop anyway? by Philip Roberts from JSConf EU 2014. This is mandatory viewing for all people touching front-end code.
There are a couple of reasons why you would do this
There is an action you don't want to run immediately but do want to run at some near future time period.
You want to allow other previously registered handlers from a setTimeout or setInterval to run
When you want to execute rest of your code without waiting previous one to finish you need to add it in anonymous method passed to setTimeout function. Otherwise your code will wait until previous is done
Example:
function callMe()
{
for(var i = 0; i < 100000; i++)
{
document.title = i;
}
}
var x = 10;
setTimeout(callMe, 0);
var el = document.getElementById('test-id');
el.innerHTML = 'Im done before callMe method';
That is the reason I use it.
Apart from previous answers I'd like to add another useful scenario I can think of: to "escape" from a try-catch block. A setTimeout-delay from within a try-catch block will be executed outside the block and any exception will propagate in the global scope instead.
Perhaps best example scenario: In today's JavaScript, with the more common use of so called Deferreds/Promises for asynchronous callbacks you are (often) actually running inside a try-catch.
Deferreds/Promises wrap the callback in a try-catch to be able to detect and propagate an exception as an error in the async-chain. This is all good for functions that need to be in the chain, but sooner or later you're "done" (i.e fetched all your ajax) and want to run plain non-async code where you Don't want exceptions to be "hidden" anymore.
AFAIK Dojo, Kris Kowal's Q, MochiKit and Google Closure lib use try-catch wrapping (Not jQuery though).
(On couple of odd occasions I've also used the technique to restart singleton-style code without causing recursion. I.e doing a teardown-restart in same loop).
To allow any previously set timeouts to execute.