this question might be duplicated but I am still not getting the answer. I am fairly new to node.js so I might need some help. Many have said that node.js is perfectly free to run incoming requests asynchronously, but the code below shows that if multiple requests hit the same endpoint, say /test3, the callback function will:
Print "test3"
Call setTimeout() to prevent blocking of event loop
Wait for 5 seconds and send a response of "test3" to the client
My question here is if client 1 and client 2 call /test3 endpoint at the same time, and the assumption here is that client 1 hits the endpoint first, client 2 has to wait for client 1 to finish first before entering the event loop.
Can anybody here tells me if it is possible for multiple clients to call a single endpoint and run concurrently, not sequentially, but something like 1 thread per connection kind of analogy.
Of course, if I were to call other endpoint /test1 or /test2 while the code is still executing on /test3, I would still get a response straight from /test2, which is "test2" immediately.
app.get("/test1", (req, res) => {
console.log("test1");
setTimeout(() => res.send("test1"), 5000);
});
app.get("/test2", async (req, res, next) => {
console.log("test2");
res.send("test2");
});
app.get("/test3", (req, res) => {
console.log("test3");
setTimeout(() => res.send("test3"), 5000);
});
For those who have visited, it has got nothing to do with blocking of event loop.
I have found something interesting. The answer to the question can be found here.
When I was using chrome, the requests keep getting blocked after the first request. However, with safari, I was able to hit the endpoint concurrently. For more details look at the following link below.
GET requests from Chrome browser are blocking the API to receive further requests in NODEJS
Run your application in cluster. Lookup Pm2
This question needs more details to be answer and is clearly an opinion-based question. just because it is an strawman argument I will answer it.
first of all we need to define run concurrently, it is ambiguous if we assume the literal meaning in stric theory nothing RUNS CONCURRENTLY
CPUs can only carry out one instruction at a time.
The speed at which the CPU can carry out instructions is called the clock speed. This is controlled by a clock. With every tick of the clock, the CPU fetches and executes one instruction. The clock speed is measured in cycles per second, and 1c/s is known as 1 hertz. This means that a CPU with a clock speed of 2 gigahertz (GHz) can carry out two thousand million (or two billion for those in the US) for the rest of us/world 2000 million cycles per second.
cpu running multiple task "concurrently"
yes you're right now-days computers even cell phones comes with multi core which means the number of tasks running at the same time will depend upon the number of cores, but If you ask any expert such as this Associate Staff Engineer AKA me will tell you that is very very rarely you'll find a server with more than one core. why would you spend 500 USD for a multi core server if you can spawn a hold bunch of ...nano or whatever option available in the free trial... with kubernetes.
Another thing. why would you handle/configurate node to be incharge of the routing let apache and/or nginx to worry about that.
as you mentioned there is one thing call event loop which is a fancy way of naming a Queue Data Structure FIFO
so in other words. no, NO nodejs as well as any other programming language out there will run
but definitly it depends on your infrastructure.
Related
So I'm not super experienced with node so bear with me.
I have 2 routes on a node.js server. During a typical day, both routes will be hit with requests at the same time. Route 1 will run smoothly but route 2 is a long running processes that returns several promises, so route 1 will take up the resource causing route 2 to pause (I have determined this is happening via data logs).
Route 1 looks like this:
app.post('/route1', function (req, res) {
doStuff().then(function(data){
res.end();
})
}
Route 2 is handling an array of data that needs to be parsed through so 1 record in the array is processed at a time
app.post('/route2', function (req, res){
async function processArray(array) {
for (const item of array) {
await file.test1()(item, res);
await file.test2()(item, res);
//await test3,test4,test5,test6
}
}
processArray(data).then(function() {
res.end();
}
}
So I'm guessing the problem is that the async/await is waiting for resources to become available before it continues to process records.
Is there a way for me to write this to where route1 will not interfere with route2?
In Node, almost everything you can await for (or call then on) is asynchronous. It does not block execution thread but rather offloads the task to another layer you don't control, and then just awaits for it to be finished while being free to work on something else. That includes working with filesystem and network requests. There are ways to block the thread still, for example, using synchronous versions of filesystem methods (like readFileSync instead of readFile) or doing heavy computations on javascript (like calculating factorial of 4569485960485096)
Given your route1 doesn't do any of this, it does not take any resources from route2. They are running in parallel. It's hard to tell without seeing the actual code, but I'm pretty sure you are getting connection timeout because your route2 is poorly written and it takes a long time to resolve (or doesn't resolve at all) for reasons not related to Node performance or blocking. Node is just chilling while waiting for your filesystem to run those endless tests 6 times in every array item (or whatever is going on there) and while this happens, browser stops waiting for response and shows you connection timeout. It's most likely that you don't need to await for every test on every array in the data instead of just running them all in parallel
Read more here https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
and here https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
NodeJs is single threaded. This is why you break the cpu/resource instensive services into micro-services.
If these route1 and route2 need to be in the same server then see if you can change the algorithm or way you handle the computation to optimize the performance or break them so that they are handled by different cores in multi-core architecture.
Again if you are talking about production situation with huge user base then its not definately good idea to put them together and run it in the same server.
See this for more information
I'm creating an nodejs web processor. I's is processing time that takes ~ 1 minute. I POST to my server and get status by using GET
this is my simplified code
// Configure Express
const app = express();
app.listen(8080);
// Console
app.post('/clean, async function(req, res, next) {
// start proccess
let result = await worker.process(data);
// Send result when finish
res.send(result);
});
// reply with when asked
app.get('/clean, async function(req, res, next) {
res.send(worker.status);
});
The problem is. The server is working so hard in the POST /clean process that GET /clean are not replied in time.
All GET /clean requests are replied after the worker finishes its task and free the processor to respond the request.
In other words. The application are unable to respond during workload.
How can I get around this situation?
Because node.js runs your Javascript as single threaded (only one piece of Javascript ever running at once) and does not time slice, as long as your worker.process() is running it's synchronous code, no other requests can be processed by your server. This is why worker.process() has to finish before any of the http requests that arrived while it was running get serviced. The node.js event loop is busy until worker.process() is done so it can't service any other events (like incoming http requests).
These are some of the ways to work around that:
Cluster your app with the built-in cluster module so that you have a bunch of processes that can either work on worker.process() code or handle incoming http requests.
When it's time to call worker.process(), fire up a new node.js process, run the processing there and communicate back the result with standard interprocess communication. Then, your main node.js process stays reading to handle incoming http requests near instantly as they arrive.
Create a work queue of a group of additional node.js processes that run jobs that are put in the queue and configure these processes to be able to run your worker.process() code from the queue. This is a variation of #2 that bounds the number of processes and serializes the work into a queue (better controlled than #2).
Rework the way worker.process() does its work so that it can do a few ms of work at a time, then return back to the message loop so other events can run (like incoming http requests) and then resume it's work afterwards for a few more ms at a time. This usually requires building some sort of stateful object that can do a little bit of work at a time each time it is called, but is often a pain to program effectively.
Note that #1, #2 and #3 all require that the work be done in other processes. That means that the process.status() will need to get the status from those other processes. So, you will either need some sort of interprocess way of communicating with the other processes or you will need to store the status as you go in some storage that is accessible from all processes (such as redis) so it can just be retrieved from there.
There's no working around the single-threaded nature of JS short of converting your service to a cluster of processes or to use something experimental like Worker Threads.
If neither of these options work for you, you'll need to yield up the processing thread periodically to give other tasks the ability to work on things:
function workPart1() {
// Do a bunch of stuff
setTimeout(workPart2, 10);
}
function workPart2() {
// More stuff
setTimeout(workPart3, 10); // etc.
}
I am new to Node and I am writing my very first node server. It should answer to a simple get request with a simple page after calling a backend rest service.
I am using express to manage the request and the axios package to make the backend request. The problem is that the server is blocking the event loop and I have problems understanding how to make the call to the backend asynchronous.
As of now the frontend server can only manage one request at a time!! I expected that if the backend service takes 10 seconds to answer everytime, the frontend server can answer two concurrent request in 10 seconds and not in 20 seconds.
Where am I wrong?
Here is an extract of the frontend node code:
app.get('/', function(req, res) {
//Making the call to the backend service. This should be asynchronous...
axios.post(env.get("BACKEND_SERVICE"),
{ "user": "some kind of input"})
.then(function(response){
//do somenthing with the data returned from the backend...
res.render('homepage');
})
}
And here it is and extract of the backend node code:
app.post('/api/getTypes', jsonParser, function (req, res) {
console.log("> API request for 'api/getTypes' SLEEP");
var now = new Date().getTime();
while(new Date().getTime() < now + 10000){ /* do nothing */ }
console.log("> API request for 'api/getTypes' WAKE-UP");
res.json({"types":"1"});
}
The problem is your busy-wait ties up the backend server such that it can't even begin to process the second request.
I assume you're trying to simulate the process of getting the types taking a while. Odds are what you're going to be doing to get the types will be async and I/O-bound (reading files, querying a database, etc.). To simulate that, just use setTimeout:
app.post('/api/getTypes', jsonParser, function (req, res) {
console.log("> API request for 'api/getTypes' SLEEP");
setTimeout(function() {
console.log("> API request for 'api/getTypes' WAKE-UP");
res.json({"types":"1"});
}, 10000);
});
That avoids hogging the backend server's only thread, leaving it free to start overlapping handling for the second (third, fourth, ...) request.
This is one of the key principles of Node: Don't do things synchronously if you an avoid it. :-) That's why the API is so async-oriented.
If you do find at some point that you have heavy CPU-burning crunching you need to do to process a request, you might spin it off as a child process of the server rather than doing it in the server process. Node is single-threaded by design, achieving very high throughput via an emphasis on asynchronous I/O. Which works great for most of what you need to do...until it doesn't. :-)
Re your comment:
The backend process will be written in another technology other than node, it will call a DB and it could take a while. I wrote that simple node rest service to simulate that. What I would like to understand is how the frontend server will react if the backend takes time to process the requests.
There's a big difference between taking time to process the requests and tying up the only server thread busy-waiting (or doing massive CPU-heavy work). Your busy-wait models doing massive CPU-heavy work, but if getting the types is going to be external to Node, you won't be busy-waiting on it, you'll be queuing a callback for an asynchronus completion (waiting for I/O from a child process, or I/O from a socket connected to a third server process, or waiting on I/O from the DB, etc.). So the setTimeout example above is a better model for what you'll really be doing.
The busy-wait keeps the front-end from completing because it goes like this:
Backend
Time Frontend Queue Backend
−−−− −−−−−−−−−− −−−−−−−−−−−−−−−−−−−−− −−−−−−−−−−−−−−−−−−−−−
0 sec Request #1 −−−−−−> Receive request #1 −−−−−> Pick up job for request #1
0 sec Request #1 −−−−−−> Receive request #2
Busy wait 10 seconds
10 sec Got #1 back <−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Send response #1
−−−−−> Pick up job for request #2
Busy wait 10 seconds
20 sec Got #2 back <−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Send response #2
So even though the front-end isn't busy-waiting, it sees 20 seconds go by because the backend busy-waits (unable to do anything else) for 10 seconds for each request.
But that's not what your real setup will do, unless the other technology you're using is also single-threaded. (If it is, you may want to have more than one of them run in parallel.)
I've written a program in Node and Express, using Request to connect to an API and downloads a bunch of data (think 3,000 API requests) (all within the usage limits of the API, mind you).
When running this in a Docker container, I'm getting a lot of getaddrinfo ENOTFOUND errors, and I'm wondering if this is a resourcing issue. My requests are like so:
request.get(url, function(err, resp, body){
// do stuff with the body here,
// like create an object and handball to a worker function
});
For the first few hundred requests this always works fine, but then I get lots nad lots of either ENOTFOUND or timeout errors, and I think the issue might be the way my code is dealing with all these requests.
I've batched them in a queue with timeouts so the requests happen relatively slowly, it helps a little bit but doesn't solve the problem completely.
Do I need to destroy the body/response objects to free up memory or something?
I've encountered similar issues with an API I was using, and it ended up being what some here suggested - rate limits. Some APIs don't return readable errors on rate limits, as they provide certain amount of resources per client, and when you've used it all up they can't even send you a bad response.
This happened even though I've stayed within the published rate limits per day, but turned out they have an unwritten limit per minute (or more like - just unable to process so many requests).
I answered it though by mocking that API with my own code, placing it in the network so it will maximize the similarities, and as my mocked code didn't do anything, I never got any errors in the NodeJS server.
Then I put it some countdowns and timeouts when it was needed.
I suggest the same to you. Remember them having a per hour limit, doesn't mean they don't have a different per second/minute limit.
Consider this simple example:
var BinaryServer = require('../../').BinaryServer;
var fs = require('fs');
// Start Binary.js server
var server = BinaryServer({port: 9000});
// Wait for new user connections
server.on('connection', function(client){
// Stream a flower image!
var file = fs.createReadStream(__dirname + '/flower.png');
client.send(file);
sleep_routine(5);//in seconds
});
When a client connects to the server I block the event for about 5 seconds (imagine that time has some complex operations). What is expect to happen if another client connects (meanwhile)? One thing that I read about NodeJS is non-blocking I/O. But in this case the second client only receive the flower after the sleeping of the first, right?
One thing that I read about NodeJS is non-blocking I/O. But in this case the second client only receive the flower after the sleeping of the first, right?
That's correct, assuming that you are doing blocking synchronous operations for five seconds straight. If you do any file system IO, or any IO for that matter, or use a setTimeout, then the other client will get their opportunity to use the thread and get the flower image. So, if you're doing really heavy cpu intensive processing, you have a few choices:
Fire it off in a separate process that runs asynchronously, EG using the built-in child_process module
Keep track of how long you've been processing for and every 100ms or so give up the thread by saving your state, and then using setTimeout to continue processing where you left off
Have multiple node processes already running, so that if one is busy there is another that can serve the second user (EG. behind a load balancer, or using the cluster module)
I would recommend a combination of 1 and 3 if this is ever a problem; but so much of node can be made asynchronous that it rarely is. Even things like computing password hashes can be done asynchronously
No - the two requests will be handled independently. So if the first request had to wait 5 seconds, and for some reason the second request only took 2 seconds, the second would return before the first.
In practice you would have your server connection called a second time before the first one had finished. But since they have all different state you would normally be unaware of that.