Edit: We can close. Isn't truly asynchronous, non-blocking javascript impossible?
var PATH = require ("path");
var URL = require ("url");
var sleep = function (ms){
var start = new Date ().getTime ();
while ((new Date ().getTime () - start) < ms);
}
require ("http").createServer (function (req, res){
if (URL.parse (req.url).pathname === "/1"){
console.log ("tab 1: I'm in");
PATH.exists ("test", function (exists){
sleep (5000);
res.writeHead (200, {"Content-Type": "text/plain"});
res.end ("1");
console.log ("tab 1: I'm done");
});
}else{
console.log ("tab 2: I'm in");
res.writeHead (200, {"Content-Type": "text/plain"});
res.end ("2");
console.log ("tab 2: I'm done");
}
}).listen (80);
Copy the content into a file.
Execute the file.
Open a new tab in browser. Set the url to localhost/1. Don't go yet.
Open a new tab in browser. Set the url to localhost/2. Don't go yet.
Go back to the first tab. Press enter and immediately after change to the second tab and press enter.
Result:
console log:
tab 1: I'm in
tab 1: I'm done
tab 2: I'm in
tab 2: I'm done
Tab 1 waits 5 seconds to receive the result "1".
Tab 2 also has to wait 5 seconds because tab 1 is sleeping for 5 seconds.
The docs says that all is asynchronous except the code. Only one thread. Only one request at a time. Requests are enqueued.
I/O calls are supposed to be asynchronous, right? Then why tab 2 has to wait to tab 1 if the callback comes from an asynchronous I/O process?
Thanks.
Because your sleep is blocking the event loop.
Replace it with setTimemout(function() { /* Code to run */ }, 5000); and watch /2 respond immediately.
The actual I/O is asynchronous, but all actions you're performing on the I/O happen in the event loop. If something is blocking the event loop, everything else has to wait, just like you said.
EDIT. For more clarity, look at the following ASCII graphic:
Event Loop Thread: ------+req1HandlerExistsCall+-------+req1Wait5Sec++++++++++++++++++++++++++req2ExistsCall+-------+req2Immediate+-------------
HTTP I/O: -+req1+--------------------------------------+req2+--------------------------------------+req1Response+--------+req2Response+
File I/O: ----------------------------+exists1+----------------------------------------------------+exists2+---------------------------
Basically, only one at a time for each thread. Because the first request handler blocks for 5 seconds (and it's essentially impossible to beat your filesystem with your fingers in a speed test), the second response doesn't even start to be handled until the first request is almost done.
You don't have any I/O calls in your code
You are calling a busy sleep method in your code ie your code is taking 5 seconds to process. And while it is processing nothing else can run on that instance
Honestly, is this a real question or are you just trying to bash node?
Sleep is implemented in blocking manner and runs on the very engine that is being single threaded.
SetTimeout method is the equivalent version of waiting some time in JavaScript.
Also consider, that in JavaScript most thing should involve a resultHandler as continuation is handled with function typed parameters to other functions to be invoked when the job is done.
Related
Below is a simple code which I wrote :-
var http = require('http');
var server = http.createServer(function (req,res) {
res.end("Hello World")
})
server.listen(5000);
setTimeout(() => {server.close() } ,4000); // put anytime duration here
The behaviour which I expected was that server should stay on for 4 seconds and then closes. But rather what is happening is that server stays on indefinitely and we can do anything, however when we leave server idle for 4 seconds, after that if we reload or do anything then the timer of 5 seconds starts and after that server closes.
Means from 0 to n seconds I kept on working, then left idle for 4 seconds, on 24th second I reloaded again and then on 29th second the server is closing.
Why is it happening like that. Basically settimeout will move to call queue after 5 seconds and then it should be moved to call stack by event loop and close the server as soon as it goes idle.We can put anytime duration in timeout we have to wait leave server idle for that duration for the settimeout to begin and after that 5 seconds it take to close it.
Why is it behaving so ?
I expected that the server shall close after 4 seconds but it is not happening so.
The issue is that setTimeout is non-blocking and does not wait for the server to finish processing any requests before closing. This means that even though server.close() is called after 5 seconds, the server continues to process any requests that were already in progress at that time. To ensure that the server closes after 5 seconds, regardless of any active requests, you can use the server.unref() method before calling setTimeout. This method tells node.js to not keep the event loop running solely for the server, so that it can close as soon as there are no other events to process.
server.unref();
setTimeout(() => {server.close() } ,5000);
First of all, I'm not experienced in asynchronous programming, so I'm sorry if I missed something obvious.
I see this question pop up a lot. I know that people don't like forcing synchronicity in Javascript, but it is necessary in this case. I'm making heavy calls to a production database that can't take too much load due to how frequently it is used. Because of this, I'm setting up my code in a loop to make a request, wait for confirmation that it is finished, sleep for 2 seconds, and then make the next request. This is because I'm going to be pulling a LOT of data from this server on a weekly basis over the course of around 10-20 minutes.
Here's the code that I have. Sleep is a function that forces the program to wait using the Date class.
var thread = function(cb){
cb();
};
do{
var x = thread(function(){
request.post(options, function(e, r, body){
console.log(e);
console.log(r.statusCode);
issues.push(body["issues"]);
maxResults = body["total"];
options.body.startAt += 25;
console.log("Going to sleep");
});
sleep(2000);
});
console.log("Waking up and moving to the next cycle");
}while(issues.length < maxResults);
console.log("Finished with the requests");
}
although I have a callback set up, my code is still running the requests asynchronously. Because I leave maxResults null, it is plainly obvious that my callback isn't working. This is my output:
Waking up and moving to the next cycle
Finished with the requests
Going to sleep
You need to make a recursive asynchronous function.
It would look something like this:
function fetch(existingIssues) {
return sendRequest().then(function() {
existingIssues.push(...);
if (existingIssues.length >= maxResults)
return existingIssues;
else
return fetch(existingIssues);
});
}
fetch([]).then(...);
As the title says...
Basically, if I have a single webworker and post it 1000 messages at once.
Each message results in the worker performing a processing intensive operation.
Am I best to post each message to the webworker sequentially after the previous one completes, or can I safely send all the requests over to the worker knowing that they will just be processed and returned one by one as they complete?
If I do this am I better off implementing a queueing system within the worker? or is it not necessary?
I understand that this single worker is only a single thread and therefore the javascript operations will indeed occur synchronously within the webworker itself, but I am concerned about contention in a similar way that performing 200 ajax requests at once would overwhelm a browser.
I hope this makes.
The worker will queue up messages (specifically, it will queue up calls to onmessage) and process each message when ever the worker's call stack is empty.
This means, however, that asynchronous operation could result in input multiple messages being processed before any any particular operation is completed. For example:
onmessage = function(e) {
var xhr = new XMLHttpRequest();
xhr.open("GET", "/" + e.data);
xhr.onload = function() {
postMessage(xhr.responseText);
}
}
If you passed in 1000 messages at once, I suspect the browser will fire off 1000 Ajax requests (or at least a lot of requests) before it runs postMessage inside of onload. This is because onload does not run until the Ajax call is complete, and each onload call will be added to the event queue behind any pending calls to onmessage that were requested before the Ajax fetch could complete.
If you wanted to ensure that only one Ajax request is out at a time, you could indeed implement an internal queue that moved only when onload was called:
var queue = [];
var busy = false;
onmessage = function(e) {
if(busy) {
queue.push(e);
}
else {
busy = true;
runAjax(e);
}
}
var runAjax = function(e) {
var xhr = new XMLHttpRequest();
xhr.open("GET", e.data);
xhr.onload = function() {
postMessage(xhr.responseText);
if(queue.length) {
// run the next queued item
runAjax(queue.shift());
} else {
busy = false;
}
}
}
In this case, the first messages goes into the else block, which sets busy = true and runs the Ajax fetch. The following 999 messages are routed to the if block, because busy is true (and will not become false until the Ajax request completed and queue is empty). The queued messages are handled only after an Ajax request is complete.
If you wanted to tolerate more than one Ajax request at the time, you could use a counter instead of boolean: use busy++ and busy-- instead of setting true and false, and test that the current number of outstanding connections has not reached your allowed max (i.e., if(busy == 5) instead of if(busy)).
The Web Worker receives each message one at a time - any queuing would be done by your own logic - for speed its best to transfer data as typed arrays especially a form called Transferable Objects - possibly you could put many (1000) messages into such a typed array (float or int) and make it a single message - the actual data transfer from browser to Web Worker or vice versa is quite fast - typed array of 10's of mbyte in milliseconds or less ... this messaging does handle objects as well albeit with a speed penalty compared to low level typed arrays
I have a bit of javascript I want to run in a webworker, and I am having a hard time understanding the correct approach to getting them to work in lock-step. I invoke the WebWorker from the main script as in the following simplified script:
// main.js
werker = new Worker("webWorkerScaffold.js");
// #1
werker.onmessage = function(msgObj){
console.log("Worker Reply")
console.log(msgObj);
doSomethingWithMsg(msgObj);
};
werker.onerror = function(err){
console.log("Worker Error:");
console.log(err);
};
werker.postMessage("begin");
Then the complimentary worker script looks like the following:
// webWorkerScaffold.js
var doWorkerStuffs = function(msg){}; // Omitted
// #2
onmessage = function (msgObj){
// Messages in will always be json
if (msgObj.data.msg === "begin")
doWorkerStuffs();
};
This code (the actual version) works as expected, but I am having a difficult time confirming it will always perform correctly. Consider the following:
The "new Worker()" call is made, spawning a new thread.
The spawned thread is slow to load (lets say hangs at "// #2")
The parent thread does "werker.postMessage..." with no recipient
... ?
The same applies in the reverse direction, where I might change the worker script to make noise outward once it is setup internally, under that scenario the main thread could hang at "// #1" and miss the incoming message as it dosen't have its comm's up.
Is there some way to guarantee that these scripts move forward in a lock-step way?
What I am really looking for is a zmq-like REP/REQ semantic, where one or the other blocks (or calls back) when 1:1 transactions can take place.
My problem is as follows:
On a page, I am doing multiple (6) ajax calls (via jQuery) to fetch some data at the same time (using php wrapper to fetch the data via the calls)
The requests are made at the same time and it takes them 10-20 seconds to complete
If user clicks on any of the links to go somewhere else (to some other page), I can see that all the calls that didn't complete get cancelled
However, the browser still waits for 20 seconds to navigate to the other page - it is like it is still waiting for the longest call to complete, even if it is cancelled
(Same issue is happening in Chrome and Firefox, ajax calls are asynchronous...I have tried to set ajax timeout, to capture readystate=o in ajax error response, even tried to do something with webworkers, to no avail)
Any insight would be helpful
Thanks!
That is due to the maximum number of connections of the browser to a single domain.
See Browserscope for mamimum per browser.
IE 8+9: 6 connections
IE 10: 8 connections
Chrome 26: 6 connections
Firefox 21: 6 connections
What you could do is collect all Defferred objects and cancel them when the use clicks on a link.
example:
// some ajax calls
doAjax({});
doAjax({});
var requests = [];
// helper function to perform and register calls
function doAjax(options) {
var jqXHR= $.ajax(options);
requests.push(jqXHR);
jqXHR.always(function(jqXHR) {
var index = requests.indexOf(jqXHR);
if (index!=-1) {
requests.splice(index, 1);
}
});
}
// function to abort all calls
// when your application has a router to provide navigation, this function
// is typically called when you navigate to another page
function abortAllPendingRequests() {
var i;
for (i=0; i<requests.length; i++) {
var xhr = requests[i];
// already finished calls (readyState 4) should not be aborted
if (xhr.readyState != 4) {
xhr.abort();
}
}
};
All is left is calling the abortAllPendingRequests function when the user tries to navigate to another page.
When you have some sort of router (e.g. Backbone.Router) you could call it there.
If your application does not have a router for navigating, but you make use of plain anchor links, you could add an onClick to the links which calls the abortAllPendingRequests function.