First of all, I am well aware that Node.js is non-blocking before anything else, but in this very specific case, it must be blocking and waiting here.
I have an authentication process that works that way (using APIs, I didn't design this so I cannot modify the way the auth works):
I send a PUT request to a REST API, I get a HTTPResponse code that determines if the API understood the request.
The server I just requested through its API sends the full response (including error codes, etc) through an XMPP protocol.
This means, when I send the request, I cannot know what happened next, and must wait for the XMPP event to trigger (basically, an on("message", callback) event).
I'd like to know how to work with this with Node.js.
Two more things to know:
1) I'm working on a client/server architecture, and my Node.js server is doing this authentication process, and sending a response through the websocket to the client and waiting for a socket answer is out of the question (not my call, but my boss wants this process to be done in one pass).
2) It must not be done with the client socket and must go through the full Node.js process for various reasons.
Thanks a lot for your help people! \o/
Sorry for not answering previously, we had some severe hardware failure at work.
Anyway, I'm not answering one of your comments directly because I found a solution I prefer, even if I thank you for your help. I've decided to use a promise and to wait for its answer to be sure to get a proper response.
Here is the code:
var answer = await new Promise((accept, reject) => {
// If there are no stanza in 30 seconds, then the process failed or the stanza got missed.
setTimeout(() => {
reject("timed out");
}, (30 * 1000));
// Waiting for the xmpp event to trigger.
xmpp.on("stanza", function(stanza) {
// Processing of the received stanza goes here.
});
});
#gkatzioura solution was interesting, but this looked a little bit heavy on bandwidth and we are working on a large scale applications, or maybe I didn't fully understand it, then it is my mistake.
#pspi solution was also interesting but this would be a problem considering the XMPP event listener is inside the request, and the PUT request needs to send a body on its end() event and here it wouldn't really work for what I want to do. I think that's because the original post I made was somewhat unclear.
Thanks a lot for your help guys! :]
I don't know enough XMPP, but would this just be case of "putting dependent logic inside callback".
request.put(..., function () {
// dependent xmpp logic inside request callback
xmpp.on(..., function () {
// xmpp and put response received, now talk back to socket client
socket.send(...);
});
});
In your case I would proceed with the event emitter (or anything in a publish subscribe fashion).
Fire your http call and inside the handler add an emitter listener with a check if the events is for the corresponding authentication.
Meanwhile your xmpp connection once it receives the authorization it shall emit a message.
The listener will receive the message successfully and will use the callback of the http call.
Related
I have a api that returns if the building is open and how much peapole are in there.
Now i want my Discord Bot to send a message when the Building opens.
How do i do that?
if the api recives a request the response is looks this :
state: {
open: false/true
}
It may be helpful to clear out the terminology: the "API" in this context is the endpoints exposed by the server and their request/response schemas (you can think of it as the fields you send and receive back). Now, this doesn't change in your case: it's the same endpoint, and the same fields. What changes is the value.
Now, you are probably doing a HTTP request to a given URL, where the server is. And in HTTP world, we say that we are requesting a resource. The resource behind https://stackoverflow.com is the homepage of this website. The resource behind the endpoint you are calling is a building's state. This resource changes overtime, it may open or closed at any time, people going in and out. But the API doesn't change in this case.
Let's reword your question, so it can be clearer: How can a client know when a HTTP resource changes? If your server only exposes this endpoint to know the state of the building, the answer is a sad "it can't". Let's say that I close the building, the server knows it somehow and now the building's state is {"open": false}. But the server doesn't have any mechanism to say to your client that the state changed, the server just waits for the client to ask what the state is, and returns. Allowing a server to send data to your client without the client requesting first adds some complexity to your architecture, and although there's a bonus (the client will know of state changes as soon as possible), in your case, it may not be necessary.
One alternative is long polling, in long polling your client makes a request to the server and the server doesn't respond immediately, it... waits. Waits for an update, like a change in building's state. When an update happens, then it sends a response. The client, in turn, requests again! And waits for the server to send an update... In practice, the client will keep up with the server state. The mentioned article for long polling gives a good example: https://javascript.info/long-polling#regular-polling
The one caveat is that the server must also support long polling. If the server just returns whatever the resource's state is, then the client will keep receiving the same state over and over. Another valid solution is instead of waiting for updates, the client keeps requesting the server for every few seconds. You may miss some updates! But in some cases, it's fine to lose track of a few updates.
Ok, enough theory. What about your case? If you want to know if a door is open or closed, but don't care to know when it happens, you can just request the server every five seconds or so:
In some pseudo javascript code, and very inspired by the long polling article mentioned before:
async function subscribe() {
let buildingState = null
while (true) {
const response = await fetch("/subscribe")
if (response.status != 200) {
// An error - let's show it
showMessage(response.statusText)
}
// Get and show the message
const message = await response.json()
// a function that returns true/false if the state is different
if (stateChanged(message, buildingState)) {
updateDiscordBot(message)
buildingState = message
}
// wait five seconds and repeat
await new Promise(resolve => setTimeout(resolve, 5000));
}
}
subscribe();
This works fine. But if you want for your client to immediately know when a door is opened or closed, this is not enough: opening and closing a door is a quite fast action, and even if you request the server every second, you may lose updates. In this case, the server needs to change. Either by implementing long polling, or something like websockets... you didn't mention that the server is in your control, but if it doesn't, it may be helpful to talk with who maintains it to work out a solution.
If you only have pull access to that API then the only way to detect state change is to periodically send a request, store response, and trigger your bot on stored response change.
I think we need some help here. Thanks in advance.
I have been doing programming in .Net for desktop applications and have used Timer objects to wait for a task to complete before the task result are shown in a data grid. Recently, we switched over to NodeJs and find it pretty interesting. We could design a small application that executes some tasks using PowerShell scripts and return the data to the client browser. However, I would have to execute a Timer on the client browser (when someone clicks on a button) to see if the file, that Timer receives from the server, has "ENDOFDATA" or not. Once the Timer sees ENDOFDATA it triggers another function to populate DIV with the data that was received from the server.
Is this the right way to get the data from a server? We really don't want to block EventLoop. We run PowerShell scripts on NodeJS to collect users from Active Directory and then send the data back to the client browser. The PowerShell scripts are executed as a Job so EventLoop is not blocked.
Here is an example of the code at NodeJs:
In the below code can we insert something that won't block the EventLoop but still respond to the server once the task is completed? As you can see in the code below, we would like to send the ADUsers.CSV file to the client browser once GetUsers.PS1 has finished executing. Since GetUSers.PS1 takes about five minutes to complete the Event Loop is blocked and the Server can no longer accept any other requests.
app.post("/LoadDomUsers", (request, response) => {
//we check if the request is an AJAX one and if accepts JSON
if (request.xhr || request.accepts("json, html") === "json") {
var ThisAD = request.body.ThisAD
console.log(ThisAD);
ps.addCommand("./public/ps/GetUsers.PS1", [{
name: 'AllParaNow',
value: ScriptPara
}])
ps.addCommand(`$rc = gc ` + __dirname + "/public/TestData/AD/ADUsers.CSV");
ps.addCommand(`$rc`);
ps.invoke().then((output) => {
response.send({ message: output });
console.log(output);
});
}
});
Thank you.
The way you describe your problem isn't that clear. I had to read some of the comments in your initial question just to be sure I understood the issue. Honestly, you could just utilize various CSV NPM packages to read and write from your active directory with NodeJS.
I/O is non-blocking with NodeJS, so you're not actually blocking the EventLoop. You can handle multiple I/O requests, since NodeJS will just create threads for each one,
and continue execution on the main thread until the I/O operations complete and send back the data to its function reference, adding them as functions to the callstack and resuming program execution from those function's references. After you get the I/O data, you just send it back to the client through the response object. There should be no timers needed.
So is the issue once the powershell script runs, you have to wait for that initial script to complete before being able to handle pending requests? I'm still a bit unclear...
I am starting to learn more about how this "web world" works and that's why I am taking the free code camp course. I already took front-end development and I really enjoyed it. Now I am on the back end part.
The back end is much more foggy for me. There are many things that I don't get so I would hope that someone could help me out.
First of all I learned about the get method. so I did:
var http = require('http');
and then made a get request:
http.get(url, function callBack(response){
response.setEncoding("utf8");
response.on("data", function(data){
console.log(data);
});
});
Question 1)
So apparently this code "gets" a response from a certain URL. but What response? I didn't even ask for anything in particular.
Moving on...
The second exercise asks us to listen to a TCP connection and create a server and then write the date and time of that connection. So here's the answer:
var server = net.createServer(function listener (socket){
socket.end(date);
});
server.listen(port);
Question 2)
Okay so I created a TCP server with net.createServer() and when the connection was successful I outputted the date. But where? What did actually happen when I put date inside of socket.end()?
Last but not least...
in the last exercise I was told to create an HTTP server (what?) to server a text file for every time it receives requests, and here's what I did:
var server = http.createServer(function callback(request, response){
var read = fs.createReadStream(location);
read.pipe(response);
});
server.listen(port);
Question 3)
a) Why did I have to create an HTTP server instead of a regular TCP? what's the difference?
b)what does createReadStream do?
c) What does pipe() do?
If someone could help me, trying to make the explanation easier would help me a lot since I am, as you can see, pretty dumb on this subject.
Thank you a lot!
This is a little broad for Stackoverflow which favors focused questions that address specific problems. But I feel your pain, so…
Questions 1:
Http.get is roughly equivalent to requesting a webpage. The url in the function is the page you are requesting. The response will include several things like the HTTP response code, but also (most importantly) the content of the page, which is what you are probably after. On the backend this is normally used for hitting APIs that get data rather than actual web pages, but the transport mechanism is the same.
Question 2:
When you open a socket, you are waiting for someone else to request a connection. (The way you do when you use http.get(). When you output data you are sending them a response like the one you received in question 1.
Question 3:
HTTP is a higher level protocol than TCP. This basically means it is more specific and TCP is more general (pedants will take issue with that statement, but it's an easy way to understand it). HTTP defines the things like GET and POST that you use when you download a webpage. Lower down in the protocol stack HTTP uses TCP. You could just use TCP, but you would have to do a lot more work to interpret the requests that come in. The HTTP library does that work for you. Other protocols like FTP also use TCP, but they are different protocol than HTTP.
For this answer, you need to understand two things. An IP address is the numeric value of a website, it's the address to the server pointing to the site. A domain name is a conversion from IP to a NAMED system which allows humans an easier way to see the names of websites, so instead of typing numbers for websites, like 192.168.1.1, we can now just type names (www.hotdog.com). That's what your get request is doing, it's requesting the site.
socket.end is a method you're calling. socket.end "Half-closes the socket. i.e., it sends a FIN packet. It is possible the server will still send some data" from the nodejs.org docs, so basically it half closes your socket at the parameter you're sending in, which is todays current date.
HTTP is hyper text transfer protocol, TCP (transmissioncontrol protocol) is a link between two computers
3a HTTP is for browsers, so that's why you did it, for a web page you were hosting locally or something.
3b createreadstream() Returns a new ReadStream object. (See Readable Stream).
Be aware that, unlike the default value set for highWaterMark on a readable stream (16 kb), the stream returned by this method has a default value of 64 kb for the same parameter.
3c pipe:
The 'pipe' event is emitted when the stream.pipe() method is called on a readable stream, adding this writable to its set of destinations.
I've a client to server Websocket connection which should be there for 40 seconds or so. Ideally it should be forever open.
The client continually sends data to server and vice-versa.
Right now I'm using this sequence:
var socket;
function senddata(data)
{
if (!socket)
{
socket = new WebSocket(url);
socket.onopen = function (evt) {
socket.send(data);
socket.onmessage = function (evt) {
var obj = JSON.parse(evt.data);
port.postMessage(obj);
}
socket.oneerror = function (evt) {
socket.close();
socket = null;
}
socket.onclose = function(evt){
socket = null;
}
}
}
else
{
socket.send(data);
}
}
Clearly as per current logic, in case of error, the current request data may not be sent at all.
To be frank it sometimes gives error that websocket is still in connecting state. This connection breaks often due to networking issues. In short it does not work perfectly well.
I've read a better design : How to wait for a WebSocket's readyState to change but does not cover all cases I need to handle.
Also I've Googled about this but could not get the correct procedure for this.
So what is the right way to send regular data through Websockets which handles well these issues like connection break etc?
An event you don't seem to cover is onclose. Which should work really well, since it's called whenever the connection terminates. This is more reliable than onerror, because not all connection disruptions result in an error.
I personally use Socket.IO, it enables real-time bidirectional event-based communication between client and server.
It is event driven. Events such as
on connection :: socket.on('conection',callback);
and
on disconnect :: socket.on('disconnect',callback);
are built in with socket.io so it can help you with your connection concerns. Pretty much very easy to use, check out their site if you are interested.
I use two-layer scheme on client: abstract-wrapper + websocket-client:
The responsibilities of the websocket-client are interacting with a server, recovering the connection and providing interfaces (event-emitter and some methods) to abstract-wrapper.
The abstract-wrapper is a high-level layer, which interacts with websocket-client, subscribes to its events and aggregating data, when the connection is temporary failed. The abstract-wrapper can provide to application layer any interface such as Promise, EventEmitter and so on.
On application layer, I just work with abstract-wrapper and don't worry about connection or data losing. Undoubtedly, it's a good idea to have here information about the status of connection and data sending confirmation, because it's useful.
If it is necessary, I can provide some code for example
This apparently is a server issue not a problem in the client.
I don't know how the server looks like here. But this was a huge problem for me in the past when I was working on a websocket based project. The connection would continuously break.
So I created a websocket server in java, and that resolved my problem.
websockets depend on lots of settings, like if you're using servlets then servlet container's settings matter, if you're using some php etc, apache and php settings matter, for example if you create a websocket server in php and php has default time-out of 30 seconds, it will break after 30 seconds. If keep-alive is not set, the connection wont stay alive etc.
What you can do as quick solution is
keep sending pings to a server after a certain amount of time (like 2 or 3 seconds, so that if a websocket is disconnected it is known to the client so it could invoke onclose or ondisconnect, I hope you know that there is no way to find if a connection is broken other than failing to send something.
check server's keep-alive header
If you have access to server, then it's timeouts etc.
I think that would help
I'm trying to make a websocket using jQuery which is triggered continually after the load of the page. The idea, is to get information continually from the server and to display them into the web page without any refresh.
What kind of event is?
Any brilliant idea, please?
setInterval can be dangerous because of the timing of your http request/response. Better to used chained setTimeouts, e.g.
var tick = function() {
// do something here
$('foo').toggle();
setTimeout(tick, 1000); // wait 1 second and call again
};
tick();
I think an HTML5 Websocket is what you are looking for.
The WebSocket specification defines an API establishing "socket" connections between a web browser and a server. In plain words: There is an persistent connection between the client and the server and both parties can start sending data at any time.
http://html5rocks.com/en/tutorials/websockets/basics