I am starting to learn more about how this "web world" works and that's why I am taking the free code camp course. I already took front-end development and I really enjoyed it. Now I am on the back end part.
The back end is much more foggy for me. There are many things that I don't get so I would hope that someone could help me out.
First of all I learned about the get method. so I did:
var http = require('http');
and then made a get request:
http.get(url, function callBack(response){
response.setEncoding("utf8");
response.on("data", function(data){
console.log(data);
});
});
Question 1)
So apparently this code "gets" a response from a certain URL. but What response? I didn't even ask for anything in particular.
Moving on...
The second exercise asks us to listen to a TCP connection and create a server and then write the date and time of that connection. So here's the answer:
var server = net.createServer(function listener (socket){
socket.end(date);
});
server.listen(port);
Question 2)
Okay so I created a TCP server with net.createServer() and when the connection was successful I outputted the date. But where? What did actually happen when I put date inside of socket.end()?
Last but not least...
in the last exercise I was told to create an HTTP server (what?) to server a text file for every time it receives requests, and here's what I did:
var server = http.createServer(function callback(request, response){
var read = fs.createReadStream(location);
read.pipe(response);
});
server.listen(port);
Question 3)
a) Why did I have to create an HTTP server instead of a regular TCP? what's the difference?
b)what does createReadStream do?
c) What does pipe() do?
If someone could help me, trying to make the explanation easier would help me a lot since I am, as you can see, pretty dumb on this subject.
Thank you a lot!
This is a little broad for Stackoverflow which favors focused questions that address specific problems. But I feel your pain, so…
Questions 1:
Http.get is roughly equivalent to requesting a webpage. The url in the function is the page you are requesting. The response will include several things like the HTTP response code, but also (most importantly) the content of the page, which is what you are probably after. On the backend this is normally used for hitting APIs that get data rather than actual web pages, but the transport mechanism is the same.
Question 2:
When you open a socket, you are waiting for someone else to request a connection. (The way you do when you use http.get(). When you output data you are sending them a response like the one you received in question 1.
Question 3:
HTTP is a higher level protocol than TCP. This basically means it is more specific and TCP is more general (pedants will take issue with that statement, but it's an easy way to understand it). HTTP defines the things like GET and POST that you use when you download a webpage. Lower down in the protocol stack HTTP uses TCP. You could just use TCP, but you would have to do a lot more work to interpret the requests that come in. The HTTP library does that work for you. Other protocols like FTP also use TCP, but they are different protocol than HTTP.
For this answer, you need to understand two things. An IP address is the numeric value of a website, it's the address to the server pointing to the site. A domain name is a conversion from IP to a NAMED system which allows humans an easier way to see the names of websites, so instead of typing numbers for websites, like 192.168.1.1, we can now just type names (www.hotdog.com). That's what your get request is doing, it's requesting the site.
socket.end is a method you're calling. socket.end "Half-closes the socket. i.e., it sends a FIN packet. It is possible the server will still send some data" from the nodejs.org docs, so basically it half closes your socket at the parameter you're sending in, which is todays current date.
HTTP is hyper text transfer protocol, TCP (transmissioncontrol protocol) is a link between two computers
3a HTTP is for browsers, so that's why you did it, for a web page you were hosting locally or something.
3b createreadstream() Returns a new ReadStream object. (See Readable Stream).
Be aware that, unlike the default value set for highWaterMark on a readable stream (16 kb), the stream returned by this method has a default value of 64 kb for the same parameter.
3c pipe:
The 'pipe' event is emitted when the stream.pipe() method is called on a readable stream, adding this writable to its set of destinations.
Related
My main function is I am creating a link-shortening app. When someone entered a long URL, it will give a short URL. If the user clicked on the short link it will search for the long URL on the DB and redirect it to the long URL.
Meantime I want to get the click count and clicked user's OS.
I am currently using current code :
app.get('/:shortUrl', async (req, res) => {
const shortUrl = await ShortUrl.findOne({short: req.params.shortUrl})
if (shortUrl == null) return res.sendStatus(404)
res.redirect(shortUrl.full)
})
findOne is finding the Long URL on the database using ShortID. I used mongoDB here
My questions are :
Are there multiple redirect methods in JS?
Is this method work if there is a high load?
Any other methods I can use to achieve the same result?
What other facts that matter on redirect time
What is 'No Redirection Tracking'?
This is a really long question, Thanks to those who invested their time in this.
Your code is ok, the only limitation is where you run it and mongodb.
I have created apps that are analytics tracker, handling billion rows per day.
I suggest you run your node code using AWS Beanstalk APP. It has low latency and scales on your needs.
And you need to put redis between your request and mongodb, you will call mongodb only if your data is not yet in redis. Mongodb has more read limitations than a straight redis instance.
Are there multiple redirect methods in JS?
First off, there are no redirect methods in Javascript. res.redirect() is a feature of the Express http framework that runs in nodejs. This is the only method built into Express, though all a redirect response consists of is a 3xx (often 302) http response status and setting the Location header to the redirect location. You can manually code that just as well as you can use res.redirect() in Express.
You can look at the res.redirect() code in Express here.
The main things it does are set the location header with this:
this.location(address)
And set the http status (which defaults to 302) with this:
this.statusCode = status;
Then, the rest of the code has to do with handling variable arguments, handling an older design for the API and sending a body in either plain text or html (neither of which is required).
Is this method work if there is a high load?
res.redirect() works just fine at a high load. The bottleneck in your code is probably this line of code:
const shortUrl = await ShortUrl.findOne({short: req.params.shortUrl})
And, how high a scale that goes to depends upon a whole bunch of things about your database, configuration, hardware, setup, etc... You should probably just test how many request/sec of this kind your current database can handle.
Any other methods I can use to achieve the same result?
Sure there are. But, you will have to use some data store to look up the shortUrl to find the long url and you will have to create a 302 response somehow. As said earlier, the scale you can achieve will depend entirely upon your database.
What other facts that matter on redirect time
This is pretty much covered above (hint, its all about the database).
What is 'No Redirection Tracking'?
You can read about it here on MDN.
please what is different
var socket = new WebSocket('ws://localhost:8181');
var socket = new WebSocket('ws://localhost:8181/websession');
what is ( Websession )
websession is just the endpoint the websocket will connect to. It's just like normal HTTP servers or REST services: You can have multiple endpoints on one server, like:
ws://localhost:8181/customers
ws://localhost:8181/prices
ws://localhost:8181/items
... and so on. (This is just an example and does not necessarily make sense for a specific use case.) In old-style HTTP, you could image them as different directories on the same server, possibly offering very different contents.
In order to use the socket correctly, you have to know your desired endpoint and use it when creating the socket. So it depends on the server whether ws://localhost:8181 or ws://localhost:8181/websession is correct (or even both of them, depending on the purpose of the individual endpoint). It's generally a good practice to give the endpoint a meaningful name, so the first one would be discouraged.
As the application seems to be running on your localhost, you should take a look at the server running at port 8181 to find out the endpoints offered. And you could possibly get used to websockets, here is one of many possible starting points.
I'm using this code to send an SSE message to the browser client.
https://nodejs.org/api/http.html#http_response_write_chunk_encoding_callback
Node server
response.writeHead(200, { 'Content-Type': 'text/event-stream', 'Access-Control-Allow-Origin': '*' });
response.write(message);
response.end();
And for the client I'm using this javascript:
var source = new EventSource('myurl');
source.addEventListener('add', addHandler, false);
source.addEventListener('remove', removeHandler, false);
Everything is working fine, but how the server knows for sure that the client actually received it ? I guess SSE is using TCP, is there any way to received the acknowledgement ?
SSEs are a one to many Push protocol. So there is no acknowledgement. You could send an AJAX request back on receipt, but there is nothing in the pattern to provide this functionality.
SSE is a one way communication protocol to push data from server to client.
There no way for a client to ack event reception.
If acknowledge is a must have, you probably need a two way communication like websockets.
I know this is many years old, but none of the answers is correct. 1) TCP does indeed ACK the push stream - its standard http! (though whether your code is at a low-enough level to detect it is a different story)
2) it's not to difficult to develope your own ACK system (I've done it! - To free up resources when last client disappears) and yes, it tends to go against the "spirit" of the protocol and duplicate to a degree the websocket paradigm...but it is wrong to say its impossible. Send a unique per-client "token" in the first message which the browser saves and starts a js "ping" timer which ajaxes a "still alive" message. In your erver code, handle the ajax and restart client-stil-alive timer. If that expires, client has gone, clean up / free resources etc.
Yes its a bit "lumpy" but it works and its not rocket-science difficulty.
just my (very late) 2c worth
The attached image was me diagnosing a case where the ACK was missing, but one every other one apart from the indicated one you can see the ACK
I have a simple socket.io client and server program, running on node.js. The client and server exchange messages between them for a few minutes, before disconnecting (like chat).
If there any function/method I can use to get the total bytes transferred (read/write), after the socket is closed?
At present I am adding up the message size for each each message sent and received by the client. But, as per my understanding, in socket.io depending on which protocol is used (websocket, xhr-polling, etc.), the size of the final payload being sent will differ due to the header/wrapper size. Hence, just adding message bytes won't give me an accurate measure of bytes transferred.
I can use monitoring tools like Wireshark to get this value, but I would prefer using a javascript utility to get this value. Search online, didn't give me any reasonable answer.
For pure websocket connections, I am being able to get this value using the functions: socket._socket.bytesRead and socket._socket.bytesWritten
Any help is appreciated!
As of socket v2.2.0 i managed to get byte data like this. Only problem these are specified when client closes browser window and reason parameter is transport error. If client uses socket.close() or .disconnect() or server uses .disconnect() then bytes are 0.
socket.on('disconnect', (reason) => {
let symbs = Object.getOwnPropertySymbols(socket.conn.transport.socket._socket);
let bytesRead = socket.conn.transport.socket._socket[symbs[3]];
let bytesWritten = socket.conn.transport.socket._socket[symbs[4]];
});
If you wanted such a feature that would work no matter what the underlying transport was below a socket.io connection, then this would have to be a fundamental feature of socket.io because only it knows the details of what it's doing with each transport and protocol.
But, socket.io does not have this feature built in for the various transports that it could use. I would conclude that if you're going to use the socket.io interface to abstract out the specific protocol and implementation on top of that protocol, then you give up the ability to know exactly how many bytes socket.io chooses to use in order to implement the connection on its chosen transport.
There are likely debug APIs (probably only available to browser extensions, not standard web pages) that can give you access to some of the page-wide info you see in the Chrome debugger so that might be an option to investigate. See the info for chrome.devtools.network if you want more info.
I'm making a multiplayer (2 player) browser game in JavaScript. Every move a player makes will be sent to a server and validated before being transmitted to the opponent. Since WebSockets isn't ready for prime time yet, I'm looking at long polling as a method of transmitting the data and node.js looks quite interesting! I've gone through some example code (chat examples, standard long polling examples and suchlike) but all the examples I've seen seem to broadcast everything to every client, something I'm hoping to avoid. For general server messages this is fine but I want two players to be able to square off in a lobby or so and go into "private messaging" mode.
So I'm wondering if there's a way to implement private messaging between two clients using nodejs as a validating bridge? Something like this:
ClientA->nodejs: REQUEST
nodejs: VALIDATE REQUEST
nodejs->ClientA: VALID
nodejs->ClientB: VALID REQUEST FROM ClientA
You need some way to keep track of which clients are in a lobby together. You can do this with a simple global array like so process.lobby[1] = Array(ClientASocket, ClientBSocket) or something similar (possibly with some additional data, like nicknames and such), where the ClientXSocket is the socket object of each client that connects.
Now you can hook the lobby id (1 in this case) onto each client's socket object. A sort of session variable (without the hassle of session ids) if you will.
// i just made a hashtable to put all the data in,
// so that we don't clutter up the socket object too much.
socket.sessionData['lobby'] = 1;
What this allows you to do also, is add an event hook in the socket object, so that when the client disconnects, the socket can remove itself from the lobby array immediately, and message the remaining clients that this client has disconnected.
// see link in paragraph above for removeByValue
socket.on('close', function(err) {
process.lobby[socket.sessionData['lobby']].removeByValue(socket);
// then notify lobby that this client has disconnected.
});
I've used socket in place of the net.Stream or request.connection or whatever the thing is.
Remember in HTTP if you don't have keep-alive connections, this will make the TCP connection close, and so of course make the client unable to remain within a lobby. If you're using a plain TCP connection without HTTP on top (say within a Flash application or WebSockets), then you should be able to keep it open without having to worry about keep-alive. There are other ways to solve this problem than what I've shown here, but I hope I got you started at least. The key is keeping a persistent object for each client.
Disclaimer: I'm not a Node.js expert (I haven't even gotten around to installing it yet) but I have been reading up on it and I'm very familiar with browser js, so I'm hoping this is helpful somehow.