I am working on a cunning plan that involves using node.js as a proxy server in front of another service.
In short:
Dispatch incoming request to a static file (if it exists)
Otherwise, dispatch the request to another service
I have the basics working, but now attempting to get the whole thing working with Sencha Connect so I can access all the kick-ass middleware provided.
All of the action happens in dispatchProxy below
connect(
connect.logger(),
connect.static(__dirname + '/public'),
(request, response) ->
dispatchProxy(request, response)
).listen(8000)
dispatchProxy = (request, response) ->
options = {host: host, port: port, method: request.method, headers: request.headers, path: request.url}
proxyRequest = http.request(options, (proxyResponse) ->
proxyResponse.on('data', (chunk) ->
response.write(chunk, 'binary')
)
proxyResponse.on('end', (chunk) ->
response.end()
)
response.writeHead proxyResponse.statusCode, proxyResponse.headers
)
request.on('data', (chunk) ->
proxyRequest.write(chunk, 'binary')
)
# this is never triggered for GETs
request.on('end', ->
proxyRequest.end()
)
# so I have to have this here
proxyRequest.end()
You will notice proxyRequest.end() on the final line above.
What I have found is that when handling GET requests, the END event of the request is never triggered and therefore a call to proxyRequest.end() is required. POST requests trigger both DATA and END events as expected.
So several questions:
Is this call to proxyRequest.end() safe? That is, will the proxyResponse still be completed even if this is called outside of the event loops?
Is it normal for GET to not trigger END events, or is the END being captured somewhere in the connect stack?
The problem is less the end event and more the data event. If a client makes a GET requests, there's headers and no data. This is different from POST, where the requester is sending data, so the on("data") handler gets hit. So (forgive me for the JS example, I'm not that familiar with coffeescript):
var http = require('http');
// You won't see the output of request.on("data")
http.createServer(function (request, response) {
request.on("end", function(){
console.log("here");
});
request.on("data", function(data) {
console.log("I am here");
console.log(data.toString("utf8"));
});
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello World\n');
}).listen(8124);
console.log('Server running at http://127.0.0.1:8124/');
If I make a curl call to this server, the data event never gets hit, because the GET request is nothing more than headers. Because of this, your logic becomes:
// okay setup the request...
// However, the callback doesn't get hit until you
// start writing some data or ending the proxyRequest!
proxyRequest = http.request(options, (proxyResponse) ->
// So this doesn't get hit yet...
proxyResponse.on('data', (chunk) ->
response.write(chunk, 'binary')
)
// and this doesn't get hit yet
proxyResponse.on('end', (chunk) ->
// which is why your response.on("end") event isn't getting hit yet
response.end()
)
response.writeHead proxyResponse.statusCode, proxyResponse.headers
)
// This doesn't get hit!
request.on('data', (chunk) ->
proxyRequest.write(chunk, 'binary')
)
// So this isn't going to happen until your proxyRequest
// callback handler gets hit, which hasn't happened because
// unlike POST there's no data in your GET request
request.on('end', ->
proxyRequest.end()
)
// now the proxy request call is finally made, which
// triggers the callback function in your http request setup
proxyRequest.end()
So yes you're going to have to manually call proxyRequest.end() for GET requests due to the logic branching I just mentioned.
my experience is that request.on('end',) is not consistently called unless it's a POST. I suspect the event (of someone making a http.request) is over before the script gets a chance to detect it.
Related
I am writing a timeout middleware with express in node.js.
app.use((req, res, next) => {
res.setTimeout(3000, () => {
console.warn("Timeout - response end with 408")
res.status(408).json({ "error": "timeout 408" });
// !!! error will happen with next function when call like `res.send()`:
// Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
next()
})
If there's an endpoint that takes more than 3000 ms, my middleware will repsond with 408. However, the next function will respond again. I don't want to check if the response has been already sent by res.headersSent api every time.
Is there a better way to handle this - like the title said - to cancel the next response in the middleware?
It's your own code in the response handler that is still running (probably waiting for some asynchronous operation to complete). There is no way to tell the interpreter to stop running that code from outside that code. Javascript does not have that feature unless you put that code in a WorkerThread or a separate process (in which case you could kill that thread/process).
If you're just trying to suppress that warning when the code eventually tries to send its response (after the timeout response has already been sent), you could do something like this:
app.use((req, res, next) => {
res.setTimeout(3000, () => {
console.warn("Timeout - response end with 408")
res.status(408).json({ "error": "timeout 408" });
// to avoid warnings after a timeout sent,
// replace the send functions with no-ops
// for the rest of this particular response object's lifetime
res.json = res.send = res.sendFile = res.jsonP = res.end = res.sendStatus = function() {
return this;
}
});
next();
});
i dont know what is 'data' and req.on() method, please help me
http.createServer( (req, res) => { .....
if( req.method === 'POST') {
if( req.url ==='/user') {
let body ='';
req.on('data', (data) => {
body += data;
});
Basically a user can send data to an HTTP server by connecting to it, send the date and then disconnect. this works for a small amount of data. by this kind of request when the user sends an HTTP request and when http.createServer( (req, res) => {} gets called, you'll have a complete body and can access it in req.body, in other words, you've got the whole data sent by the user.
But sometimes the user wants to send a huge file(a big image, video, etc), the problem is, that it's problematic to send that huge data with a single request, instead, the user sends the data with a stream of data chunks. the user connects to the server and http.createServer( (req, res) => {} gets called, then the user starts to send the data in chunks, now with every chunk of data that is being sent and is being received by the server, req.on('data', (data) => {}) gets called, and it adds the received data to the body of the request. when the streams are finished, you'll have a complete body that contains that big file or any kind of big data that the user sent, and after finishing streams, the user disconnects from the server.
The req.on method is part of node's Event Emitter model. It provides methods to bind functions to certain events (on in this case) and ways to dispatch events to that functions
The data event is one of those examples. It is dispatched when there's data available in the buffer for the request to read, basically.
You can find more information on Event Emitters here and about the http events, you can find it here.
The req.on method is a function which allows to attach event listener for the event named which is passed as the first argument emitter.on.
the req object is type of http.ClientRequest which extends the EventEmitte which as some function which are related to Publish–subscribe pattern more.
in your case which i repeat bellow
req.on('data', (data) => {
body += data;
});
You are attaching a listener which will be executed every time the req object dispatch in their internal work flow the data event.
I am trying to implement this library here, which generates QR codes and all other kinds of codes.
The problem I have is making a request where I have access to both req and res object, since I will need to pass these to the library. In the documentation, they are recommending
http.createServer(function(req, res) {
// If the url does not begin /?bcid= then 404. Otherwise, we end up
// returning 400 on requests like favicon.ico.
if (req.url.indexOf('/?bcid=') != 0) {
res.writeHead(404, { 'Content-Type':'text/plain' });
res.end('BWIPJS: Unknown request format.', 'utf8');
} else {
bwipjs.request(req, res); // Executes asynchronously
}
}).listen(3030);
The problem is I already have a server created, and I simply want to call the library in a get request, without creating another server. I have tried
http.get('http://localhost:3030/?bcid=azteccode&text=thisisthetext&format=full&scale=2', (req, res) => {
bwipjs.request(req, res); // Executes asynchronously
}
)
which obviously didn't work as the callback only takes response as an argument.
I would like to use bare node in the implementation as this is the way my server is implemented, and I don't want to add a library (like Express) just for this scenario.
You are heavily misunderstanding the role of http.get
http.get is used to do HTTP GET call to that certain url. It's basically what axios or request or ajax or xhr or postman or browser does.
The url param of http.get is not route. It's literally is the url you want to hit.
If you want to handle specific route you have to do it in the http.createServer() handler itself.
Like,
http.createServer(function(req, res) {
if (req.url.indexOf('/?bcid=') != 0) {
//do something
} else if (req.method == "get" && req.url.indexOf('/?bcid=') != 0){
bwipjs.request(req, res); // Executes asynchronously
} else {
//do something else
}
}).listen(3030);
Check out the req or http.IncomingMessage for available properties that you can use.
Issue clarification
When we use .emit() or .send() and we also want to confirm message reception (so called acknowledgements) we simply write something like this:
socket.emit('someEvent', payload, callback);
What this question is all about is a callback part. That's the great stuff as it allows to generally send back some data as a response with no extra events emitted. All that server needs to do is to handle the request in a proper way:
socket.on('someEvent', (payload, callback) => {
doSomeStuff();
callback(someData);
);
That works just fine when we deal with a success case. But what shall we do in these cases:
1) Callback was not sent from the client side / callback's not a function and there's a need to respond from the server side with something like 'Error: no callback is provided. Usage: ...'
Example:
Client side - socket.emit('someEvent'); or socket.emit('someEvent', 1);
Server side - socket.on('someEvent', callback => callback());
or
2) While handling the request something went wrong (e.g. an unsuccessful validation result) and we need to report this in a way like: 'No payload is provided or it is invalid'
Example:
Server side -
socket.emit('someEvent', payload, callback => {
checkPayload();
callback(someData);
});
Client side - socket.on('someEvent', invalidPayload, callback);
Question: is there a mechanism to create custom callback from responder's side?
My workings and workarounds
1) As for the missing callback or that one which is not a function I've concluded that I can only validate it and then invoke it only in case of its validity. So the server side is undergoing some changes:
socket.emit('someEvent', callback => callback instanceof Function && callback()); //check callback correctness
Pros: there won't be an internal error if a callback is not a function as expected.
Cons: in case of invalid callback a client won't be noticed about it.
2) As for the case when we need to send some error back I've only found a workaround to return a specific, agreed in advance, falsy value like null so that it means that no data can be returned.
socket.emit('someEvent', payload, callback => {
checkPayload();
callback(someData || null); //send falsy, error-like value instead
});
Pros: a client will be noticed about some error by getting null.
Cons: from server side there's no simple middleware function that validates the input data and returns error before the main logic is being executed.
I've thought about middlewares for reaching the needed functionality, but there's no, so to say, 'event level middlewares' yet, only on the whole namespace and socket levels. Shall I try to filter events by their names on the socket level to attach the needed functionality and send error in a way like next(new Error(...));? In this case there can be a work with error event listening, I guess.
socket.io / socket.io-client versions used: 2.3.0
1) Callback was not sent from the client side / callback's not a function and there's a need to respond from the server side with something like 'Error: no callback is provided. Usage: ...'
The client and server have to agree how to do this. If the client doesn't provide a callback, then the server argument will be undefined so you can detect that from the server.
So, the proper way to do it is this:
// client
socket.emit('someMsg', someData, function(response) {
console.log(`Got ${response} from server`);
});
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
fn("got your data");
}
});
});
So, if the client does not pass the callback, then fn on the server side will be undefined. So, you are correct to test for that before calling it.
2) As for the case when we need to send some error back I've only found a workaround to return a specific, agreed in advance, falsy value like null so that it means that no data can be returned.
Yes, you have to agree in advance how to send an error back. The cleanest way to send an error back would probably be to wrap your response in an object and use a .error property on that object.
// client
socket.emit('someMsg', someData, function(response) {
if (response.error) {
console.log(`Got error ${response.error} from server`);
} else {
console.log(`Got data ${response.data} from server`);
}
});
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
// no error here
fn({error: null, data: "Got your message"});
}
});
});
What you're seeing here is that socket.io is not really a request/response type protocol and socket.io has tried to shoehorn in a bit of a response around which you have to build your own structure.
Or, you can send an error object if there's an error:
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
// send an error here
fn({error: new Error("xxx Error")});
}
});
});
From server side there's no simple middleware function that validates the input data and returns error before the main logic is being executed.
I don't really understand what you're trying to use middleware for or to validate? the only place this data is present is on your message handler so any server-side validation you want to do on what the client sent needs to be there. You can certainly do that validation before you've send a response.
Shall I try to filter events by their names on the socket level to attach the needed functionality and send error in a way like next(new Error(...));? In this case there can be a work with error event listening, I guess.
Socket.io doesn't work like Express and I don't really see why you'd try to make it work that way. There is no next() involved in receiving a socket.io message so I'm not sure what you're trying to do there. There is an option for middleware when the socket.io connection is first made, but not for subsequent messages sent over that connection.
Is there a way to send response from server even if no callback is provided from client side?
If the client does not provide a callback, then the only way to send a response back to the client would be to send another message. But, the whole point of sending a response is if you have a cooperating client that is listening and expecting a response so the client may as well use the callback if they want the response. If the client doesn't want the response and won't code anything to receive it, there's nothing you can do about that.
Hello guys i just started learning node.js today and search a lot off stuff on the internet , then try to code in node.js i use these two codes to show me the same result but the last one is show the error on my browser something likes "can not find the page".So please explain to me why?
// JScript source code
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World\n');
}).listen(1337, "127.0.0.1");
console.log('Server running at http://127.0.0.1:1337/');
This is working but
// Include http module.
var http = require("http");
// Create the server. Function passed as parameter is called on every request made.
// request variable holds all request parameters
// response variable allows you to do anything with response sent to the client.
http.createServer(function (request, response) {
// Attach listener on end event.
// This event is called when client sent all data and is waiting for response.
request.on("end", function () {
// Write headers to the response.
// 200 is HTTP status code (this one means success)
// Second parameter holds header fields in object
// We are sending plain text, so Content-Type should be text/plain
response.writeHead(200, {
'Content-Type': 'text/plain'
});
// Send data and end response.
response.end('Hello HTTP!');
});
}).listen(1337, "127.0.0.1");
This one is not working
Why?
The link of the last one that's not working
http://net.tutsplus.com/tutorials/javascript-ajax/node-js-for-beginners/
Thank you for all the answers, but i still don't understand about the problems.
the last one that is not working just has request.on?
request is an instance of http.IncomingMessage, which implements the stream.Readable interface.
Documentation at http://nodejs.org/api/stream.html#stream_event_end says:
Event: 'end'
This event fires when no more data will be provided.
Note that the end event will not fire unless the data is completely consumed. This can be done by switching into flowing mode, or by calling read() repeatedly until you get to the end.
var readable = getReadableStreamSomehow();
readable.on('data', function(chunk) {
console.log('got %d bytes of data', chunk.length);
})
readable.on('end', function() {
console.log('there will be no more data.');
});
So in your case, because you don't use either read() or subscribe to the data event, the end event will never fire.
Adding
request.on("data",function() {}) // a noop
within the event listener would probably make the code work.
Note that using the request object as a stream is only necessary for when the HTTP request has a body. E.g. for PUT and POST requests. Otherwise you can consider the request to have finished already, and just send out the data.
If the code your posting is taken literally from some other site, it may be that this code example was based on Node 0.8. In Node 0.10, there have been changes in how streams work.
From http://blog.nodejs.org/2012/12/20/streams2/
WARNING: If you never add a 'data' event handler, or call resume(), then it'll sit in a paused state forever and never emit 'end'.
So the code you posted would have worked on Node 0.8.x, but does not in Node 0.10.x.
The function you are applying to the HTTP server is the requestListener which supplies two arguments, request, and response, which are respectively instances of http.IncomingMessage and http.ServerResponse.
The class http.IncomingMessage inherits the end event from the underlying readable stream. The readable stream is not in flowing mode, so the end event never fires, therefore causing the response to never be written. Since the response is already writable when the request handler is run, you can just directly write the response.
var http = require('http');
http.createServer(function(req, res) {
res.writeHead(200, {
'Content-Type': 'text/plain'
});
res.end('Hello HTTP!');
}).listen();