I am trying to proxy an api call from client side through my server for some third party service, reasons for this being CORS issues and adding in a secret key on the server side. I usually get it done in the following way:
app.use('/someService', (req, res) => {
let url = `https://example.com/${config.SECRET_KEY}/endpoint${req.url}`
req.pipe(request(url).pipe(res))
})
this way I can use any ajax library on the client side and perform get request for example: get: '/someService/params' and it would normally go through and perform that request and then pipe it back. However now I started getting:
Error: write after end
in express and I am not entirely sure about what could be causing it.
Your piping is wrong. As it is now, you're piping to res twice (.pipe() returns the argument passed to it for chainability).
Instead try this:
req.pipe(request(url)).pipe(res)
I should point out however that properly proxying the HTTP response is not quite that simple since currently this line will always respond with HTTP status code 200, no matter what the remote server for the middle request responds with. Also, any headers from that response will not be sent to res. With that in mind, you could naively try something like:
var proxiedRes = req.pipe(request(url));
proxiedRes.on('response', function(pres) {
res.writeHead(pres.statusCode, pres.headers);
// You will want to add a `pres` 'error' event handler too in case
// something goes wrong while reading the proxied response ...
pres.pipe(res);
});
Related
I'm wondering if there's any way to listen for console messages and act on console messages when they're received. Mainly, is there any way to do this without an external module, and using the http module?
The goal is to trigger a NodeJS function or code snippet on an event like click in the HTML. If there's also a way to do this, then that's great. But once again, I'd like to do this without an external module, and just use those that are built-in to NodeJS.
Use onclick() function in JavaScript to trigger a function call when clicking on a element. Then use fetch to make a api call to the nodejs server.
I know #Haris Wilson already got the answer, but I'd just like to provide a code example.
Instead of trying to catch a console message and then execute a function if we find it, we can use fetch() to make a request to whatever URL we need, and this can allow us to make other requests.
In this case, we can use the url module and the http module to parse the url and serve the API and website, respectively.
const url = require('url')
const http = require('http')
const requestListener = async function (req, res) {
// Basic server setup
res.writeHead(200, {
'Content-Type': 'text/html'
});
res.end(/** Content here */)
// API
if (url.parse(req.url, true).pathname === '/APIcall') {
let arguments = url.parse(req.url, true).query
// Preform necassary actions here
}
}
We can now use onClick to call a function inside our webpage JavaScript, and use fetch([API URL]) to give our NodeJS data to preform an action. We can use URL params to do this, such as https://localhost:8080/APIcall?data=someData&moreParam=more-data, where ?data=someData&moreParam=more-data are the URL params.
I have a server side task that takes some time to compute, and I'd like to periodically send updates to the client. I store and send the information as an object (via JSON), and it isn't an array where I can send data sequentially. Rather, I want to send some new information, and update others as the calculation continues.
From other posts on here I realize that:
response.json(object) is a nice and easy way to send an object json in one go, with headers set and everything. However, this - like response.send() - terminates the connection:
var app = express()
app.get('/', (request, response) => {
response.json( { hello:world } );
})
Alternatively, one could set the headers manually, and then use response.write with JSON.stringify
response.setHeader('Content-Type', 'application/json');
response.write(JSON.stringify({ hello:world } ));
response.end();
The above two methods work for sending an object in one go, but ideally what I'd like to do is send incremental updates to my object. E.g.
response.setHeader('Content-Type', 'application/json');
response.write( JSON.stringify( { hello:[world], foo:bar } ) );
// perform some operations
response.write( JSON.stringify( { hello:[world, anotherWorld], foo:cat } ) );
response.end()
However, what is happening on the clientside is:
After the first response.write, the client receives { hello:[world], foo:bar } but does not trigger my callback
After the second response.write, I can see the data received is { hello:[world], foo:bar }{ hello:[world, anotherWorld], foo:cat } still does not trigger my callback
My callback is only called after response.end(), and then I get an exception when trying to parse it as JSON, because it isn't a valid JSON anymore, but a bunch of JSONs stuck back to back with no comma or anything: Uncaught (in promise) SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 1 column XXX of the JSON data.
Ideally my client callback would be triggered upon receiving each write, and it would remove that bit of data from the buffer so to speak, so the next incoming json would be standalone.
Is what I want to do possible? If so, how?
My fetch code btw:
fetch(url)
.then(response => response.json()) // parse the JSON from the server
.then(returnInfo => {
onReturn(returnInfo);
});
For your use-case, you can consider using WebSockets to deliver incremental updates to your UI. There are 3 stages of WebSockets connections. Connect, message and disconnect. One page load your front-end maintains persistent connection with backend. You can send first JSON data on connect and then when your backend has updates, send data in your message call back. I have written a blog post that implements WebSockets using Python and Javascript. However, you can implement similar logic using NodeJS
https://blog.zahidmak.com/create-standalone-websocket-server-using-tornado/
Issue clarification
When we use .emit() or .send() and we also want to confirm message reception (so called acknowledgements) we simply write something like this:
socket.emit('someEvent', payload, callback);
What this question is all about is a callback part. That's the great stuff as it allows to generally send back some data as a response with no extra events emitted. All that server needs to do is to handle the request in a proper way:
socket.on('someEvent', (payload, callback) => {
doSomeStuff();
callback(someData);
);
That works just fine when we deal with a success case. But what shall we do in these cases:
1) Callback was not sent from the client side / callback's not a function and there's a need to respond from the server side with something like 'Error: no callback is provided. Usage: ...'
Example:
Client side - socket.emit('someEvent'); or socket.emit('someEvent', 1);
Server side - socket.on('someEvent', callback => callback());
or
2) While handling the request something went wrong (e.g. an unsuccessful validation result) and we need to report this in a way like: 'No payload is provided or it is invalid'
Example:
Server side -
socket.emit('someEvent', payload, callback => {
checkPayload();
callback(someData);
});
Client side - socket.on('someEvent', invalidPayload, callback);
Question: is there a mechanism to create custom callback from responder's side?
My workings and workarounds
1) As for the missing callback or that one which is not a function I've concluded that I can only validate it and then invoke it only in case of its validity. So the server side is undergoing some changes:
socket.emit('someEvent', callback => callback instanceof Function && callback()); //check callback correctness
Pros: there won't be an internal error if a callback is not a function as expected.
Cons: in case of invalid callback a client won't be noticed about it.
2) As for the case when we need to send some error back I've only found a workaround to return a specific, agreed in advance, falsy value like null so that it means that no data can be returned.
socket.emit('someEvent', payload, callback => {
checkPayload();
callback(someData || null); //send falsy, error-like value instead
});
Pros: a client will be noticed about some error by getting null.
Cons: from server side there's no simple middleware function that validates the input data and returns error before the main logic is being executed.
I've thought about middlewares for reaching the needed functionality, but there's no, so to say, 'event level middlewares' yet, only on the whole namespace and socket levels. Shall I try to filter events by their names on the socket level to attach the needed functionality and send error in a way like next(new Error(...));? In this case there can be a work with error event listening, I guess.
socket.io / socket.io-client versions used: 2.3.0
1) Callback was not sent from the client side / callback's not a function and there's a need to respond from the server side with something like 'Error: no callback is provided. Usage: ...'
The client and server have to agree how to do this. If the client doesn't provide a callback, then the server argument will be undefined so you can detect that from the server.
So, the proper way to do it is this:
// client
socket.emit('someMsg', someData, function(response) {
console.log(`Got ${response} from server`);
});
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
fn("got your data");
}
});
});
So, if the client does not pass the callback, then fn on the server side will be undefined. So, you are correct to test for that before calling it.
2) As for the case when we need to send some error back I've only found a workaround to return a specific, agreed in advance, falsy value like null so that it means that no data can be returned.
Yes, you have to agree in advance how to send an error back. The cleanest way to send an error back would probably be to wrap your response in an object and use a .error property on that object.
// client
socket.emit('someMsg', someData, function(response) {
if (response.error) {
console.log(`Got error ${response.error} from server`);
} else {
console.log(`Got data ${response.data} from server`);
}
});
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
// no error here
fn({error: null, data: "Got your message"});
}
});
});
What you're seeing here is that socket.io is not really a request/response type protocol and socket.io has tried to shoehorn in a bit of a response around which you have to build your own structure.
Or, you can send an error object if there's an error:
// server
io.on('connection', socket => {
socket.on('someMsg', (data, fn) => {
console.log(`Got data ${data} from client, sending response`);
// if client wants a response, send the response
if (fn) {
// send an error here
fn({error: new Error("xxx Error")});
}
});
});
From server side there's no simple middleware function that validates the input data and returns error before the main logic is being executed.
I don't really understand what you're trying to use middleware for or to validate? the only place this data is present is on your message handler so any server-side validation you want to do on what the client sent needs to be there. You can certainly do that validation before you've send a response.
Shall I try to filter events by their names on the socket level to attach the needed functionality and send error in a way like next(new Error(...));? In this case there can be a work with error event listening, I guess.
Socket.io doesn't work like Express and I don't really see why you'd try to make it work that way. There is no next() involved in receiving a socket.io message so I'm not sure what you're trying to do there. There is an option for middleware when the socket.io connection is first made, but not for subsequent messages sent over that connection.
Is there a way to send response from server even if no callback is provided from client side?
If the client does not provide a callback, then the only way to send a response back to the client would be to send another message. But, the whole point of sending a response is if you have a cooperating client that is listening and expecting a response so the client may as well use the callback if they want the response. If the client doesn't want the response and won't code anything to receive it, there's nothing you can do about that.
To keep things clean in my express route page I have a local function that is called in every route and it passes the sql query together with the req and res objects.
This works fine for sending a successful result and calling res.send works.
The problem that I'm having is I can't seem to find a way to get res.status to work and no matter the syntax it simply times-out and gives no error whatsoever in the console OR on the front end.
The tricky thing is, when it's inside the specific route it does work but the error message does not seem to get sent through instead it's just blank body?
`async function queryDatabase(queryParam, req, res) {
try {
const cp = new sql.ConnectionPool(config);
await cp.connect();
let result = await cp.request().query(queryParam);
cp.close();
res.send(result.recordset);
} catch (err) {
res.statusMessage = `Database error: ${err}`;
res.status(520);
}
}`
res.status(520) only sets the status value in the response object. It does not actually send the response. So, to send the response, you have several options. In the more recent versions of Express, you can use this shortcut:
res.sendStatus(520);
This will both set the status and send the response.
But, you can also do this in any version of Express:
res.status(520).end();
Which also sets the status and then sends the response.
You should end your response, use res.status(520).end() instead of res.status(520)
I am working on a NodeACS app in which I have to send xml as request from Java HTTP client and to receive response after some manipulation. The Java HTTP Client is working fine so far but the issue is with Node JS file. The req.on('data'...) event is not firing in my case.
Following is the code of my JS file:
function index(req, res) {
console.log(req.headers); //Its getting printed on console
req.on('data', function (data) {
console.log("Inside 1"); //Not printed on console
....REST OF THE CODE....
});
req.on('end', function () {
res.writeHead(200);
res.end();
});
}
In the above code after getting request the index function is called and printing the output for console.log(req.headers); but as mentioned above the script is not running after that. Please help me out What am I missing in this.
For testing I have installed cURL on my system and sending POST request to server using cURL request using following command:
curl -X POST -d #output.xml http://localhost:7788/
Can you confirm you aren't consuming the body prior to this? If the body has been consumed already by another middleware like body-parser you would need to restream the body via something like connect-restreamer. If it were consumed, the data event would not be emitted.
If that checks out, check to see if the "end" event is being emitted. Could be a sign of the content-length header being set to the wrong value.