Sending Many POST requests simultaneously Nodejs - javascript

I am new to Nodejs so excuse me for any mistake .. :)
Let me explain what i am trying to do :
basically i am making a push notification service for my platform .. i will explain further..
I have two NodeJs servers (using express) :
SERVER 1 :
it gets everything needed from the database such as ( device registration , identifier ..) and should send to the second server.
SERVER 2 : This server Receives a JSON ( contains everything i need ) to create the FCM and APNS payload and then send to the convenient provider (FCM,APNS).
what i am using : i am using axios to send POST requests.
The issue : since the 1st server will be sending big amount of requests ( usually 5K or more -- it's dynamic) at the same time , axios cannot handle that , and I've tried many other alternatives to axios but faced the same thing.
My question : How can i send that amount of requests without any issues ?
PS: when i send few requests ( 100 or bit more) i face no errors ...
I hope everything is clear and i would really appreciate any help.
Code Example of the Request with Axios :
PS: it always falls in the "[Request Error] ..."
try
{
axios.post(endpoint,{sendPushRequest})
.then( response => {
console.log(response.data);
})
.catch( er => {
if (er.response) {
console.log("[Response Error] on sending to Dispatcher...");
// The request was made and the server responded with a status code
// that falls out of the range of 2xx
console.log(er.response.data);
console.log(er.response.status);
console.log(er.response.headers);
} else if (er.request) {
console.log("[Request Error] on sending to Dispatcher...");
// The request was made but no response was received
// `error.request` is an instance of XMLHttpRequest in the browser and an instance of
} else {
// Something happened in setting up the request that triggered an Error
console.log('[Error]', er.message);
}
console.log(er.config);
});
}
catch (e) {
console.log('[ Catch Error]', e);
}

Usually, for doing this kind of asynchronous stuff you should use any queuing service as if the second server gets busy which it might in case of handling such a huge number of rest APIs your user would miss the notification.
Your flow should be like:
SERVER 1: it gets everything needed from the database such as ( device registration, identifier ..) and should push/publish to any queuing service such as rabbitMQ/Google pubsub etc.
SERVER 2: Instead of having rest APIs, this server should pull messages from the queue recursively and then Receives a JSON ( contains everything I need ) to create the FCM and APNS payload and then send to the convenient provider (FCM, APNS).
This is beneficial because even if anything happens to your server like busy/crashes the message would persist in the queue and on restarting the server you would be able to do your work(sending a notification or whatever).

Related

How to constantly send message from a node js to my front end

How to constantly update my front end dashboard with new information from the back end.
I have been searching for a solution online, but couldn't stumble on any.
I already know how to send static variables with ejs, but I cant figure out how to update my front end with new messages from the server.
I am working with express for the server and ejs for templating, plus server side java script.
I want to consonantly send messages to the user. Something like page 3 of 100......, 10 of 100..... and so forth. If you have experience with node Js, kindly help me out. Thanks.
You could use Long pooling to solve your problem, Long pooling is,
A request is sent to the server
The server doesn’t close the connection until it has a message to
send
When a message appears – the server responds to the request with it
The browser makes a new request immediately.
The situation when the browser sent a request and has a pending connection with the server is standard for this method. Only when a message is delivered, the connection is reestablished.
If the connection is lost, because of, say, a network error, the browser immediately sends a new request.A sketch of client-side subscribe function that makes long requests:
async function subscribe() {
let response = await fetch("/subscribe");
if (response.status == 502) {
// Status 502 is a connection timeout error,
// may happen when the connection was pending for too long,
// and the remote server or a proxy closed it
// let's reconnect
await subscribe();
} else if (response.status != 200) {
// An error - let's show it
showMessage(response.statusText);
// Reconnect in one second
await new Promise(resolve => setTimeout(resolve, 1000));
await subscribe();
} else {
// Get and show the message
let message = await response.text();
showMessage(message);
// Call subscribe() again to get the next message
await subscribe();
}
}
subscribe();
Hope this hepls!

How to return multiple updates of a JSON using expressjs and nodejs

I have a server side task that takes some time to compute, and I'd like to periodically send updates to the client. I store and send the information as an object (via JSON), and it isn't an array where I can send data sequentially. Rather, I want to send some new information, and update others as the calculation continues.
From other posts on here I realize that:
response.json(object) is a nice and easy way to send an object json in one go, with headers set and everything. However, this - like response.send() - terminates the connection:
var app = express()
app.get('/', (request, response) => {
response.json( { hello:world } );
})
Alternatively, one could set the headers manually, and then use response.write with JSON.stringify
response.setHeader('Content-Type', 'application/json');
response.write(JSON.stringify({ hello:world } ));
response.end();
The above two methods work for sending an object in one go, but ideally what I'd like to do is send incremental updates to my object. E.g.
response.setHeader('Content-Type', 'application/json');
response.write( JSON.stringify( { hello:[world], foo:bar } ) );
// perform some operations
response.write( JSON.stringify( { hello:[world, anotherWorld], foo:cat } ) );
response.end()
However, what is happening on the clientside is:
After the first response.write, the client receives { hello:[world], foo:bar } but does not trigger my callback
After the second response.write, I can see the data received is { hello:[world], foo:bar }{ hello:[world, anotherWorld], foo:cat } still does not trigger my callback
My callback is only called after response.end(), and then I get an exception when trying to parse it as JSON, because it isn't a valid JSON anymore, but a bunch of JSONs stuck back to back with no comma or anything: Uncaught (in promise) SyntaxError: JSON.parse: unexpected non-whitespace character after JSON data at line 1 column XXX of the JSON data.
Ideally my client callback would be triggered upon receiving each write, and it would remove that bit of data from the buffer so to speak, so the next incoming json would be standalone.
Is what I want to do possible? If so, how?
My fetch code btw:
fetch(url)
.then(response => response.json()) // parse the JSON from the server
.then(returnInfo => {
onReturn(returnInfo);
});
For your use-case, you can consider using WebSockets to deliver incremental updates to your UI. There are 3 stages of WebSockets connections. Connect, message and disconnect. One page load your front-end maintains persistent connection with backend. You can send first JSON data on connect and then when your backend has updates, send data in your message call back. I have written a blog post that implements WebSockets using Python and Javascript. However, you can implement similar logic using NodeJS
https://blog.zahidmak.com/create-standalone-websocket-server-using-tornado/

Streaming Node.js response to client over XHR

I have an Express.js server with a route that is responsible for parsing and importing a CSV file.
I'd like to report the status of the import process in realtime back to the user's browser.
Streaming responses from Express.js is straight forward:
router.get( '/test', function( _req, _res, _next ) {
_res.write( 'File import initiated.' );
// ... processing CSV in loop
_res.write( 'Processing row x.' );
// artificial delay to simulate some i/o
setTimeout( function() {
_res.write( 'File import completed successfully.' );
_res.end();
}, 2000 )
} );
Sending a HTTP request to the endpoint via CURL works as expected. In the above example two streamed responses
are received immediately, followed by the final response after two seconds.
This is not the case in the browser (Chrome) however. Testing both with jQuery $.ajax, and a raw XHR call, the entire
response appears to be buffered and returned all at once when the response is complete.
How is this type of communication normally handled? I assume socket.io is an option, but surely there's a simpler
way to utalise the build int XHR or XHR2 features of the browser?
You can give back a reference id to the client, and the client calls the correct http api's to gather the status of that csv processing request. You might want some backend support to enable this. Something like some tables in the database that can track this status.
router.get( '/test', function( _req, _res, _next ) {
SaveFile(_res.file).then(function(id) {
_res.end(JSON.stringify({id: id});
});
});
router.get( '/status', function( _req, _res, _next ) {
GetStatus(_res.params.id).then(function(status) {
_res.end(JSON.stringify(status));
});
});
Else, if you still want realtime, employ websockets. You've to design the form of communication you expect.

Swift Alamofire, only sends 4 post requests

I am trying to develop an app which sends a HTTP POST request to a NodeJS server (on the localhost). I am using the networking library 'Alamofire' just to try and simplify things.
My issue arises when I try and send multiple requests, I set up a simple button which sends the post request. When I run the app i can click the button and see that it sends the post requests when I click it, but only for the first 4 button presses/post requests.
It may be possible that my server is not set up correctly and not sending a response back or something along those lines.
I get 4 post's each time i open the app - everytime i close and re open the app i get another 4.
The same goes for when I restart the server.
I have tested the server using a pre-existing app (HTTP utility) and it appears to work fine.
When I use Alamofire i use JSON encoding, could this cause the issue???
Here's my swift code:
import UIKit
import Alamofire
class ViewController: UIViewController {
#IBAction func postBTN(sender: AnyObject) {
post(["xx":[10]])
}
override func viewDidLoad() {
super.viewDidLoad()
}
func post(postParrams: [String : AnyObject]) {
Alamofire.request(.POST, "http://10.0.0.10:3000", parameters: postParrams, encoding: .JSON)
}
}
UPDATE *** I added the following code below the 'Alamofire.request' to print the response data:
.validate()
.responseJSON { response in
print(response.request)
print(response.response)
print(response.result)
switch response.result {
case .Success:
print("Validation Successful")
case .Failure(let error):
print("the error is", error)
}
}
The error code that prints is:
the error is Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={NSUnderlyingError=0x7fd5a3d0e170 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, NSErrorFailingURLStringKey=http://localhost:3000/, NSErrorFailingURLKey=http://localhost:3000/, _kCFStreamErrorDomainKey=4, _kCFStreamErrorCodeKey=-2102, NSLocalizedDescription=The request timed out.}
From this i gather that the response result is a 'failure' and there are 2 error codes: Code=-1001, CodeKey=-2102

Send user details (session token) within every AJAX requests (Sencha Touch 2)

i am building a Sencha Touch 2 Application with userspecific datasets.
Architecture of the App:
Sencha Touch App <=====> Java Server backend with REST Services
( many AJAX requests =) )
What i actually have is:
Login the user with username/password
The app gets initialized and the loginform comes into play. After submitting the form as a AJAX request, the server backend checks the userdata and calls the client callback function.
And what i want to do is:
The callback function should
create a cookie with the sessiontoken or
store the sessiontoken within the localstorage (http://docs.sencha.com/touch/2-0/#!/api/Ext.data.proxy.LocalStorage) or
store the sessiontoken within js variable
Okay, shouldn't be the problem.
But how can i achieve the following:
Most of the data is specific for one user and should be returned by the REST service if needed (by clicking on the navigation,...). How can i send the sessiontoken (see above) within every AJAX request - so the server can provide the suitable datasets (assuming the token is valid)?
Send cookies within AJAX requests
I have already read that cookies gets automaticly added to the request if the url is on the same space, right? The Java Server is on the same domain (localhost:8080) but the cookies aren't available - instead of requests on urls like 'app.json'. I thought that cross-domain-requests are really domain specific?
Send paramaters within AJAX requests
Because the cookies aren't avi i thought about the possiblity of 'manually' adding parameters to the ajax requests. The App will contain many AJAX requests and thats why i dont want to add the token manually - i tried to override the requests function of Ext.Ajax but i failed ;-( :
(function() {
var originalRequest = Ext.data.Connection.prototype.request;
Ext.override(Ext.data.Connection, {
request : function(options) {
alert("Do sth... like add params");
return originalRequest.apply(this, options);
}
});
})();
ERROR:
Uncaught Error: [ERROR][Ext.data.Connection#request] No URL specified
I also tried to add a listener
Ext.Ajax.add({
listeners : {
beforerequest : function( conn, options, eOpts ){
alert("Do sth... like add params");
}
}
});
ERROR:
Uncaught TypeError: Object [object Object] has no method 'add'
Any idea about how i can add the token?
Or any better way of handling these case?
Thanks!
Finally i successfully used:
function addAjaxInterceptor(context)
{
Ext.Ajax.on('beforerequest', function(conn, options, eOptions)
{
// add the param to options...
}, context);
}
Executed from the app (=> addAjaxInterceptor(this)).
But the following solution is more suitable for my situation i think:
Ext.Ajax._defaultHeaders = {
// params as json
};
(Cause Ext.Ajax is a singleton object and i dont change the params for every request)

Categories

Resources