I have a 3rd party service to implement, that provides restful API with long polling... they are serving live events, and displaying it as quick as possible is crucial.
Sadly that service does not provide push notifications, so we need to deal with what we have....
so one of the APIs has a long-polling functionality, so in theory the idea is, I query the API, establish an open channel for 30 seconds and wait for the changes... (and publish the changes on the FE)... and then timeout the connection, establish a new one and repeat the process.
Should be pretty straight forward....
But so far, I couldn't find anything in angular's docs about long polling... the only thing I found was polling in connection with rxJS... so setting an inteval on how often I query the API... (which in my case would be every 30s)... but nothing about on leaving the channel open and listen for the changes...
found this thred:
How to implement http long polling in Angular 2
but it is not talking about this problem.
I don't want to end up querying the API every second.
Any ideas?
I considered implementing signalR (which might not really help here), but it relies on jQuery... which I don't want to add to the bundle unless is 100% necessary!
I think you misunderstood the concept of long-pulling..
Long-pulling where the client polls the server requesting new information. The server holds the request open until new data is available. Once available, the server responds and sends the new information. When the client receives the new information, it immediately sends another request, and the operation is repeated. This effectively emulates a server push feature.
It you want to keep the connection alive you need to use webSoket.
WebSockets provide a persistent connection between a client and server that both parties can use to start sending data at any time. The client establishes a WebSocket connection through a process known as the WebSocket handshake. This process starts with the client sending a regular HTTP request to the server
But I didn't understood from your question why you can't send another fetch request when the long-pulling completes, something like:
myLongPullingFunc(){
this.http.get(longPullingURL)
.subscribe(res=>{
if(sholdFetchData){
useFetchedData(res);
this.myLongPullingFunc();
}else
doSomethingElse()
})
}
You can read more about long-pulling here and on webSokects here
Related
I've seen how to make a post request from JavaScript to get data from the server, but how would I do this flipped. I want to trigger a function in the flask server that will then dynamically update the variable on the JavaScript side to display. Is there a way of doing this in a efficient manner that does not involve a periodic iteration. I'm using an api and I only want to the api to be called once to update.
There are three basic options for you:
Polling - With this method, you would periodically send a request to the server (maybe every 5 seconds for example) and ask for an update. The upside is that it is easy to implement. The downside is that many requests will be unnecessary. It sounds like this isn't a great option for you.
Long Polling - This method means you would open a request up with the server and leave the request open for a long period of time. When the server gets new information it will send a response and close the request - after which the client will immediately open up a new "long poll" request. This eliminates some of the unnecessary requests with regular polling, but it is a bit of a hack as HTTP was meant for a reasonably short request response cycle. Some PaaS providers only allow a 30 second window for this to occur for example.
Web Sockets - This is somewhat harder to setup, but ultimately is the best solution for real time server to client (and vice versa) communication. A socket connection is opened between the server and client and data is passed back and forth whenever either party would like to do so. Javascript has full web socket support now and Flask has some extensions that can help you get this working. There are even great third party managed solutions like Pusher.com that can give you a working concept very quickly.
Basically I'm working on an application that posts huge amounts of data to web API(third party API).
I am working on nodejs to connect to MsSQL server and fetch data, process it and post it in desired format to the web API.
Scenario is: In nodejs, I have script that does the above job. It is currently initiated (or should I say triggered?) by a button on web page using axios POST with all necessary parameters. Eg below:
axios.post('/api/v1/fetch-new-labors', {
startDate: 'somedate',
endDate: 'someDate'
}).then(...handles further processing & posting to api)
The process takes around 2-3 mins to finish. Meanwhile, if the page is refreshed, it obviously needs to restart the whole process by clicking the button.
Question: I am sure there is a way to let the process run(on serverside I presume) that takes care of fetching, processing and posting even if the client side page is refreshed/reloaded and at the same time, keep the client side informed with progressbar or x out of y records posted kind of thing. I thought of web sockets but I wasn't sure if there's a prefered way to achieve this. I'm not looking for the whole code/process, I am looking for someone to guide me towards overall concept/idea.
tl;dr: Long-running jobs usually avoid using the traditional request-response cycle, opting instead for some variation of the Pub/Sub pattern described below:
Accept request, then start processing
You should respond to the user immediately with an HTTP 202: Accepted, signalling you accepted the request, then start the processing.
You can perform some initial checks before responding (Does the user have other jobs? Does the request pass basic validation checks?) but you should not start processing the actual long-running job before responding.
You can use a simple HTTP request to create jobs.
Push status updates from server to subscribed clients
On page load the client subscribes to updates from the server.
Using WebSockets, push server-to-client status notifications regarding the progress. Don't forget to also handle and display errors to the client.
At this point, you'll probably need a way to uniquely identify each client across refreshes. You can easily solve this by storing a UUID via LocalStorage when a user first visits your app/website. If your app requires logins, then you can use the logged-in user's ID instead.
Check if user has already running jobs before accepting a new one
When the user refreshes you can send an initial message via WebSockets again notifying the user if he has any running jobs and what their progress is.
Based on your OP, I think you might want to disable the "Create Job" button if there's a pending job.
You can use other mechanisms for bidirectional server-client communication (such as long-polling/Server-sent Events) if you want, although WebSockets should be the most straightforward and flexible solution.
I'd personally go for a batteries-included WebSocket library such as socket.io.
I need to refresh a part of my view without refreshing the whole page.
At my index.html page I have three panels, wich one shows the number of Tickets by it's status, I need to refresh this number every time a new ticket is created or updated. I used Java with Spring Boot and Thymelaf to build my application.
This is my view:
This is the way I'm doing it now:
model.addAttribute("resolvedTickets", atendimentoService.findAllTicketsByStatus(STATUS_RESOLVED).size());
I have tried to use web sockets but i can't figure out how to get this and refresh the panels.
In a standard web interaction, the client (i.e. your web browser) sends a request to your server. Your server receives the request, and sends back the information to show in your browser and then terminates the connection.
WebSockets are a way to create a persistent, two-way connection between the client and the server, but it requires cooperation from both. A lot of shared servers don't allow WebSockets, so you first have to make sure your server is capable of providing WebSockets. (I see from your screenshot that you're running on Heroku, which should have no problem running WebSockets.)
On the server side, you need to set up handling for incoming WebSocket requests. I don't know what language you've coded your server in, so I can't provide any guidance, but there are plenty of libraries that do the server-side part of WebSockets in most languages.
On the client side, you need to set up your WebSocket client. MDN has a great guide on WebSockets that explains what you'll need to do. Basically, all you'll have to do is listen for incoming messages and increment your counter.
var count = 0;
var exampleSocket = new WebSocket("ws://example.com/socket");
exampleSocket.onmessage = function(event) {
count++;
document.getElementById('myTicketCounter').innerHTML = count;
}
For some things, WebSockets are overkill. If you find that this is too much work for too little reward, you can also just set up an AJAX call to fire every few minutes that pings another page on your server and returns the number of tickets and updates accordingly. It won't be instantaneous, but if you don't need down-to-the-second resolution, it'll probably suffice. You can adjust the interval to be as long or as short as you want (to an extent; bombarding your server with constant requests will slow you down a bit).
I notice an instant messenger using javascript/http/xmlhttprequest rather than tcp/websocket to communicate.
such as http://w.qq.com/login.html
I monitored its communication way in chrome developer tool.
All are http requests. One request to server every minute.
The thing I confused is that if I send a message to the http client, it receives the message immediately.
As I know http client can not get message from others, it has to send request and get response.
Is there any way to make http client to get message from others without using tcp/socket or sending request and geting response?
Your comment welcome
I don't get all this asian hieroglyphs, but they probably use long-polling: client asks server for new data and server holds answer (don't send anything, and don't close connection) while data is not ready (new message came) or until huge timeout expired. As soon as connection closed, client process response and sends next long poll request. The more common term for "permanent" connect via http called Comet. As you can see, Comet can be implemented via many techniques. As for me, the most modern is ajax streaming. It requires modern browser of course, but worth it. It's not so hard to implement streaming by yourself, but I believe there are few js libs which do the job for you.
Upd:
Here is pretty good explanation with code source
and
here is one of many questions about streaming on SO (the answer in the bottom is most interesting I guess)
I've been googling for hours for this issue, but did not find any solution.
I am currently working on this app, built on Meteor.
Now the scenario is, after the website is opened and all the assets have been loaded in browser, the browser constantly makes recursive xhr calls to server. These calls are made at the regular interval of 25 seconds.
This can be seen in the Network tab of browser console. See the Pending request of the last row in image.
I can't figure out from where it originates, and why it is invoked automatically even when the user is idle.
Now the question is, How can I disable these automatic requests? I want to invoke the requests manually, i.e. when the menu item is selected, etc.
Any help will be appriciated.
[UPDATE]
In response to the Jan Dvorak's comment:
When I type "e" in the search box, the the list of events which has name starting with letter "e" will be displayed.
The request goes with all valid parameters and the Payload like this:
["{\"msg\":\"sub\",\"id\":\"8ef5e419-c422-429a-907e-38b6e669a493\",\"name\":\"event_Coll_Search_by_PromoterName\",\"params\":[\"e\"]}"]
And this is the response, which is valid.
a["{\"msg\":\"data\",\"subs\":[\"8ef5e419-c422-429a-907e-38b6e669a493\"]}"]
The code for this action is posted here
But in the case of automatic recursive requests, the request goes without the payload and the response is just a letter "h", which is strange. Isn't it? How can I get rid of this.?
Meteor has a feature called
Live page updates.
Just write your templates. They automatically update when data in the database changes. No more boilerplate redraw code to write. Supports any templating language.
To support this feature, Meteor needs to do some server-client communication behind the scenes.
Traditionally, HTTP was created to fetch dead data. The client tells the server it needs something, and it gets something. There is no way for the server to tell the client it needs something. Later, it became needed to push some data to the client. Several alternatives came to existence:
polling:
The client makes periodic requests to the server. The server responds with new data or says "no data" immediately. It's easy to implement and doesn't use much resources. However, it's not exactly live. It can be used for a news ticker but it's not exactly good for a chat application.
If you increase the polling frequency, you improve the update rate, but the resource usage grows with the polling frequency, not with the data transfer rate. HTTP requests are not exactly cheap. One request per second from multiple clients at the same time could really hurt the server.
hanging requests:
The client makes a request to the server. If the server has data, it sends them. If the server doesn't have data, it doesn't respond until it does. The changes are picked up immediately, no data is transferred when it doesn't need to be. It does have a few drawbacks, though:
If a web proxy sees that the server is silent, it eventually cuts off the connection. This means that even if there is no data to send, the server needs to send a keep-alive response anyways to make the proxies (and the web browser) happy.
Hanging requests don't use up (much) bandwidth, but they do take up memory. Nowadays' servers can handle multiple concurrent TCP connections, so it's less of an issue than it was before. What does need to be considered is the amount of memory associated with the threads holding on to these requests - especially when the connections are tied to specific threads serving them.
Browsers have hard limits on the number of concurrent requests per domain and in total. Again, this is less of a concern now than it was before. Thus, it seems like a good idea to have one hanging request per session only.
Managing hanging requests feels kinda manual as you have to make a new request after each response. A TCP handshake takes some time as well, but we can live with a 300ms (at worst) refractory period.
Chunked response:
The client creates a hidden iFrame with a source corresponding to the data stream. The server responds with an HTTP response header immediately and leaves the connection open. To send a message, the server wraps it in a pair of <script></script> tags that the browser executes when it receives the closing tag. The upside is that there's no connection reopening but there is more overhead with each message. Moreover, this requires a callback in the global scope that the response calls.
Also, this cannot be used with cross-domain requests as cross-domain iFrame communication presents its own set of problems. The need to trust the server is also a challenge here.
Web Sockets:
These start as a normal HTTP connection but they don't actually follow the HTTP protocol later on. From the programming point of view, things are as simple as they can be. The API is a classic open/callback style on the client side and the server just pushes messages into an open socket. No need to reopen anything after each message.
There still needs to be an open connection, but it's not really an issue here with the browser limits out of the way. The browser knows the connection is going to be open for a while, so it doesn't need to apply the same limits as to normal requests.
These seem like the ideal solution, but there is one major issue: IE<10 doesn't know them. As long as IE8 is alive, web sockets cannot be relied upon. Also, the native Android browser and Opera mini are out as well (ref.).
Still, web sockets seem to be the way to go once IE8 (and IE9) finally dies.
What you see are hanging requests with the timeout of 25 seconds that are used to implement the live update feature. As I already said, the keep-alive message ("h") is used so that the browser doesn't think it's not going to get a response. "h" simply means "nothing happens".
Chrome supports web sockets, so Meteor could have used them with a fallback to long requests, but, frankly, hanging requests are not at all bad once you've got them implemented (sure, the browser connection limit still applies).