I have a JavaScript application that regularly saves new and updated data. However I need it to work on slow connection as well.
Data is submitted in one single HTTP POST request. The response will return newly inserted ids for newly created records.
What I'm finding is that data submitted is fully saved, however sometimes the return result times out. The browser application therefore does not know the data has been submitted successfully and will try to save it again.
I know I can detect the timeout in the browser, but how can I make sure the data is saved correctly?
What are some good methods of handling this case?
I see from here https://dba.stackexchange.com/a/94309/2599 that I could include a pending state:
Get transaction number from server
send data, gets saved as pending on server
if pending transaction already exists, do not overwrite data, but send same results back
if success received, commit pending transaction
if error back, retry later
if timeout, retry later
However I'm looking for a simpler solution?
Really, it seems you need to get to the bottom of why the client thinks the data was not saved, but it actually was. If the issue is purely one of timing, then perhaps a client timeout just needs to be lengthened so it doesn't give up too soon or the amount of data you're sending back in the response needs to be reduced so the response comes back quicker on a slow link.
But, if you can't get rid of the problem that way, there are a bunch of possibilities to program around the issue:
The server can keep track of the last save request from each client (or a hash of such request) and if it sees a duplicate save request come in from the same client, then it can simply return something like "already-saved".
The code flow in the server can be modified so that a small response is sent back to the client immediately after the database operation has committed (no delays for any other types of back-end operations), thus lessening the chance that the client would timeout after the data has been saved.
The client can coin a unique ID for each save request and if the server sees the same saveID being used on multiple requests, then it can know that the client thinks it is just trying to save this data again.
After any type of failure, before retrying, the client can query the server to see if the previous save attempt succeeded or failed.
You can have multiple retries count as a simple global int.
You can also automatically retry, but this isn't good for an auto save app.
A third option is use the auto-save plugins for jQuery.
Few suggestions
Increase the time out, don't handle timeout as success.
You can flush output of each record as soon as you get using ob_flush and flush.
Since you are making request in regular interval. Check for connection_aborted method on each API call, if client has disconnected you can save the response in temp file and on next request you can append the last response with new response but this method is more resource consuming.
Related
The project that I am working on is to receive a request where in the main and/or most part of that request consists of data coming from a database. Upon receiving, my system proceeds with its function which is to parse all the data and ultimately concatenates the needed information to form a query, then insert those data using the mentioned query into my local database.
It is working fine and no issue at all. Except for the fact that it takes too long to process when the request has over 6,000,000 characters and over 200,000 lines (or maybe less but still with large numbers).
I have this tested with my system being used as a server (the supposed setup in production), and with Postman as well, but both drops the connection before the final response is built and sent. I have already tested and seen that although the connection drops, my system still proceeds with processing the data even up to the query, and even until it sends its supposed response. But since the request dropped somewhere in the middle of the processing, the response is ignored.
Is this about connection timeout in nodejs?
Or limit in 'app.use(bodyParser.json({limit: '10mb'}))'?
I really only see 1 way around this. I have done similar in the past. Allow the client to send as much as you need/want. However, instead of trying to have the client wait around for some undetermined amount of time (at which point the client may timeout), instead send an immediate response that is basically "we got your request and we're processing it".
Now the not so great part but it's the only way I've ever solved this type of issue. In your "processing" response, send back some sort of id. Now the client can check once in a while to see if it's request has been finished by sending you that id. On the server end you store the result for the client by the id you gave them. You'll have to make a few decisions about things like how long a response id is kept around and if it can be requested more than once, things like that.
I have a question in terms of code and NOT user experience, I have the JS:
$(document).on( "click", "input:radio,input:checkbox", function() {
getContent($(this).parent(),0);
});
The above JS gets the contents from radios and checkboxes, and it refreshes the page to show dependencies. For example if I check on yes, and the dependency is on yes, show text box, the above works!
What I want to know is, if there is a better way to do the same thing, but in a more friendly way, as this is at times, making the pages slow. Especially if I do a lot of ticks/checks in one go, I miss a few, as the parent refreshes!
If you have to hit your server to getContent() then it will automatically be slow.
However, you can save a lot if you send all the elements once instead of hitting the server each time a change is made.
Yet, if creating one super large page is not an option, then you need to keep your getContent() function, but there is one possible solution, in case you did not already implement such, which is to cache all the data that you queried earlier.
So you could have an object (a map) which has keys defining the data you're interested in. If the key is defined, then the data is already available and your return and use that data directly from the cache. Otherwise, you have to hit the server.
One thing to do, you mentioned slowness as you 'tick' things back and forth, is to not send more than one request at a time to the server (with a timeout in case the server never replies). So the process here is:
Need data 'xyz'
Is that data already cached? if yes, then skip step (3 and 4)
If a request being worked on? if yes, push the data on the request stack and return
Send a request to the server, which blocks any further request until answer for 'xyz' is received
Receive the answer and cache the data in an object (map) and release the request queue
Make use of data as required
I check the request queue, if not empty pop the next request and start processing from (2)
The request process is expected to be run on a timer because (1) it can time out and (2) it need to run in the background (not GUI preemptive)
I have a web application with a big order form: 300+ input fields for item amounts, each with a button to increase and decrease the amount. Each change in an input field is sent to the server via Ajax. The buttons are "debounced": Clicks are collected and the new amount is sent 200 ms after the last click. Now it seems that some requests fail, probably due to bad network conditions, but it could also be a server problem. This means that the displayed amount and the the amount stored on the server differ. What strategies can I use to keep client and server in sync? At the moment I see two options:
Error handling on the client - when a request fails, re-send it (with a maximum number of tries).
Calculate a checksum/hash of all amounts and send it together with the amount. If the server calculates a different amount, it returns an error code and all field contents are sent to the server.
Any other ideas or recommendations?
You can maintain a ChangeLog (a list of ids of records that are to be synced with the server) on the client side (just ids, not the record itself).
Then during a sync process make sure the server sends back an acknowledgement of having successfully processed a record.
If success, remove the record from the ChangeLog. If not, it stays and will be re-sent during the next sync process.
This is the protocol used by SyncML standard and may apply to your case.
Typically an ajax interaction would involve sending the request, providing feedback to the user that the request is in process, then once the response arrives handle it.
Waiting for the response is obviously unavoidable when the next action requires the data sent from the server but what if the interaction is an update to the some data on the server such as sorting the order of a list. Would it be bad practice to assume success? So you'd make the request and simply update the DOM based on the assumption that the sort will be successful. I'd imagine you'd then have to provide a rollback method should the request fail as well as notify the user but 99% of the time the request should go through and appear instantaneous to the user.
Is this a common pattern and are there any other factors that should be considered other than a rollback and notification method?
Any advice would be much appreciated,
Rich
You probably want to take a look at the command pattern http://en.wikipedia.org/wiki/Command_pattern. It looks like you want to modify some data and assume that the server gets modified as well. If the AJAX handler fails than you can rollback the command (and notify the user).
If the user is to receive feedback about the success or not, then it would be bad practice to assume success.
Just return a success/fail response from the server, and let the user know that their action was successful. You're not really losing anything by this extra trip back from the server, and any request could potentially fail.
I'm just wondering..is it possible to receive multiple responses from a single ajax call?
I'm thinking purely for aesthetic purposes to update the status on the client side.
I have a single ajax method that's called on form submit
$.ajax({
url: 'ajax-process.php',
data: data,
dataType: 'json',
type: 'post',
success: function (j) {
}
});
I can only get one response from the server-side. Is it possible to retrieve intermittent statuses? Such as:
Default (first): Creating account
Next: Sending email confirmation
Next: Done
Thanks for your help! :)
From a single ajax call, I don't think it is possible.
What you could do is check frequently where the process is (it's what is used for the upload bars in gmail for example). You do a first ajax request to launch the process, and then a series of ajax request to ask the server how he is doing. When the server answers "I'm done", you're good to go, and until that you can make the server respond and say the current state.
There is something called comet which you can set up to "push" requests to client, however it is probably way more than what you are wanting to invest in, time-wise.
You can open up a steady stream from the server, so that it continues to output, however I'm not sure how client-side script can handle these as individual "messages". Think about it like a server that outputs some info to the browser, does more work, outputs some more to the browser, does more work, etc. This shows up more or less in real time to the browser as printed text. It is one long response, but it is still one response. I think ajax only handles a response once it finished being sent, but maybe someone else will know more than me on the topic.
But you couldn't have the server output several individual responses without reloading itself, at least not with PHP, because once you start outputting the response, the response has begun and you can't chop that up without finishing the response, which happens when the script is done executing.
Your best bet is with the steady stream, but again, I'm not sure how ajax handles getting responses in chunks.
Quick Update
Based on the notes for this plugin:
[http://plugins.jquery.com/project/ajax-http-stream]
things don't look promising. Specifically:
Apparently the trend is to disallow access to the xmlhttprequest.responseText before the request is complete (stupid imo). Sorry there's nothing I can do to fix this
Thus, not only can you not get what you want in one request, you probably can't get it multiple requests, unless you want to break up the actual server-side process into several parts, and only have it continue to the next step when an ajax function triggers it.
Another option would be to have your script write it's status at specific points to another file on the server, call it "status.xml" or "status.txt". Have your first ajax function initialize the process, and have a second ajax function that queries this status file and outputs that to the user.
It is possible, but it has more to do with your backend script. As Anthony mentioned there is a tech called comet. Another term I've heard is called "Long polling". The idea is that you delay the time in which your php(insert language of choice) script finished processing.
In php you can do something like this:
while($response !== 'I'm done'){
sleep(1);
}else{
return $some_value;
exit();
}
This code stops your script from completely finishing. sleep(1) allows the script to stop and lets the server rest for 1 millisecond, before it loops back through. You can adjust the sleep time based on your needs. In php the amount of time the script sleeps is not counted agains your server timeout time.
You'll obviously need to make more checks for you code. You'll probably also want to allow for an abort script call. Something like sending a get request to kill the backend script. Maybe on the javascript unload event.
In the tests that I've done. I made the initial ajax call, and when the value was returned, I made another ajax call, that way your back end script wont time out.
I've only played around with this on my local server, so i'm not sure how real world this is, but it works.