I am writing an application that allows a user to dynamically select which columns they want to pull from our database (all public internal data) as well as customize their logic (WHERE Clause).
The result sets from these queries can be anything from a 10 records to our whole table of over 35,000 records.
On load of the result page, I trigger an AJAX function which runs the query that user has chosen and this triggers to stored procedure to run the query. This data is then passed to the AJAX success function and rendered to the DOM so it can be manipulated, exported etc.
--
The Issue:
When people choose to download the entire table (not often but certain departments have a need), we are having timeout issues which can be expected.
I am trying to find out where the bottle neck is to try and improve the experience.
When I run the query directly on the database for the 35,000 record result set, it takes about 12 seconds over VPN so I don't think the issue is with the database it self.
You can see in the image above where the timeout happened on the POST call.
I am trying to understand how the process flows for fetching / writing data to the DOM. Is the connection that is fetching the data from the database timing out or do we successfully have that data but we are timing out trying to write the 35,000 table rows?
Example Code:
var output = '';
$.ajax({
url: "api/fetchDashboardRender",
type: 'POST',
cache: false,
data: {
dashboardID: dashboardID
},
error: function(err) {
alert(err.statusText);
},
success: function(data) {
// Example .. Loop over the data in the result set
$(data).each(function(){
// Append each table row to a variable
output += '';
});
// Once done, append the variable to the DOM
$('#results').append(output);
}
});
Questions:
While I understand there is no need to write that much data to the DOM, there are reasons they want it done and I am following instruction.
Does the success function of an AJAX call include the DOM
manipulation or is it it successful once the data has been retrieved
from the source?
Is there a better way to approach this type of situation outside of making multiple calls in a paginated approach for batches of data?
Can you help me identify the main bottle neck here as to where things are being hung up?
Thanks for any pointers.
Related
I'm helping with an open source project. It's a small Go webserver running on a device containing a Raspberry Pi. I want to be able to have a user click a button on an html screen, which calls a routine in Go, which returns 2 values, a boolean and a string.
What we are wanting to do is see which network interfaces are up on the raspberry pi e.g. is the lan connection up?
To do this I really need to ping a site from each interface. This takes a few seconds for each of 3 interfaces: Lan, WiFi, and 3G.
I can do this when the page is requested and fill in an html template as the page loads, but it means waiting maybe 10 to 15 secs for the page to load, so it seems like something is broken.
So I want to be able to list each of the 3 interfaces on the page and have the user click 'test' which then calls a routine in the underlying Go webserver.
I then need to be able to display the results from the call in a couple of text areas for each interface.
What I have tried:
I have tried registering a Go function (in this case IsLANConnectionUp) using funcmap from the net/html package and calling it from the html template from a JavaScript function, like this:
<button onclick = "getLANStatus()" class="btn btn-primary">Test</button>
<script>
function getLANStatus() {
var status = document.getElementById('status');
{{ if IsLANConnectionUp }}
status.innerHTML = "Lan is up!"
{{ else }}
status.innerHTML = "Lan is down!"
{{ end }}
}
</script>
But having the template code inside the javascript code doesn't seem to work. Also, I'd like the text output from the ping command (which my Go function getLANStatus and I don't know how to extract that data from the function call. The documentation says only one value can be returned.
Searching on StackOverflow I see this: calling Golang functions from within javascript code
$.ajax({
url: "http://www.example.com/signup",
data: {username: "whatever"} //If the request needs any data
}).done(function (data) {
// Do whatever with returned data
});
But it says things like "// Do whatever with the returned data" I'm new to web programming and so don't know how to use that code. If this is the way to go, could someone please expand on this a little?
Any help would be much appreciated.
So couple different concepts here.
Render: On the initial request to your html that generates the Test button. Your go server will render that html 1 time and return it to your browser. It does not re-request dynamically unless you wire some stuff up to make the web page change.
Client: So when someone clicks your button, the function getLANStatus will be ran. You will want that function to do a few things
Through ajax, communicate with your go server through an api that will return the status of your connections as a json object. Something like
{
"3g": "up",
"lan": "down",
"wifi": "up"
}
Second, in the done part of your ajax, you will manipulate something in the DOM in order to convey that the status of the interfaces is what it is. You could do that by finding the element, then changing the text to what is returned by the object.
As a simple first step, you can alert the payload in the function that would look like this
$.ajax({
url: "http://YOUR_GO_SERVER_IP_OR_DNS:PORT/interfaces_status.json"
}).done(function (data) {
alert(data);
console.log(data);
debugger;
});
Then if you request that with the console open in chrome, you will be able to directly play with the returned data so that you know what all it reponds to.
I have a javascript array which stores seat numbers (in a cinema), which are selected by the user via clicking and added to the array each time using a function. I want the page to show the total cost of these seats, which means accessing an sql table inside some php.
So I have e.g. seatNumbers = ["a1", "d6", "e3"] and three sql query like 'select cost from seat where seat_number='a1';'. The function that adds to the array on clicking a seat and prints the seats is something like:
var seatArray = [];
function addSeat(seat) {
seatArray.push(seat);
document.getElementById("textarea").innerHTML="Seats : ";
for (x in arraytest) {
document.getElementById("textarea").innerHTML+=arraytest[x];
document.getElementById("textarea").innerHTML+=" ";
}
}
And I want to show the total cost in the HTML.
I'm wondering if what I'm trying to achieve is possible? What would be the general method and would I need to load a new page instead? And if it's not possible, what would be a better way to go about this?
Your question is very broad, and to answer it, quite some stuff needs to be known and used.
What you want is very possible though. In fact, there are technologies that in their core focus on providing solutions to problems like yours! What you need is some solid info (and possibly experience with) AJAX and maybe even REST. By using AJAX your page wont have to reload, and in your situation AJAX is probably the best choice anyway.
To point you in the right direction: AJAX javascript W3Schools Tutorial and PHP AJAX W3Schools tutorial
Then, use jQuery to make it all a LOT easier: jQuery (i'd go for 1.x)
You'll have to create an API that accepts an HTTP (preferably GET) request and returns the cost for the seat that you refer to in your URL like (more REST like, should return a whole seat object with price included): /seat/200, or (not REST like)/seat/cost/200).
Your choice if you want to follow (if you haven't read up on it, possibly confusing) REST rules. In your situation i'd just begin with some good old AJAX, it just works and is even better suited for stuff like this.
You should use AJAX. Try using jQuery library and ajax function.
Covert Your array with seats on JSON string and send it by AJAX to page which check the whole price. Then, You can update Your HTML code with total cost.
var json_data = '{...}';
$.ajax({
url: "total_cost.php",
dataType: "json",
type: 'POST',
data: { json: json_data },
success: function(response) {
// .. on success
var json_response = jQuery.parseJSON(response);
var cost = json_response.total_cost;
}
});
I am currently working on a web based time tracking software. I'm developing in grails, but this question is solely related to javascript and asynchronous requests.
The time tracking tool shall enable users to choose a day for the current month, create one or multiple activities for each day and save the entire day. Each activity must be assigned to a project and a contract.
Upon choosing "save", the partial day is saved to the database, the hours are calculated and a table is updated at the bottom of the page, showing an overview of the user's worked hours per month.
Now to my issue: There may be a lot of AJAX request. Patient users might only click the "create activity" button just once and wait until it is created. Others, however, might just keep clicking until something happens.
The main issue here is updating the view, although i also recognized some failed calls because of concurrent database transaction (especially when choosing "save" and "delete" sequentially). Any feedback on that issue -- requests not "waiting" for the same row to be ready again -- will be apreciated as well, yet this is not my question.
I have an updateTemplate(data, day) function, which is invoked onSuccess of respective ajax calls in either of my functions saveRecord(), deleteRecord(), pasteRecords(), makeEditable() (undo save). Here is the example AJAX call in jquery:
$.ajax({
type: "POST",
url: "${g.createLink(controller:"controller", action:"action")}",
data: requestJson,
contentType:"application/json; charset=utf-8",
async: true,
success: function(data, textstatus) {updateTemplate(data["template"], tag); updateTable(data["table"]);},
});
In the controller action, a JSON object is rendered as a response, containing the keys template and table. Each key has a template rendered as a String assigned to it, using g.render.
Now, what happens when I click on create repeatedly in very short intervalls, due to the asynchronous calls, some create (or other) actions are executed concurrently. The issue is that updateTemplate just renders data from the repsonse; the data to render is collected in the create controller action. But the "last" request action only finds the objects created by itself. I think this is because create actions are run concurrently
I figure there is something I'm either overcomplicating or doing something essentially wrong working with a page that refreshs dynamically. The only thing I found that helps are synchronous calls, which works, but the user experience was awful. What options do I have to make this work? Is this really it or am I just looking for the wrong approach? How can I make this all more robust, so that impatient users are not able to break my code?
*********EDIT:********
I know that I could block buttons or keyboard shortcuts, use synchronous calls or similar things to avoid those issues. However, I want to know if it is possible to solve it with multiple AJAX requests being submitted. So the user should be able to keep adding new activities, although they won't appear immediately. There is a spinner for feedback anyway. I just want to somehow make sure that before the "last" AJAX request gets fired, the database is up to date so that the controller action will respond with the up-to-date gsp template with the right objects.
With help of this Stackoverflow answer, I found a way to ensure that the ajax call -- in the javascript function executed lastly -- always responds with an up-to-date model. Basically, I put the javascript functions containing AJAX calls in a waiting queue if a "critical" AJAX request has been initiated before but not completed yet.
For that I define the function doCallAjaxBusyAwareFunction(callable) that checks if the global variable Global.busy is 'true' prior to executing the callable function. If it's true, the function will be executed again until Global.busy is false, to finally execute the function -- collecting the data from the DOM -- and fire the AJAX request.
Definition of the global Variable:
var Global = {
ajaxIsBusy = false//,
//additional Global scope variables
};
Definition of the function doCallAjaxBusyAwareFunction:
function doCallAjaxBusyAwareFunction(callable) {
if(Global.busy == true){
console.log("Global.busy = " + Global.busy + ". Timout set! Try again in 100ms!!");
setTimeout(function(){doCallAjaxBusyAwareFunction(callable);}, 100);
}
else{
console.log("Global.busy = " + Global.busy + ". Call function!!");
callable();
}
}
To flag a function containing ajax as critical, I let it set Global.busy = true at the very start and Global.busy = false on AJAX complete. Example call:
function xyz (){
Global.busy = true;
//collect ajax request parameters from DOM
$.ajax({
//desired ajax settings
complete: function(data, status){ Global.busy = false; }
}
Since Global.busy is set to true at the very beginning, the DOM cannot be manipulated -- e.g. by deletes while the function xyz collects DOM data. But when the function was executed, there is still Global.busy === true until the ajax call completes.
Fire an ajax call from a "busy-aware" function:
doCallAjaxBusyAwareFunction(function(){
//collect DOM data
$.ajax({/*AJAX settings*/});
});
....or fire an ajax call from a "busy-aware" function that is also marked critical itself (basically what I mainly use it for):
doCallAjaxBusyAwareFunction(function(){
Global.busy = true;
//collect DOM data
$.ajax({
//AJAX SETTINGS
complete: function(data, status){ Global.busy = false; }
});
});
Feedback is welcome and other options too, especially if this approach is bad practice. I really hope somebody finds this post and evaluates it, since I don't know if it should be done like that at all. I will leave this question unanswered for now.
This post by #BenjaminRH (How to change/edit the text of a paragraph/div using jQuery?) provides exactly the sort of functionality I'm trying to build on.
By clicking on a button, a new paragraph is created on the page, which can be edited and saved to the page.
I want to save it to a database. When I look at the page source after editing, I don't see the changes there, which doesn't surprise me... but I don't know where to "capture" the changed text so that I can validate and post to mySQL.
JQuery is a javascript library - which runs client side. If you wanted to save that data into the database - you would have to send it to the server (php/asp/mvc etc) using ajax and then insert the data into the database.
See the jquery Ajax function for details on how to accomplish sending data asynchronously.
Create the data in javascript that you want to show and save in database.
Wrap the data in JSON and use ajax to POST the data to the server side code
Server-side retrieve the posted data and parse it into something usable
Server-side write a script to insert the data into the database.
Handle any errors that may occur.
Pseudo-code:
// step 1
var someData = 'this is my data';
$("#myDiv").html(someData);
// step 2
$.ajax({
type: "POST",
dataType: 'json', // <-- if the data is complex and needs to be object-oriented
url: "some.php", // <-- that is the file that will handle the post server-side.
data: JSON.stringify({ someData }) // <-- just pass someData if your data is not complex
})
.always(function(jqXHR, textStatus) {
if (textStatus != "success") {
// step 5: handle error.
alert("Error: " + jqXHR.statusText); //error is always called .statusText
} else {
alert("Success: " + jqXHR.response); //might not always be named .response
}});
OK, I've managed to solve it for myself, without using ajax. I took the example from (How to change/edit the text of a paragraph/div using jQuery?) and modified it by placing the elements in an (html) form.
The second modification was to use <textarea> elements, not <p> elements, as <p> elements cannot be posted.
As #caspian pointed out (in the comments) those two steps do populate the $_POST array and away we go.
I am working on a project for school and it's about jQuery, PHP, symfony and all that web stuff :). It is actually my first time using jQuery and even though I have already learned something, there are still many thing I dont know how to solve.
In this project, one of the things I am supposed to make is basically filtering some results. I have a database and when the site is loaded, all items from the database are displayed in a table. Then above the table, there is a text field. When I type in it, the items in the table are filtered as I type. But sometimes, when I write for example "HP", the jQuery first finishes the request for "HP" and displays the result and then it finished the request for "H" and so it overwrites the result of "HP" and this shouldnt happen. If I make the requests synchronous, it prevents me from writing in the text field when a request is being processed. How can I make it so that the requests are completed in the order they were called?
The jQuery code of this part of the project looks like this:
var oldTerm;
function filter() {
var term = $('#term').val();
//alert(term);
if(term != oldTerm) {
$.ajax({
url: url.replace('--', term).replace(/\/$/, '')
//musi bejt synchronni
}).done(function(data) {
$('#items_table tbody').html(data);
alert(term);
// udalost se musi registrovat po nacteni radku!
$('.edit_button').click(createForm);
});
oldTerm = term;
}
}
$(document).ready(function() {
oldTerm = null;
$('#term').keyup(filter);
});
I think your best shoot is to make a queue for the ajax call maybe use a lock too. You can also use this : Queue AJAX calls