I have a table using DataTables , it contains a large number of rows and so this causes the page to load very slowly as i assume the browser waits till the table is filled before displaying the page.
I would like to only load one page of the table (10 rows), and only show further data when the user browses through the table, showing a loading sign would be great too.
I have researched and heard of a DataTables function called 'deferRender' which is supposed to do what i need, but i can't get it to work with my table.
My table has 8 columns + the html is generated using PHP that builds the table from data in a text file:
<?php
$tdcount = 1; $numtd = 8; // number of cells per row
$str = "<table id=\"table1\" class=\"table1 table table-striped table-bordered\">
<thead>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
</thead>
<tbody>
";
$f = fopen("tabledata.txt", "r");
if ( $f === FALSE ) {
exit;
}
while (!feof($f)) {
$arrM = explode(",",fgets($f));
$row = current ( $arrM );
if ($tdcount == 1)
$str .= "<tr>"; $str .= "<td>$row </td>";
if ($tdcount == $numtd) {
$str .= "</tr>";
$tdcount = 1;
} else {
$tdcount++;
}
}
if ($tdcount!= 1) {
while ($tdcount <= $numtd) {
$str .= "<td> </td>"; $tdcount++;
} $str .= "</tr>";
}
$str .= "</tbody></table>";
echo $str;
I then use the following code to turn it into a datatable:
<script>
$(document).ready(function() {
$('#table1').basictable({
forceResponsive: false
});
$('#table1').DataTable( { "order": [[ 0, "desc" ]] } );
});
</script>
I have read the instructions here: https://datatables.net/examples/server_side/defer_loading.html
and know i need to add parameters to the JS:
"processing": true,
"serverSide": true,
"ajax": "scripts/server_processing.php",
"deferLoading": 57
and use a server_processing script, however the example only shows how to use it when connecting to a DB, and not when the data is loaded from a text file with php.
How can i acheive this?
This will focus purely on the DataTables aspects of a "server-side" solution. How you write the server-side logic needed to support it is out of scope for this answer. But I hope these notes will at leasst clarify what that logic needs to be, and how you can approach it.
Assumptions
Assume you have a text file containing 1,000 rows of data like this (or a million - but too many rows to send to the browser and to DataTables, all at once). The text file is a simple pipe-delimited file, with three fields:
id|name|description
1|widget_1|This is a description for widget 1
2|widget_2|This is a description for widget 2
3|widget_3|This is a description for widget 3
...
1000|widget_1000|This is a description for widget 1000
You want to send 10 items at a time to DataTables, using server-side processing.
Your data maps to a simple JSON structure, like this - an array of objects (each object is one record):
[
{
"id": 1,
"name": "widget_1",
"description": "This is a description for widget 1"
},
{
"id": 2,
"name": "widget_2",
"description": "This is a description for widget 2"
},
... // more records...
]
The DataTable Definition
Your datatable definition looks like this - it is deliberately very simple, at this stage:
<body>
<div style="margin: 20px;">
<table id="demo" class="display dataTable cell-border" style="width:100%">
</table>
</div>
<script type="text/javascript">
$(document).ready(function() {
$('#demo').DataTable({
serverSide: true,
ajax: {
url: 'http://localhost:7000/data',
type: 'POST'
},
columns: [
{ title: 'ID',
data: 'id' },
{ title: 'Name',
data: 'name' },
{ title: 'Description',
data: 'description' }
]
});
});
</script>
</body>
Initial Response
When the web page is first displayed, it will send an initial POST request to the URL (http://localhost:7000/data), and it will expect to receive a JSON response from the web server, containing the data to be displayed.
Because DataTables is using serverSide: true, DataTables will expect the JSON to have a specific structure, as described here.
Specifically, the server has to add all of the mandatory fields (draw, recordsTotal, recordsFiltered, and data) to the JSON it sends to DataTables.
In our case it would look like this - note that it is just our previously mentioned JSON structure, with a few extra metadata fields added:
{
"draw": 1,
"recordsTotal": 1000,
"recordsFiltered": 1000,
"data": [{
"id": 1,
"name": "widget_1",
"description": "This is a description for widget 1"
}, {
"id": 2,
"name": "widget_2",
"description": "This is a description for widget 2"
}, {
"id": 3,
"name": "widget_3",
"description": "This is a description for widget 3"
}, {
"id": 4,
"name": "widget_4",
"description": "This is a description for widget 4"
}, {
"id": 5,
"name": "widget_5",
"description": "This is a description for widget 5"
}, {
"id": 6,
"name": "widget_6",
"description": "This is a description for widget 6"
}, {
"id": 7,
"name": "widget_7",
"description": "This is a description for widget 7"
}, {
"id": 8,
"name": "widget_8",
"description": "This is a description for widget 8"
}, {
"id": 9,
"name": "widget_9",
"description": "This is a description for widget 9"
}, {
"id": 10,
"name": "widget_10",
"description": "This is a description for widget 10"
}]
}
It is the server's responsibility to build this JSON - the first 10 records of the server's data set. The server also tells DataTables that it has a total of 1,000 records, and that it has not filtered out any data (yet) - hence there are also a total of 1,000 records after filtering.
DataTables needs all of this information, so it knows how many pagination buttons to display, and what pagination data to show.
Note that it is entirely the server's responsibility to do all this work - that's why it's called "server-side" processing.
The client (browser) only has 10 records to render - so that happens quickly.
(I just noticed that the screenshot mentions "500 records" - that's a mistake in my server-side code - there is no filter, so I need to fix that).
Subsequent Requests
When a user clicks on a page navigation button (e.g. page "4"), that triggers a new request from DataTables to the server. DataTables builds this request automatically, using the fields described here.
The request is sent as form data.
In our example, the request looks like this:
"Form data": {
"draw": "5",
"columns[0][data]": "id",
"columns[0][name]": "",
"columns[0][searchable]": "true",
"columns[0][orderable]": "true",
"columns[0][search][value]": "",
"columns[0][search][regex]": "false",
"columns[1][data]": "name",
"columns[1][name]": "",
"columns[1][searchable]": "true",
"columns[1][orderable]": "true",
"columns[1][search][value]": "",
"columns[1][search][regex]": "false",
"columns[2][data]": "description",
"columns[2][name]": "",
"columns[2][searchable]": "true",
"columns[2][orderable]": "true",
"columns[2][search][value]": "",
"columns[2][search][regex]": "false",
"order[0][column]": "1",
"order[0][dir]": "asc",
"start": "30",
"length": "10",
"search[value]": "",
"search[regex]": "false"
}
These fields tell the server everything it needs to know, so it can prepare the correct response.
In our case the most important fields are these:
"start": "30",
"length": "10"
Start at row 30, and provide 10 records.
It is, again, the server's responsibility to prepare a JSON response which accurately reflects the requested data.
In our case this means the server needs to have logic to read through the text file to the correct starting point (data row 31 - remember the offset starts at zero), and 10 rows in total (rows 31 through 40).
Other fields in the above request from DataTables describe how the data is to be sorted, and filtered. In our case there is no filter "search[value]": "", - and the data is to be sorted by the first column in ascending order.
Final Notes
I have deliberately not described the following:
1) How your server-side code handles the creation of the JSON responses it sends back to DataTables;
2) How your server-side code parses the form requests it receives from DataTables.
That all depends entirely on what your server-side technology is. DataTables doesn't care. It's just passing JSON messages - it is decoupled from the server-side implementation - as it should be.
Regarding the "defer render" option described here, that is an enhancement you may choose to add if you feel you need it. But I would recommend getting a more basic server-side implementation working first.
Not particulary relavant to any language, but I used Node.js as an example to make a minimum working project for what's explained above:
The data: (put this at the root of where you have the working directory of node.js service)
[ { "id": 1,
"name": "widget_1",
"description": "This is a description for widget 1"
},{ "id": 2,
"name": "widget_2",
"description": "This is a description for widget 2"
},{ "id": 3,
"name": "widget_3",
"description": "This is a description for widget 3"
},{ "id": 4,
"name": "widget_4",
"description": "This is a description for widget 4"
},{ "id": 5,
"name": "widget_5",
"description": "This is a description for widget 5"
},{ "id": 6,
"name": "widget_6",
"description": "This is a description for widget 6"
},{ "id": 7,
"name": "widget_7",
"description": "This is a description for widget 7"
},{ "id": 8,
"name": "widget_8",
"description": "This is a description for widget 8"
},{ "id": 9,
"name": "widget_9",
"description": "This is a description for widget 9"
},{ "id": 10,
"name": "widget_10",
"description": "This is a description for widget 10"
},{ "id": 11,
"name": "widget_11",
"description": "This is a description for widget 11"
},{ "id": 12,
"name": "widget_12",
"description": "This is a description for widget 12"
},{ "id": 13,
"name": "widget_13",
"description": "This is a description for widget 13"
},{ "id": 14,
"name": "widget_14",
"description": "This is a description for widget 14"
},{ "id": 15,
"name": "widget_15",
"description": "This is a description for widget 15"
},{ "id": 16,
"name": "widget_16",
"description": "This is a description for widget 16"
},{ "id": 17,
"name": "widget_17",
"description": "This is a description for widget 17"
},{ "id": 18,
"name": "widget_18",
"description": "This is a description for widget 18"
},{ "id": 19,
"name": "widget_19",
"description": "This is a description for widget 19"
},{
"id": 20,
"name": "widget_20",
"description": "This is a description for widget 20"
}]
The HTML: (put this under your working directory/public)
<!DOCTYPE html>
<html>
<body>
</head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/1.13.1/css/jquery.dataTables.css">
<script type="text/javascript" charset="utf8" src="https://cdn.datatables.net/1.13.1/js/jquery.dataTables.js"></script>
</head>
<div style="margin: 20px;">
<div id = 'pageinfo' style='display:none'></div>
<table id="demo" class="display dataTable cell-border" style="width:100%"></table>
</div>
<script type="text/javascript">
$(document).ready(function() {
var table = $('#demo').DataTable({
serverSide: true,
ajax: {
url: 'http://localhost:3000/data',
data: $('#pageinfo').text(), //post pagination parameters to server
type: 'POST'
},
columns: [
{ title: 'ID',
data: 'id' },
{ title: 'Name',
data: 'name' },
{ title: 'Description',
data: 'description' }
],
lengthMenu: [5, 10, 20],
pageLength: 5
});
function getinfo(){
var info = table.page.info();
return info;
}
$('.dataTables_wrapper').on('click',function(e){
var info = table.page.info();
$('#pageinfo').text(JSON.stringify(info)) //get the pagination parameters
})
});
</script>
</body>
<html>
And the server:
const fs = require('fs');
const data = JSON.parse(fs.readFileSync("data.json", 'utf8'));
var express = require('express');
var app = express();
var bodyParser = require('body-parser')
app.use( bodyParser.json() ); // to support JSON-encoded bodies
app.use(bodyParser.urlencoded({ // to support URL-encoded bodies
extended: true
}));
app.use('/static', express.static(__dirname + '/public')); //directory for static html docs
app.post('/data', function(req, res){
var resp = {"recordsTotal": 20,"recordsFiltered": 20,"start":0,"length":5}
if(req.body.start){
resp.draw = JSON.parse(req.body.draw);
resp.data = data.slice(JSON.parse(req.body.start),
JSON.parse(req.body.start)+JSON.parse(req.body.length));
}else{
resp.draw = 1;
resp.data = data.slice(resp.start,resp.start+resp.length);
}
res.end(JSON.stringify(resp))
})
app.listen(3000, function () {
console.log('listening on port 3000');
});
Then you'd be able to just start the server, and browse the result at http://localhost:3000/static/dtable.html
Read through these codes, or probably dig in to a few intermediate outputs, and you'll have a better idea of how DataTable works under a minimum setup for server-side processing.
I need help because I'm losing my mind haha ...
I have the main array products with this (it's just a sample) :
[
{
"from": "country",
"maker": "name of maker",
"id": "1969",
"image": "image.jpg",
"label": "355",
"name": "name of product",
"price": "12.90",
"subscriber_price": "8.90",
"url_path": "url",
"occasion": null,
"colour": "31",
"origin": "397",
},
{
"from": "country",
"maker": "name of maker",
"id": "2043",
"image": "image.jpg",
"label": "362",
"name": "name of product",
"price": "24.90",
"subscriber_price": "24.90",
"url_path": "url",
"occasion": "51,376,155,39",
"colour": "31",
"origin": "395"
}
]
I'm working this the Picker Component. So, what I'm doing is :
I have a Picker to select products with their "colour".. then I have another one to filter the selected products (only with colour:31 for example) with their "origin" and finally I want to filter them through their "label" ...
The fact is I have 3 Pickers, 3 functions to select them and it's working but the problem is I'm erasing with a setState my render of "displayProducts". So, when I have selected the 3 options, I can't go back..
For example, I choose "colour:31" with "origin:397" and "label:355" .. I can't go back and tell : finally I want "origin:395" because it doesn't exist anymore, etc... and one "colour" can have different "label, origin, ..."
I'm doing something like this but it's only available for ONE option and not multiple options and without keeping a solution to find again my filtered products :
onChangeGetOrigin(originValue) {
this.setState(() => ({
activeOrigin: originValue,
displayProducts: this.state.displayProducts.filter(product => product.origin == originValue)
}));
}
Do anyone can understand what I'm saying ? :-D
You can maintain two arrays. One contains the complete list of products and the other one is a derived array after applying the filters. You can use the derived list for display and original array for selection.
I have a Kendo grid where I'm trying to add a delete feature. My datasource looks like:
var datasource = new kendo.data.DataSource({
transport: {
read: {
url: Router.action("Admin", "GetScansForMailItem", { mailItemIdnt: detailinit.data.MailItemIdnt }),
dataType: "json"
},
destroy: {
url: Router.action("Admin", "DeleteScan"),
type: "post"
}
},
model: {
id: "ScanIdnt",
fields: {
ScanIdnt: {editable: false, nullable: false}
}
},
pageSize: 5
});
I added the model part because this answer, however it made no difference.
The actual grid looks like:
.kendoGrid({
dataSource: datasource
scrollable: false,
sortable: true,
pageable: true,
editable: "inline",
columns: [{
field: "ScanIdnt",
title: "Scan ID"
}, {
field: "CreatedDate",
title: "Created",
template: "#= kendo.parseDate(CreatedDate, 'yyyy/MM/dd') #"
}, {
field: "ScanDocumentRelativePath",
title: "File Path",
template: "<a href='/CAMP/Admin/Download?scanIdnt=#= ScanIdnt #'>#= ScanDocumentRelativePath.substring(1) #</a>"
}, {
field: "ScanUserIdnt",
title: "Scanned By"
},{
command: "destroy",
title: ""
}]
});
Strangely, clicking the delete button removes the from the gird on the UI, but there is absolutely no Ajax call is made the the destroy URL. I can't seem to figure out why. Any ideas?
EDIT I'd like to point out that this grid is in fact a nested grid inside of another grid (like here) I discovered that the parent grid handles actually makes a call, but to the wrong function. For some reason, it clicking delete on a to level item calls the read function of the nested grid, however, the nested grids do nothing
Figured it out (sorta). While I think there were many issues with my code and the grid, It seems that when it came down to it, Kendo didn't like how I had my data.
In the Kendo docs related to hierarchical grids, the data for the child grid is stored in a field of the data for the parent. For example, given the following JSON:
"ParentItems": [
{
"Id": 12345 ,
"Name": "Test1",
"ChildItems": [
{"Id": 1, "Name": "Test"},
{"Id": 2, "Name": "Test"}
]
},
{
"Id": 12346 ,
"Name": "Test2",
"ChildItems": [
{"Id": 1, "Name": "Test"},
{"Id": 2, "Name": "Test"}
]
}
]
In the parent grid, each ParentItem would display it's respective ChildItems in the child grid.
On the other hand, I was pulling both data sets separately. Basically, I pulled the ParentItems like:
"ParentItems": [
{
"Id": 12345,
"Name" : "Test1"
},
{
"Id": 12346,
"Name" : "Test2"
}
]
And then made a second request to pull the child items, based on the parent's id.
"ChildItems": [
{"Id": 1, "Name": "Test", "ParentId": "12345"},
{"Id": 2, "Name": "Test", "ParentId": "12345"}
]
I was able to modify the server side code to serve the data like in the very first example and managed to get things working. The specific document that helped me out can be found here
I have a dhtmlx gantt chart in my page, normally it would have work perfectly but now when I have a JSON file with nested array all my output would be unrecognized instead. I'm trying to populate the official name.
Can Anyone help me with this? Thank you very much.
JSON
{
"data": [
{
"assign_to": {
"id": 3,
"employee_id": "28141",
"official_name": "Hal Jordan",
},
"task": {
"id": 1,
"name": "Modeling",
"description": "3d modeling work"
},
"start_date": "2017-06-15",
"end_date": "2017-06-19"
},
{
"assign_to": {
"id": 3,
"employee_id": "28144",
"official_name": "Kyle Rayner",
},
"task": {
"id": 8,
"name": "Composting",
"description": null
},
"start_date": "2017-06-01",
"end_date": "2017-06-08"
}
]
}
Javascript
gantt.config.columns = [
{name: "assign_to.official_name", label: "Assign To", align: "left", width: 70},
];
function initializeGantt() {
gantt.init("my_scheduler");
gantt.load("/dashboard/ganttchart_list/5/?format=json");
}
initializeGantt();
HTML
<div id="my_scheduler" style='width:1405px; height:245px;'></div>
dhtmlx gantt doesn't support nested structure, so you'll need to preprocess your items before loading them to gantt.
Since you know which format you have and which format is expected by gantt (https://docs.dhtmlx.com/gantt/desktop__supported_data_formats.html#json ) it shouldn't be a problem.
The most important thing
Make sure data items have 'id' property
when you define column name like this:
{name: "assign_to.official_name", label: "Assign To", align: "left", width: 70},
column name can't be mapped to a property of a nested object, so you'll either have to flatten structure during loading, very roughly, it can be done in following way:
http://snippet.dhtmlx.com/129d2e043
Or define a column template which would take value from nested object:
http://snippet.dhtmlx.com/9e8e65e85
please check the console output for a result dataset
I have a ExtJS grid with a store mapped to it.
The JSON response for the store contains 20 fields, but I have mapped only 10 fields in the store. Is it possible for me to obtain the other 10 fields without me having to map them in the store or is it necessary for me to map all the fields to the store. Doing this on row select event of the grid. As for the code, I am able to create the grid, map the store, fire the event and even get the record for the selected row with the mapped 10 field.
Any suggestions ??
As requested by Shekhar :
Store definition
Ext.define('gdoc.store.PrintPaperStore', {
extend: 'Ext.data.Store',
constructor: function(cfg) {
var me = this;
cfg = cfg || {};
me.callParent([Ext.apply({
autoLoad: true,
storeId: 'PrintPaperStore',
proxy: {
type: 'ajax',
url: 'urlToRetrieveTheJSON',
reader: {
type: 'json',
root: 'root'
}
},
fields: [{
mapping: 'value',
name: 'value'
},
{
mapping: 'display',
name: 'display'
}
]
}, cfg)]);
}
});
The JSON Response :
{
"root": [{
"value": "COATED115",
"display": "Coated recycled paper (115gms)",
"description": "Good quality",
"extra": "EXTRA INFO"
}, {
"value": "COATED135",
"display": "Coated recycled paper (135gms)",
"description": "Good quality"
}, {
"value": "OFFSET",
"display": "Offset recycled paper (80gms)",
"description": "Good quality",
"extra": "EXTRA INFO"
}, {
"value": "OTHER",
"display": "Other paper (client to order)",
"description": "Good quality",
"extra": "EXTRA INFO"
}]
}
As you can see I have mapped only value and display from the JSON response in the store.
Is it possible for me to retrieve values of description and extra without me having to map them in the store.
You can access the .raw property on the model, which will contain the raw JSON from the reader.
No. The model, or store, must set all the fields you want to have available from the JSON data.
I'm confused as to why you don't want to map the extra fields? It's it purely time saving and hoping there was a quicker way to get access to the data? Or are you expecting the fields names to be dynamic?
If it's the dynamic reason, take a look at this post for a solution. Dynamic model with ExtJS 4