I scraped this page https://www.capfriendly.com/teams/bruins, specifically looking for the tables under the tab Cap Hit (Fowards, Defense, GoalTenders).
I used Python and BeautifulSoup4 and CSV as the output format.
import requests, bs4
r = requests.get('https://www.capfriendly.com/teams/bruins')
soup = bs4.BeautifulSoup(r.text, 'lxml')
table = soup.find(id="team")
with open("csvfile.csv", "w", newline='') as team_data:
for tr in table('tr', class_=['odd', 'even']): # get all tr whose class is odd or even
row = [td.text for td in tr('td')] # extract td's text
writer = csv.writer(team_data)
writer.writerow(row)
This is the output that I get:
['Krejci, David "A"', 'NMC', 'C', 'NHL', '30', '$7,250,000$7,250,000NMC', '$7,250,000$7,500,000NMC', '$7,250,000$7,500,000NMC', '$7,250,000$7,000,000Modified NTC', '$7,250,000$7,000,000Modified NTC', 'UFA', '']
['Bergeron, Patrice "A"', 'NMC', 'C', 'NHL', '31', '$6,875,000$8,750,000NMC', '$6,875,000$8,750,000NMC', '$6,875,000$6,875,000$6,000,000NMC', '$6,875,000$4,375,000$3,500,000NMC', '$6,875,000$4,375,000$1,000,000Modified NTC, NMC', '$6,875,000$4,375,000$1,000,000Modified NTC, NMC', 'UFA']
['Backes, David', 'NMC', 'C, RW', 'NHL', '32', '$6,000,000$8,000,000$3,000,000NMC', '$6,000,000$8,000,000$3,000,000NMC', '$6,000,000$6,000,000$3,000,000NMC', '$6,000,000$4,000,000$3,000,000Modified NTC', '$6,000,000$4,000,000$1,000,000Modified NTC', 'UFA', '']
['Marchand, Brad', 'M-NTC', 'LW', 'NHL', '28', '$4,500,000$5,000,000Modified NTC', '$6,125,000$8,000,000$4,000,000NMC', '$6,125,000$8,000,000$3,000,000NMC', '$6,125,000$7,500,000$4,000,000NMC', '$6,125,000$5,000,000$1,000,000NMC', '$6,125,000$6,500,000$4,000,000NMC', '$6,125,000$5,000,000$3,000,000Modified NTC']
As you can see data from different tabs is concatenated together:
'$7,250,000$7,000,000Modified NTC'
Somebody advised me to use javascript to scrape the table and that it should solve my problem?
Based on the source code, this is some text in specific rows that is conditionally visible depending on what tab you're on (as your title states). The class .hide is added to the child element in the td when it is intended to be hidden on that specific tab.
When you're parsing the td elements to retreive the text, you could filter out those elements which are suppose to be hidden. In doing so, you can retrieve the text that would be visible as if you were viewing the page in a web browser.
In the snippet below, I added a parse_td function which filters out the children span elements with a class of hide. From there, the corresponding text is returned.
import requests, bs4, csv
r = requests.get('https://www.capfriendly.com/teams/bruins')
soup = bs4.BeautifulSoup(r.text, 'lxml')
table = soup.find(id="team")
with open("csvfile.csv", "w", newline='') as team_data:
def parse_td(td):
filtered_data = [tag.text for tag in td.find_all('span', recursive=False)
if 'hide' not in tag.attrs['class']]
return filtered_data[0] if filtered_data else td.text;
for tr in table('tr', class_=['odd', 'even']):
row = [parse_td(td) for td in tr('td')]
writer = csv.writer(team_data)
writer.writerow(row)
I am using datatables 1.10.11.
As per the documentation, i can set the rowId using following syntax:
$('#myTable').DataTable( {
rowId: 'staffId'
} );
I am not creating datatable using Ajax.
I have two text boxes and "Add" button on left hand side and one datatable on right hand side. When i click on "Add", a new row is added to datatable. I hope that makes sense.
I need unique id attribute for each row for my use. I have tried doing so but no success at all.
According to documentation, this feature is available since DataTables 1.10.8
Did anyone solved this problem? Any help will be appreciated.
Thanks.
For add ID to tr element you con use rowId and especify the column that contain your id.
$('#myTable').DataTable(
{
columns: [
{data:"staffId"},//Remember specified the column
{data:"position"},
{data:"office"},
{data:"age"},
{data:"date"},
{data:"salary"}
],
rowId: 'staffId' //Reference to column data
});
For your button add, you should create the object and add to Table using row.add and draw for show in your table, for example:
$("#btnAdd").on("click",function(){
var rowID = $("#rowId").val();//Get value of TextBox
var new_object = {
staffId: rowID, //New rowId
position: 'Developer',
office: "Peru",
age: 25,
date: "2016/03/25",
salary: "$ 58.200"
};
myTable.row.add(
new_object
).draw();
});
Result: https://jsfiddle.net/cmedina/7kfmyw6x/65/
I have a page where I send an ajax request to a server. There is a table on the page which displays some data. The server returns a json object which is a list of objects and it doesn't contain any layout for the page.
I want to update only table rows by returned json. How can I do this without using third-party libraries and only using jquery? I just want a rough idea and example.
Here is the demo fiddle. (simple version)
NEW: See the updated fiddle (advanced version).
Let's say you have this JSON data retrieved:
var jsonData = [
{ field1: 'value a1', field2: 'value a2', field3: 'value a3', field4: 'value a4' },
{ field1: 'value b1', field2: 'value b2', field3: 'value b3', field4: 'value b4' },
{ field1: 'value c1', field2: 'value c2', field3: 'value c3', field4: 'value c4' }
];
And you have this table:
<table id="data-table">
<tr><td>There are no items...</td></tr>
</table>
Now, you need a method that can parse the values in a customisable order and presence. For example, if you can pass a field array to the parser function; you can set the order of the fields and leave out a field or two if you want to:
loadTable('data-table', ['field1', 'field2', 'field3'], jsonData);
Notice that field4 is not parsed.
The loadTable function loops through the items of the returned array and create a <tr> with each field value inside a <td>. Here is the simple function:
function loadTable(tableId, fields, data) {
//$('#' + tableId).empty(); //not really necessary
var rows = '';
$.each(data, function(index, item) {
var row = '<tr>';
$.each(fields, function(index, field) {
row += '<td>' + item[field+''] + '</td>';
});
rows += row + '<tr>';
});
$('#' + tableId).html(rows);
}
UPDATE:
Added support for:
Table headers
Default text (for empty list)
Appending lists
Clearing the table
etc...
You can now simply include an empty table and the dynamicTable will take care of the rest:
<table id="data-table"></table>
Call the dynamicTable.config() method to configure the table:
var dt = dynamicTable.config('data-table', //id of the table
['field2', 'field1', 'field3'], //field names
['header 1', 'header 2', 'header 3'], //set to null for field names to be used as header names instead of custom headers
'There are no items to list...'); //default text for no items
Then call dt.load(data); to load new data (removes the previous list if there is one),
OR call dt.load(data, true); to append the new data at the end of the previous list.
Calling dt.clear(); method will clear the whole list.
Updated fiddle here.
You can iterate through your JSON objects.
$.each(JSON_Object, function(key, value) {
// Write your code here
});
Then you can simply append your table with data.
$('#yourTableName tr:last').after('<tr><td>John</td><td>$500</td></tr>');
Since you specifically mentioned that you don't need 3rd party libraries, you can do something like above. However if you need dataset with all the features, you can consider some plugin like http://datatables.net/.
If the table in question is associated to an oData service (e.g. as is standard in Fiori) with an item reference in the table XML such as items="{/ReportSet}" then a request to update the table with a filter will automatically update the table items i.e. in JS:
eTable.bindItems(sPathR,template,null,this.instanceFilter);
sPathR is the entity set e.g. /ReportSet in the above
template should be the item template in the table
this.instanceFilter is the defined filter set prior to the call
we are just started with Sigma Grid ,and it is awesome in its functionality when we compared to other Grids.
But i encountered some problem with Sigma Grid ,or may be with javascript.
I dont know whether the problem is with Grid or with my code.
I have a table with 3 fields namely MailID,MailName,MailData.
MailID is int ,MailName and MailData contains HTML content and it save as string in database.
When i load the Grid,i have some problems.
Problem 1 :
As i said above the Maildata contain html content,the following image is just a example with <*b> ,u can see that the HTML is automatically rendering on the grid itself ,i need the exact string.
please check the following image.
Problem 2 :
as u can see i have links on the grid,for edit,send,delete but on one filed its damaged.[check the image below ]
the code i used to render links is following .
{id: 'mailid' , header: "Action", width :120 , resizable : false, sortable : false , printable : false ,
renderer : function(value ,record,columnObj,grid,colNo,rowNo){
var no= record[columnObj.fieldIndex];
var cod = (record['maildata']);
return 'Edit | Delete';
Problem 3 :
The third value of MailData is 5 and it is integer ,when i alert the value its shows it correctly.
check the following image.
But when i alert the second value of maildata it giving error ,the second value of MailData is "hai newuser" ,it showing the following error on firebug.
missing ) after argument list
alert(hai newuser)
check the image below.
But when i alert 9th value of MailData it run correctly ,the content is <b>poy</b> ,this one is also save as string,but the grid automatically BOLD [which i dnt like].Check the image below.
also there are some others the 7the value contain ;".: etc ,also /b ,when i alert the data it showing the following error,
unexpected end of XML source
alert(<b>jjfdslkdjflsdnfsldfnf
dsOptions and ColOptions are following .
var dsOption= {
fields :[
{name : 'mailid' },
{name : 'mailname',type:"text" },
{name : 'maildata',type:"text" }
],
recordType : 'object',
}
function my_renderer(value ,record,columnObj,grid,colNo,rowNo)
{
var no= record[columnObj.fieldIndex];
return "<img src=\"./images/flag_" + no.toLowerCase() + ".gif\">";
}
function showalert(no)
{
$(document).ready(function()
{
$.post("http://localhost/power/index.php/power/give",{ name: no}, function(data)
{
//alert("Data Loaded: " + data);
$("#editor").show("fast");
$( '#txtar' ).ckeditor();
$('#txtar' ).val( data.maildata );
//$("#editor").html(data);
},"json"
);
});
}
var colsOption = [
{id: 'mailid' , header: "Mail ID" , width :60},
{id: 'mailname' , header: "Mail Name" , width :160 ,type:"text"},
{id: 'maildata' , header: "Mail Data" , width :190,type:"text" },
{header: "Group" , width :70,
editor : { type :"select" ,options : {'php':'php','asp':'asp'}
,defaultText : 'php' } },
{id: 'mailid' , header: "Action", width :120 , resizable : false, sortable : false , printable : false ,
renderer : function(value ,record,columnObj,grid,colNo,rowNo){
var no= record[columnObj.fieldIndex];
var cod = (record['maildata']);
return 'Edit | Delete';
} }
];
I am littlebit new in Javascript and Sigmagrid,i think that i am doing something worst with codes,pls help me to success.
Thank you.
Note : i posted the same Question on Sigma Grid Forum too,i think that it is not a problem.
Problem 2
The string cod contains a >
Problem 3
The string hai newuser needs to be contained in " or ' or it is considered a variable name
Basically you have to decide -- are you going to validate the html or not. If you don't validate the HTML then html errors in the data will show as errors on your page. You could also HTML escape the html so you will see the HTML codes -- this is probably the best plan.
Other sites use (like this one) use markdown -- this is easier to validate -- then they generate the actual HTML before display.
In addition you are having problems with the alert. Alert displays strings not HTML so you will see what you are seeing -- different results than expected depending on the HTML.
I would take a step back and ask yourself -- what is the type of the data, how am I going to display it. How am I going to validate that if it is HTML it is valid.
There are the problems you need to address -- your examples all stem from this problem.