I'm trying to scrape a website and there is a table with clickable elements and text. I managed to use this to grab the innerText of the table elements:
const result = await page.$$eval('tableselector tr', rows => {
return Array.from(rows, row => {
const columns = row.querySelectorAll('td');
return Array.from(columns, column => column.innerText);
});
});
I've tried just returning columns and using result[row][column].getProperty('innerText').jsonValue() to try and grab the innerText but it doesn't work. Could someone explain where I'm going wrong?
EDIT:
Here is an HTML Segment that represents the structure of the table I am trying to scrape.
<table id = "table_id">
<body>
<!-- input button is the clickable element I want to grab -->
<tr class = "GridRowStyle">
<td>input button</td><td>text2</td><td>text3</td><td>text4</td><td>text5</td><td>text6</td><td>text7</td>
</tr>
<tr class = "GridAlternatingStyle">
<td>input button</td><td>text2</td><td>text3</td><td>text4</td><td>text5</td><td>text6</td><td>text7</td>
</tr>
<tr class = "GridRowStyle">
<td>input button</td><td>text2</td><td>text3</td><td>text4</td><td>text5</td><td>text6</td><td>text7</td>
</tr>
</body>
Related
I used cheerio for the first time today
This is a simplified version of the html source I want.
<div id="country-table">
<!-- div duplicate cause style -->
<div>
<div>
<table>
<tbody>
<tr>
<td>1</td>
<td>USA</td>
<td>1.6</td>
<td>75.8</td>
<td>132,000</td>
</tr>
<tr>
<td>2</td>
<td>INDIA</td>
<td>12123</td>
<td>1322</td>
<td>123213</td>
</tr>
<tr>
<td>3</td>
<td>BRAZIL</td>
<td>3123</td>
<td>213123</td>
<td>134</td>
</tr>
<tr>
<!-- and more... -->
</tbody>
</table>
</div>
</div>
</div>
and i tried to this:
const axios = require("axios").default;
const cheerio = require("cheerio").default;
axios.get("https://coronaboard.kr").then((html) => {
const arr = [];
const $ = cheerio.load(html.data, { xml: true, xmlMode: true });
const data = $("#country-table>div>div>table>tbody").each((index, item) => {
arr.push(item);
});
console.log(arr);
});
I want to put information in td into tr.
ex){number:x,name:USA,confirmed:x,and more...}
If anyone knows how to do it, please answer me!
If you're wanting to extract the data from the table, then this will help. Follow the comments to help you understand how it works.
var $ = cheerio.load(html.data);
// targets the specific table with a selector
var html_table = $('#country-table>div>div>table');
// gets table cell values; loops through all tr rows
var table_data = html_table.find('tr').map(function() {
// gets the cells value for the row; loops through each cell and returns an array of values
var cells = $(this).find('td').map(function() {return $(this).text().trim();}).toArray();
// returns an array of the cell data collected
return [cells];
}).toArray();
// output table data
console.log('table_data', table_data);
The initial text of A, B, C, D, and the number need to be removed in the frontend because I require it in the backend.
The HTML structure of table row is like this:
<tr ng-repeat="(key, field) in nbd_fields" ng-show="field.enable && field.published" class="ng-scope">
<td class="ng-binding">A,B,C,D: 1 - Auswahl Wunschkarte : <b class="ng-binding">Wähle eine Option</b>
</td>
<td ng-bind-html="field.price | to_trusted" class="ng-binding"></td>
</tr>
Before Input:
Current Output:
If you notice that the selected option is also not visible. Is it because of the $(window).load() ?
Required Output:
Code that I am using:
jQuery(".ng-scope td.ng-binding:first-child").text(function(i, oldVal) {
return oldVal.replace(/^[^-]+ - /,"");
});
});
How can I make it so that it does not affect the <b> tag inside?
I used the above code for the steps heading with a different selector on the same page* and it worked because it did not have any children to alter.
I had to wrap it around $(window).load() so that the changes are applied after the table is loaded. $(document).ready() did not work with it. Not sure why?
(Edit: Modified to accommodate restated requirement in comment below.)
To strip "everything up to and including the '-'" from the text of first column table cells while leaving the rest untouched:
// strip "everything up to and including the '-'"
// from table cell contents
function stripPrefix(tblCell) {
// only evaluate first td in tr
if (tblCell.previousElementSibling) {
return;
}
const tNode = tblCell.firstChild;
// ignore if table cell is empty
if (!tNode) {
return;
}
const chars = tNode.nodeValue.split('');
const iFirstDash = chars.indexOf('-');
if (iFirstDash === -1) { return; }
tNode.nodeValue = chars.slice(iFirstDash+1).join('');
}
function stripAllPrefixes() {
const tds = document.getElementsByTagName('td');
for (const td of tds) {
stripPrefix(td);
}
}
td {
border: 1px solid gray;
}
<h4>Strip "everything up to and including the '-'" from Table Cells</h4>
<table>
<tr>
<td>A,B,C,D: 1 - Auswahl Wunschkarte : <b>Wähle eine Option</b></td>
<td></td>
</tr>
<tr>
<td>B,C,D,E: 20 - A different leader : <b>should also be stripped</b></td>
<td></td>
</tr>
<tr>
<td>Oops no dash here <b>Just checking</b></td>
<td></td>
</tr>
</table>
<button onclick="stripAllPrefixes();">Strip All</button>
It does not effect the b tag, your code is working, you just need to use the right method and do the replacement to the HTML code and not the text nodes:
jQuery(".nbd-field-header label, .nbo-summary-table .ng-binding").html(function(i, oldVal) {
return oldVal.replace(/^[^-]+ - /,"");
});
I'm using Datatables and mark ids of my table with
<tr data-id='1'>
tags. I want to get the ids of selected rows. I tried this but it doesn't seem to work:
var $issueID = $(my_table.rows('.selected').nodes()).data('id');
$.each($issueID, function (value, index ) {
alert(value);
});
If I want to do it for a single row it works fine if I use
row().node()
but I can't get it right for many rows.
This should do the trick:
var selectedIds = [];
var my_table = $('#my_table').DataTable();
my_table.rows('.selected').every( function() {
selectedIds.push(this.data().id);
});
As Mike mentioned in a comment, notice that a capital D which is used to initialise the DataTable here. $().DataTable() returns a DataTables API instance, while $().dataTable() will also initialise a DataTable, but returns a jQuery object.
While searching for the same answer I came across this article. I modified the code in your question to find a working solution.
var inactiveRecord = $(my_table.rows('.selected').nodes());
$.each(inactiveRecord, function (idx, value) {
alert($(value).data('id'));
});
You should use a Class to do this in addition to your data-id.
JQUERY
$('.row').each( function() {
var value = $(this).attr('data-id');
alert(value);
})
HTML
<tr class="row" data-id="1">
<td></td>
</tr>
<tr class="row" data-id="2">
<td></td>
</tr>
<tr class="row" data-id="3">
<td></td>
</tr>
or without a Class you could just use
$('tr').each( function() {
var value = $(this).attr('data-id');
alert(value);
})
I recommend adding a class to tr so you don't accidentally get it mixed up with other rows that may not need to be counted.
Ok, so I have this partial view (part of an MVC application) which just displays data from a view in the database.
#model IEnumerable<TransportPlanner.Models.TRANSPORT_PLANNER_VIEW>
<table>
<tr>
<th>#Html.CheckBox("HeaderCheck")</th>
<th>#Html.DisplayNameFor(model => model.Status)</th>
....
<th>#Html.DisplayNameFor(model => model.Volume)</th>
<th>#Html.DisplayNameFor(model => model.Weight)</th>
....
</tr>
#foreach (var item in Model) {
<tr>
<th>#Html.CheckBox("RowCheck")</th>
<td>#Html.DisplayFor(modelItem => item.Status)</td>
....
<td>#Html.DisplayFor(modelItem => item.Volume)</td>
<td>#Html.DisplayFor(modelItem => item.Weight)</td>
....
</tr>
}
</table>
I want to be able to find a way in which I can get the values for the Volume and Weight fields of only the checked rows (after checking them), and add them to get the total (which is displayed but not stored).
For example, once I get the results displayed on screen, and I check 3 rows (having the values of 'weight' as 5, 10 and 15), then value displayed should be '30' (sum of the weights). Similarly, if I remove the checkbox for the row having weight as '5', then the value displayed should be '25'.
My front end skills are almost non-existent, and I have scourged over the internet for nearly 12 hours but not found a way to do it. I know that I need to either use JavaScript (or some flavour of it like JQuery) or Ajax (if I need the values updates dynamically as I check/uncheck them).
What is the best way to achieve this, without updating my model? I don't have the luxury of time because I am the only developer at my workplace, and this is the first step of a huge task I need to complete in 3 weeks.
Your #Html.CheckBox("RowCheck") is generating invalid html (duplicate id attributes). Replace it with
<input type="checkbox" class="checkbox" />
Then add class names to the <td> elements for Volume and Weight
<td class="volume">#Html.DisplayFor(modelItem => item.Volume)</td>
<td class="weight">#Html.DisplayFor(modelItem => item.Weight)</td>
And assuming you want to display the totals in the table footer, add the following html to the <table> element (note you should also use the <thead> and <tbody> in your table
<table>
....
<tfoot>
<tr>
<td></td>
....
<td id="totalvolume"><td>
<td id="totalweight"><td>
....
</tr>
</tfoot>
</table>
Then you can use javascript/jquery to handle the change event of the checkboxes, and sum the values in each row. Using jquery:
$('.checkbox').change(function() {
var totalVolume = 0;
var totalWeight = 0;
var selected = $('.checkbox:checked'); // get the selected checkboxes
$.each(selected, function(index, item) { // loop through the selected checkboxes
var row = $(this).closest('tr'); // get the containing tr element
totalVolume += Number(row.find('.volume').text());
totalWeight += Number(row.find('.weight').text());
})
// display the totals
$('#totalvolume').text(totalVolume);
$('#totalweight').text(totalWeight);
});
Note, the above code assumes the values of properties Volume and Weight are numeric (without any formatting)
How can get a row's value on mouse click or checking the checkbox preferably from the below given html table?
Here is the js for getting values for my table from a xml using spry
var ds1 = new Spry.Data.XMLDataSet("xml/data.xml", "rows/row");
var pv1 = new Spry.Data.PagedView( ds1 ,{ pageSize: 10 , forceFullPages:true, useZeroBasedIndexes:true});
var pvInfo = pv1.getPagingInfo();
Here is the Div with spry region containing the table that gets populated from pv1 (see js part)
<div id="configDiv" name="config" style="width:100%;" spry:region="pv1">
<div spry:state="loading">Loading - Please stand by...</div>
<div spry:state="error">Oh crap, something went wrong!</div>
<div spry:state="ready">
<table id="tableDg" onclick="runEffect('Highlight', 'trEven', {duration: 1000, from: '#000000', to: '#805600', restoreColor: '#805600', toggle:true}, 'Flashes a color as the background of an HTML element.')"
style="border:#2F5882 1px solid;width:100%;" cellspacing="1" cellpadding="1">
<thead>
<tr id="trHead" style="color :#FFFFFF;background-color: #8EA4BB">
<th width="2%"><input id="chkbHead" type='checkbox' /></th>
<th width="10%" align="center" spry:sort="name"><b>Name</b></th>
<th width="22%" align="center" spry:sort="email"><b>Email</b></th>
</tr>
</thead>
<tbody spry:repeat="pv1">
<tr class="trOdd"
spry:if="({ds_RowNumber} % 2) != 0" onclick="ds1.setCurrentRow('{ds_RowID}');"
style="color :#2F5882;background-color: #FFFFFF">
<td><input type="checkbox" id="chkbTest" class = "chkbCsm"></input></td>
<td width="10%" align="center"> {name}</td>
<td width="22%" align="center"> {email}</td>
</tr>
<tr class="trEven" name="trEven" id="trEven"
spry:if="({ds_RowNumber} % 2) == 0" onclick="ds1.setCurrentRow('{ds_RowID}');"
style="color :#2F5882;background-color: #EDF1F5;">
<td><input type="checkbox" class = "chkbCsm"></input></td>
<td id="tdname" width="10%" align="center"> {name}</td>
<td width="22%" align="center"> {email}</td>
</tr>
</tbody>
</table>
</div>
</div>
I am trying the below code but still I am not getting the alert and hence none of the answers are also not working. I know the syntax n all are everything correct, but i am not able to figure out what is the problem here!
//inside $(document).ready(function()
$("#chkbHead").click(function() {
alert("Hi");
});
My page has other tables too for aligning some contents. So when I use the below code it works perfectly on those tables except the one in the question. It might be the problem because there are only 2 tr in the table which gets populated by a spry dataset and hence not getting identified properly. May be, I am not sure, just trying to help improve my understanding
$('tr').click(function() {
alert("by");
});
The values of a Row you will get with:
$('#tableDg tbody tr').live( 'click', function (event) {
$(this).find('td').each( function( index, item ) {
if ( $(this).has(':checkbox') ) {
alert( $(this).find(':checkbox').val() );
} else {
alert( $(this).text() );
}
};
});
What exactly do you mean by value of a table row? You can get the inner html of a table row like this:
var html = '';
$('tr').click(function() {
html = $(this).html();
});
You can get attributes of the table row (e.g. it's Id) like so:
var id = '';
$('tr').click(function() {
id = $(this).attr('id');
});
Alternatively you can get the value of nested elements such as a text input like so:
var text = '';
$('tr').click(function() {
text = $(this).find('#myTextBox').val();
});
EDIT
This is how to change the checked attribute of a checkbox nested in a table row:
$('tr').click(function() {
$(this).find('input:checkbox').attr('checked', 'checked');
// alternatively make it unchecked
$(this).find('input:checkbox').attr('checked', '');
});
EDIT
As the table rows are being loaded dynamically - the $().click() event binding method will not work, because when you are calling it - the table rows do not exist, so the click event cannot be bound to them. Instead of using $().click use the jQuery live method:
$('tr').live('click', function() {
// do stuff
});
This binds the click event to all current table rows and all table rows that may be added in the future. See the jQuery docs here
you have to use Spry Observer,
something like this:
function funcObserver(notificationState, notifier, data) {
var rgn = Spry.Data.getRegion('configDiv');
st = rgn.getState();
if (notificationState == "onPostUpdate" && st == 'ready') {
// HERE YOU CAN USE YOUR JQUERY CODE
$('#tableDg tbody tr').click(function() {
$(this).find('input:checkbox').attr('checked', 'checked');
// alternatively make it unchecked
$(this).find('input:checkbox').attr('checked', '');
});
}
}
Spry.Data.Region.addObserver("configDiv", funcObserver);