I produce a complex HTML summary report from data in a database which could be a summary of maybe 200,000 rows in the database. The user can click a link to request an Excel version.
When they do a JS script extracts the key components of the report and stuffs them into a form in a hidden iframe. This form submits to a server-side script which generates the Excel version of the report (without the graphics etc).
As the calculations for the report are complex and "costly" it makes sense not to run them again to create the Excel version as all the data is on the page already. Also the user may have customised the report once it is loaded and I can use JS to pass those preferences to the form as well so the Excel doc reflects them too.
The way I am doing this is to include the following for each component of the report that transfers to a row in the Excel version. I've hijacked an HTML tag that isn't otherwise used.
<code id="xl_row_211865_2_x" class="rowlabel">Musicals}{40%}{28.6%}{6</code>
The code element above is a summary of the row below in the HTML report which becomes one row in the Excel doc and includes the label and various data elements. There may be a thousand or more such elements in one report.
As the data contains text I've had to use something like }{ as a field separator as this is unlikely to occur in any real text in the report. I have code set to display:none in the CSS.
When the user wants an Excel version of their report the JS code searches the HTML for any <code> elements and puts their className and innerHTML in the form. The className indicates how to format the row in Excel and the data is then put into adjacent cells on the Excel row.
The HTML report shows one percentage base (they can toggle between them) but the user preference when requesting an Excel version was to include both.
Is there a better way of doing this?
(As this is a part of a complex web app no user is going to turn CSS off or lack javascript or they wouldn't get this far)
ADDED: I can't use HTML5 as the users are corporates often on older browsers like IE6
Use the new data- attributes
http://www.javascriptkit.com/dhtmltutors/customattributes.shtml
<div data-row="[["Musicals",40,28.6,6], ...]">
The div could be the TD tag or TR tag or any other relevant tag already related to the row and the " is the escaped ".
That makes the data hidden from view and also ensures that there will come standard solutions to process the data.
Also for encoding data I would suggest using JSON as that is also a standard that is easy to use.
Standard solutions:
1) Use a Javascript data block:
<script>
var mydata = {
'Musicals': ['6','40%','28.6%'],
"That's Life": ['2','13.2%','0.5%'],
...etc....
}
</script>
2) Use element attributes:
(see http://ejohn.org/blog/html-5-data-attributes/ for more info)
<div class='my_data_row' data-name='Musicals' data-col1='6' data-col2='40%' data-col3='26.6%'>
...and then use Javascript to load the attributes as required.
This second option would be used when the data is related to the element in question. You wouldn't normally want to use this for data that's going to be used elsewhere; I would say that in your case, the simple Javascript data block would be a far better solution.
If you do go with the data attributes as per the second option, note the use of the 'data-' prefix on the attributes. This is an HTML5 specification that keeps user-defined attributes separate from normal HTML ones. See the linked page for more info on that.
You could try the new html5 feature localStorage instead of using hidden html fields, that is if you're sure that your users use only latest modern browsers.
Anyway, an improvement on your code would be to actually store the data in JSON format:
Instead of
Musicals}{40%}{28.6%}{6
you would use something like
{
"label": "Musicals",
"percentage1": 40,
"percentage2": 28.6,
"otherLabel": 6
}
This way you can build javascript objects just by evaluating (eval) or parsing (JSON.parse) the innerHTML of the hidden element, in a faster way than you interpret your own curly brackets protocol.
My point to solve that in a better way would be take these results and save in some temporal XML file in the server, show the contents in the browser and when the user request for the Excel version, you only need to take the temporal XML.
Take a look to Linq-to-XML, because its fluent-style programming would help you in reading the XML file in few lines and then creating such Excel file.
Another solution would be serialize your object collection to JSON and deserialize them with the DataContractJsonSerializer. That would make the size of temp file smaller than XML approach.
Related
I have successfully setup pagedown on a site I am using, but I have run into an issue when trying to edit HTML that has already been created. I would like to take a HTML chunk that was created using pagedown, convert it back to markdown and place it in the editor.
I looked around but didn't see this covered in the documentation. I took a look in the Markdown.Converter.js file to see if there was a makeMarkdown function to match the makeHTML function but I didn't see anything.
How do I go about converting HTML back to markdown for editing?
As far as I know, no, there is no existing solution that will convert html to markdown. There are a few problems that would need to be solved before that can be done, for example, representing floats, text alignment, font sizes, etc in markdown. That leaves you with two options:
Store the markdown in the database, then convert the markdown to html on the fly. This has the advantage of being able to easily edit the text and reduces the amount of data you're storing in the database.
the second option is to store both the markdown and the html in the database. This uses more disk space, however will result in less resources being used to retrieve the html because you no longer have to convert markdown to html on the fly.
Both options are viable, each with their own advantages. I usually use the first option so that i don't have duplicate data in the database, but the second option is likely easier to use because the display-system that displays the content won't be required to have a markdown processor, instead it just pulls the generated html directly from the database.
I'll likely move to the second option instead in future projects because it makes the data more portable. If you were to access the database in a different server-language, you wouldn't need a markdown processor written in that language to get the html.
The title phrases it badly so here's a longer description :
I have an application that exports data in html format. ( 500 rows, 20 columns)
It looks terrible with lots of useless columns.
I want to use something like datatables to make a more usable table, i.e. paging/sorting/filtering/hiding columns
The option I'm trying first is to insert the table from the exported html file using the .load() function from jquery. Then I loop through the table deleting/modifying columns.
This seems very slow (I suspect my looping and searching) so I'm looking for improvements.
One idea is to pre-convert my exported html file to json (using notepad++ macros or something like that) and then build the table that I want from that json file.
Any opinions on whether I can expect a large performace boost, or potential problems to look out for ?
Many thanks / Colm
JSON should be faster, when its loaded its ready to go without all of the text parsing you would need to do with a text file. Lots of other jquery addons available to make it easy for you once it is in JSON.
I think this is not about which loads data faster but which solution is better for your problem. Datatables is very flexible and you can load from different sources. Take a look at the "Data Sources" and "Server side processing" in the examples: http://datatables.net/examples/
Datatables uses mostly JSON format. To process your data need to find the best approach; convert your exported html file, process the file with javascript to convert data (jquery can help you here), etc..
This page gives some real world examples of loading data in json vs data in a html table. Fairly conclusive, see the post from sd_zuo on July 2010, a fourfold increase in speed loading from json and then just building the table that you want to display.
Granted the page deals specifically with the slowness of the innerHtml function in IE8 but I think I'll give it a go in json and see how it compares across a couple of browsers.
P.S. This page gives good advice on fast creation of html using raw javascript and then only using jquery to insert one full row at a time
I have some data that I need to associate with specific element such as an individual table row. This data contains information such as the current state, and a unique identifier that correlates to an SQL row. When the user interacts with the element I want to read out the unique identifier, and with that identifier, issue an AJAX request to let the user change the state of that element.
After researching, it seems that there are two camps as to how to embed this element specific information.
1) Using a data- attribute in the html5. My understanding is this will work in modern web browsers as well as older browsers that don't support html5. But while this works, it does not following the HMTL spec ( less than HTML5) and so it won't validate if you run it through a HTML syntax checker.
2) Store the additional data into a javascript array, object etc. The extra work with this is now you need a way to correlate the javascript data to the html element.
What are the pros and cons of using these two different approaches to storing the data? And what approach would you recommend?
Thanks!
I wouldn't worry about the data- attributes not passing validators. The attribute is in HTML5 because people were using similar, non-standardized, attributes for a long time specifically to solve this problem. Go ahead and start writing "HTML5" by using the parts of the spec that work, i.e. don't break in a certain browser, and using the HTML5 doctype. The W3C validator at least already supports the doctype.
As for which method to use, I think it really boils down to when you want to parse the information in the JavaScript interpreter: on page load or when the data is needed. I imagine it depends on just how much information you have as to which will be most efficient. But you can't go wrong with adding it to the HTML with a data- attribute or two.
Personally, I like adding the information to the HTML with data- attributes. In the scenario you describe, I would use data-state and data-rid (or similar) so that I don't have to further parse the information (it sounds like you are thinking of putting two bits of information in one data- attribute). This way, your table of information is truly complete: the data is presented to the user and the markup contains further information that could be necessary to a parser.
I'd definitely go with option (1) and either not worry about those attributes not validating or just validate your document as html5. It's simple. It works.
The "separation of concerns" theory that leads some people to option (2) is nonsensical for this sort of situation because if you put the data into a JS object of some kind you still need a way to tie it back to the actual html elements so then not only are the two not really separate, they're more complicated than they need to be on the client side, and the server side code needed to produce it in the first place is more complicated.
In option (2), no special correlation is needed any more than in option (1)—rather, less. You can put the data into a property of the DOM object that corresponds to the element. Why not make use of the basic feature of JavaScript that you can add properties to an object?
You've tagged your question with jQuery, so I'll assume you have it. You can use the .data() method to store arbitrary data and associate it with an element.
$("tr").first().data('sqlId', 1234);
assert($("tr").first().data('sqlId') === 1234);
I once again need to do something that sounds simple but is infact frustratingly evading me.
On my company's intranet site we have a large table of data that has a javascript filter applied to it so that managers and other interested parties can quickly locate the rows that are relevant to them. The filter I am using can be found at http://tablefilter.free.fr/ .
My issue arises when I need to have a button to export the filtered results to Excel so that the managers can access the data offline. There are many straight forward options for exporting the HTML table to excel but I have been unable to figure out how to get JUST the filtered results to export. (Note: This intranet site will only be accessed via IE)
There is a function as part of the javascript table filter, GetFilteredData(), that will grab the filtered data cells and input these into an array, i think called filteredData[]. This array is formated as such: [rowindex,[value1,value2,value3...]].
So how do I get this array into an Excel or csv file? Again, this will be accessed only by IE so activeX objects and controls are acceptable.
Also I should probably note that I cannot use server-side technologies so please limit your responses to the confines of HTML, javascript and activeX. Thanks!
FYI: DataTables has a nice plugin called TableTools which can export table to csv on client-side. It's achieved using Flash. If you are satisfied with the filter of DataTables, I think this would be a good solution.
http://www.datatables.net/extras/tabletools/
If you are using IE starting at version 8, you can use data: URLs. Just generate the URL and point the borwser there using location.href. The data in CSV can be generated by javascript and base64 encoded.
You might want to consider an approach that relies on string manipulation.
Once you have this array, you can turn it into a JSON string. Try this popular lightweight library (json2.js):
https://github.com/douglascrockford/JSON-js
Example:
text = JSON.stringify(['e', {pluribus: 'unum'}]);
// text is '["e",{"pluribus":"unum"}]'
You should have something like
var dataString = '["rowindex",["value1","value2","value3"]]'
At this point you could use some regex replacement on the string to format it to either style.
For years I've been reading about XML and I have just not quite grokked it. Most documents I see about it simply explain the syntax (extraordinarily easy to understand) and say that it's portable: I've worked with Unix my whole life so the idea of putting things in plain text to be portable is hardly revolutionary. My specific question is that I have a document (my CV) that I would like to present to web visitors in several formats: as a webpage, as a pdf, or even as plain text. Is XML and Javascript the right approach to take?
What I need is for the document to be easily editable, conversion easy and just easy general upkeep. For example, when I publish a paper, I'd like to take less than five minutes to add the info and then have everything go automatically from there.
Give me your opinions: I also use LaTeX compulsively, so my current approach has been just to have my CV in LaTeX and to convert it to a web-page using LaTeXML. However, I sorta have the feeling that with everybody jumping up and down about XML and Javascript, that there might be something good to learn about it.
I would also like to simplify maintaining my homepage by not duplicating the same footer for every single page that I set up.
Thanks,
Joel
Edit: I'll also take any book recommendations!
I think this is a slight misunderstanding of the combination of JavaScript and XML.
XML, in and of itself is an excellent means of representing data. It's largely human-readable, and easily parsed with libraries in nearly every programming language. That is the main benefit of XML.
Using XML with JavaScript is certainly a solution, but I think it's a matter of the question you're asking. JavaScript can parse XML, and allow you to obtain and manipulate data from your XML document. If you want to grab data from a server without reloading your HTML page (synchronously or asynchronously), then using JavaScript and XML is a valid way to do that.
If you want to, however, display your XML as a webpage, you would likely be better off using XML and XSLT [wikipedia], or perhaps PHP and XPath, to transform the document into browser-readable HTML. On the other hand, you could use nearly any language to convert the XML to a plain-text file, rich text file, or store it in a normalized database.
To sum up, XML is a great way to store data, because it can be used in so many different ways, and by so many different languages. It's an answer to many different questions; you just have to figure out which questions you're asking.
To elaborate on my comment
The transformation to whatever output you desire is depending on how you store your CV on your server and whether you have the possibility to process it on the server. If you store it in XML, you can transform it to desired (binary) output using server based tools - that would for php be pdf and word (on windows server platform) for example. XML would be interesting from a mark-up point of view since it would make it clear where the table of contents, headers, lists of experience and so one would be found.
JavaScript cannot transform something into PDF or word, that has to be done on the server. What javascript can do is to get a text from the server in XML or JSON using AJAX and manipulate this into what the user sees on the screen. For XML that can be done with XSL(T) too. If you want for self-education purposes to use JavaScript, JSON is very nice since it is in my opinion more readable than XML and it creates a populated javascript object with the least work.
Footer in javascript: in the page have
<script type="text/javascript" src="footer.js"></script> and in footer.js, you can for example do
var footerText = 'Here goes whatever you want';
document.write(footerText);
Comparison between XML and JSON
I've got a webpage with browser-side XSLT transformation up and running for years. It's a playground, only some words in german. See how easy it is to build this on heese.net/test. You can switch between "Beispiel" (=Demo) and XSL. The sourcecode of the page in the iframe is the XML. You can do this serverside with 3 lines of PHP-code.
On Javascript: you can use it with XSLT and I show this on my site, but it can't interact. First the XSLT builds an HTML page out of your XML data and after this job is completely done the Javascript in the resultig HTML document begins to work.
Parsing XML with Javascript is a different task.