Develop Reference

Is it possible to write protect old data of JSON Files and only enable appending?

I need to store some date stamped data in a JSON file. It is a sensor output. Each day the same JSON file is updated with the additional data. Now, is it possible to put some write protection on already available data to ensure that only new lines could be added to the document and no manual tampering should occur with it?
I suspect that creating checksums after every update may help, but I am not sure how do I implement it? I mean if some part of JSON file is editable then probably checksum is also editable.
Any other way for history protection?

Write protection normally only exists for complete files. So you could revoke write permissions for the file, but then also appending isn't possible anymore.
For ensuring that no tampering has taken place, the standard way would be to cryptographically sign the data. You can do this like this, in principle:
Take the contents of the file.
Add a secret key (any arbitrary string or random characters will do, the longer the better) to this string.
Create a cryptographical checksum (SHA256 hash or similar).
Append this hash to the file. (Newlines before and after.)
You can do this again every time you append something to the file. Because nobody except you knows your secret key, nobody except you will be able to produce the correct hash codes of the part of the file above the hash code.
This will not prevent tampering but it will be detectable.
This is relatively easily done using shell utilities like sha256sum for mere text files. But you have a JSON structure in a file. This is a complex case because the position in the file does not correlate with the age of the data anymore (unlike in a text file which is only being appended to).
To still achieve what you want you need to have an age information on the data. Do you have this? If you provide the JSON structure as #Rohit asked for we might be able to give more detailed advice.

How can I save a very large in-memory object to file?

I have a very large array with thousands of items
I tried this solution:
Create a file in memory for user to download, not through server
of creating an anchor
text file
~~JSON.stringify on the array caused the tab to freeze~~ Correction: Trying to log out the result caused the tab to freeze, stringify by itself works fine
The data was originally in string form but creating an anchor with that data resulted in a no-op, I'm assuming also because the data was too big, because using dummy data successfully resulted in a file download being triggered
How can I get this item onto my filesystem?
edit/clarification:
There is a very large array that I can only access via the the browser inspector/console. I can't access it via any other language

Javascript does not allow you to read or write files, except for cookies, and I think the amount of data you are using exceeds the size limit for cookies. This is for security reasons.
However languages such as php, python and ruby allow the reading and writing of files. It appears you are using binary data, so use binary files and write functions.
As to the choice of language : if you already know one use that, or whichever you can get help with. Writing a file is a very basic operation and all three languages are equally good. If you don't know any of these languages you can literally copy and paste the code from their websites.

Ajax - best technique for processing large amount of data

I would like to know, what's the best method to retrieve and use a large amount of dynamic data.
For example:
I have a big website with a lot of fields, which dynamically create popups. The popups are created with a Javascript template engine, which needs JSON encoded data.
Now what I can do:
Every time i request a popup, the client fetches the JSON data via AJAX
I can create a Javascript var via PHP, which stores the data for all possible popups in the HTML code
Or I can fetch the data via AJAX and cache it, in a Javascript var
So which one of these is the best one?
What are the disadvantages of them?
Or how would you attach/load the data for these popups?
BTW does anybody know why all the facebook popups are so smooth? It seems that they are created asynchronously, but they are so fast - like they were already embedded.

Pre-emptive caching.
Basically your 'pop-ups' (god knows why you have so many - there must be a better way :-D hehe) will have a pattern or logical order or whatever.
Using a combination of:
Loading the Main / Most likely to be first used pop-ups data and storing that in a var.
I would highly recommend trying to do this with JSON or similar and store data for 10-20 pop-ups together - downside is performance - have to parse whole file for 1 pop-up (but modern browsers / PCs - not much issue) - plus side number of http requests - the killer of site speed.
You COULD** start loading data for a button etc. on HOVER (as well as click) - milliseconds make prizes you know!
Finally - just ajax the data in and keep it small - the more you can strip out of the ajax call and pre-load (image sprites on page load etc. etc.) the faster your site will respond.
However without knowing:
how often the data will update
what sort of data you are sending (is it all graphs, all text etc.)
how many of these pop-ups you have
how often a new pop-up will be loaded
what device(s) you users will be using
etc.
I can only give wild stabs in the dark!

I am working on the same thing now and find a good introduction blog http://blog.mariusschulz.com/2014/02/05/passing-net-server-side-data-to-javascript, hope it can give you some suggestions.

How to save a files contents in Javascript/jQuery

Basically, I want to upload ONLY a CSV file via Javascript or jQuery.
I want to try and do this without any PHP involved.
I need to use a HTML upload form, and then save only it's contents to a multidimensional array or a string.
I do not need to save the uploaded file to the server, I just need to save it's contents to a string as stated.
I have looked far and wide online, yet everything involves PHP.
Is this possible with just Javascript or jQuery?
Thanks in advance

This uses a library I wrote and released under the GPLv3 License: html5csv
The example below uploads a CSV file into the browser, where it is available as an array of arrays.
The library supports various block operations, such as make a table, edit, plot, fit, call a function, save in browser session storage or local storage.
JSFIDDLE
html
Choose a CSV file to load into the application:
<input id='foo' type='file'>
<hr />
js (requires jQuery and html5csv.js)
CSV.begin('#foo').
table('output', {header:1, caption:'Uploaded CSV Data'}).
go();
Here, go() can take a function callback
(e,D), where e will contain an error string or null, and D is an object that may contain D.rows[0][0],...,D.rows[n-1][m-1] for a n x m matrix of data. Row 0 may be a header row.
Asynchronicity is used, in fact enforced in places. So beware that like AJAX, this code will return immediately to the subsequent line, and is best read as setting up a workflow of what to do when the previous step becomes ready.
Saving/Restoring
You can save data into the user's browser localStorage object with .save('local/someKey'). somewhere in the workflow, and data existing in the array at that point will be stored in HTML5 local storage (perhaps even compressed if you include the LZString library as documented), until the browser user deletes it.
Then in the same page or another page on the same web site you can get the data back out with CSV.begin('local/someKey')...
Using the data
You should put any code you want to use the data into a function that can fit either the callbacks expected by html5csv's call or go as documented on the html5csv site.

The jQuery CSV plugin can use client-side file handling (no need for server-side script like PHP):
https://code.google.com/p/jquery-csv/#Client-Side_File_Handling

You can use plugin which allow you to parse CSV into Array.
http://code.google.com/p/jquery-csv/
Features
Convert a CSV String to an array
Convert a multi-line CSV string to a 2D array
Convert a multi-line CSV string to an array of objects (ie header:value pairs)
Convert an array of values to CSV (under development)
Convert an array of objects to CSV (under development)
Hooks/Callbacks to extend the default parsing process
Customizable delimiter (default: ") and separator (default: ,) characters
Node.js support (ie CommonJS importing and async callback support)

To do the upload, you need to be able to read the file off the disc. You can do this with the HTMl5 File API. I'm sure there are jQuery libraries to simplify this, but that's the underlying tech.
Someone else posted a question (and solution) on how to do that with jQuery: html5's file api example with jquery?
Once you've got access to the file in the browser, use a CSV library to work with it.

XML and Javascript: the right tool for the job?

For years I've been reading about XML and I have just not quite grokked it. Most documents I see about it simply explain the syntax (extraordinarily easy to understand) and say that it's portable: I've worked with Unix my whole life so the idea of putting things in plain text to be portable is hardly revolutionary. My specific question is that I have a document (my CV) that I would like to present to web visitors in several formats: as a webpage, as a pdf, or even as plain text. Is XML and Javascript the right approach to take?
What I need is for the document to be easily editable, conversion easy and just easy general upkeep. For example, when I publish a paper, I'd like to take less than five minutes to add the info and then have everything go automatically from there.
Give me your opinions: I also use LaTeX compulsively, so my current approach has been just to have my CV in LaTeX and to convert it to a web-page using LaTeXML. However, I sorta have the feeling that with everybody jumping up and down about XML and Javascript, that there might be something good to learn about it.
I would also like to simplify maintaining my homepage by not duplicating the same footer for every single page that I set up.
Thanks,
Joel
Edit: I'll also take any book recommendations!

I think this is a slight misunderstanding of the combination of JavaScript and XML.
XML, in and of itself is an excellent means of representing data. It's largely human-readable, and easily parsed with libraries in nearly every programming language. That is the main benefit of XML.
Using XML with JavaScript is certainly a solution, but I think it's a matter of the question you're asking. JavaScript can parse XML, and allow you to obtain and manipulate data from your XML document. If you want to grab data from a server without reloading your HTML page (synchronously or asynchronously), then using JavaScript and XML is a valid way to do that.
If you want to, however, display your XML as a webpage, you would likely be better off using XML and XSLT [wikipedia], or perhaps PHP and XPath, to transform the document into browser-readable HTML. On the other hand, you could use nearly any language to convert the XML to a plain-text file, rich text file, or store it in a normalized database.
To sum up, XML is a great way to store data, because it can be used in so many different ways, and by so many different languages. It's an answer to many different questions; you just have to figure out which questions you're asking.

To elaborate on my comment
The transformation to whatever output you desire is depending on how you store your CV on your server and whether you have the possibility to process it on the server. If you store it in XML, you can transform it to desired (binary) output using server based tools - that would for php be pdf and word (on windows server platform) for example. XML would be interesting from a mark-up point of view since it would make it clear where the table of contents, headers, lists of experience and so one would be found.
JavaScript cannot transform something into PDF or word, that has to be done on the server. What javascript can do is to get a text from the server in XML or JSON using AJAX and manipulate this into what the user sees on the screen. For XML that can be done with XSL(T) too. If you want for self-education purposes to use JavaScript, JSON is very nice since it is in my opinion more readable than XML and it creates a populated javascript object with the least work.
Footer in javascript: in the page have
<script type="text/javascript" src="footer.js"></script> and in footer.js, you can for example do
var footerText = 'Here goes whatever you want';
document.write(footerText);
Comparison between XML and JSON

I've got a webpage with browser-side XSLT transformation up and running for years. It's a playground, only some words in german. See how easy it is to build this on heese.net/test. You can switch between "Beispiel" (=Demo) and XSL. The sourcecode of the page in the iframe is the XML. You can do this serverside with 3 lines of PHP-code.
On Javascript: you can use it with XSLT and I show this on my site, but it can't interact. First the XSLT builds an HTML page out of your XML data and after this job is completely done the Javascript in the resultig HTML document begins to work.
Parsing XML with Javascript is a different task.