I am fairly new to web development. I am using HTML and Javascript, specifically the Jexcel Javascript library, to create a spreadsheet webpage. The spread sheet is sourced from a CSV file stored in the working directory of my project. The problem is, when I make changes to that CSV file, they do not show up in my project until I rename the file (and change the filepath in my code respectively). I suspect this has to do with the webpage caching the source CSV file, but I am not sure. How would I go about disabling caching/making my table source from the CSV file everytime? I am using Tomcat Web Server as well. Thank you.
The code that is sourcing my spreadsheet is the following:
<script>
$('#spreadsheet').jexcel({
allowInsertRow: false,
allowInsertColumn: false,
allowDeleteRow: false,
allowDeleteColumn: false,
csv : 'data/test.csv',
csvHeaders : true,
defaultColWidth : '200px',
});
</script>
When changes are made to test.csv I want those changes to show the next time the page is refreshed, but this is not currently happening.
Update: Restarting the Web Server does make the changes show up, so I believe this is a caching issue. I am using Java Servelets on the Server Side of my code.
Browser side caching of resources is a common issue.
To prevent it, the easiest way is to append a version hash to resource url to force reload when modified, so you still benefit of cache feature when file hasn't changed.
Without information about the languages you use on server side, it's very difficult to help you more.
Here is the fonction I generally use to generate resources url :
function assetv($path) {
$hash = 'undefined';
if (file_exists($path)) {
$hash = filemtime($path);
if (!$hash) {
$hash = md5_file($path);
}
}
return urlFromPath($path) . "?v=" . $hash;
}
Of course, the "urlFromPath" depends on your app ;-)
[EDIT]
The browser loads your csv file using an url (data/test.csv) that is "used as cache key" (approximately).
So to force reload when file was modified, you need to change the url, for instance with a version number parameter (file name doesn't changes) :
data/test.csv?v=1
data/test.csv?v=2
As server side only is aware of file state, you need to generate the versionned url on server side when building your HTML.
Finally, instead of managing a version number manually, you can use a "natural" attribute of the file, as it's last modification time or a hash of it's content.
Thank you all for the suggestions. Ultimately I found out the problem was due to the fact that the csv file I was sourcing my data from was in the project folder and Tomcat does not update changes to that file immediately. I solved this by reading the data from a file stored outside the working directory on my machine.
Related
I am creating browser based video editing tool. I want a user to first download a ~70mb javascript file and store it somewhere on his computer. I want to link that file when my website is opened. How can I achieve that.
EDIT
What i meant is that there are various files like js1.js,js2.js... all sums upto 70mb . So i will offer a zip folder to download and only link js1 or js2 file etc depending on the effects user wish to apply
i am sorry to inform you but i think there is something really wrong with what you are trying to do.
A "solution" would be to just cache the javascript on the user's browser so any subsequent requests parse the cache instead of requesting the resource again from the server.
You should know however that if you are in need to download ~70mb of a javascript file you are doing something wrong. I have a whole web app project that when published the total size is around 60mb, all files required to properly run included, and its a damn big codebase in there.
I find it very hard to believe there is ever a need for a single javascript file to be that big, in any case maybe a simple caching should do the trick
That is actually done automatically. Once you add a <script> tag with a link to a local js file (also stored on the server) the file is loaded automatically.
See HTML <script> src Attribute for more information on that.
You can only reference to js files on the server. Files on the server could look like this:
index.html
somefancyjsfile.js
You can then reference from inside your html file to the js file via the <script> tag.
I'm not sure though if the size is not a bit too much...
I am new to HTML/Javascript, as well as coding in general so bear with me :). I am trying to create a "Spot the Difference" game in html5 using javascript. Everything is local (on my machine). I have two pictures, of the same size, one with differences. To generate data about the clickable fields, I have a java program that reads both of the images and outputs all of the positions in which pixels are different into a XML file. My plan was to then use this XML file with my javascript to define where the user could click. However, it appears (correct me if I'm wrong) that javascript cannot read local XML files for security reasons. I do not want to use an ActiveXObject because I plan on putting this onto mobile devices via phone gap or a webkit object. Does anyone have a better approach to this problem, or perhaps a way to read local XML files via javascript? Any help would be greatly appreciated, thanks.
If you are planning to put this into a smart phones (iOS and Android) and read local files, I have done similar things with JSON (yes, please don't use XML).
Convert your output to JSON
Put this as part of your application package. For example, in Android, I put it as part of the .apk in /appFiles/json
Create a custom content provider that would read the local file. I create mine as content:// but you create whatever scheme you want. You could leverage android.content.ContentProvider in order to achieve custom URL Scheme. iOS has its own way to create custom scheme as well. The implementation simply read your local storage and give the content
To read it from Javascript, I simply call ajax with the custom scheme to get the json file. For example content://myfile/theFile.json simply redirect me to particular directory in local storage with /myfile/theFile.json appended to it
Below is the sample to override openFile() in the ContentProvider
public ParcelFileDescriptor openFile (Uri uri, String mode) {
try {
Context c = getContext();
File cacheDir = c.getCacheDir();
String uriString = uri.toString();
String htmlFile = uriString.replaceAll(CUSTOM_CONTENT_URI, "");
// Translate the uri into pointer in the cache
File htmlResource = new File(cacheDir.toString() + File.separator + htmlFile);
File parentDir = htmlResource.getParentFile();
if(!parentDir.exists()) {
parentDir.mkdirs();
}
// get the file from one of the resources within the local storage
InputStream in = WebViewContentProvider.class.getResourceAsStream(htmlFile);
// copy the local storage to a cache file
copy(in, new FileOutputStream(htmlResource));
return ParcelFileDescriptor.open(htmlResource, ParcelFileDescriptor.MODE_READ_WRITE);
} catch(Exception e) {
throw new RuntimeException(e);
}
}
I hope it helps
I would suggest modifying your java program to output a JSON formatted file instead of XML. JSON is native to JavaScript and will be much simpler for you to load.
As for actually loading the data, i'm not sure what the best option is as you say you want to evenutally run this on a mobile device. If you were just making a normal website you could setup a web server using either Apache or IIS depending on your OS and put the files in the document root. Once you've done that you can load the JSON file via Ajax, which can easily be googled for.
Not sure if this helps any.
Since this is a local file, you can do this with jQuery
$.ajax({
type: "GET",
url: "your.xml",
dataType: "xml",
success: function(xml){
///do your thing
}
});
http://api.jquery.com/jQuery.ajax/
If this has been asked before, I apologize but this is kinda of a hard question to search for. This is the first time I have come across this in all my years of web development, so I'm pretty curious.
I am editing some HTML files for a website, and I have noticed that in the src attribute of the script tags that the previous author appended a question mark followed by data.
Ex: <script src="./js/somefile.js?version=3.2"></script>
I know that this is used in some languages for value passing in GET request, such as PHP, but as I far as I ever knew, this wasn't done in javascript - at least in calling a javascript file. Does anyone know what this does, if anything?
EDIT: Wow, a lot of responses. Thanks one and all. And since a lot of people are saying similar things, I will post an global update instead of commenting everyone.
In this case the javascript files are static, hence my curiosity. I have also opened them up and did not see anything attempt to access variables on file load. I've never thought about caching or plain version control, both which seam more likely in this circumstance.
I believe what the author was doing was ensuring that if he creates version 3.3 of his script he can change the version= in the url of the script to ensure that users download the new file instead of running off of the old script cached in their browser.
So in this case it is part of the caching strategy.
My guess is it's so if he publishes a new version of the JavaScript file, he can bump the version in the HTML documents. This will not do anything server-side when requested, but it causes the browser to treat it as a different file, effectively forcing the browser to re-fetch the script and bypass the local cache of the file.
This way, you can set a really high cache time (like a week or a month!) but not sacrifice the ability to update scripts frequently if necessary.
What you have to remember is that this ./js/somefile.js?version=3.2 doesn't have to be a physical file. It can be a page which creates the file on the fly. So you could have it where the request says, "Hey give me version 3 of this js file," and the server side code creates it and writes it to the output stream.
The other option is to force the browser to not cache the file and pull down the new one when it makes the request. Since the URI changed, it will think the file is completely new.
A (well-configured) web server will send static files like JavaScript source code once and tell the web browser to cache that file locally for a certain period of time (could be a day, a week, a month, or longer). When the browser sees another request for that same file, it will just use that version instead of getting new code from the server.
If the URL changes -- for example by adding a query string -- then the browser suspects that its cached version is no good and gets a new one. As such, the ? helps developers say "Oops, I changed this file, make sure the browser gets a new copy."
In this case it's probably being used to ensure the source file isn't cached between versions.
Of course, it could also be used server side to generate the javascript file, without knowing what you have on the other end of the request, it's difficult to be definitive.
BTW, the ?... portion of the url is called the query string.
this is used to guarantee that the browser downloads a new version of the script when available. The version number in the url is incremented each time a new version is deployed so that the browser see it as a different file.
Just because the file extension is .js doesn't mean that the target is an actual .js file. They could set up their web server to pass the requested URL to a script (or literally have a script named somefile.js) and have that interpret the filename and version.
The query string has nothing to do with the javascript. Some server side code is hosting up a different version depending on that querystring it appears.
You should never assume anything about paths in a URL. The extension on a path in a URL doesn't really tell you anything. URLs can be completely dynamic and served by some server side code or can rewritten in web servers dynamically.
Now it is common to add a querystring to urls when loading javascript files to prevent client side caching. If the page updates and references a new version of the script then the page can bust through and cause the client to refresh it's script.
I recently read on Meta about some browser not flushing their cache even after reading a script url of this form
myscript.js?v=1234
so to go around the problem i am thinking about implementing a solution i also read but without any details given to it. something like myscript-1234.js and reroute to the actual correct file, but i have a doubt now.
Should i rewrite that url to myscript.js or to myscript.js?v=1234 ? I am actually confused as to how it even going to make a difference to have a rewriting.
Your rewriting should not redirect to any other URL (which would the be fetched by the browser), but should be "internal" on your server.
What I mean is that when receiving a request for "myscript-1234.js", your server should instead serve the content of the myscript.js file ; which will always be the last version.
In the end :
For the client the is a different URL each time you update the file on the server : myscript-1234.js, myscript-1235.js, myscript-1236.js, ...
This is why the browser will try to re-fetch the file from the server : as it's not the same name, it will not have the file in cache
But, for the server, there is always one and only one file : myscript.js
And you're using some "rewrite" rule so thr server just removes the -XYZ portion of the file name before trying to read it from disk.
I've got a site where individual pages might require some javascript or CSS files hooked into their heads. I'm trying to keep everything client side when it comes to managing this process, rather than getting on the FTP and sorting everything out in the code so I need to be able to upload css and js files.
I've got CCK filefield up and running, and it works with css files, but it refuses to upload .js files. It instead seems to view every .js as ".js.txt" and then the file appears on the server as thisismyfile.js.txt
Not ideal...
Does anyone know how to work around this. Is it a mime type problem with Drupal or the server, or is Drupal set up to avoid script uploads and n00b hack attacks.
Once the files are uploaded I intend to use PHP mode on the page or node to call drupal_add_css and drupal_add_js.
Looking at the field_file_save_file() function in field_file.inc from filefield module, you can find the following snippet
// Rename potentially executable files, to help prevent exploits.
if (preg_match('/\.(php|pl|py|cgi|asp|js)$/i', $file->filename) && (substr($file->filename, -4) != '.txt')) {
$file->filemime = 'text/plain';
$file->filepath .= '.txt';
$file->filename .= '.txt';
}
So yes, it's a 'security thing', as Jeremy guessed.
You could patch that RegEx for an immediate 'fix', but that would remove this otherwise useful security check completely for all filefields used on the site.
So a more specific workaround might be a better approach. Since you want to add the files via drupal_add_js() calls from code anyways, you might as well do the renaming there, adding some kind of verification to make sure you can 'trust' the file (e.g. who uploaded it, whatever).
Edit: Concerning options to rename (and alternatives) when calling drupal_add_js():
For renaming the file, look into the file_move() function. A problem with this would be that it won't update the corresponding entry in the files table, so you would have to do that also, if the move operation succeeded. (The filefield just stores the 'fid' of the corresponding entry in the files table, so you'd need to find it there by 'fid' and change the 'filename', 'filepath' and 'filemime' entries according to your rename/move)
Alternatively, you could just load the content of the *.js.txt file and add that string with the 'inline' option of drupal_add_js(). This would be less 'elegant' and could be a performance hit, but if those are not important criteria in your specific case, it is less trouble.
Yet another option would be just passing the *.js.txt file as is to drupal_add_js(), ignoring the 'wrong' extension. A short local test showed that this works (at least in firefox). This might be the 'least effort' solution, but would need some additional testing concerning different browser behavior concerning usage of 'misnamed' js files.
Allowing Drupal to upload javascript files would be a security risk, which is also why it doesn't allow you to do it, but instead appends the .txt extension. The reason is that js files are executable along with php, pl, py, cgi, asp. So if Drupal could upload those files to the server, it would be possible for evil doers to upload a file and run it doing all kinds of nasty things on your server, basically anything is possible. Best thing would be to find a different way of uploading files which are secure.
I had a similar need, and found a way to get around the security by first changing the 'allow_insecure_uploads' variable value by running this line of code in your hook_install:
variable_set('allow_insecure_uploads', 1);
Then in a module add this function
/**
* Implementation of FileField's hook_file_insert().
*/
function MODULE_NAME_file_insert(&$file) {
//look for files with the extenstion .js.txt and rename them to just .js
if(substr($file->filename, -7) == '.js.txt'){
$file_path = $file->filepath;
$new_file_path = substr($file_path, 0, strlen($file_path)-4);
file_move($file_path, $new_file_path);
$file->filepath = $file_path;
$file->filename = substr($file->filename, 0, strlen($file->filename)-4);
$file->filemime = file_get_mimetype($file->filename);
$file->destination = $file->filepath;
$file->status = FILE_STATUS_TEMPORARY;
drupal_write_record('files', $file);
}
What this does is in the hook_insert call it checks if a file has the extension ".js.txt". If it does it copies it to a new location and renames it. This is after the security check so its ok. I don't think you need to worry about the cache clear deleting your js files as long as you don't put them in the files/js directory. Create your own directory for you module and you should be ok.
I faced this situation when I wanted to allow .js file to be upload as is (without .txt and with 'application/javascript' mimetype) for a specific field. Also, I didn't wanted to alter Drupal core... of course.
So I needed to create a module implementing hook_file_presave(). This also work for Multiupload File Widget, since its hook is on file_save().
Note that you would have to replace MYMODULE_NAME and MYFIELD_NAME by your own values.
function MYMODULE_NAME_file_presave($file) {
// Bypass secure file extension for .js for field_additional_js field only
if((isset($file->source) && strpos($file->source, "MYFIELD_NAME") !== FALSE) && substr($file->filename, strlen($file->filename) - 7) == ".js.txt") {
// Define new uri and save previous
$original_uri = $file->uri;
$new_uri = substr($file->destination, null, -4);
// Alter file object
$file->filemime = 'application/javascript';
$file->filename = substr($file->filename, null, -4);
$file->destination = file_destination($new_uri, FILE_EXISTS_RENAME);
$file->uri = $file->destination;
// Move fil (to remove .txt)
file_unmanaged_move($original_uri, $file->destination);
// Display message that says that
drupal_set_message(t('Security bypassed for .js for this specific field (%f).', array('%f' => $file->filename)));
}
}
Drupal also "munges" javascript files. To prevent Drupal from automatically adding underscores to the filename there is a hidden variable that is checked before the filename is "munged".
Setting the variable to 1 solves the issue for me (along with altering the REGEX in includes/file.inc).
I hate hacking core, but this seems like a poor design to me. Javascript files are not server side scripts like php, py, pl, cgi, and asp.
You can use the allowed file extensions settings to prevent php and other server side scripts from being uploaded.
eg:
variable_set('allow_insecure_uploads', 1);
See:
http://api.drupal.org/api/function/file_munge_filename/6
So uploading .js files to the files directory is pretty much impossible.
Even if you manage to get .js files uploaded cleanly, these files will get deleted when the cache is cleared.
Any js files that live inside the files directory will be deleted whenever the drupal_clear_js_cache() function is executed.
http://api.drupal.org/api/function/drupal_clear_js_cache/6
So Drupal sees .js files living in the file uploads directory as temporary.
Now I understand why they are appending ".txt", it is to prevent them from being removed when the cache is cleared.
So as a compromise I guess I will just be uploading .js files manually (via FTP) to the /misc folder. :(