I've got a site where individual pages might require some javascript or CSS files hooked into their heads. I'm trying to keep everything client side when it comes to managing this process, rather than getting on the FTP and sorting everything out in the code so I need to be able to upload css and js files.
I've got CCK filefield up and running, and it works with css files, but it refuses to upload .js files. It instead seems to view every .js as ".js.txt" and then the file appears on the server as thisismyfile.js.txt
Not ideal...
Does anyone know how to work around this. Is it a mime type problem with Drupal or the server, or is Drupal set up to avoid script uploads and n00b hack attacks.
Once the files are uploaded I intend to use PHP mode on the page or node to call drupal_add_css and drupal_add_js.
Looking at the field_file_save_file() function in field_file.inc from filefield module, you can find the following snippet
// Rename potentially executable files, to help prevent exploits.
if (preg_match('/\.(php|pl|py|cgi|asp|js)$/i', $file->filename) && (substr($file->filename, -4) != '.txt')) {
$file->filemime = 'text/plain';
$file->filepath .= '.txt';
$file->filename .= '.txt';
}
So yes, it's a 'security thing', as Jeremy guessed.
You could patch that RegEx for an immediate 'fix', but that would remove this otherwise useful security check completely for all filefields used on the site.
So a more specific workaround might be a better approach. Since you want to add the files via drupal_add_js() calls from code anyways, you might as well do the renaming there, adding some kind of verification to make sure you can 'trust' the file (e.g. who uploaded it, whatever).
Edit: Concerning options to rename (and alternatives) when calling drupal_add_js():
For renaming the file, look into the file_move() function. A problem with this would be that it won't update the corresponding entry in the files table, so you would have to do that also, if the move operation succeeded. (The filefield just stores the 'fid' of the corresponding entry in the files table, so you'd need to find it there by 'fid' and change the 'filename', 'filepath' and 'filemime' entries according to your rename/move)
Alternatively, you could just load the content of the *.js.txt file and add that string with the 'inline' option of drupal_add_js(). This would be less 'elegant' and could be a performance hit, but if those are not important criteria in your specific case, it is less trouble.
Yet another option would be just passing the *.js.txt file as is to drupal_add_js(), ignoring the 'wrong' extension. A short local test showed that this works (at least in firefox). This might be the 'least effort' solution, but would need some additional testing concerning different browser behavior concerning usage of 'misnamed' js files.
Allowing Drupal to upload javascript files would be a security risk, which is also why it doesn't allow you to do it, but instead appends the .txt extension. The reason is that js files are executable along with php, pl, py, cgi, asp. So if Drupal could upload those files to the server, it would be possible for evil doers to upload a file and run it doing all kinds of nasty things on your server, basically anything is possible. Best thing would be to find a different way of uploading files which are secure.
I had a similar need, and found a way to get around the security by first changing the 'allow_insecure_uploads' variable value by running this line of code in your hook_install:
variable_set('allow_insecure_uploads', 1);
Then in a module add this function
/**
* Implementation of FileField's hook_file_insert().
*/
function MODULE_NAME_file_insert(&$file) {
//look for files with the extenstion .js.txt and rename them to just .js
if(substr($file->filename, -7) == '.js.txt'){
$file_path = $file->filepath;
$new_file_path = substr($file_path, 0, strlen($file_path)-4);
file_move($file_path, $new_file_path);
$file->filepath = $file_path;
$file->filename = substr($file->filename, 0, strlen($file->filename)-4);
$file->filemime = file_get_mimetype($file->filename);
$file->destination = $file->filepath;
$file->status = FILE_STATUS_TEMPORARY;
drupal_write_record('files', $file);
}
What this does is in the hook_insert call it checks if a file has the extension ".js.txt". If it does it copies it to a new location and renames it. This is after the security check so its ok. I don't think you need to worry about the cache clear deleting your js files as long as you don't put them in the files/js directory. Create your own directory for you module and you should be ok.
I faced this situation when I wanted to allow .js file to be upload as is (without .txt and with 'application/javascript' mimetype) for a specific field. Also, I didn't wanted to alter Drupal core... of course.
So I needed to create a module implementing hook_file_presave(). This also work for Multiupload File Widget, since its hook is on file_save().
Note that you would have to replace MYMODULE_NAME and MYFIELD_NAME by your own values.
function MYMODULE_NAME_file_presave($file) {
// Bypass secure file extension for .js for field_additional_js field only
if((isset($file->source) && strpos($file->source, "MYFIELD_NAME") !== FALSE) && substr($file->filename, strlen($file->filename) - 7) == ".js.txt") {
// Define new uri and save previous
$original_uri = $file->uri;
$new_uri = substr($file->destination, null, -4);
// Alter file object
$file->filemime = 'application/javascript';
$file->filename = substr($file->filename, null, -4);
$file->destination = file_destination($new_uri, FILE_EXISTS_RENAME);
$file->uri = $file->destination;
// Move fil (to remove .txt)
file_unmanaged_move($original_uri, $file->destination);
// Display message that says that
drupal_set_message(t('Security bypassed for .js for this specific field (%f).', array('%f' => $file->filename)));
}
}
Drupal also "munges" javascript files. To prevent Drupal from automatically adding underscores to the filename there is a hidden variable that is checked before the filename is "munged".
Setting the variable to 1 solves the issue for me (along with altering the REGEX in includes/file.inc).
I hate hacking core, but this seems like a poor design to me. Javascript files are not server side scripts like php, py, pl, cgi, and asp.
You can use the allowed file extensions settings to prevent php and other server side scripts from being uploaded.
eg:
variable_set('allow_insecure_uploads', 1);
See:
http://api.drupal.org/api/function/file_munge_filename/6
So uploading .js files to the files directory is pretty much impossible.
Even if you manage to get .js files uploaded cleanly, these files will get deleted when the cache is cleared.
Any js files that live inside the files directory will be deleted whenever the drupal_clear_js_cache() function is executed.
http://api.drupal.org/api/function/drupal_clear_js_cache/6
So Drupal sees .js files living in the file uploads directory as temporary.
Now I understand why they are appending ".txt", it is to prevent them from being removed when the cache is cleared.
So as a compromise I guess I will just be uploading .js files manually (via FTP) to the /misc folder. :(
Related
I am fairly new to web development. I am using HTML and Javascript, specifically the Jexcel Javascript library, to create a spreadsheet webpage. The spread sheet is sourced from a CSV file stored in the working directory of my project. The problem is, when I make changes to that CSV file, they do not show up in my project until I rename the file (and change the filepath in my code respectively). I suspect this has to do with the webpage caching the source CSV file, but I am not sure. How would I go about disabling caching/making my table source from the CSV file everytime? I am using Tomcat Web Server as well. Thank you.
The code that is sourcing my spreadsheet is the following:
<script>
$('#spreadsheet').jexcel({
allowInsertRow: false,
allowInsertColumn: false,
allowDeleteRow: false,
allowDeleteColumn: false,
csv : 'data/test.csv',
csvHeaders : true,
defaultColWidth : '200px',
});
</script>
When changes are made to test.csv I want those changes to show the next time the page is refreshed, but this is not currently happening.
Update: Restarting the Web Server does make the changes show up, so I believe this is a caching issue. I am using Java Servelets on the Server Side of my code.
Browser side caching of resources is a common issue.
To prevent it, the easiest way is to append a version hash to resource url to force reload when modified, so you still benefit of cache feature when file hasn't changed.
Without information about the languages you use on server side, it's very difficult to help you more.
Here is the fonction I generally use to generate resources url :
function assetv($path) {
$hash = 'undefined';
if (file_exists($path)) {
$hash = filemtime($path);
if (!$hash) {
$hash = md5_file($path);
}
}
return urlFromPath($path) . "?v=" . $hash;
}
Of course, the "urlFromPath" depends on your app ;-)
[EDIT]
The browser loads your csv file using an url (data/test.csv) that is "used as cache key" (approximately).
So to force reload when file was modified, you need to change the url, for instance with a version number parameter (file name doesn't changes) :
data/test.csv?v=1
data/test.csv?v=2
As server side only is aware of file state, you need to generate the versionned url on server side when building your HTML.
Finally, instead of managing a version number manually, you can use a "natural" attribute of the file, as it's last modification time or a hash of it's content.
Thank you all for the suggestions. Ultimately I found out the problem was due to the fact that the csv file I was sourcing my data from was in the project folder and Tomcat does not update changes to that file immediately. I solved this by reading the data from a file stored outside the working directory on my machine.
I am creating browser based video editing tool. I want a user to first download a ~70mb javascript file and store it somewhere on his computer. I want to link that file when my website is opened. How can I achieve that.
EDIT
What i meant is that there are various files like js1.js,js2.js... all sums upto 70mb . So i will offer a zip folder to download and only link js1 or js2 file etc depending on the effects user wish to apply
i am sorry to inform you but i think there is something really wrong with what you are trying to do.
A "solution" would be to just cache the javascript on the user's browser so any subsequent requests parse the cache instead of requesting the resource again from the server.
You should know however that if you are in need to download ~70mb of a javascript file you are doing something wrong. I have a whole web app project that when published the total size is around 60mb, all files required to properly run included, and its a damn big codebase in there.
I find it very hard to believe there is ever a need for a single javascript file to be that big, in any case maybe a simple caching should do the trick
That is actually done automatically. Once you add a <script> tag with a link to a local js file (also stored on the server) the file is loaded automatically.
See HTML <script> src Attribute for more information on that.
You can only reference to js files on the server. Files on the server could look like this:
index.html
somefancyjsfile.js
You can then reference from inside your html file to the js file via the <script> tag.
I'm not sure though if the size is not a bit too much...
I am trying to make a compiler in which users make code at abc.com/newProject and their output is in an iframe, that iframe need to be served files that are made at the abc.com/newProject. So I will be doing a POST of JSON obj at abc.com/compile-project that will create files and those will be used by the iframe, after being used those should get deleted. Files are basically JS files that iframe will fetch using script in header.
So a pseudo-code will look something like this:-
app.post('/compile-project', function(req, res){
//Directory created using node tmp
//files created in the directory
//These files are accessible using <script src="/js/file1.js"></script>
//when the current connection requests the files they get deleted
});
Any help will be appreciated thanks.....
I am trying to make a compiler [...] Any help will be appreciated thanks.....
I would strongly discourage you from doing that if you don't know what you're doing (and considering the fact that you're asking how to save a file then apparently you don't).
The requirements that you described are extremely simple but you need to have much deeper understanding of everything that's going on to avoid serious security problems that you will encounter with no doubt along the way.
What you describes can be done without even using a file system, since all your files are served only once so it doesn't make much sense to store them in actual files. But even if you insist on the file system then all you need is to use fs.mkdtemp to create a temporary directory, use something like the uuid module for unique IDs to use in the filenames, then use fs.writeFile to write a file. This is all you need for the file upload endpoint. Now in the download endpoint all you need is to use fs.readFile to read the file and fs.unlink to remove it. That's it.
Now, it will surely get you into trouble of failures on browser reloads, back button not working, and finally security issues of people being able to serve any random code from your servers leading to vulnerabilities too numerous to even list here.
Take a look at the source code of repl.it and JS Bin on GitHub:
https://github.com/replit/repl.it
https://github.com/jsbin/jsbin
to appreciate the scope of the project that you are willing to undertake.
I've been looking around for quite some time now and I cant seem to find a way to get a list of subfolders from a specific directory.
An example would be, if I'm at www.mysite.com/projects and inside projects there are several folders containing individual project files.
the reason I want to do this is I was going to make a script that would add new project's names to a menu using the sub folder names.
Am I missing something ? Is this even possible with JQuery or JavaScript ?
I've gone as far as getting pathnames and locations and also had a look at ActiveXObjects but cant get anything to work on either my PC or on the server.
Any help would be appreciated.
There is no such thing as a directory in HTTP. Only resources.
Some of those resources might be an HTML document that lists some other resources (which are in a particular directory on a file system on a computer running the HTTP server). Most HTTP servers will generate such documents for you automatically.
You need to have your server generate a suitable response for a suitable request. Then use (since you mention jQuery) the ajax() method to make that request.
Then you need to parse the response. You can either use the default directory index page and then parse the HTML returned, or you can write a server side program to generate the data in a nicer format (such as JSON).
That said…
the reason I want to do this is I was going to make a script that would add new project's names to a menu using the sub folder names.
You would almost certainly be better off doing that on the server. You'll get more reliable, faster and search engine friendly results.
ActiveX is a technology enabling JScript (the Microsoft implementation of JavaScript) to have more access to the clients computer and it only works on Internet Explorer.
Folders on the server are not like folders in a filesystem. Any folder/subfolder has the potential to contain an index.html which outputs some text (not necessarily the list of subfolders it contains).
Also most webserver configurations have an active options of not showing the subfolder list even if there is not index.html present.
What you can do is place an index.php file in that folder with the following code:
<?php
$directories = scandir('.');
header('Content-Type: application/json');
echo json_encode($directories);
And you can receive this content as such:
$.getJSON('http://domain.com/path/to/folder/', function(directories) {
do_something(directories);//
});
If a user tries to drag and drop a folder to my file uploader control for uploading it, then I need to show an error message to the user saying that only files can be uploaded. The problem is I couldn't distinguish a file from a folder.
One way I thought was to check for file type property of jQuery. Supposing that the name of the file is "test.txt", then file type would return "text/plain". For a normal folder name such as "TestFolder", file type would be blank and its file size would return 0. However if the folder name included an extension like "TestFolder.txt", then file type would return "text/plain".
So I could have checked for file type and file size but it would not work correctly for folder name like "TestFolder.txt". Could any one suggest me a good solution to fix this using jQuery or other methods?
The ability to determine if a folder has been dropped is dependent on the user agent's support of the Filesystem API. If the user agent DOES support the Filesystem API (currently only Chrome 21+), you can make use of the Entry interface, which has two child interfaces: DirectoryEntry and FileEntry. The interface itself has isDirectory and isFile functions. Without support for this interface, there is no way to determine if dropped items are folders when examining the DataTransfer object.
Since you are not going to allow folder to drag and drop, first check if it's folder or file, second only if it's not folder then check for file extension. But IE < 11 does not support file API to handle. Hope it helps.
As far as I'm aware the browser (And javascript inherently) doesn't have access to any file access methodologies for security purposes so you cannot check what the file actually is.
I have worked with this problem myself in the past and the best option I have found that works is to handle the file server-side, and return the error back to the page after upload has completed.
I would be happy to see better alternatives myself though.