The client is supposed to see just a directory and its contents on the server (FS_ROOT).
And the server is supposed to convert the paths that it receives from the client to real paths that exist and do the file operations that the client requested on them:
I made these 2 functions to handle that and I want to ask if they are secure enough. I mean there should be no way for the client to fool the server to do something outside FS_ROOT
function fromVirtualPath(virtPath){
if(virtPath === '/' || virtPath === '.')
return FS_ROOT;
virtPath = virtPath.trim();
if(virtPath[0] === '/')
virtPath = virtPath.substr(1);
const absPath = path.resolve(FS_ROOT, virtPath);
if(absPath.indexOf(FS_ROOT) !== 0)
throw new Error('Outside root dir - no permissions!');
return absPath;
}
function toVirtualPath(absPath){
return '/' + path.relative(FS_ROOT, absPath);
}
Example real path: /www/site.com/public_html/yo
Client should see: /yo
About fromVirtualPath I would simply move the line virtPath = virtPath.trim(); to be the first line of the function, then it's ok.
If the values passed to toVirtualPath are always return values of fromVirtualPath, yes it is secure enough; other wise we could check if the value is a good absPath.
function fromVirtualPath(virtPath) {
virtPath = virtPath.trim();
if (virtPath === '/' || virtPath === '.')
return FS_ROOT;
if (virtPath[0] === '/')
virtPath = virtPath.substr(1);
const absPath = path.resolve(FS_ROOT, virtPath);
if (absPath.indexOf(FS_ROOT) !== 0)
throw new Error('Outside root dir - no permissions!');
return absPath;
}
function toVirtualPath(absPath) {
if (absPath.indexOf(FS_ROOT) !== 0)
throw new Error('Bad absolute path!');
return '/' + path.relative(FS_ROOT, absPath);
}
Your code is a bit insecure until you make use of the techniques provided by NODE.JS in the mentioned article. Try implementing the following code,
function fromVirtualPath(virtPath) {
virtPath = virtPath.trim();
if (virtPath === '/' || virtPath === '.')
return FS_ROOT;
if (virtPath.indexOf('\0') !== -1)
throw new Error('That was evil.');
const absPath = path.join(FS_ROOT, virtPath);
if (absPath.indexOf(FS_ROOT) !== 0)
throw new Error('Outside root dir - no permissions!');
return absPath;
}
function toVirtualPath(absPath) {
return '/' + path.relative(FS_ROOT, absPath);
}
The following article from NODE.JS will be really helpful to you.
"How can I secure my code?"
Poison Null Bytes
Poison null bytes are a way to trick your code into seeing another
filename than the one that will actually be opened.
if (filename.indexOf('\0') !== -1) {
return respond('That was evil.');
}
Preventing Directory Traversal
This example assumes that you already checked the
userSuppliedFilename variable as described in the "Poison Null
Bytes" section above.
var rootDirectory = '/var/www/'; // this is your FS_ROOT
Make sure that you have a slash at the end of the allowed folders name
you don't want people to be able to access /var/www-secret/, do you?.
var path = require('path');
var filename = path.join(rootDirectory, userSuppliedFilename);
Now filename contains an absolute path and doesn't contain ..
sequences anymore - path.join takes care of that. However, it might
be something like /etc/passwd now, so you have to check whether it
starts with the rootDirectory:
if (filename.indexOf(rootDirectory) !== 0) {
return respond('trying to sneak out of the web root?');
}
Now the filename variable should contain the name of a file or
directory that's inside the allowed directory (unless it doesn't
exist).
Security is a complex matter. And you can never be sure.
Despite the fact that I couldn't find any flows in #RahulVerma answer I'll add my 2 cents...
The link that #RahulVerma posted is official but not a documentation per se. And in the documentation there is nothing about Poison Null Bytes ...strange isn't it.
And that makes you think: maybe, just maybe, when the fs and/or path modules were written authors didn't put enough effort into security considerations, or just missed that. Yes, maybe there are some good reasons for you and not the fs/path to handle the \0. But also wouldn't it be better if everyone was protected from \0 by default? And only for some rear occasions you could explicitly set an option to allow \0 in paths.
So... what am I trying to say is: security is hard even for the best of us, and without proper peer review (currently, less than 100 views on this question do not strike me as a "proper peer review") or, better yet, a history of successful time in production, you should not be satisfied with these answers (my included) saying "It's OK, if you add this or that".
Why don't you use some code that already was tested in battles instead of trying to write a secure code by yourself?
E.g serve-static is used in Express.
(Probably it doesn't meet your needs - it's static after all, but you get the idea)
Even if you don't want another dependency in your project you can at least study and copy from the implementation that proved itself. (But, yes, it doesn't seem different from the #RahulVerma answer)
That said. I'd like to point out that:
If you'd copy the implementation, you can make a mistake while doing so.
Even if your code is safe, consider how safe do you manage your code. Will it be safe tomorrow?
Even well tested libraries and engines can, and often do, have bugs, and fall prey to 0day exploits
Oh! Just found: https://github.com/autovance/ftp-srv/issues/167
It's about the library that was suggested in another question of yours.
So, if you decide (or if you'll be assured) that now you code is surely safe don't stop on that! Add an extra layer of security to it anyway:
restrict the server's access to folders outside of the /www/site.com/public_html/ on an OS level.
The following principles can be applied to secure client access to paths relative to the web root:
Restrict access outside of your public web root folder to your
service. Rationale: begin with ZERO trust.
Split the path provided by the user into parts. This will remove leading '/' and all '/' separators leaving only the parts of the path. Better yet, use whitelisting for path parts to restrict acceptable characters in a path part using a regular expression. Rationale: sanitize user input
Validate each part sequentially for existence assuming that the first part starts from the web root as it is intended. Disallow .. (parent dir) in part names (to prevent traversal outside the web root folder). Rationale: sanitize user input and validate user input
Avoid using symbolic links under the web root folder (to prevent
traversal outside the web root folder). Rationale: reduce attack surface
Fail early with an error upon encountering the first invalid part. Rationale: reduce attack surface
To optimize system calls, you can do the check for .. and part whitelisting in one pass. If there are any .. in the path or offending parts, return an error. Otherwise, split the parts and rebuild the absolute path string by concatenating them with your web root and do one existence check instead of multiple folder existence checks along the path.
Instead of trying to validate every path yourself, let the operating system do it for you! This is a good example of an application that could use a chroot.
Here is an example of an npm library which creates a chroot.
> var chroot = require("chroot")
> var fs = require("fs")
> chroot('/virtual/root/here', 'nobody')
> fs.readdir(".", function(err, files) { console.log(files); }) // Lists virtual root
> fs.readdir("..", function(err, files) { console.log(files); }) // Also lists virtual root
> fs.readdir("/", function(err, files) { console.log(files); }) // ALSO lists virtual root
Should you run this script as root, it immediately changes the user to "nobody" and sandboxes you to your virtual root. This prevents the script from accessing anything outside it, and the program can't chroot out either, as it's no longer running as root.
Now that you are chrooted into your virtual root, using "/" will give you a directory listing of your virtual root - essentially, you can use your virtual path directly in fs.readdir()!
Need to access some specific files outside the new root? Use microservices! You can run a node.js instance in the background as your file accessor, and communicate between your main server and your file accessor. Having two nodejs instances not only allows your background task to sandbox itself, but also allows you to make use of multithreading.
Yours is a basic java code. In real time scenarios, these basic java code should not be deployed on server side and we can't expect
secuirty out of this.
To add the security check to this java code, many APIs come as part of Spring framework but since we are writing java code then we can
make use of java NIO package only, API name WatchService and WatchEvent
class DirectoryWatchTest {
public static void main(String[] args) {
try {
WatchService watchService = FileSystems.getDefault().newWatchService();
Path path = Paths.get("C:/");
/**
* The register() method of the Path class takes a WatchService object and an event type for which the
* application needs to get notified.
*
* The supported event types are:
* ENTRY_CREATE: indicates if a directory or file is created.
* ENTRY_DELETE: indicates if a directory or file is deleted.
* ENTRY_MODIFY: indicates if a directory or file is modified.
* OVERFLOW: indicates if the event might have been lost or discarded. This event is always implicitly
* registered so we don't need to explicitly specify it in the register() method. */
path.register(watchService, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
while (true) {
WatchKey key;
try {
key = watchService.take();
} catch (InterruptedException ex) {
return;
}
/**
* The whole work flow:
* A Watchable object is registered with a watch service by invoking its register method,
* returning a WatchKey to represent the registration.
*
* When an event for an object is detected, the key is signalled, and if not currently signalled,
* it is queued to the watch service so that it can be retrieved by consumers that invoke the poll or
* take methods to retrieve keys and process events.
*
* pollEvents List<WatchEvent<?>> pollEvents() method retrieves and removes all pending events for
* this watch key, returning a List of the events that were retrieved. Note that this method does not
* wait if there are no events pending. */
for (WatchEvent<?> event : key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
#SuppressWarnings("unchecked")
WatchEvent<Path> ev = (WatchEvent<Path>) event;
Path fileName = ev.context();
System.out.println(kind.name() + ": " + fileName);
if (kind == ENTRY_MODIFY && fileName.toString().equals("DirectoryWatchTest.java")) {
System.out.println("My source file has changed!!!");
System.out.println("My source file has changed!!! - Modified");
}
}
/**Once the events have been processed the consumer invokes the key's reset method to reset the
* key which allows the key to be signalled and re-queued with further events.*/
boolean valid = key.reset();
if (!valid) {
break;
}
}
} catch (IOException ex) {
System.err.println(ex);
}
}
}
This kind of basic security check can be put in java code. The user will be able to watch the url unless and until we don't get
hold of protocol and hide it via #PutMapping or implementing security based API's in this but for that we need framework based API's
enter code here
Related
The short story:
Is there simple way to normalize file path in JavaScript, like in Java we have Paths.get("/a/b/../c").normalize() so /a/../../c would become /c. I seen many same questions here, but they are Node.js solutions, and I need pure JS or JQuery approach so it can be used in browser.
The long story:
I have a file server with web UI, that allows to browse files and download them. UI is written in spring and accessible at mysite.com/ui/
The file storage located at mysite.com/files/ which is plain Apache directory, so its possible to get direct link to file.
The real storage directory on server is /var/www/files
Path passing to back-end as mysite.com/ui/?path=/a/../../c, so back-end will normalize path variable separately to /c and then append it to base dir and so retrieving content of /home/storage/c, so it works perfectly.
The problem comes when user tries to download file like this with direct link. I.e. if user tries to download /a/../../c/d file from file server root, it appending to base storage url, which mysite.com/files/, and it becomes mysite.com/files/a/../../c/d so it will point to /var/www/d instead of /var/www/files/d so file can't be downloaded even if it is visible from web UI.
So I need to normalize relative file path first on front-end like on back-end when retrieving content of directory but I don't know how it can be done in JS. Is there function for it, or I have to write my own one?
So I ended up writing my own function on JS. It might be not what "path normalization" stands for, but anyway its exactly what I need
function normalizePath(path) {
// remove multiple slashes
path = path.replace(/\/+/g, '/');
// remove leading slash, will be added further
if (path.startsWith("/"))
path = path.substring(1)
// remove trailing slash
if (path.endsWith("/"))
path = path.slice(0, -1);
let segments = path.split("/");
let normalizedPath = "/";
for (let segmentIndex = 0; segmentIndex < segments.length; segmentIndex++) {
if (segments[segmentIndex] === "." || segments[segmentIndex] === "") {
// skip single dots and empty segments
continue;
}
if (segments[segmentIndex] === "..") {
// go up one level if possible
normalizedPath = normalizedPath.substring(0, normalizedPath.lastIndexOf("/") + 1);
continue;
}
// append path segment
if (!normalizedPath.endsWith("/"))
normalizedPath = normalizedPath + "/"
normalizedPath = normalizedPath + segments[segmentIndex];
}
return normalizedPath;
}
Still, I won't mark this as accepted answer as it's more of a quick fix, and I'm not JS expert so it definitely must be more elegant solution.
I am creating a Node.JS app that allows users to edit various documents. A sub-directory is created on the server for each user using their user ID for the sub-directory name. I don't want to use a database at this time because I am creating a prototype on a tight deadline so I'm using a file based system for now to get things done quickly.
Note, users access the system from a web browser from one of my web pages.
When a user creates a document they specify the document name. Right now, on the server side the document name is sanitized of any characters that are not supported by the operating system (Linux), but that's it. My concern is that a user might try to access a directory that doesn't belong to them using relative path components inserted into the document name, in an attempt to "walk up and down" the directory tree to break out of the sub-directory reserved from them.
I have read several horror stories of users figuring out clever ways to do this via exotic UTF-8 codes, etc., so I'm looking for a Node.JS code sample or library that has a function that eliminates all relative path elements from a file name in a thorough and robust way.
How can I, on the server side, make absolutely sure that a user created document name submitted in a POST from the browser is a primary file name and nothing else?
I'm using express, I'm not sure if this is bulletproof but it seems to be doing what I need it to (making sure that a file exists before processing the request)
const fs = require('fs');
const path = require('path');
checkUserSuppliedFilenameExists(req, res) {
if(!req.body.fileName) {
return res.status(400).json({ message: "missing required parameters" });
}
/**
* strip any potentially harmful stuff from the start of the string
* will return the following:
* "foo.txt" => "foo.txt"
* "foo" => "foo"
* "../../foo.txt" => "foo.txt"
* "/etc/foo.txt" => "foo.txt"
* undefined => TypeError
* null => TypeError
*/
let suppliedFilename = path.basename(req.body.fileName);
// build the path to the file in the expected directory
// using __dirname to build relatively from the script currently executing
let filePath = path.resolve(__dirname, `../my-specified-directory/${suppliedFilename}`);
// check if it exists
if(!fs.existsSync( filePath ) ) {
return res.status(400).json({ message: "file doesn't exist stoopid" });
}
}
I'm developing some webGL project in Unity that has to load some external images from a directory, it runs all fine in the editor, however when I build it, it throws a Directory Not Found exception in web console. I am putting the images in Assets/StreamingAssets folder, that will become StreamingAssets folder in the built project (at root, same as index.html). Images are located there, yet browser still complains about not being able to find that directory. (I'm opening it on my own computer, no running web server)
I guess I'm missing something very obvious, but it seems like I could use some help, I've just started learning unity a week ago, and I'm not that great with C# or JavaScript (I'm trying to get better...) Is this somehow related to some javascript security issues?
Could someone please point me in the right direction, how I should be reading images(no writing need to be done) in Unity WebGL?
string appPath = Application.dataPath;
string[] filePaths = Directory.GetFiles(appPath, "*.jpg");
According to unity3d.com in webGL builds everything except threading and reflection is supported, so IO should be working - or so I thought:S
I was working around a bit and now I'm trying to load a text file containing the paths of the images (separated by ';'):
TextAsset ta = Resources.Load<TextAsset>("texManifest");
string[] lines = ta.text.Split(';');
Then I convert all lines to proper path, and add them to a list:
string temp = Application.streamingAssetsPath + "/textures/" + s;
filePaths.Add(temp);
Debug.Log tells me it looks like this:
file://////Downloads/FurnitureDresser/build/StreamingAssets/textures/79.jpg
So that seems to be allright except for all those slashes (That looks a bit odd to me)
And finally create the texture:
WWW www = new WWW("file://" + filePaths[i]);
yield return www;
Texture2D new_texture = new Texture2D(120, 80);
www.LoadImageIntoTexture(new_texture);
And around this last part (unsure: webgl projects does not seem easily debuggable) it tells me: NS_ERROR_DOM_BAD_URI: Access to restricted URI denied
Can someone please enlighten me what is happening? And most of all, what would be proper to solution to create a directory from where I can load images during runtime?
I realise this question is now a couple of years old, but, since this still appears to be commonly asked question, here is one solution (sorry, the code is C# but I am guessing the javascript implementation is similar). Basically you need to use UnityWebRequest and Coroutines to access a file from the StreamingAssets folder.
1) Create a new Loading scene (which does nothing but query the files; you could have it display some status text or a progress bar to let the user knows what is happening).
2) Add a script called Loader to the Main Camera in the Loading scene.
3) In the Loader script, add a variable to indicate whether the asset has been read successfully:
private bool isAssetRead;
4) In the Start() method of the Loading script:
void Start ()
{
// if webGL, this will be something like "http://..."
string assetPath = Application.streamingAssetsPath;
bool isWebGl = assetPath.Contains("://") ||
assetPath.Contains(":///");
try
{
if (isWebGl)
{
StartCoroutine(
SendRequest(
Path.Combine(
assetPath, "myAsset")));
}
else // desktop app
{
// do whatever you need is app is not WebGL
}
}
catch
{
// handle failure
}
}
5) In the Update() method of the Loading script:
void Update ()
{
// check to see if asset has been successfully read yet
if (isAssetRead)
{
// once asset is successfully read,
// load the next screen (e.g. main menu or gameplay)
SceneManager.LoadScene("NextScene");
}
// need to consider what happens if
// asset fails to be read for some reason
}
6) In the SendRequest() method of the Loading script:
private IEnumerator SendRequest(string url)
{
using (UnityWebRequest request = UnityWebRequest.Get(url))
{
yield return request.SendWebRequest();
if (request.isNetworkError || request.isHttpError)
{
// handle failure
}
else
{
try
{
// entire file is returned via downloadHandler
//string fileContents = request.downloadHandler.text;
// or
//byte[] fileContents = request.downloadHandler.data;
// do whatever you need to do with the file contents
if (loadAsset(fileContents))
isAssetRead = true;
}
catch (Exception x)
{
// handle failure
}
}
}
}
Put your image in the Resources folder and use Resources.Load to open the file and use it.
For example:
Texture2D texture = Resources.Load("images/Texture") as Texture2D;
if (texture != null)
{
GetComponent<Renderer>().material.mainTexture = texture;
}
The directory listing and file APIs are not available in webgl builds.
Basically no low level IO operations are supported.
How can I tell if a file-system path is a hard link with Node.js? The function fs.lstat gives a stats object that, when given a hard link will return true for stats.isDirectory() and stats.isFile() respectively. fs.lstat doesn't offer up anything to note the difference between a normal file or directory and a linked one.
If my understanding of how linking (ln) works is correct, then a linked file points to the same place on the disk as the original file. This would mean that both the original and linked version are identical, and there is no way to tell the difference between the original file and the linked.
The functionality I'm looking for is as follows:
This is hypothetical pseudo-code for demonstration & communication purposes.
fs.writeFileSync('./file.txt', 'hello world')
fs.linkSync('./file.txt', './link.txt')
fs.isLinkSync('./file.txt') // => false
fs.isLinkSync('./link.txt') // => true
fs.linkChildrenSync('./file.txt') // => ['./link.txt']
fs.linkChildrenSync('./link.txt') // => []
fs.linkParentSync('./link.txt') // => './file.txt'
fs.linkParentSync('./file.txt') // => null
Alright.. just for fun...
You may have an option for finding the files via inode in a certain directory.
Once you grab the inode ID from the stat object..
fs.stat('./okay.file', function(err, stats){
var inodeID = stats.ino; // Double check that this is correct
});
You can then iterate over all the files in the folder and check with a conditional if the inode ID matches. Get all files in a directory. If it doesn't, you can assume there is no link (IN that current directory).
However, it doesn't look like we could search for a file by the inode id. see: nodejs open nfs files by inode (or a the fastest way to reopen a file)
fs.lstat: https://nodejs.org/api/fs.html#fs_fs_lstat_path_callback
Stats object: https://nodejs.org/api/fs.html#fs_class_fs_stats
Sorry but that is not possible you can't differ between original and hard linked file. They are the same on your linux system and poinzing to the same inode.
I have a Win32-DLL (C++) which is loaded as a plugin in another application. The DLL starts a nw.js instance (ShellExecuteEx and SEE_MASK_NOCLOSEPROCESS) and ends it at DLL unloading (by the hInstance of ShellExecuteEx). I need a way to send a string (plain ansi) to the nw-process and retrieve an answer (also string). The old way was a simple http-request with the response in the body. But the environment changes during the development, the "package" app-dll-nw runs multiple times by the same user and multiple users run on the same machine (terminal server). So port listing is "impossible" (yeah random ports or singleton nw, but no).
I found different ways:
socket - port listing problem
wm_copydata/wm_... - need a custom nw-plugin with hidden window (no native nw way); no request-response-system
RPC - port listing problem
DDE - no native javascript way (found a module, which uses .net); In my old delphi days DDE was a not so simple task and it failed multiple times with no logic.
shared memory - no experience; expectations: asynchronous, trigger?, no native javascript way
shared file - no experience; expectations: asynchronous, trigger (watcher on file change) but problems with synchronization, native js way possible
named pipe - no experience; expectations: win32-api and like a chat system (in-pipe [send broadcast] and out-pipe [receive broadcast], or both in one)? If yes, I can use one name about all instances and use unique identifiers and wait for the right answer.
What is a nice and simple way to communicate like the http-way but w/o networking?
Update 1: The node module "net" is able to create a server for a named pipe. The first test, sending a string from the dll to nw, was successful.
var server = net.createServer(function(stream) {
stream.on('data', function(c) {
console.log('data:', c.toString());
});
stream.on('end', function() {
//server.close();
});
});
server.listen('\\\\.\\pipe\\MyAppDynamicGUID');
Update 2 - My Solution
With named pipe and a simplified version of https://msdn.microsoft.com/en-us/library/windows/desktop/aa365592(v=vs.85).aspx I found a working methode.
Server in nw.js:
var server = net.createServer(function(req) {
req.on('data', function(c) {
console.log(c.toString());
req.write('123|Hello World', 'ascii');
});
});
server.listen('\\\\.\\pipe\\MyAppDynamicGUID');
The client in C++ (no permanent connection, strange string handling, simplified error handling):
static std::string PipenameA = "\\\\.\\pipe\\MyAppDynamicGUID";
#define BUFSIZE 512
std::string SendPipeRequestA(std::string sRequest) {
DWORD dwToWrite, dwWritten, dwRead;
BOOL bSuccess;
char chBuf[BUFSIZE];
std::vector<char> buffer;
HANDLE hPipe = CreateFileA(PipenameA.c_str(), GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, NULL);
if (hPipe == INVALID_HANDLE_VALUE)
return "-1|Pipe-Error 1 (connect)";
dwToWrite = (lstrlenA(sRequest.c_str())+1)*sizeof(char);
bSuccess = WriteFile(hPipe, sRequest.c_str(), dwToWrite, &dwWritten, NULL);
if (!bSuccess)
return "-1|Pipe-Error 2 (write)";
do {
bSuccess = ReadFile(hPipe, chBuf, BUFSIZE*sizeof(char), &dwRead, NULL);
if (!bSuccess && GetLastError() != ERROR_MORE_DATA)
break;
buffer.insert(buffer.end(), chBuf, chBuf + dwRead);
} while (!bSuccess);
std::string sResponse(&buffer[0]);
CloseHandle(hPipe);
return sResponse.c_str();
}
// Jonny
The answers you will get will be opinion based, be aware of that.
you can inject the data into the JS module as command line argument
for example
start nw.js MyData
and get it insinde the JS with process.argv.
now, sending the data back to the C++ executables/DLLs is a bit tricky.
if you shell-execute the process, you can have the handle to it.
you can print the data into the stdout from the JS part , and read it in the native app by getting the STDOUT handle from the process handle.
Register your nw.js app with a custom url should be an elegant way.
Such as "github://", "thunder://", "twitter://"
On windows you may have a look at:
https://msdn.microsoft.com/en-us/library/aa767914(v=vs.85).aspx
With custom url you can take simple arguments to nw.js at single-instance mode. See:
https://github.com/nwjs/nw.js/wiki/Handling-files-and-arguments#open-file-with-existing-app
If more data required maybe base64 can help, or even more by LZ-String compress method.