I'm trying to understand how to deal(in a secure way) with Path Traversal.
For example an application receives from a client a file name via REST API in JSON, look for it in the non-accessible(by outside) directory and retrieve a response with the file:
app.get('/', (req, res) => {
const fileName = req.body.fileName;
// some code...
fs.stat(`./nonAccessibleDir/${fileName}`, async function(err, stat) {
// some code...
});
// some code...
}
The problem with the above approach is that a client can send something like "../" in the fileName request and it will "eat" it without an issue. How can one deal with this kind of scenarios, what and how I should fix this vulnerability, please?
Update:
Sorry, but I forgot to mention that I know I should check the input I receive, but what if I need to pass the "/" and "." in the input? Also, if I don't need this characters, is that all I need to check to remove the Path Traversal vulnerability?
An easy way would be to validate the fileName through a regex that detects any ../ segments and returns an error if any are present.
if (fileName.match(/\.\.\//g) !== null) {
// return an api error
}
You could have quite a tight validation rule that prevents any forward slashes in fileName at all, making it only possible to point to a file directly in your desired directory.
Related
I'm doing a link shortener. The link shortener works very well but now I'm trying to do API for it. The problem is that if I pass the URL argument in the get URL it's not working. I tried a lot of things but it's not working. When I do like http://localhost:3500/api/create/google.com it works but when I do http://localhost:3500/api/create/https://google.com it's not working because of the https://. These are the last 3 inputs via my API that failed: http://https:google.com, google.com, http://
I'm using express and mongoose. Here's my code:
app.get('/api/create/:shortUrl(*)', async (req, res, next) => {
if (req.params.shortUrl.includes("https://") || req.params.shortUrl.includes("http://") || req.params.shortUrl.includes("/")) {
req.params.shortUrl = req.params.shortUrl.replace("https://", "").replace("http://", "").replace("/", "")
}
if (req.params.shortUrl == "") {
res.send("invalid URL")
}
await shorturl.create({full: `http://${req.params.shortUrl}`})
const shortUrls = await shorturl.find().sort({_id: -1}).limit(-1)
const latest = shortUrls[0]
res.send("https://p33t.link/" + latest.short)
});
You have to properly encode portions of the URL that contain restricted characters such as : and // that aren't part of the actual protocol to make it a legal URL. So, the idea is that you encode the "parameter" before appending it to the URL. Presumably, you would use encodeURIComponent(), depending upon exactly where you're placing it in the URL.
After parsing the core part of the URL, a web server will decode the remaining components of the URL and give you back the original characters. I would suggest that your particular use would probably work better as a query parameter rather than a part of the path which would give you this when properly encoded:
http://localhost:3500/api/create?u=https%3A%2F%2Fgoogle.com
And, you could then use:
app.get('/api/create', (req, res) => {
console.log(req.query.u);
...
});
I have the following route in my express application :
app.get('/api/:URI', (req, res) => {
doStuff();
}
The URI parameter passed is an URI encoded on the client side with encodeURIComponent()
It works fine except when the URI contains a hash.
Example: http://foo.bar/foobar/bla#blabla-313fe4ce-4f8d-48b7-b0f3-a59844402ee8
In this case the route is ignored.
On the browser side I receive a code 301, then the result of the next valid route.
If I remove the hash or, weirder, if I disable the cache on the browser side it works perfectly.
Is there any way express can ignore the hash ?
Edit : It's absolutely not a Can I use an at symbol (#) inside URLs? duplicate, the question is more about express routing and/or about browsers cache issues than about allowed characters in an URL.
Is there any way express can ignore the hash ?
I tried using the OR operator. For example,
app.get('/blog' || '/blog#top', (request, response) => {
...
});
So it works even if #top is present or not.
app.get('/images/:filename', (req, res, next) => {
console.log(req.params.filename);
const newUrl = `/static/images/${req.params.filename}`;
req.url = newUrl;
next();
});
Above code works fine if filename does not contain a special character .
For example localhost/images/myImage.jpg will print out myImage.jpg.
If filename contains a . such as localhost/images/my.Image.jpg, console will not print out anything. Is there a way to fix this?
Edit: Added code below. I'm not sure what it does.
app.get(
/^.+\.(jpg|jpeg|gif|png|bmp|ico)$/,
(req, res) => {
res.status(404).end();
}
);
Edit 2: My bad! Like someone's comment, I found the problem was actually from Service Worker. It was apparently messing with caching.
Could somebody explain what ^.+\.$ does in the second piece of code above? I'll mark the answer for that one instead.
I assume it's express.js? If so, :parms is not the best candidate to pass complex values (e.g. something more complex that simple alphaliteral, ehm, params). You could try to pass it to req.query isntead of params and properly escape it client-side and unescape server-side
Please refer to this documentation:
Serving static files in Express
I need to get number of items of specific directory in Node.js
If I get the items like
var dirItems = fs.readdirSync(__dirname+'/my_dir');
and then get specific item like
dirItems[1]
everything is ok
But if I try to get their number like
dirItems.length
or
Object.keys(dirItems).length
the page doesn't work in the browser
How to get the number of directory items?
UPDATED
My full code:
var http = require('http'),
fs = require('fs');
http.createServer(function(req, res) {
var dirItems = fs.readdirSync(__dirname+'/my_dir');
res.writeHead(200, {'Content-Type': 'text/html'});
res.end(dirItems.length);
}).listen(80, 'localhost');
I was able to reproduce the error nyou get.
res.end() for the basic http server class is very picky about what you send it. You must give it a string (the error you got should have been a big clue here).
So, change this:
res.end(dirItems.length);
to this:
res.end(dirItems.length.toString());
And, it works for me. I was able to reproduce your original error and then make it work by making this simple change.
Logically, you can only send string data as an http response so apparently res.end() isn't smart enough to attempt a string conversion on its own. You have to do it yourself.
FYI, if you use a higher level framework like Express, it is more tolerant of what you send it (it will attempt a string conversion in a situation like this).
Here is how I would do it:
const fs = require('fs');
const dir = './somedir';
fs.readdir(dir, (err, files) => {
console.log(files.length);
});
I can't seem to pass a whole url e.g. "http://example.heroku.com/http://www.facebook.com"
app.get('/:url', function(req, res){
var url = req.params.url;
// do something with url...
}
I always get an error that says "Cannot GET /http://www.facebook.com".
How do I get past this?
Some characters (like /) have special meaning in URLs and need to be encoded.
http://example.heroku.com/http%3A%2F%2Fwww.facebook.com
Most programming languages have a function (possibly via a third party library) which can encode that for you. In JavaScript, for instance, that is encodeURIComponent.
You can use regular expression. For example:
// http://localhost:3000/mountpoint/http://www.facebook.com
app.get( /^\/mountpoint\/(.*)/, function(req, res) {
var url = req.params[0];
res.json(url);
});
Thanks for the comments, answers but I've found out I could use the wildcard and I was able to get the whole URL parameter without running into the 'Cannot GET /' error
'/*'