Can you please explain in detail what is happening in these lines of code?
Why we are storing a function in the object with a callback?
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, '/tmp/my-uploads')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
var upload = multer({ storage: storage })
The multer library is going to call the destination function when it wants to know what directory you want to put the uploads into. It will call the filename function when it wants to now what filename you want to give a particular upload.
The reason both of these are functions is so that you have the option of examining the incoming request to compute a directory or filename that is appropriate for that specific request. For example, if you wanted to store all files uploaded by one particular user in a directory for that specific user, you might look at the req.session object to determine which user it is making the upload and then you would compute the appropriate directory for that user.
The destination option may be provided as a plain string instead of a function if the value is known and is constant for all requests. For example, you could do this:
var storage = multer.diskStorage({
destination: '/tmp/my-uploads',
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
The reason that you can communicate the final result via a callback rather than just a plain return value is in case you need to do some asynchronous operation (like look something up in a database) in order to compute what the directory or filename should be. Communicating the result back by calling their supplied cb(...) function allows you do asynchronous operations and communicate back the result asynchronously.
Related
im having problems to understand this function structure, I would thank a lot a simple explanation. To put into context Im creating a storageEngine in a const called storage, that is what multer library uses to save files from a web application. But what I don`t understand is why the function diskStorage receives like an object with two atributes (destination and filename), which in turn are asigned a function() that calls a cb(req, file, cb) that finally gives the value that is required. I mean I know what destination and filename will be used for in multer, but what i dont understand is how it is created here, the JS structure.
const storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, "public/assets");
},
filename: function (req, file, cb) {
cb(null, file.originalname);
},
});
const upload = multer({ storage });
I have a problem in that sometimes my form (submitted in a standard way via the Node.Js app) takes too long to be processed.
On the backend (app.js) the logic looks like this:
var multer = require('multer')
var storage = multer.diskStorage({
destination: function(req, file, cb) {
cb(null, './temp')
},
filename: function(req, file, cb) {
cb(null, file.originalname)
},
})
var upload = multer({
storage: storage
})
app.post(
'/import',
upload.array('uploadedFiles', 50),
imports.submit
)
So what happens then is that the POST request happens again (probably after not receiving any response) and the files get resubmitted. My log shows /POST --/--.
How can I avoid having that happen?
I tried to use the following code:
$('#submitform').on('submit', function(e){
e.preventDefault();
$(this).find('input[type=submit]').attr('disabled', 'disabled');
$('#message').html('Importing the data');
$("#submitform").unbind('submit').submit()
});
But it simply does not submit the form.
How could I fix this issue?
I can't use AJAX as the response I get from the node.js app as I'd have to rewrite too much backend.
This is kind of hard for me to explain but I am trying to shorten my code by creating one link instead of 50 for downloading a form. I will try to explain this a bit better using my code.
I have 5 get requests that do exactly the same thing but download a different file.
router.get('/form1', function (req, res) {
var file = __dirname + '/../public/forms/form1.pdf';
res.download(file);
});
router.get('/form2', function (req, res) {
var file = __dirname + '/../public/forms/form2.pdf';
res.download(file);
});
etc...
and my front-end link are;
FORM 1
FORM 2
etc...
Is there anything I can do to make this a more dynamic? The only way I can think of is something like this;
router.get('/:formName', function (req, res) {
// some how do a "req.params.formName"
var file = __dirname + '/../public/forms/' + req.params.formName + '.pdf';
res.download(file); // Set disposition and send it.
});
But I don't know how I will get the formName or if thats even possible.
Here are some more options to clarify:
Option 1: If you have a folder on the server with a fairly manageable directory structure, simply use express.static to map the physical folder to a virtual one with automatic download:
app.use('/download', express.static(path.join(__dirname, '/public/forms/')))
This will result in any link from the front-end with href='/download/something.pdf' working as long as that file is on the server in the path you mapped (i.e. in /public/forms).
Option 2 (which David E above answered in essence): In your original code, if you wanted to generate a path handler for a link that looks like /download/form1, /download/form2, it's a very minor modification:
router.get('/download/:formNumber', function (req, res) {
var file = __dirname + '/public/forms/' + req.params.formNumber + '.pdf';
res.download(file);
});
Option 3: You want to authenticate access to the files and potentially support multiple, complex URL schemes to a single handler that can lookup the appropriate physical path and send the file:
router.get('/download/:path[forms|images|files]/:formNumber1', fileRequestHandler);
router.get('/public/downloadFolder2/:formNumber2', fileRequestHandler);
router.get('/public/downloadFolder3/:formNumber3', fileRequestHandler);
function fileRequestHandler(req, res) {
// Check authentication here - example below from Passport
if(!req.isAuthenticated())
return res.status(401).send({err: 'Unauthorized'});
// Check which form number is supplied and map to appropriate physical file
if(req.params.formNumber1) {
// in this example, req.params.path can be one of three allowed sample values - forms or images or files
var file = __dirname + '/public/' + req.params.path + '/' + req.params.formNumber + '.pdf';
res.download(file);
} else if(req.params.formNumber2) {
// etc.
}
}
Note: Ezra Chang's point about path validity is important.
This response assumes that your route lives in index.js.
router.get('/form/:formName', (req, res, next) => {
res.download(`${__dirname}/../public/forms/${req.params.formName}.pdf`);
});
FORM 2
Be careful about your path. I don't know whether you can start at a directory, go up a level, then down again.
I have a simple upload application written in NodeJS using Multer, which works fine. Here's the code:
var express = require('express'),
bodyParser = require('body-parser'),
qs = require('querystring'),
multer = require('multer'),
logger = require('morgan');
var config = require('./config'),
oauth = require('./oauth');
function extractExtension(filename) {
return filename.split('.').pop();
}
function getRandom(min, max) {
return Math.floor(Math.random() * max) + min;
}
var app = express();
//app.use(cors());
// Add headers
app.use(function(req, res, next) {
// Website you wish to allow to connect
res.setHeader('Access-Control-Allow-Origin', 'http://localhost:8080');
// Request methods you wish to allow
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, PATCH, DELETE');
// Request headers you wish to allow
res.setHeader('Access-Control-Allow-Headers', 'X-Requested-With,Authorization,content-type');
// Set to true if you need the website to include cookies in the requests sent
// to the API (e.g. in case you use sessions)
res.setHeader('Access-Control-Allow-Credentials', true);
// Pass to next layer of middleware
next();
});
// Multer
var momentUpload = multer({
dest: './uploads/',
limits: {
fileSize: 256 * 1024 * 1024
},
rename: function(fieldname, filename) {
return Date.now() + '-' + getRandom(100000, 999999) + '.' + extractExtension(filename);
},
onFileUploadStart: function(file) {
console.log(file.originalname + ' is starting ...')
},
onFileUploadComplete: function(file) {
console.log(file.fieldname + ' uploaded to ' + file.path)
}
}).single('file');
app.set('port', 4000);
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
extended: true
}));
app.post('/file/upload', [oauth.ensureAuthenticated, momentUpload, function(req, res) {
console.log(req.body); // form fields
console.log(req.file); // form files
res.status(204).end();
}]);
// Start the Server
app.listen(app.get('port'), function() {
console.log('Metadata store env: ' + config.METADATASTORE_ENV);
console.log('Express server listening on port ' + app.get('port'));
firebase.authenticate();
console.log('Connected to Firebase');
});
The problem is, however, that the configuration of Multer doesn't seem to be working at all. The destPath works, and files appear in the folder I provided (./uploads/). Larger file sizes are allowed (e.g. a file of 400MB, while the options clearly state 256MB), and the callback functions aren't fired once. There is no error message. Any idea what I'm doing wrong here? I followed guides on Google and on the official page, but can't get it to work.
First of all, multer has changed its API recently, so it no longer accepts rename, onFileUploadStart or onFileUploadComplete!
We can take a look at the API here https://github.com/expressjs/multer, so lets analyse the new way things work!
NOTE: If you haven't updated your multer version, I strongly advise you to, because older versions are suspected to have a security breach.
Basic Usage
Multer accepts an options object, the most basic of which is the dest
property, which tells Multer where to upload the files. In case you
omit the options object, the files will be kept in memory and never
written to disk.
By default, Multer will rename the files so as to avoid naming
conflicts. The renaming function can be customized according to your
needs.
The options object also accepts fileFilter(a function to control which files are uploaded) and limits(an object that specifies the size limits) parameters.
So, your code would look like this (concerning only the multer part and considering you'd want to use all options which you don't have to):
// Multer
var momentUpload = multer({
dest: './uploads/',
limits: {
fileSize: 256 * 1024 * 1024
},
fileFilter: function(req, file, cb) {
// The function should call `cb` with a boolean
// to indicate if the file should be accepted
// To reject this file pass `false`, like so:
cb(null, false)
// To accept the file pass `true`, like so:
cb(null, true)
// You can always pass an error if something goes wrong:
cb(new Error('I don\'t have a clue!'))
}
}).single('file');
If you want to have more control of the storing of files you can use a storage engine. You can create your own or simply use the ones available.
The available ones are: diskStorage to store files on disk or memoryStorage to store files in the memory as Buffer objects.
Since you clearly want to store files in the disk, let's talk about the diskStorage.
There are two options available: destination and filename.
destination is used to determine within which folder the uploaded
files should be stored. This can also be given as a string (e.g.
'/tmp/uploads'). If no destination is given, the operating system's
default directory for temporary files is used.
Note: You are responsible for creating the directory when providing
destination as a function. When passing a string, multer will make
sure that the directory is created for you.
filename is used to determine what the file should be named inside the folder. If no filename is given, each file will be given a random name that doesn't include any file extension.
So, your code (concerning only the multer part) would look like this:
// Multer
//Storage configuration
var storageOpts = multer.diskStorage({
destination: function (req, file, cb) {
//Choose a destination
var dest = './uploads/';
//If you want to ensure that the directory exists and
//if it doesn't, it is created you can use the fs module
//If you use the following line don't forget to require the fs module!!!
fs.ensureDirSync(dest);
cb(null, dest);
},
filename: function (req, file, cb) {
//here you can use what you want to specify the file name
//(fieldname, originalname, encoding, mimetype, size, destination, filename, etc)
cb(null, file.originalname);
}
});
var momentUpload = multer({
storage: storageOpts,
limits: {
fileSize: 256 * 1024 * 1024
},
fileFilter: function(req, file, cb) {
// The function should call `cb` with a boolean
// to indicate if the file should be accepted
// To reject this file pass `false`, like so:
cb(null, false)
// To accept the file pass `true`, like so:
cb(null, true)
// You can always pass an error if something goes wrong:
cb(new Error('I don\'t have a clue!'))
}
}).single('file');
Hope it helped! :)
I have an express.js app, and I am using drywall in order to manage the user system.
When a user signs up, I want a directory to be generated for that user, and I want that user to be able to upload files to that directory and view those files through his or her account.
I am not entirely sure, but I think that most likely for the directory generation I will have to do that inside views/signup/index.js, and that the user can only upload files to his or her directory if logged in.
However, I'm a bit stuck when it comes to saving and displaying the files. I have little experience with server side code, so implementing actions such as accessing files is slightly beyond my scope.
Thanks in advance to those who help.
So first you should create a folder for each user by using fs.mkdir:
http://nodejs.org/api/fs.html#fs_fs_mkdir_path_mode_callback
Let's say you want to create these folders into your app root / images:
Example:
var fs = require('fs');
fs.mkdir(__dirname + '/images/' + userId, function(err) {
if (err) {
/* log err etc */
} else {
/* folder creation succeeded */
}
});
You should probably use the userId for the folder name (since it's easier than trying to strip out the bad characters from the username itself, and this will also work in the future if the user changes his username).
The second thing you need to do is to allow the user to upload files (but only if he is logged in and into the right folder). It's better to not include the bodyParser middleware for all routes, but instead include the json && urlencoded middleware for all routes (http://www.senchalabs.org/connect/json.html && http://www.senchalabs.org/connect/urlencoded.html) and the multipart middleware only for the upload url ( http://www.senchalabs.org/connect/multipart.html && example: https://github.com/visionmedia/express/blob/master/examples/multipart/index.js ).
An example:
app.post('/images', express.multipart({ uploadDir: '/tmp/uploads' }), function(req, res, next) {
// at this point the file has been saved to the tmp path and we need to move
// it to the user's folder
fs.rename(req.files.image.path, __dirname + '/images/' + req.userId + '/' + req.files.image.name, function(err) {
if (err) return next(err);
res.send('Upload successful');
});
});
Note: in the example above I've taken into consideration that req.userId is populated with the id of the user by an auth middleware.
Showing the images to the user if he has the rights to see them (the auth middleware should be applied for this path as well):
app.get('/images/:user/:file', function(req, res, next) {
var filePath = __dirname + '/images/' + req.userId + '/' + req.params.file;
fs.exists(filePath, function(exists) {
if (!exists) { return res.status(404).send('Not Found'); }
// didn't attach 'error' handler here, but you should do that with streams always
fs.createReadStream(filePath).pipe(res);
});
});
Note: in production you might want to use send instead, that example was just demoing the flow ( https://github.com/visionmedia/send ).