I use multer to parse multiple files sent as multipart/data-form with axios
...
const storage = multer.diskStorage({
destination: './gallery',
filename(req, file, cb) {
(1) ....
},
});
const upload = multer({ storage });
router.post('/products', upload.array('images'), (req, res, next) => {
Product.create(...)
.then((product) => {
(2) ...
})
.catch(..)
})
...
at this point everything is fine and my images are saved.
the problem is that i want to make a loop in (1) or (2) and name my files like this
files.forEach((file, index) => {
// rename file to => product_id + '_' + index + '.jpeg'
}
For example if i have 3 files they will be named to
5a9e881c3ebb4e1bd8911126_1.jpeg
5a9e881c3ebb4e1bd8911126_2.jpeg
5a9e881c3ebb4e1bd8911126_3.jpeg
where 5a9e881c3ebb4e1bd8911126 is the id of the product document saved by mongoose.
how to solve this naming issue ?
is multer the best solution cause i want full control over my files ?
Is there a better approach with another node package ?
is it good to send images as multipart/data-form or data URL base64 ?
This is easy, as long as you understand how express works. So before jumping to solution its important to have a clear understanding.
When you have a express code like below
router.post('/abc', function(req, res) {res.send('hello world');})
Express passes the request from chains of middlewares/functions. Now each function gets req, res, next parameters. The next is function, which a middleware is suppose to call when the processing is complete. If the middleware decides not to call next the request ends there and no more middlewares are called further.
When we used function(req, res) {res.send('hello world');}, we didn't take the next parameter at all, which means we are not interested in any other code to do anything. Now getting back to our problem
router.post('/products', upload.array('images'), (req, res, next) => {...}
You have used upload.array('images') first and then your actual product creation code. So I would show two approaches to solve this problem
One more middleware to rename the files
router.post('/products', upload.array('images'), (req, res, next) => {
Product.create(...)
.then((product) => {
req.product = product
next();
})
.catch(..)
}, (req, res, next) => {
//Get the product id using req.product
//Move the files as per the name you desire
})
Reverse the processing order
In this approach you first create the product and then let image processing happen. I have created a sample for the showing the same
let express = require('express');
let bodyParser = require('body-parser');
app = express();
let multer = require('multer');
const storage = multer.diskStorage({
destination: './gallery',
filename: (req, file, cb) => {
console.log('Product id - ' + req.product_id);
cb(null, req.product_id + '.js');
},
});
const upload = multer({ storage });
app.all('/', (req, res, next) => {
console.log('Hello you');
promise = new Promise((resolve) => {
// simulate a async product creation
setTimeout(() => resolve(1234), 100);
});
promise.then((product_id) => {
console.log('create the product and get the new product id')
// set the product id in the request object, so the multer
// filename function can access it
req.product_id = product_id;
res.send('uploaded files');
if (next)
next();
});
}, upload.array('images'));
module.exports = {
app
};
app.listen(8020);
And testing it using postman works fine
Edit: 19-Mar-2018
For multiple files you can easily update your filename function code like below
const storage = multer.diskStorage({
destination: './gallery',
filename: (req, file, cb) => {
req.file_id = req.file_id || 0;
req.file_id++;
console.log('Product id - ' + req.product_id);
cb(null, req.product_id +'_' + req.file_id +'.js');
},
});
This will make sure that you get all the files for that product. Now coming to your questions
how to solve this naming issue ?
This answer already does that
is multer the best solution cause i want full control over my files ?
I can't say, as long it works and does what you want, it should be good enough
Is there a better approach with another node package ?
I couldn't find lot of packages. But you can explore this if you want
is it good to send images as multipart/data-form or data URL base64 ?
I would use multipart/data-form, so that no base64 conversion is needed at client side. But again this is a matter of opinion as well.
You can't set the name purely in (1) since at that point you do not know the ID of the product yet.
You can't set the name purely in (2) since at that point the files have already been saved (with filename generated by your filename(req, file, cb) function).
So I think the best solution might be to move the files after they are uploaded.
This could be done in (2). When you process the files in the router, req.files will be an array of files that have already been uploaded.
In your promise callback for Product.create, you have access to the product (which you need for the ID) and the list of files (which you need for the number).
For that, you could use fs.rename(oldPath, newPath, callback).
https://nodejs.org/docs/latest/api/fs.html#fs_fs_rename_oldpath_newpath_callback
Something like this should work:
Product.create(...).then((product) => {
req.files.forEach((file, index) => {
// file.path is the full path to the file that was uploaded.
// newPath is where you want to put it.
// Let's use the same destination and just change the filename.
const newPath = file.destination + product.id + '_' + index
fs.rename(file.path, newPath)
})
})
Related
Hello sorry for my english, i have a little problem. i try to upload many images but in back side i have just one image, (i use React express formidable cloudinary) here is my code front :
const [arrayFiles, setArrayFiles] = useState([]);
const handleFiles = (e) => {
let arrayUpload = [...arrayFiles];
arrayUpload.push(e.target.files[0]);
setArrayFiles(arrayUpload);
};
const handleSubmit = async (e) => {
arrayFiles.forEach((file) => {
formData.append("image", file);
});
const response = await axios.post(
"http://localhost:3100/offer/publish",
formData
);
here is my code back but req.files => just one image
my page route :
router.post("/offer/publish", async (req, res) => {
console.log(req.files);
const result = await cloudinary.uploader.upload(req.files.image.path, {
folder: `api/leboncoin/offers/${newOffer._id}`, // _id vient de la création du newOffer au dessus
public_id: "preview",
cloud_name: process.env.CLOUDINARY_NAME,
});
my page index.js:
page index.js :
const express = require("express");
const formidable = require("express-formidable");
const mongoose = require("mongoose");
const cloudinary = require("cloudinary").v2;
const cors = require("cors");
const app = express();
app.use(cors());
app.use(formidable({ multiples: true }));
You only get one file in req.file as you've set your multer.single
Using multer
There are 3 ways you can handle multiple file upload, each with a slightly different taste.
Assume you have a base multer
const storage = multer.diskStorage({
destination: "public/data/",
filename: function(req, file, cb){
// You may change this to however you want, only affect your file name
crypto.randomBytes(20, (err, buf) => {
cb(null, buf.toString("hex") + path.extname(file.originalname))
})
}
});
const upload = multer({ storage: storage });
Use .any()
Accepts all files that comes over the wire. An array of files will be stored in req.files.
WARNING: Make sure that you always handle the files that a user uploads. Never add multer as a global middleware since a malicious user could upload files to a route that you didn't anticipate. Only use this function on routes where you are handling the uploaded files.
router.post("/offer/publish",upload.any(), async (req, res) => {
console.log(req.files); // Should give you an array of files
// Do anything else
});
Use .array(fieldname[, maxCount])
Accept an array of files, all with the name fieldname. Optionally error out if more than maxCount files are uploaded. The array of files will be stored in req.files.
router.post("/offer/publish",upload.array('someFieldName', 10), async (req, res) => {
console.log(req.files); // Should give you an array of files
// Do anything else
});
Use .fields(fields)
Accept a mix of files, specified by fields. An object with arrays of files will be stored in req.files.
fields should be an array of objects with name and optionally a maxCount. Example:
router.post(
"/offer/publish",
upload.fields([
{
name: "image",
maxCount: 1,
},
{
name: "audio",
maxCount: 1,
},
]),
async (req, res) => {
console.log(req.files.image[0]);
console.log(req.files.audio[0]);
// Do anything else
}
);
For your case, I would recommend going with Option 2.
I'm new both to Express.js and to StackOverflow; I'm sorry if this is a duplicate question. I checked, but didn't see anything relevant.
So, I'm using Multer + Express to allow a user to upload an image, which will be named '${username}.{extension}', to a server-side /uploads/ folder. I don't want users to be able to save more than one image on the server (i.e. no "user1.jpg" and "user1.png"). To accomplish this, I wrote the following middleware:
function deleteUserImage(req){
const acceptedExtensions = ['.png', '.jpg', '.jpeg', '.tif', '.tiff', '.JPG', '.bmp'];
acceptedExtensions.forEach(char => {
if(fs.existsSync(`./uploads/${req.cookies.username+char}`)){
fs.unlinkSync(`./uploads/${req.cookies.username+char}`);
}
})
}
I then was able to get the functionality I wanted with the following routes:
app.post('/process_upload-image', (req, res, next) => { //User sends post req w/ image file
deleteUserImage(req) //images for that user are cleared.
next();
})
app.post('/process_upload-image', upload.single('user-image'), (req, res, next) => {
res.redirect('/welcome'); //user is redirected after multer uploads the image.
})
I was wondering if this was best-practices, however, since you end up with two routes listening at the same URI? Is there a way to pass req to deleteUserImage(), then call upload.single()...all in one route?
Thanks!
You can simply chain multiple middlewares on one route. Change the deleteUserImage function to:
function deleteUserImage(req, res, next){
const acceptedExtensions = ['.png', '.jpg', '.jpeg', '.tif', '.tiff', '.JPG', '.bmp'];
acceptedExtensions.forEach(char => {
if(fs.existsSync(`./uploads/${req.cookies.username+char}`)){
fs.unlinkSync(`./uploads/${req.cookies.username+char}`);
}
})
next()
}
and then remove the first route and change the second one to:
app.post('/process_upload-image', deleteUserImage, upload.single('user-image'), (req, res, next) => {
res.redirect('/welcome');
})
My uploaded form has a file and a field "filepath" which is dynamically generated and contains the desired filepath of the file I'm uploading.
Example:
filepath: "assets/images/asset-01-02/"
I'm trying to set this field as a variable so I can save the file to this directory to keep uploaded files organized.
Current code:
const cors = require('cors');
const express = require('express');
const app = express();
const formidable = require('formidable');
app.use(cors());
app.options('*', cors());
app.post('/upload', (req, res) => {
var form = new formidable.IncomingForm();
form.parse(req, (err, fields, files) => {
if (err) {
console.log('Error', err)
throw err
}
console.log(fields.filepath); //Output e.g. "assets/images/asset-01-02/"
})
form.on('fileBegin', (name, file) => {
//Need to use fields.filepath after '/public/uploads/'
//e.g. __dirname + '/public/uploads/' + fields.filepath + file.name;
file.path = __dirname + '/public/uploads/' + file.name;
});
form.on('file', (name, file) => {
console.log('Uploaded ' + file.name);
});
});
app.listen(80, function(){
console.log('Listening on Port 80...');
});
I need to get the fields.filepath value passed to the form.on('fileBegin') function but I'm not sure how to. I haven't come across any examples of this specific issue.
As far as I am understanding your question, you are trying to send 'filepath' with 'multipart/form-data' or from client to server with uploaded files.
and you are trying to catch and set this 'filepath' using form.parse 'fields' into form.parse callback.
your code do not work simply because form.on('fileBegin') callback will execute before form.parse callback.
here is execution order for callback in Formidable.(can change due to asynchronous functions)
form.on('fileBegin')
form.on('file')
form.parse()
form.on('end')
instead passing 'filepath' with 'form-data', send it with query like localhost/upload?filepath=assets/images/asset-01-02
and get it like
var filepath = req.query.filepath;
in your code
I'm using Multer to manage image uploads, and when running my app on my local machine everything works perfectly. But now that I'm trying to deploy to Heroku I get this error in my application logs:
Unhandled rejection Error: EROFS: read-only file system, mkdir
'/public'
...which seems like it's trying to create a directory rather than using the existing one (like it does on my local machine)
This is my the code I'm using for Multer:
var storage = multer.diskStorage({
destination: (req, file, cb) => {
cb(null, 'public/img/') },
filename: (req, file, cb) => {
cb(null, new Date().toISOString() + file.originalname)
} })
var upload = multer({storage: storage})
router.post('/:propertyid', upload.single('img'), function(req, res) {
Property.update({
img: req.file.filename
}, {
where: {
id: req.params.propertyid,
}
}).then(updatedProperty => {
res.redirect('/admin/' + updatedProperty)
})
});
The public/img directory already exists and when testing it, it runs fine with correct filenames and end up in the correct place. Any help is appreciated! Thanks!
Heroku dynos have an ephemeral filesystem. This answer by Naaman Newbold, explains very well what that means.
TL;DR:
Dynos' filesystem shouldn't be used to store any permanent storage or data besides what is deployed. In order to do use a storage, AWS S3 can be used.
You can Use fs-extra to create folder it will sync and create folder like below in below code I am creating directory by user_id and uploading multiple images so i have used fs.mkdirsSync(path) in my code
let fs = require('fs-extra');
let storage = multer.diskStorage({
destination: function (req, file, cb) {
let Id = req.body.id;
let path = `tmp/daily_gasoline_report/${Id}`;
fs.mkdirsSync(path);
cb(null, path);
},
filename: function (req, file, cb) {
// console.log(file);
let extArray = file.mimetype.split("/");
let extension = extArray[extArray.length - 1];
cb(null, file.fieldname + '-' + Date.now() + "." + extension);
}
})
I have a case where I need to store images in different directories . So I have set the multer as.
app.use(multer({
dest: path.join(__dirname, '`public/assets/img/profile`'),
rename: function (fieldname, filename, req, res) {
if(req.session.user) return req.session.user.id;
else if(req.session.doctor) return req.session.doctor.id;
}
}));
However I need more destinations to store images.
public/assets/img/picture1
I have seen similar questions but I could not understand any of them.
Any help would be greatfull.
Your example is from quite old version of multer. I strongly recommend you to use latest version (due to security reasons).
If you are sure that you need old version then just add to your multer options:
app.use(multer({
//...
changeDest: function(dest, req, res) {
return dest + '/user1';
}
//...
}));
More details you will get in documentation (link to old version) link
Newest version of multer works a little different. This would be to big offtopic to write in detail how to use new version of multer. You easily will find in stackoverflow answer or from actual version of documentation link
I only write how to change destination directory (example from docs):
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, '/tmp/my-uploads')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
var upload = multer({ storage: storage })
app.post('/profile', upload.single('picture'), function (req, res, next) {
// req.file is the `picture` file
// req.body will hold the text fields, if there were any
})