Develop Reference

Express Route returns status code 500 with generic 'Error' when file is sent as 'multipart/form-data'

DETAILS:
I have an express route which is set up to accept files as multipart/formdata and upload them to an S3 bucket. I am using multer to filter image types, as well as store them temporarily on the server through the creation of an upload directory. The files are removed shortly after upload success. The array of files are named images as per multer configuration, and accepts a maximum of 3 images.
The code works perfectly on my local machine. I test through POSTMAN and can upload 1-3 files and get the proper response. If there are no files attached, the correct response is triggered as well, all with status code 200.
PROBLEM:
The exact same codebase is deployed on Amazon ECS with Docker, but somehow keeps failing consistently with status code 500 and a generic 'Error' message that is not found in the codebase. Using logs I have determined that multer is not the cause, as it passes through the filter. It appears to be failing somewhere between the multer middleware and the route itself, with an exception.
Exception: Using POSTMAN, if a multipart/formdata POST request is made with no files I.E empty images array, the route is triggered properly and the message "You did not attach any images" is returned as a response.
I have been unable to figure out the issue and appreciate it if some guidance can be provided on this issue!
CODE SNIPPETS:
filesController:
files.post(
"/multiple",
upload.array("images", 3),
async (req: ExpressRequest, res: ExpressResponse) => {
try {
const files: { [fieldname: string]: Express.Multer.File[] } | Express.Multer.File[] =
req.files;
console.log("FILES", files);
// execute only if there are files
if (files.length > 0) {
const dataPromises = (files as Array<Express.Multer.File>).map(
async (file: Express.Multer.File) => {
// check if file.mimetype here is 'image/heic', and convert into jpeg accordingly
const fileNameWithoutExt = file.filename.split(".")[0];
try {
if (file.mimetype == "image/heic") {
await convertFile(file, fileNameWithoutExt, 0.2);
const response = await uploadFilePath(
S3_IMAGE_BUCKET,
`./uploads/${fileNameWithoutExt}.jpeg`,
`${fileNameWithoutExt}.jpeg`
);
console.log("HEIC File Upload Response", response);
fs.unlinkSync(`./uploads/${fileNameWithoutExt}.jpeg`);
fs.unlinkSync(file.path);
return {
fileName: `${fileNameWithoutExt}.jpeg`,
metaData: response.$metadata,
};
} else {
const response = await uploadFile(S3_IMAGE_BUCKET, file);
console.log("JPEG File Upload Response", response);
fs.unlinkSync(file.path);
return {
fileName: file.filename,
metaData: response.$metadata,
};
}
} catch (err) {
console.error("Error for file conversion/upload", err, err.stack);
res.status(500).send({
message: "Upload failed due to conversion or something.",
error: err,
stack: err.stack,
});
}
}
);
const fileData = await Promise.all(dataPromises);
const fileNames = fileData.map((data: any) => data.fileName);
const statusCodes = fileData.map((data: any) => data.metaData.httpStatusCode);
if (statusCodes.find((statusCode) => statusCode === 200)) {
res.status(200).send({
filePath: `/image/`,
fileNames,
});
} else {
res.status(403).send({
message: "Upload failed. Please check credentials or file has been selected.",
});
}
} else {
res.status(200).send({
message: "You did not attach any images",
});
}
} catch (err) {
res.status(500).send({
message: "Upload failed. Please check credentials or file has been selected.",
});
}
}
);
multer configuration:
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
cb(null, "uploads/");
},
filename: (req, file, cb) => {
console.log("MULTER STORAGE STARTED")
const date = new Date().toISOString().substring(0, 10);
// const name = `${req.body.first_name}_${req.body.last_name}`;
// cb defines the name of the file when stored
const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-";
const nanoid = customAlphabet(alphabet, 20);
cb(null, `${date}_${nanoid()}_${file.originalname}`);
console.log("FILE NAME CREATED, MULTER STORAGE STOPPED")
},
});
/* Accept jpeg or png files only */
// NOTE: file type rejection works, but there is no error message displayed if file is rejected. logic in route continues to be executed
const fileFilter = (
req: Request,
file: Express.Multer.File,
cb: (error: Error | null, accepted: boolean) => void
) => {
console.log("======== FILE FILTER ========", file);
if (
file.mimetype === "image/jpeg" ||
file.mimetype === "image/png" ||
file.mimetype === "image/heic"
) {
cb(null, true);
console.log("FILTER PASSED")
} else {
console.log("FILTER FAILED");
cb(null, false);
}
};
/* Only accepts filesize up to 5MB */
// the first parameter is super important that determines where the data is stored on the server
const upload = multer({
dest: "uploads/", // default simple config to upload file as binary data
storage, // enable if storage of actual file is required.
// limits: { fileSize: 1024 * 1024 * 5 },
fileFilter,
});
SCREENSHOTS:
response with no images in form data
response with images in form data

Can you make sure the upload directory exists in your Docker container? Multer will not create it if it doesn't exist. It might be failing silently between your storage function and the actual writing of the files to disk.
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
cb(null, "uploads/");
},
should be something like:
import { access, constants } from "fs";
import { join } from "path";
...
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
const currentDirectory: string = process.cwd();
const uploadDirectory: string = join(currentDirectory, 'uploads/');
// can we write to this path?
access(uploadDirectory, constants.W_OK, (err) => {
if (err) {
console.error(err);
cb(err, null);
}
cb(null, uploadDirectory);
})
},

How to use multer function in controller instead of using in route

I want to Use multer function in controller file But the problem is req.body got undefine when I use postman form-data body to upload images This is link to code that how I am using multer Function but I want use it in my controller
how I want to use it as you can see in below code
const multerHelper = require("../helpers/multer_helper");
Documents: async (req, res) => {
console.log(req.body)
if (!req.body.id) {
console.log(req.body)
logger.warn(error.MANDATORY_FIELDS);
return res.status(500).send(error.MANDATORY_FIELDS)
}
try {
multerHelper.createUserImage
let result = error.OK
logger.info(result);
return res.status(200).send(result)
} catch (err) {
logger.warn(err);
console.log(err);
return res.status(500).send(error.SERVER_ERROR)
}
}
but it throws error like req.body undefine
req.body empty image
postman form-data image
Anyone know how to do it

You can use multer functions and objects in only routes not in controller if you want to use it in controller you have to write storage and upload functions in controllers, here I have used multer error handling and I am uploading multiple images
Documents: async (req, res) => {
if (!req.headers.room_id) {
logger.warn(error.MANDATORY_FIELDS);
return res.status(500).send(error.MANDATORY_FIELDS)
}
try {
let storage = multer.diskStorage({
destination: function (req, file, cb) {
let id = req.headers.room_id;
let path = `tmp/daily_gasoline_report/${id}`;
fsextra.mkdirsSync(path);
cb(null, path);
},
filename: function (req, file, cb) {
// console.log(file);
let extArray = file.mimetype.split("/");
let extension = extArray[extArray.length - 1];
cb(null, file.fieldname + '-' + Date.now() + "." + extension);
}
})
var upload = multer({ storage: storage }).array('images', 100);
upload(req, res, function (err) {
if (err) {
console.log(err);
return res.end("Error uploading file.");
} else {
res.end("File has been uploaded");
}
});
let result = error.OK
logger.info(result);
return res.status(200).send(result)
} catch (err) {
logger.warn(err);
console.log(err);
return res.status(500).send(error.SERVER_ERROR)
}
}

Do you have a parser?
How do you parse the data?
Your gonna need to use something that gives you this data something like:
https://www.npmjs.com/package/express-fileupload
this package helps you to parse the form data and the files data itself.

Express file upload and view

I am using express-fileupload to upload the images. The images are saved in my local directory. I want to insert the name of the file to the mongodb if possible. Finally I want the image to be displayed in my frontend.
function insertRecord(req,res){
if(req.files){
const file=req.files.filename
filename=file.name
file.mv("./upload"+filename,function(err){
if(err)
console.log(err)
})
}
const user=new User()
user.name=req.body.name
user.address=req.body.address
user.email=req.body.email
user.mobile=req.body.mobile
user.filename=req.body.filename
user.save((err,docs)=>{
if(!err){
res.redirect('/user/list')
}
else {
if (err.name == 'ValidationError') {
handleValidationError(err, req.body);
res.render("./users/addOrEdit", {
viewTitle: "Insert User",
user: req.body
});
}
else
console.log('Error during record insertion : ' + err);
}
});
}
I am not sure whether the way to insert the name of the file to the mongodb is correct or not. Anyway, that is optional but I am not understanding how can I display the uploaded images which are present in the local directory.
I tried to save the image as base64 but the record is not saved to the database now.
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, 'uploads/')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
var upload = multer({ storage: storage })
router.post('/',upload.single('myImage'),function(req,res){
if (req.body._id == '')
insertRecord(req, res);
else
updateRecord(req, res);
})
function insertRecord(req,res){
var img = fs.readFileSync(req.file.path);
var encode_image = img.toString('base64');
var finalImg = {
contentType: req.file.mimetype,
image: new Buffer(encode_image, 'base64')
};
const user=new User()
user.name=req.body.name
user.address=req.body.address
user.email=req.body.email
user.mobile=req.body.mobile
user.save(finalImg,(err,docs)=>{
if(!err){
res.redirect('/user/list')
}
else {
if (err.name == 'ValidationError') {
handleValidationError(err, req.body);
res.render("./users/addOrEdit", {
viewTitle: "Insert User",
user: req.body
});
}
else
console.log('Error during record insertion : ' + err);
}
});
}

Edit: I think there is a problem in the code: it should be `'./upload/'+filename' not without the second slash.
In order to show the images, you have to open a static route in Express. Example: app.use('/images', express.static(PATH)). Then you can, in the frontend, call it as <img src="URL/images/FILENAME" />
From your code, it is not possible to understand what kind of data you are sending to the server. As far as I understand, you're trying mv the string filename. In order to transfer files (such as images), you should have form-data instead of JSON data or you should encode the image file into Base64 to transfer it as text (not the filename, the whole file).
Check Multer out for this kind of job. It is described well in the README.md. Apart from that, until you submit the form, the image won't be available in the front-end. If you want to preview the image before uploading it's a separate process which you can learn more in here.

How to fix "MulterError: Unexpected field" in a nodejs express server?

I'm setting up a server to upload zip files from the client. The server runs with express and multer to do this. When uploading a file, the server throws a "MulterError: Unexpected field" error and I cannot figuere out what is cousing it.
I've tried with png images and It works just fine. But with the zip files it just doesn't work.
const multerConfig = {
//specify diskStorage (another option is memory)
storage: multer.diskStorage({
//specify destination
destination: function(req, file, next){
next(null, './public/zip-storage');
},
//specify the filename to be unique
filename: function(req, file, next){
console.log(file);
const ext = file.mimetype.split('/')[1];
//set the file fieldname to a unique name containing the original name, current datetime and the extension.
next(null, file.fieldname + '-' + Date.now() + '.'+ext);
}
}),
// filter out and prevent non-image files.
fileFilter: function(req, file, next){
if(!file){
next();
}
// only permit zip mimetypes
const zip = file.mimetype.startsWith('application');
if(zip){
console.log('zip uploaded');
next(null, true);
}else{
console.log("file not supported")
errorReq = true;
return next();
}
}
};
/* ROUTES
**********/
app.get('/', function(req, res){
res.render('index.html');
});
var errorDetection = function(){
if(!errorReq){
errorReq = false;
return('complete.html');
} else{
errorReq = false;
return('errorupload.html');
}
}
app.post('/upload', multer(multerConfig).single('photo'),function(req, res){
//Here is where I could add functions to then get the url of the new photo
//And relocate that to a cloud storage solution with a callback containing its new url
//then ideally loading that into your database solution. Use case - user uploading an avatar...
res.redirect('complete.html');
}
);
// RUN SERVER
app.listen(port,function(){
console.log(`Server listening on port ${port}`);
});
This is the error:
MulterError: Unexpected field
at wrappedFileFilter (/home/axentiva-miguel/Documentos/ServerAPP/node_modules/multer/index.js:40:19)
at Busboy.<anonymous> (/home/axentiva-miguel/Documentos/ServerAPP/node_modules/multer/lib/make-middleware.js:114:7)
at emitMany (events.js:147:13)
at Busboy.emit (events.js:224:7)
at Busboy.emit (/home/axentiva-miguel/Documentos/ServerAPP/node_modules/busboy/lib/main.js:38:33)
at PartStream.<anonymous> (/home/axentiva-miguel/Documentos/ServerAPP/node_modules/busboy/lib/types/multipart.js:213:13)
at emitOne (events.js:116:13)
at PartStream.emit (events.js:211:7)
at HeaderParser.<anonymous> (/home/axentiva-miguel/Documentos/ServerAPP/node_modules/dicer/lib/Dicer.js:51:16)
at emitOne (events.js:116:13)

In the code
app.post('/upload', multer(multerConfig).single('photo'),function(req, res){
//Here is where I could add functions to then get the URL of the new photo
//And relocate that to a cloud storage solution with a callback containing its new URL
//then ideally loading that into your database solution. Use case - user uploading an avatar...
res.redirect('complete.html');
}
I was putting the value single('photo'), when in the HTML form I sent, a 'zip' value in there. Changing that made the code work as expected :)

Req.file is undefined when uploaded from ReactJS to NodeJS

I am trying to send an image in a POST req from react to nodejs. However my back-end is not receiving the file and req.file is undefined. When I test the server image upload code with postman, everything works fine so I suspect something is wrong on the front-end. Anybody know what's wrong?
Here's my code:
Image_Upload.js (submit image func, fileList[0] is an image):
onSubmitImage = e => {
const config = {
headers: {
"content-type": "multipart/form-data"
}
};
const formData = new FormData();
formData.append(
"image",
this.state.fileList[0]
);
axios
.post("/api/profile/img_upload", formData)
.then(res => console.log("Response: " + res))
.catch(err => console.log(err.response.data));
};
profile.js (receiving image endpoint):
router.post(
"/img_upload",
passport.authenticate("jwt", { session: false }),
(req, res) => {
upload(req, res, err => {
console.log(req);
if (err) {
res.status(400).json({ Error: err });
} else {
if (req.file == undefined) {
res.status(404).json({ error: "no file selected" });
} else {
res.json({ fileLoc: req.file.location });
}
}
});
}
);
img_upload.js (multer setup):
const upload = multer({
storage: storage,
limits: { fileSize: 1000000 },
fileFilter: function(req, file, cb) {
checkFileType(file, cb);
}
}).single("image");
Any help is appreciated
EDIT:
I am noticing that when nodejs receives the req from the front-end, it is in req.body instead of req.file. So I am positive the problem is in react. How can I get react to send it as req.file instead of req.body?
Image request from Reacdt Front-end to Nodejs backend

upload that you defined is actually an express middleware which must take 3 arguments: request, response and next, respectively. next() calls the subsequent middleware in case no errors or next(error) otherwise. I highly recommend checking the answers to this question to get a better idea.
What you have done here is that you're calling a middleware as if it was a method. To fix that, config multer first
const multer = require('multer');
const upload = multer({
storage: 'path/here/',
limits: { fileSize: 1000000 },
fileFilter: function(req, file, cb) {
checkFileType(file, cb);
}
});
then
router.post("/img_upload",
passport.authenticate("jwt", { session: false }),
upload.single('image'),
(req, res) => {
// Your code here
});