Develop Reference

How to resolve path issues while moving files in node.js?

I am trying to get a file from html form and store it in another folder. It's basically cloud function, and I am new to both node.js and firebase so don't know what I am doing wrong. What I manage to do is:
const fileMiddleware = require('express-multipart-file-parser');
app.post("/sendMail", (req, res) => {
const {
fieldname,
filename,
encoding,
mimetype,
buffer,
} = req.files[0];
console.log(req.files[0].originalname);
var fs = require('fs')
var oldPath = req.files[0].originalname;
var newPath = '/functions/'+oldPath;
fs.rename(oldPath, newPath, function (err) {
if (err) throw err
console.log('Successfully renamed - AKA moved!')
});
});
Whenever I try to move file, I got path issues. The error is as follows:
[Error: ENOENT: no such file or directory, rename 'C:\Users\Maisum Abbas\now\functions\sendMail.txt'
> 'C:\functions\sendMail.txt'] {
> errno: -4058,
> code: 'ENOENT',
> syscall: 'rename',
> path: 'C:\\Users\\Maisum Abbas\\now\\functions\\sendMail.txt',
> dest: 'C:\\functions\\sendMail.txt'
> }
Also, this is the path where I want to actually move the file but oldpath is already setup like this.
C:\Users\Maisum Abbas\now\functions\sendMail.txt

Since I needed to attach a file with email, it was causing path issues. I tried it with multer and it works. What I did:
//call libraries here
var storage = multer.diskStorage({
destination: function (req, file, callback) {
callback(null, 'resume/');
},
filename: function (req, file, callback) {
callback(null, file.fieldname + '-' + Date.now());
}
});
var upload = multer({ storage : storage}).single('filetoupload');
app.post("/careerMail", (req, res) => {
const { name } = req.body;
const { email } = req.body;
const { phone } = req.body;
upload(req,res,function(err) {
if(err) {
return res.end("Error uploading file.");
}
});
const dest = 'mymail';
const mailOptions = {
from: email, // Something like: Jane Doe <janedoe#gmail.com>
to: dest,
subject: 'Candidate Application', // email subject
html: `<div>
<strong>From:</strong> ` +
name +
`<br /><br />
<strong>Email:</strong> ` +
email +
`<br /><br />
<strong>Phone:</strong> ` +
phone +
`<br /><br />
</div>
`,// email content in HTML
attachments: [
{
filename: req.files[0].originalname,
content: req.files[0].buffer.toString("base64"),
encoding: "base64"
}
]
and rest of the code...

I suggest rethinking this approach altogether. You won't be able to move files around in a deployed function. The nodejs runtime filesystem doesn't allow any files to be written anywhere in the filesystem, except for os.tmpdir() (which is /tmp on Linux).
If you need to write a file temporarily, you should definitely only use that tmp space. Be aware that files written there occupy memory and should be deleted before the function terminates, or you could leak memory.
You can read files that you deployed with your code, but you should do that through relative paths.

I ran into same problem while moving file. I sort this problem by using a function to get the application root folder and then concatenate rest of the location.
//place this file on application root.
//import where you need to get the root path.
const path = require('path');
module.exports = (function(){
return path.dirname(require.main.filename || process.mainModule.filename);
})();
//taking your case move location.
const rootPath = //require the above module.
const newPath = rootPath + /functions/' +oldPath;
fs.rename(oldPath, newPath, function (err) {
if (err) throw err
console.log('Successfully renamed - AKA moved!')
});

Transform photo with imagemin before uploading to s3 using multer-s3-transform

I was trying to use multer-s3-transform in uploading my photo to s3, but before that I need to transform the image to much lower file size using imagemin. I was able to do it using sharp but still want to try it using imagemin to specifically set the quality.
This is what I did using sharp. I need to find a way how to do it using imagemin
const multerS3Obj = multerS3({
s3 : s3,
bucket : config.amazon.s3.bucketName,
acl : "public-read",
contentType : multerS3.AUTO_CONTENT_TYPE,
metadata : function(req, file, cb) {
const metadataObj = Object.assign({}, req.body);
metadataObj.content_type = file.mimetype;
metadataObj.filename = file.originalname;
cb(null, metadataObj);
},
shouldTransform: function(req, file, cb) {
cb(null, /^image/i.test(file.mimetype));
},
transforms: [
{
key: function(req, file, cb) {
const refType = req.params.refType,
refId = req.params.refId,
subfolder = `uploads/${refType}/${refId}/`;
cb(null, subfolder + file.originalname);
},
transform: function(req, file, cb) {
cb(null, sharp().resize(null,null));
}
}
]
});
This is what I did using imagemin but it didn't work
const buff = async (image, path) => {
const files = await imagemin([image], path, {
plugins : [
imageminJpegtran(),
imageminPngquant({quality: '65-80'})
]
})
}
this is the transform part in the first code
transform: async function(req, file, cb) {
const files = await buff(file.originalname, 'location')
cb(null, sharp().resize(null,null));
}
I keep getting error: uncaughtException: dest.on is not a function

I was able to solved it. instead of using imagemin to set the quality of the image found out that sharp supports setting of image quality
let quality = ''
if (file.mimetype === 'image/jpeg') {
quality = sharp().jpeg({quality: 50})
} else if (file.mimetype === 'image/png') {
quality = sharp().png({quality: 50})
}

How to download files using multer-sftp

I have this code, but surfing the web, i cannot found some way to download a file from the remote server. I'm able to upload the files in the remote server but I don't have any idea how download from there.
var storage = sftpStorage({
sftp: {
host: '171.16.....',
port: xxxx,
username: 'username',
password: 'xxxxxxxxxxxxxxxx'
},
destination: function(req, file, cb) {
cb(null, 'uploads')
},
filename: function(req, file, cb) {
cb(null, Date.now() + file.originalname)
}
});
var upload = multer({ storage: storage })
This is the route to upload the files: (Works great!)
router.post('/upload-image', upload.single('file'), listarController.uploadImage);
This is the route to download the files - locally: (Searching a method...)
router.get('/get-file/:file', listarController.getFile);
The method to download locally:
controller.getFile = (req, res) => {
var file = req.params.file;
var path_file = './uploads/' + file;
fs.exists(path_file, (exists) => {
if (exists) {
return res.sendFile(path.resolve(path_file))
} else {
return res.status(200).send({
message: "The image doesn't exist."
})
}
})
}
Some suggestion? Thanks in advance

I solved it with the following libraries:
1. multer-sftp: To upload the files.
2. ftp: To download and rename the files.

can't change name to uploaded file using multer

I am trying to change the name of the image im uploading to the server using multer which gives the file a random name.
I user multer.diskStorage method to do so as described in the documentation but it keeps saving the file with random names
CODE:
const storage = multer.diskStorage({
destination: (req, file, cb) => {
cb(null,'./uploads/')
},
fileName: (req,file,cb) => {
cb(null, file.originalname)
}
})
const upload = multer({storage: storage})
router.post('/', upload.single('carImage') ,(req, res) => {
res.send(req.file);
}
RESPONSE :
{
fieldname: 'carImage',
originalname: 'testOCR8.jpg',
encoding: '7bit',
mimetype: 'image/jpeg',
destination: './uploads/',
filename: '229f70c20e5550dbe638db49791ef17d',
path: 'uploads/229f70c20e5550dbe638db49791ef17d',
size: 1712380
}

im uploading to the server using multer which gives the file a random name
You made a typo. It is filename not fileName. This is the standard behavior as per the docs.
filename is used to determine what the file should be named inside the folder. If no filename is given, each file will be given a random name that doesn't include any file extension.
So, your code should be
const storage = multer.diskStorage({
destination: (req, file, cb) => {
cb(null,'./uploads/')
},
filename: (req,file,cb) => { // notice the change 'filename'
cb(null, file.originalname)
}
});
const upload = multer({storage: storage});

Try a different approach for using StorageMulter. Try the following -
var StorageMulter = multer.diskStorage({
destination: function(req, file, callback) {
callback(null, "./temp");
},
filename: function(req, file, callback) {
var uploadFileName = "x.jpg"; //Manipulate this variable accordingly
callback(null, uploadFileName);
}
});
var upload = multer({
storage: StorageMulter
});
app.post("/api/uploaddocument", function(req, res) {
upload(req, res, function(err) {
if (err) {
return res.end("Something went wrong!"+ err);
}
});
});

Multer trying to create a new folder when one already exists

I'm using Multer to manage image uploads, and when running my app on my local machine everything works perfectly. But now that I'm trying to deploy to Heroku I get this error in my application logs:
Unhandled rejection Error: EROFS: read-only file system, mkdir
'/public'
...which seems like it's trying to create a directory rather than using the existing one (like it does on my local machine)
This is my the code I'm using for Multer:
var storage = multer.diskStorage({
destination: (req, file, cb) => {
cb(null, 'public/img/') },
filename: (req, file, cb) => {
cb(null, new Date().toISOString() + file.originalname)
} })
var upload = multer({storage: storage})
router.post('/:propertyid', upload.single('img'), function(req, res) {
Property.update({
img: req.file.filename
}, {
where: {
id: req.params.propertyid,
}
}).then(updatedProperty => {
res.redirect('/admin/' + updatedProperty)
})
});
The public/img directory already exists and when testing it, it runs fine with correct filenames and end up in the correct place. Any help is appreciated! Thanks!

Heroku dynos have an ephemeral filesystem. This answer by Naaman Newbold, explains very well what that means.
TL;DR:
Dynos' filesystem shouldn't be used to store any permanent storage or data besides what is deployed. In order to do use a storage, AWS S3 can be used.

You can Use fs-extra to create folder it will sync and create folder like below in below code I am creating directory by user_id and uploading multiple images so i have used fs.mkdirsSync(path) in my code
let fs = require('fs-extra');
let storage = multer.diskStorage({
destination: function (req, file, cb) {
let Id = req.body.id;
let path = `tmp/daily_gasoline_report/${Id}`;
fs.mkdirsSync(path);
cb(null, path);
},
filename: function (req, file, cb) {
// console.log(file);
let extArray = file.mimetype.split("/");
let extension = extArray[extArray.length - 1];
cb(null, file.fieldname + '-' + Date.now() + "." + extension);
}
})