I am using express-fileupload to upload the images. The images are saved in my local directory. I want to insert the name of the file to the mongodb if possible. Finally I want the image to be displayed in my frontend.
function insertRecord(req,res){
if(req.files){
const file=req.files.filename
filename=file.name
file.mv("./upload"+filename,function(err){
if(err)
console.log(err)
})
}
const user=new User()
user.name=req.body.name
user.address=req.body.address
user.email=req.body.email
user.mobile=req.body.mobile
user.filename=req.body.filename
user.save((err,docs)=>{
if(!err){
res.redirect('/user/list')
}
else {
if (err.name == 'ValidationError') {
handleValidationError(err, req.body);
res.render("./users/addOrEdit", {
viewTitle: "Insert User",
user: req.body
});
}
else
console.log('Error during record insertion : ' + err);
}
});
}
I am not sure whether the way to insert the name of the file to the mongodb is correct or not. Anyway, that is optional but I am not understanding how can I display the uploaded images which are present in the local directory.
I tried to save the image as base64 but the record is not saved to the database now.
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, 'uploads/')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
var upload = multer({ storage: storage })
router.post('/',upload.single('myImage'),function(req,res){
if (req.body._id == '')
insertRecord(req, res);
else
updateRecord(req, res);
})
function insertRecord(req,res){
var img = fs.readFileSync(req.file.path);
var encode_image = img.toString('base64');
var finalImg = {
contentType: req.file.mimetype,
image: new Buffer(encode_image, 'base64')
};
const user=new User()
user.name=req.body.name
user.address=req.body.address
user.email=req.body.email
user.mobile=req.body.mobile
user.save(finalImg,(err,docs)=>{
if(!err){
res.redirect('/user/list')
}
else {
if (err.name == 'ValidationError') {
handleValidationError(err, req.body);
res.render("./users/addOrEdit", {
viewTitle: "Insert User",
user: req.body
});
}
else
console.log('Error during record insertion : ' + err);
}
});
}
Edit: I think there is a problem in the code: it should be `'./upload/'+filename' not without the second slash.
In order to show the images, you have to open a static route in Express. Example: app.use('/images', express.static(PATH)). Then you can, in the frontend, call it as <img src="URL/images/FILENAME" />
From your code, it is not possible to understand what kind of data you are sending to the server. As far as I understand, you're trying mv the string filename. In order to transfer files (such as images), you should have form-data instead of JSON data or you should encode the image file into Base64 to transfer it as text (not the filename, the whole file).
Check Multer out for this kind of job. It is described well in the README.md. Apart from that, until you submit the form, the image won't be available in the front-end. If you want to preview the image before uploading it's a separate process which you can learn more in here.
Related
DETAILS:
I have an express route which is set up to accept files as multipart/formdata and upload them to an S3 bucket. I am using multer to filter image types, as well as store them temporarily on the server through the creation of an upload directory. The files are removed shortly after upload success. The array of files are named images as per multer configuration, and accepts a maximum of 3 images.
The code works perfectly on my local machine. I test through POSTMAN and can upload 1-3 files and get the proper response. If there are no files attached, the correct response is triggered as well, all with status code 200.
PROBLEM:
The exact same codebase is deployed on Amazon ECS with Docker, but somehow keeps failing consistently with status code 500 and a generic 'Error' message that is not found in the codebase. Using logs I have determined that multer is not the cause, as it passes through the filter. It appears to be failing somewhere between the multer middleware and the route itself, with an exception.
Exception: Using POSTMAN, if a multipart/formdata POST request is made with no files I.E empty images array, the route is triggered properly and the message "You did not attach any images" is returned as a response.
I have been unable to figure out the issue and appreciate it if some guidance can be provided on this issue!
CODE SNIPPETS:
filesController:
files.post(
"/multiple",
upload.array("images", 3),
async (req: ExpressRequest, res: ExpressResponse) => {
try {
const files: { [fieldname: string]: Express.Multer.File[] } | Express.Multer.File[] =
req.files;
console.log("FILES", files);
// execute only if there are files
if (files.length > 0) {
const dataPromises = (files as Array<Express.Multer.File>).map(
async (file: Express.Multer.File) => {
// check if file.mimetype here is 'image/heic', and convert into jpeg accordingly
const fileNameWithoutExt = file.filename.split(".")[0];
try {
if (file.mimetype == "image/heic") {
await convertFile(file, fileNameWithoutExt, 0.2);
const response = await uploadFilePath(
S3_IMAGE_BUCKET,
`./uploads/${fileNameWithoutExt}.jpeg`,
`${fileNameWithoutExt}.jpeg`
);
console.log("HEIC File Upload Response", response);
fs.unlinkSync(`./uploads/${fileNameWithoutExt}.jpeg`);
fs.unlinkSync(file.path);
return {
fileName: `${fileNameWithoutExt}.jpeg`,
metaData: response.$metadata,
};
} else {
const response = await uploadFile(S3_IMAGE_BUCKET, file);
console.log("JPEG File Upload Response", response);
fs.unlinkSync(file.path);
return {
fileName: file.filename,
metaData: response.$metadata,
};
}
} catch (err) {
console.error("Error for file conversion/upload", err, err.stack);
res.status(500).send({
message: "Upload failed due to conversion or something.",
error: err,
stack: err.stack,
});
}
}
);
const fileData = await Promise.all(dataPromises);
const fileNames = fileData.map((data: any) => data.fileName);
const statusCodes = fileData.map((data: any) => data.metaData.httpStatusCode);
if (statusCodes.find((statusCode) => statusCode === 200)) {
res.status(200).send({
filePath: `/image/`,
fileNames,
});
} else {
res.status(403).send({
message: "Upload failed. Please check credentials or file has been selected.",
});
}
} else {
res.status(200).send({
message: "You did not attach any images",
});
}
} catch (err) {
res.status(500).send({
message: "Upload failed. Please check credentials or file has been selected.",
});
}
}
);
multer configuration:
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
cb(null, "uploads/");
},
filename: (req, file, cb) => {
console.log("MULTER STORAGE STARTED")
const date = new Date().toISOString().substring(0, 10);
// const name = `${req.body.first_name}_${req.body.last_name}`;
// cb defines the name of the file when stored
const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz-";
const nanoid = customAlphabet(alphabet, 20);
cb(null, `${date}_${nanoid()}_${file.originalname}`);
console.log("FILE NAME CREATED, MULTER STORAGE STOPPED")
},
});
/* Accept jpeg or png files only */
// NOTE: file type rejection works, but there is no error message displayed if file is rejected. logic in route continues to be executed
const fileFilter = (
req: Request,
file: Express.Multer.File,
cb: (error: Error | null, accepted: boolean) => void
) => {
console.log("======== FILE FILTER ========", file);
if (
file.mimetype === "image/jpeg" ||
file.mimetype === "image/png" ||
file.mimetype === "image/heic"
) {
cb(null, true);
console.log("FILTER PASSED")
} else {
console.log("FILTER FAILED");
cb(null, false);
}
};
/* Only accepts filesize up to 5MB */
// the first parameter is super important that determines where the data is stored on the server
const upload = multer({
dest: "uploads/", // default simple config to upload file as binary data
storage, // enable if storage of actual file is required.
// limits: { fileSize: 1024 * 1024 * 5 },
fileFilter,
});
SCREENSHOTS:
response with no images in form data
response with images in form data
Can you make sure the upload directory exists in your Docker container? Multer will not create it if it doesn't exist. It might be failing silently between your storage function and the actual writing of the files to disk.
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
cb(null, "uploads/");
},
should be something like:
import { access, constants } from "fs";
import { join } from "path";
...
const storage = multer.diskStorage({
// potential error, path to store files, callback
destination: (req, file, cb) => {
// cb acceptes two arguments: 1. err 2. destination folder wrt to server.js
const currentDirectory: string = process.cwd();
const uploadDirectory: string = join(currentDirectory, 'uploads/');
// can we write to this path?
access(uploadDirectory, constants.W_OK, (err) => {
if (err) {
console.error(err);
cb(err, null);
}
cb(null, uploadDirectory);
})
},
I've been following some tutorials on how to upload a file using node and I've had success in actually uploading the file (I've primarily used https://www.geeksforgeeks.org/file-uploading-in-node-js/). However, every time it uploads the file, it changes to a new page. Is there a way to stay on the same page after uploading the photo, or do I have to create an HTML file with the same name with the same HTML as before the upload? The working code is below:
const express = require("express")
const path = require("path")
const multer = require("multer")
const app = express()
app.use(express.static(__dirname + '/public'));
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, "uploads")
},
filename: function (req, file, cb) {
cb(null, file.fieldname + "-" + Date.now()+".jpg")
}
})
var upload = multer({
storage: storage,
fileFilter: function (req, file, cb){
var filetypes = /jpeg|jpg|png/;
var mimetype = filetypes.test(file.mimetype);
var extname = filetypes.test(path.extname(
file.originalname).toLowerCase());
if (mimetype && extname) {
return cb(null, true);
}
cb("Error: File upload only supports the "
+ "following filetypes - " + filetypes);
}
}).single("myfile");
app.get("/",function(req,res){
res.render("Signup");
})
app.post("/uploadCode",function (req, res, next) {
upload(req,res,function(err) {
if(err) {
res.send(err)
}
else {
res.send("Success, Image uploaded!")
}
})
})
app.listen(8000,function(error) {
if(error) throw error
console.log("Server running on port 8000")
})
If you do the upload from a form where the browser submits the form for you automatically via a submit button, then it will be a browser form POST and the browser will automatically display whatever that POST request returns as the response.
If you, instead, upload form a form with a Javascript Ajax call to do the form POST, then the form response just comes back to your Javascript and nothing happens to the current web page in the browser. Your Javascript can receive that response and then decide what to do with it (if anything).
You've shown your server-side code here, but it's actually the client-side that controls what happens after the response from the upload is received back from the server. So, the changes would have to be made on the client-side to submit your form via Javascript.
Hello everyone I'm trying to create an app with a backend where the user will be able to upload a profile pic. I'm using nodejs for the back with the framework Expressjs.
However, when I try to upload an image to my server from my react-native app nothing gets saved into my images folder.
I'm on iOS right now, I don't know if it changes anything.
When I tried to do the same but without react-native directly sending POST request using Postman I was able to do so.
I don't really know why I thought it would be extension name or something but it does not change anything.
Here is my backend code first :
I'm using Multer for the upload
const storage = multer.diskStorage({
destination : function(req, file, cb ){
cb(null,'./images/');
},
filename : function(req,file,cb){
cb(null, Date.now() + file.originalname);
}
})
const upload = multer({ storage : storage,
//filer only valide file == image
fileFilter: function(req,file,cb){
let ext = path.extname(file.originalname);
if(ext != '.png' && ext != '.jpg' && ext !='jpeg'){
return cb('Only images are allowed ')
}
cb(null, true)
},
limits:{fieldSize: 25*1024*1024}
})
// route for photo TEST
router.post('/upload',upload.single('photo'), (req,res)=> {
console.log('file',req.files);
//console.log('body', req.body);
res.status(200).json({
message:'success!',
});
});
And now this is what I do on my react-native app
const createFormData = (photo, body) =>{
const data = new FormData();
data.append('photo', {
name:photo.fileName,
type:photo.type,
uri :
Platform.OS === 'android' ? photo.uri : photo.uri.replace('file://', ''), // to be sure it works with android
});
Object.keys(body).forEach(key => {
data.append(key, body[key]);
});
return data;
handleUploadPhoto = () =>{
fetch('http://localhost:5050/api/new/upload',{
method: 'POST',
body: createFormData(this.state.dataPhoto,{userId : 'test'}),
})
.then(response => response.json())
.then(response => {
console.log('upload succes', response);
alert('Photo updated succesfully!')
})
.catch(error => {
console.log('upload failed', error);
alert('Upload is not possible right now!');
})
};
I do get the alert("Photo updated succesfully on my screen" so I guess it works on the front but not on the back...
I load the image using react-native-image-picker
I don't get any error message but no files are saved into my images folder.
Thanks in advance for any help
Are you not seeing anythign related to console.log('file', req.files) which you printed ?
Verify when you upload , you are getting the file + the uploaded file related ojbects in the node console.
Enable the console body log and also view it console.log('body', req.body)
I'm creating a temporary JSON file in my NodeJS backend which holds the information the user has filled in a form. At the end of the form when user clicks on the download button, I run some Python script in NodeJS to validate the data and then create a temporary file of this JSON data and return it to user as a HTTP GET response.
Right now I'm using a timer to delete this temporary file after 10 seconds, which is bad. I want to know how to detect when the user has fully downloaded the file to their local disk from the browser so I can delete this temporary file in backend.
The client Angular code:
$scope.downloadForm = function() {
var data = formDataFactory.getDataForSubmission();
var url = '/FormSubmission/DownloadData';
// Below POST call will invoke NodeJS to write the temporary file
$http.post(url, data)
.success(function(data, status) {
$scope.downloadPath = data.filePath;
$scope.downloadFile = data.fileName;
url = '/tmp/forms/' + $scope.downloadFile;
// If the temporary file writing is successful, then I get it with a GET method
$http.get(url)
.success(function(data, status) {
$log.debug("Successfully got download data");
$window.location = $scope.downloadPath;
})
.error(function(data, status) {
$log.error("The get data FAILED");
});
})
.error(function(data, status) {
$log.error("The post data FAILED");
});
}
$scope.download = function() {
$scope.downloadForm();
setTimeout(function() { //BAD idea
$scope.deleteForm($scope.downloadPath);
}, 10000);
}
The server NodeJS code:
// POST method for creating temporary JSON file
router.post('/FormSubmission/DownloadData', function(req, res) {
if (!req.body) return res.sendStatus(400); // Failed to get data, return error
var templateString = formTmpPath + 'form-XXXXXX.json';
var tmpName = tmp.tmpNameSync({template: templateString});
fs.writeFile(tmpName, JSON.stringify(req.body, null, 4), function(err) {
if (err) {
res.sendStatus(400);
} else {
res.json({ fileName: path.basename(tmpName), filePath: tmpName, out: ''});
}
});
});
// Get method for downloading the temporary form JSON file
router.get('/tmp/forms/:file', function(req, res) {
var file = req.params.file;
file = formTmpPath + file;
res.download(file, downloadFileName, function(err) {
if (err) debug("Failed to download file");
});
});
Update:
I'm trying to use a stream now to send the data back, but for some reason this get method is called twice!? Can't understand why!!
// Get method for downloading the temporary form JSON file
router.get('/tmp/forms/:file', function(req, res) {
var filename = "ipMetaData.json";
var file = req.params.file;
file = formTmpPath + file;
var mimetype = mime.lookup(file);
const stats = fs.statSync(file);
res.setHeader('Content-disposition', 'attachment; filename=' + filename);
res.setHeader('Content-type', mimetype);
res.setHeader('Content-Length', stats.size);
console.log("Will send the download response for file: ", file);
//var path = __dirname + "\\..\\tmp\\forms\\form-auSD9X.json";
console.log("Creating read stream for path: " + file);
var stream = fs.createReadStream(file);
// This will wait until we know the readable stream is actually valid before piping
stream.on('open', function () {
// This just pipes the read stream to the response object (which goes to the client)
stream.pipe(res);
});
// This catches any errors that happen while creating the readable stream (usually invalid names)
stream.on('error', function(err) {
console.log("Caught an error in stream"); console.log(err);
res.end(err);
});
stream.on('end', () => {
console.log("Finished streaming");
res.end();
//fs.unlink(file);
});
});
if I understand your problem correctly, you can do this in different ways, but easiest way is first, remove the timer to remove the file, and remove it after the download completes from the backend as follows
router.get('/tmp/forms/:file', function(req, res) {
var file = req.params.file;
file = formTmpPath + file;
res.download(file, downloadFileName, function(err) {
if (err) debug("Failed to download file");
else {
// delete the file
fs.unlink(file,function(err){
if(err) debug(err);
})
}
});
});
The problem was with doing a get call and then change location to the file path which has the same path. I changed my API path and used the stream .on('end', callback) to remove the file.
// If the temporary file writing is successful, then I get it with a GET method
$http.get(url) --> this URL should be different from $window.location
.success(function(data, status) {
$log.debug("Successfully got download data");
$window.location = $scope.downloadPath;
})
.err
Is it possible to only allow download from Amazon's S3 for anyone who have some token and a link to some file in my bucket?
This token, for example, can be generated by my upload backend server, and appended to the uploaded file.
What possibilities do I have here?
Edit
When I said download from S3, I mean directly from S3 not through my server, only upload happens through server
You can generate presigned URLs using minio-py like this:
from minio import Minio
client = Minio('s3.amazonaws.com',
access_key='YOUR-ACCESSKEYID',
secret_key='YOUR-SECRETACCESSKEY')
downloadURL = client.presigned_get_object('mybucket', 'myobject')
You can use downloadURL on browser to download files directly from S3.
Of course it is possible, just warp the access with some authentication from your end
var backendTokenAuthenticator = require('/auth')
app.get('/download/:file',function(req, res){
if(req.query.token && req.params.file){
backendTokenAuthenticator(req.query.token,req.params.file,function(err,isAuth){
if(err){
res.status(500).end()
}else if(!isAuth){
res.status(403).end('permission denied')
}else{
var s3 = new AWS.S3();
s3.getObject({
Bucket: "myBucket",
Key: req.params.file
},
function (error, data) {
if (error) {
console.log("Failed to retrieve an object: " + error);
res.status(404).end('File not found');
} else {
res.setHeader('Content-Length', data.ContentLength);
res.setHeader('Content-Type', 'application/octet-stream');
res.end(data.Body)
}
});
}
})
}else{
res.status(403).end('token must be supplied')
}
});