Develop Reference

res.download file from Amazon S3

I am trying to download a file from outside of my root directory however every time I try, it tries to take it from the root directory. I will need the user of my site to be able to download these files.
The file has initially been uploaded to Amazon S3 and I have accessed it using the getObject function.
Here is my code:
app.get('/test_script_api', function(req, res){
var fileName = req.query.File;
s3.getObject(
{ Bucket: "bucket-name", Key: fileName },
function(error, s3data){
if(error != null){
console.log("Failed to retrieve an object: " + error);
}else{
//I have tried passing the S3 data but it asks for a string
res.download(s3data.Body);
//So I have tried just passing the file name & an absolute path
res.download(fileName);
}
}
);
});
This returns the following error:
Error: ENOENT: no such file or directory, stat '/home/ec2-user/environment/test2.txt'
When I enter an absolute path it just appends this onto the end of /home/ec2-user/environment/
How can I change the directory res.download is trying to download from?
Is there an easier way to download your files from Amazon S3?
Any help would be much appreciated here!

I had the same problem and I found this answer:
NodeJS How do I Download a file to disk from an aws s3 bucket?
Based on that, you need to use createReadStream() and pipe().
R here more about stream.pipe() - https://nodejs.org/en/knowledge/advanced/streams/how-to-use-stream-pipe/
res.attachment() will set the headers for you.
-> https://expressjs.com/en/api.html#res.attachment.
This code should work for you (based on the answer in the above link):
app.get('/test_script_api', function (req, res) {
var fileName = req.query.File;
res.attachment(fileName);
var file = s3.getObject({
Bucket: "bucket-name",
Key: fileName
}).createReadStream()
.on("error", error => {
});
file.pipe(res);
});
In my case, on the client side, I used
This made sure that the file is downloading.

NodeJS download file from AWS S3 Bucket

I'm trying to make an endpoint in NodeJS/Express for downloading content from my AWS S3 Bucket.
It works well, I can download the file in the client side but I can also see the stream preview in the Network tab which is annoying...
QUESTION
I'm wondering if what I'm doing is correct and a good practice.
Also would like to know if it's normal to see the output stream in the Network tab.
How should I properly send I file from S3 to my client application using NodeJS/Express?
I'm pretty sure other websites requests don't let you preview the content with a: "Fail to load response data".
This is what I do in my NodeJS application to get the stream file from AWS S3:
download(fileId) {
const fileObjectStream = app.s3
.getObject({
Key: fileId
})
.createReadStream();
this.res.set("Content-Type", "application/octet-stream");
this.res.set(
"Content-Disposition",
'attachment; filename="' + fileId + '"'
);
fileObjectStream.pipe(this.res);
}
And in the client side I can see this:

I think the issue is with the header :
//this line will set proper header for file and make it downloadable in client's browser
res.attachment(key);
// this will execute download
s3.getObject(bucketParams)
.createReadStream()
.pipe(res);
So code should be like this (This is what I am doing it in my project handling file as res.attachment or res.json in case of error so client can display error to end user) :
router.route("/downloadFile").get((req, res) => {
const query = req.query; //param from client
const key = query.key;//param from client
const bucketName = query.bucket//param from client
var bucketParams = {
Bucket: bucketName,
Key: key
};
//I assume you are using AWS SDK
s3 = new AWS.S3({ apiVersion: "2006-03-01" });
s3.getObject(bucketParams, function(err, data) {
if (err) {
// cannot get file, err = AWS error response,
// return json to client
return res.json({
success: false,
error: err
});
} else {
res.attachment(key); //sets correct header (fixes your issue )
//if all is fine, bucket and file exist, it will return file to client
s3.getObject(bucketParams)
.createReadStream()
.pipe(res);
}
});
});

Upload stream to azure blob storage

I would upload an audio file to azure blob storage.
First I make a http request to url to get the file .
Then I would "directly" save it to azure blob storage without the need to store it in server then upload it.
Here's my code:
request
.get(url, {
auth: {
bearer: token
}
})
.on("response", async function(response) {
const res = await blobService.createAppendBlobFromStream(
"recordings", // container name
"record.wav",
response, // should be stream
177777, // stream length
function(err, res) {
try {
console.log(res);
} catch (err) {
console.log(err);
}
}
);
});
Actually when I upload a file in blob storage and check my database I get an empty file with no data inside, I think I'm not sending the data stream correcty.
What I expect is to get the audio file in blob storage with data inside that I get from the get request
I should also specify the stream length but I don't know how to get it I putted a random number but it should be the right stream length. I have checked response object but I havn't found that infomation.

I think you can't upload a file directly so first of all create a folder in your blob storage then create a file.
Read the selected file data and write it into the created file using file stream or else.
here is the upload file code,
app.post('/upload',function(req,res){
if(req.files.fileInput){
var file = req.files.fileInput,
name = file.name,
type = file.mimetype;
var uploadpath = __dirname + '/uploads/' + name;
file.mv(uploadpath,function(err){
if(err){res.send("Error Occured!")}
else {res.send('Done! Uploading files')}
});
}
else {
res.send("No File selected !");
res.end();
};
})

Using SAS URL upload browser file to Azure Blob

My Requirement is to copy, list of the files from client(browser) to azure blob using SAS URL. So I am using azure-sdk-for-node, but its look like its not working for me. Can someone recommend any solution or any other library ?.
var azure = require('azure-storage');
var blobSvc = azure.createBlobServiceWithSas('https://XXXXXXXXXX', 'XXXXXXXXX"');
blobSvc.createBlockBlobFromBrowserFile('testmigration', 'taskblob', 'abc.txt', function (error, result, response){
if (!error) {
console.log('file uploaded failed', error);
} else {
console.log('file uploaded success');
}})
ERROR : blobSvc.createBlockBlobFromBrowserFile is not a function

How to get all download url from firebase storage?

I'm a newbie and I am trying to get all download url from firebase storage and display it in a table or a list .

The download URL isn't provided in the snapshot you receive when your file is uploaded to Storage. It'd be nice if Firebase provided this.
To get the download URL you use the method getDownloadURL() on your file's storage reference. There are two types of storage references, firebase.storage.ref() and firebase.storage.refFromURL(. The former uses the path to the file, which is provided in the snapshot after your file is uploaded to Storage. The latter uses either a Google Cloud Storage URI or a HTTPS URL, neither of which is provided in the snapshot. If you want to get the download URL immediately after uploading a file, you'll want to use firebase.storage.ref() and the file's path, e.g.,
firebase.storage.ref('English_Videos/Walker_Climbs_a_Tree/Walker_Climbs_a_Tree_3.mp4');
Here's the full code for uploading a file to Storage and getting the download URL:
firebase.storage().ref($scope.languageVideos + "/" + $scope.movieOrTvShow + "/" + $scope.videoSource.name).put($scope.videoSource, metadata) // upload to Firebase Storage
.then(function(snapshot) {
console.log("Uploaded file to Storage.");
firebase.storage().ref(snapshot.ref.location.path).getDownloadURL() // get downloadURL
.then(function(url) {
var $scope.downloadURL = url;
})
.catch(function(error) {
console.log(error);
});
})
.catch(function(error) {
console.log(error);
});