Develop Reference

Send all image files from node.js to frontend

How can I send all image files from my backend nodejs server folder to my Reactjs client? I have created a web upload site where each user can signin and upload their files. So whenever a user signs-in, I want all the files he/she has uploaded to be visible on website.
res.sendFile didn't help. I found out that it does not send multiple files at once.
So far it is only sending 1 single file (console log shows all the files) that is visible on the client side.
Nodejs:
function getFiles (dir, files_){
files_ = files_ || [];
var files = fs.readdirSync(dir);
for (var i in files){
var name = dir + '/' + files[i];
if (fs.statSync(name).isDirectory()){
getFiles(name, files_);
} else {
files_.push(name);
}
}
return files_;
}
app.get('/loadit', verifyToken, (req, res) => {
var loadFiles = getFiles(__dirname + /data/);
jwt.verify(req.token, 'secretkey', (err, decoded) => {
if(err) {
res.sendStatus(403);
} else {
loadFiles.map((data1) => {
console.log(data1);
return res.sendFile(data1)
})
}
})
});
Is there any different approach for doing the same task? I also thought about sending all the images link as a json list to the frontend (reactjs) and then requesting images link from my nodejs server. I don't know if that is a good idea at all.

You can generate an archive file (like zip) on the fly at the server to download all images, e.g. with https://github.com/archiverjs/node-zip-stream to make a zip file with all images.
If you want to show all images, add an API to get a list of filenames and an other one to get a specific image file.

You can never send more than one file at a time, if you want to send all the images of the user, you should send a json array with all of his images, and on the frontend fetch them one by one.

send file from node to client for download

I am using an ajax call from browser
so, on button click
a function is called
for route '/file'
app.get('/filez',function(req,res){
var id = req.params.id;
console.log('id is : ',id);
var video = ytdl(url, { filter: (format) => format.container === 'mp4' })
.pipe(fs.createWriteStream('video.mp4'));
res.download('video.mp4');
now, the file is being downloaded to the server at the moment.
but what I want to do is to send the file so the client can download it from browser.
i dont want the file to be downloaded to the server.
Here's the ajax request I made from the browser using a button click.
and I want to get the file as the response which can be downloaded to the client computer.
function myAjaxCall(){
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function(){
if(xhr.readyState ===XMLHttpRequest.DONE && xhr.status ===200){
console.log('response has come',xhr.response);
return xhr.response;
}
};
xhr.open('GET','/filez',true);
xhr.send();
};
EXPLANATION : well, not really. i was thinking more along these lines - so on button click an ajax call is sent to server and it's supposed to get back a response. I want the file to be sent via this response.
so, how do I accomplish this?

pass the get request in a href tag and it will download the file to the client , you are writing the file in server side so it is being created their , you have to have the file been created on the server side and then serve the file through res.download to the client side and later delete the file from the server side to download the file in clinet side
suppose your url looks like http://getsomefile.com/xyz?newfile=video.mp4 then in your client sides a tag you can have this url as on some domain http://getsomefile.com with route xyz to serve the file
<a href="http://getsomefile.com/xyz?newfile=video.mp4" download>
where i am assuming on client side you have some route defined as
app.get('/xyz,function(req,res){
var file = req.query.newfile;
})
and then you can use fs.unlink to delete the files through a cron like or have a manual cleaner used or do something like this
app.get('/xyz', function(req, res){
var file = req.query.newfile
res.download(realFilepath, file , function(err) {
if (!err) {
fs.unlink(realFilepath);
}
});
});
realFilepath is the actual path of file kept in server , which you can gracefully delete after the file has been served.

Downloading Torrent with Node.JS

I was wondering if anyone had an example of how to download a torrent using NodeJS? Essentially, I have an RSS Feed of torrents that I iterate through and grab the torrent file url, then would like to initiate a download of that torrent on the server.
I've parsed and looped through the RSS just fine, however I've tried a few npm packages but they've either crashed or were just unstable. If anyone has any suggestions, examples, anything... I would greatly appreciate it. Thanks.
router.get('/', function(req, res) {
var options = {};
parser.parseURL('rss feed here', options, function(err, articles) {
var i = 0;
var torrent;
for (var title in articles.items) {
console.log(articles.items[i]['url']);
//download torrent here
i++;
}
});
});

You can use node-torrent for this.
Then, to download a torrent:
var Client = require('node-torrent');
var client = new Client({logLevel: 'DEBUG'});
var torrent = client.addTorrent('a.torrent');
// when the torrent completes, move it's files to another area
torrent.on('complete', function() {
console.log('complete!');
torrent.files.forEach(function(file) {
var newPath = '/new/path/' + file.path;
fs.rename(file.path, newPath);
// while still seeding need to make sure file.path points to the right place
file.path = newPath;
});
});
Alternatively, for more control, you can use transmission-dæmon and control it via its xml-rpc protocol. There's a node module called transmission that does the job! Exemple:
var Transmission = require('./')
var transmission = new Transmission({
port : 9091,
host : '127.0.0.1'
});
transmission.addUrl('my.torrent', {
"download-dir" : "/home/torrents"
}, function(err, result) {
if (err) {
return console.log(err);
}
var id = result.id;
console.log('Just added a new torrent.');
console.log('Torrent ID: ' + id);
getTorrent(id);
});

If you are working with video torrents, you may be interested in Torrent Stream Server. It a server that downloads and streams video at the same time, so you can watch the video without fully downloading it. It's based on torrent-stream library.
Another interesting project is webtorrent. It's a nice torrent library that works in both: NodeJs & browser and has streaming support. From my experience, it doesn't have very good support in the browser, but should fully work in NodeJS.

Meteor - What's the best way to get files from a post request?

I'm currently using the redactor wysiwyg, which has an image upload implementation that sends a post request to a given url then expects back some JSON like: { "filelink": "/static/img.jpg" } in order to display the uploaded image.
My current approach has been to create a server-side route, which would get the image from the request body, send it to s3, save the meta data (collectionFS), then return the necessary JSON via the response.
I've instantiated redactor like this:
Template.editor.rendered = function() {
$("#editor").redactor({
imageUpload: "s3"
});
};
And the server-side router looks like this:
Router.route('/s3', function () {
this.response.setHeader("Content-Type", "text/html");
var data = JSON.stringify(this.request.body;
var res = this.response;
res.end(data);
}, {where: 'server'}
);
Unfortunately, this returns a blank JSON object. I have tried with request.files and request.body.files, but these don't work.
I know the route is working, because I can send plain html via the response. And, I can definitely see binary data of the uploaded file in the post request in firebug, but I can't seem to get Meteor to get those files.

The POST data is not available at the time the middleware is called, but instead it is received in data events. You need to handle those events to get your file data. Example:
Router.route('...', function() {
var buffers = [];
var totalLength = 0;
this.request.on('error', function(err) {
// handle network error here
});
this.request.on('data', function(chunk) {
buffers.push(chunk);
totalLength += chunk.length;
if(totalLength > SOME_LARGE_CONSTANT) {
// handle data overflow here
}
});
this.request.on('end', function() {
var data = Buffer.concat(buffers);
// handle properly received data here
});
});

save incoming file from s3 w/nodejs & knox?

This is likely very basic because the docs leave it out... from knox docs:
"Below is an example GET request on the file we just shoved at s3, and simply outputs the response status code, headers, and body."
client.get('/test/Readme.md').on('response', function(res){
console.log(res.statusCode);
console.log(res.headers);
res.setEncoding('utf8');
res.on('data', function(chunk){
console.log(chunk);
});
}).end();
Easy enough, but how do I save the incoming data as a local file? new BufferList() or something?
I'm trying to build an 'on-the-fly' image resizing service that loads images from s3 or cloudfront and returns them sized based on the request. The browser then caches the sized images instead of the full ones straight from s3. Of course, I need this basic bit working first! Any ideas?
Thanks guys!

It doesn't look like knox supports the stream API, so you can't use stream.pipe() and get proper backpressure. However, chances are your disk will be faster than S3, so this probably doesn't matter.
In the "response" callback, open up a writable stream from the filesystem module with var outstream = fs.createWriteStream(filename);. In the "data" callback, call outstream.write(chunk); Hopefully there is a "end" callback you can use the close the write stream as well.

As an alternative to answer above, you can save the incoming file to buffer like this:
var buffer = '';
client.get('/test/Readme.md').on('response', function(res){
res.setEncoding('utf8');
res.on('data', function(chunk){
buffer += chunk;
});
res.on('end', function(){
// do something with the buffer such as save it to file,
// or directly resize the image here.
// eg. save to file:
fs.writeFile('downloaded_readme.md', buffer, 'utf8', function (err) {
});
});
}).end();