I'm trying to do a post request onto my api, the api works perfectly ( I am able to post files, but not through a url), but now I'm trying to post through an url.
this is the code I have now, I removed some lines that aren't relevant to the question or were for testing.
request({
url: url + "gettoken"
, json: true
}, function (error, response, body) {
user = body;
var rs = fs.createReadStream(up.url);
var ws = request.post(url + "upload?token=" + `${user.token}&key=${user.key}&filename=${filename}`);
ws.on('drain', function () {
rs.resume();
});
rs.on('end', function () {
console.log(filename);
});
ws.on('error', function (err) {
console.error('cannot send file ' + err);
});
rs.pipe(ws);
})
Can anyone please help me.
So the idea is to upload a file that's located at up.url to another server at url + "upload?...".
Since fs.createReadStream is meant to read local files, and not URL's, you need something that can create a stream from a URL (or rather, retrieve that URL and stream the response).
You can also use request for that:
request({
url: url + "gettoken",
json: true
}, function (error, response, body) {
const user = body;
const rs = request.get(up.url);
const ws = request.post(url + "upload?token=" + `${user.token}&key=${user.key}&filename=${filename}`);
rs.on('end', function () {
console.log(filename);
});
ws.on('error', function (err) {
console.error('cannot send file ' + err);
});
rs.pipe(ws);
});
Typically, file uploads work through multipart/form-data, but your code doesn't suggest that being used here. If it is, the code would become something like this:
const ws = request.post(url + "upload?token=" + `${user.token}&key=${user.key}&filename=${filename}`, {
formData : {
the_file : rs
}
});
// no `rs.pipe(ws)`
Related
I am making a GET call with the following URL
https://auth.ebay.com/oauth2/authorize?client_id=CLIENT_ID&response_type=code&redirect_uri=REDIRECT_URI&scope=https://api.ebay.com/oauth/api_scope
This URL will redirect me to a "success.php" website from my server. With that redirection, it adds in params to the URL. For example: https://www.example.com/success.php?code=12345.
I need to get that code param from this redirection. How can I do that?
I tried to do a basic .get() call, but it doesnt seem to work..
https.get(url, (resp) => {
let data = '';
// A chunk of data has been received.
resp.on('data', (chunk) => {
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
console.log(JSON.parse(data).explanation);
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
I have tried other ways that I thought would work from research on here, like waiting for the redirect, etc. Nothing seems to work.
It is a query param ( seems to me you are working with an oAuth flow, reading about how these flows work would also help you work out an approach to this)
So I would capture it the following way
app.get('/callback', function (req, res) {
var code = req.query.code || null;
console.log(code);
}
where /callback is the URL you are redirected to and where you can capture the code to request the authorization token
Based on the code you already have it seems you might want the following.
var https = require('https');
var url = 'https://auth.ebay.com/oauth2/authorize?client_id=CLIENT_ID&response_type=code&redirect_uri=REDIRECT_URI&scope=https://api.ebay.com/oauth/api_scope';
https.get(url, (resp) => {
var location = new URL(resp.headers.location);
var code = location.searchParams.get('code');
console.log(code);
}).on("error", (err) => {
console.log("Error: " + err.message);
});
My answer assumes you are writing the code that is making the request while #Jordi Riera assumes you are writing code to process the request. Might you tell us which it is?
I'm certain I'm missing something obvious, but the gist of the problem is I'm receiving a PNG from a Mapbox call with the intent of writing it to the file system and serving it to the client. I've successfully relayed the call, received a response of raw data and written a file. The problem is that my file ends up truncated no matter what path I take, and I've exhausted the answers I've found skirting the subject. I've dumped the raw response to the log, and it's robust, but any file I make tends to be about a chunk's worth of unreadable data.
Here's the code I've got at present for the file making. I tried this buffer move as a last ditch after several failed and comparably fruitless iterations. Any help would be greatly appreciated.
module.exports = function(req, res, cb) {
var cartography = function() {
return https.get({
hostname: 'api.mapbox.com',
path: '/v4/mapbox.wheatpaste/' + req.body[0] + ',' + req.body[1] + ',6/750x350.png?access_token=' + process.env.MAPBOX_API
}, function(res) {
var body = '';
res.on('data', function(chunk) {
body += chunk;
});
res.on('end', function() {
var mapPath = 'map' + req.body[0] + req.body[1] + '.png';
var map = new Buffer(body, 'base64');
fs.writeFile(__dirname + '/client/images/maps/' + mapPath, map, 'base64', function(err) {
if (err) throw err;
cb(mapPath);
})
})
});
};
cartography();
};
It is possible to rewrite your code in more compact subroutine:
const fs = require('fs');
const https = require('https');
https.get(url, (response)=> { //request itself
if(response) {
let imageName = 'image.png'; // for this purpose I usually use crypto
response.pipe( //pipe response to a write stream (file)
fs.createWriteStream( //create write stream
'./public/' + imageName //create a file with name image.png
)
);
return imageName; //if public folder is set as default in app.js
} else {
return false;
}
})
You could get original name and extension from url, but it safer to generate a new name with crypto and get file extension like i said from url or with read-chunk and file-type modules.
Trying to allow users to upload image files to the Node.js server in a MEAN Stack application. I am using ng-file-upload for the client side angular directive. That seems to be working good enough. I run into an error when I pass the image to the server.
I use an API route to handle the work on the server side. The server will be responsible for saving the file to disk with node-multiparty module. It seems to hit route but when it tries to emit a close event I get the error. throw new Error('"name" and "value" are required for setHeader().'
The file I want is in my temp folder but it doesn't get saved to the target directory on my server plus I get the header error after the file should have been saved. So I need to stop the error and save the file with fs.rename() to the target image directory.
Here is the code that is breaking.
file api.js
// router to save images
router.route('/img/upload')
.post(function (req, res) {
console.log("image upload hits the router")
var options = {};
var count = 0;
var form = new multiparty.Form(options);
//save file to disk
form.on('file', function (name, file) {
var uploadDirectory = 'img/user/profile/';
var oldPath = file.path;
var newPath = uploadDirectory + file.originalFilename;
fs.rename(oldPath, newPath, function (err) {
if (err) throw err;
console.log('renamed complete');
});
});
// Close emitted after form parsed
form.on('close', function () {
console.log('Upload completed!');
res.setHeader('text/plain'); // Here is the line that gives an error.
res.end('Received ' + count + ' files');
});
// Parse req
form.parse(req);
});
So this is what I got to work for me
The actual line that gave me an error was setHeaders. It appears I needed to put the name and value as strings separated by a comma. This works perfectly for me now. I hope it saves everyone time coding.
// post
.post(function (req, res) {
var options = {};
var count = 0;
var form = new multiparty.Form(options);
form.on('error', function (err) {
console.log('Error parsing form: ' + err.stack);
});
//save file to disk
form.on('file', function (name, file) {
var uploadDirectory = '/img/user/profile/';
var oldPath = file.path;
var newPath = uploadDirectory + file.originalFilename;
fs.rename(oldPath, newPath, function (err) {
if (err) throw err;
console.log('renamed complete');
});
});
// Close emitted after form parsed
form.on('close', function () {
console.log('Upload completed!');
res.setHeader('Content-Type', 'text/plain');
res.end('Received ' + count + ' files');
});
// Parse req
form.parse(req);
});
I have the following code, which will retrieve a text file from an external server, and search the file for a specific string.
The function:
function checkStringExistsInFile(String, cb) {
var opt = {
host: 'site.com',
port: 80,
path: '/directory/data.txt',
method: 'GET',
};
var request = http.request(opt, function(response){
response
.on('data', function(data){
var string = data+"";
var result = ((string.indexOf(" "+ String +" ")!=-1)?true:false);
cb(null, result);
})
.on('error', function(e){
console.log("-ERR: Can't get file. "+JSON.stringify(e));
if(cb) cb(e);
})
.on('end', function(){
console.log("+INF: End of request");
});
});
request.end();
}
And this is where I call the function, and do something with the results.
checkStringExistsInFile(String, function(err, result){
if(!err) {
if(result) {
//there was a result
} else {
//string not present in file
}
} else {
// error occured
}
});
This worked great in the beginning (small text file), but my textfile is getting larger (4000 characters+) and this is not working anymore.
What can I do to solve this? Should I safe the temporary save the file on my server first, should I open the file as a stream?
It would be appreciated if you can support your answer with a relevant example. Thanks in advance!
Documentation :
If you attach a data event listener, then it will switch the stream into flowing mode, and data will be passed to your handler as soon as it is available.
If you just want to get all the data out of the stream as fast as possible, this is the best way to do so.
http://nodejs.org/api/stream.html#stream_event_data
Data event is emitted as soon as there are data, even if the stream is not completely loaded. So your code just look for your string in the first lines of your stream, then callbacks.
What to do ?
Your function should only call callback on the end() event, or as soon as it finds something.
function checkStringExistsInFile(String, cb) {
var opt = {
host: 'site.com',
port: 80,
path: '/directory/data.txt',
method: 'GET',
};
var request = http.request(opt, function(response){
var result = false;
response
.on('data', function(data){
if(!result) {
var string = data+"";
result = (string.indexOf(" "+ String +" ")!=-1);
if(result) cb(null, result);
}
})
.on('error', function(e){
console.log("-ERR: Can't get file. "+JSON.stringify(e));
if(cb) cb(e);
})
.on('end', function(){
console.log("+INF: End of request");
cb(null, result)
});
});
request.end();
}
Right now I'm using this script in PHP. I pass it the image and size (large/medium/small) and if it's on my server it returns the link, otherwise it copies it from a remote server then returns the local link.
function getImage ($img, $size) {
if (#filesize("./images/".$size."/".$img.".jpg")) {
return './images/'.$size.'/'.$img.'.jpg';
} else {
copy('http://www.othersite.com/images/'.$size.'/'.$img.'.jpg', './images/'.$size.'/'.$img.'.jpg');
return './images/'.$size.'/'.$img.'.jpg';
}
}
It works fine, but I'm trying to do the same thing in Node.js and I can't seem to figure it out. The filesystem seems to be unable to interact with any remote servers so I'm wondering if I'm just messing something up, or if it can't be done natively and a module will be required.
Anyone know of a way in Node.js?
You should check out http.Client and http.ClientResponse. Using those you can make a request to the remote server and write out the response to a local file using fs.WriteStream.
Something like this:
var http = require('http');
var fs = require('fs');
var google = http.createClient(80, 'www.google.com');
var request = google.request('GET', '/',
{'host': 'www.google.com'});
request.end();
out = fs.createWriteStream('out');
request.on('response', function (response) {
response.setEncoding('utf8');
response.on('data', function (chunk) {
out.write(chunk);
});
});
I haven't tested that, and I'm not sure it'll work out of the box. But I hope it'll guide you to what you need.
To give a more updated version (as the most recent answer is 4 years old, and http.createClient is now deprecated), here is a solution using the request method:
var fs = require('fs');
var request = require('request');
function getImage (img, size, filesize) {
var imgPath = size + '/' + img + '.jpg';
if (filesize) {
return './images/' + imgPath;
} else {
request('http://www.othersite.com/images/' + imgPath).pipe(fs.createWriteStream('./images/' + imgPath))
return './images/' + imgPath;
}
}
If you can't use remote user's password for some reasons and need to use the identity key (RSA) for authentication, then programmatically executing the scp with child_process is good to go
const { exec } = require('child_process');
exec(`scp -i /path/to/key username#example.com:/remote/path/to/file /local/path`,
(error, stdout, stderr) => {
if (error) {
console.log(`There was an error ${error}`);
}
console.log(`The stdout is ${stdout}`);
console.log(`The stderr is ${stderr}`);
});