Sorry for the vague question.. but I'm not sure quite what the problem is.
I have a node http server that I'm using to serve JSON data to a web app. It works great, but my JSON strings are starting to get large(10-12 MB), so I want to add compression with zlib.
The JSON data is in a string variable and I want to compress and then write to the response object... but the results that go back to the client seem to always have with perfect headers, and no content. Here is my deliverResponse function:
var deliverResult = function (data, response, callback, acceptEncoding){
var payload = callback + '(' + JSON.stringify(data) + ');';
if (acceptEncoding.match(/\bdeflate\b/)) {
response.writeHead(200, { 'Content-Encoding': 'deflate', 'Content-Type': 'text/javascript; charset=UTF-8' });
zlib.deflate(payload, function(err, result){
if(!err){
//console.log(result.toString('utf8')); // I have data on the console here
response.write(result.toString('utf8')); // No data sent here
}
});
} else if (acceptEncoding.match(/\bgzip\b/)) {
response.writeHead(200, { 'Content-Encoding': 'gzip', 'Content-Type': 'text/javascript; charset=UTF-8' });
zlib.gzip(payload, function(err, result){
if(!err){
response.write(result.toString('utf8'));
}
});
} else {
writelog('INFO', 'Returning data without compression\n');
response.writeHead(200, { 'Content-Type': 'text/javascript; charset=UTF-8' });
response.write(payload);
}
response.end();
}
The http server examples with zlib use streams and the pipe function, but I'm not sending a file as I generate the JSON data in the app from a database, so I am basing this on the convenience method examples. My troubleshooting so far I know that the response object is good, and that result.toString('utf8') outputs gobeldy-gook as expected. If I don't send an acccept-encoding header to the server, it sends plain text perfectly - so it had got to be the compression functions.
Anyone have any idea about this? I'm pretty sure it has to to with my lack of understanding about streams, pipes, buffers and the zlib object, and it's probably just a syntax issue, so hopefully someone who understands all this can help me out :)
Cheers
Solved....
Stupid problem.. the response.write is being called in a async function, so it's doing the write.end() before the response.write and an empty response is sent... replaced the response.write with response.end in the callback and it works perfectly :)
Related
I'm incredibly new at trying to do anything relating to webservers and the like and I've gotten stuck trying to send data from a javascript that runs on a website to a server that runs locally. I've been able to get them to communicate, but all the data I get on the server is always just "[object object]" instead of the string that I send from the browser.
This is how the server is looking currently, very bareboned:
http.createServer(function(request, response){
response.writeHead(200, {"Content-Type": "text/plain"});
console.log("request recieved: " + response + request);
}).listen(8001);
And this is what i send from the browser:
var url = "http://localhost:8001";
$.ajax({
url: url,
type: "POST",
data: "Hello friend",
contentType: "text/plain",
});
I've also tried something like this which results in the same problem:
var http = new XMLHttpRequest();
var sendData = "HELLO";
http.open("POST", "http://localhost:8001", true);
http.setRequestHeader("Content-type", "text/plain");
http.send(sendData);
I've also tried to use JSON.stringify on the server-side to try to get the string, but this returns an error stating that it cannot be used on a 'circular object'.
I'm sorry if this question is really stupid, as I said, I'm a complete beginner and this is the first time I've tried to do something remotely similar. I've tried researching what to do differently, but after trying countless options I felt that I needed to ask here instead.
Thanks in advance! And if any more information is needed I'd be happy to try to expand on the issue!
When you use concatenation (+) operator then javascript will convert object to string. Default String representation of object is [object object]. That's why it's printing [object object].
http.createServer(function(request, response){
response.writeHead(200, {"Content-Type": "text/plain"});
console.log("request recieved: " , response , request);
}).listen(8001);
And if you want to get the body then you have the listen to data event and consume the data.
http
.createServer(function(request, response) {
let body = "";
request.on("data", chunk => {
body += chunk.toString(); // convert Buffer to string
});
request.on("end", () => {
console.log(body);
response.end('ok');
});
})
.listen(8001);
I will recommend to use express package to avoid all boilerplate code.
You can access the body with request.body. You can't JSON.stringify the whole request because, as you saw, it holds ciruclar deps, but you can do so with the request.body.
I'am learning AngularJs and I've tried to write a very basic script sending an http request to Ebay public API, I've signed up and got my API keys, I've read the docs several times and wrote this basic code :
$scope.getQueryUrl = function () {
// Some unrelated code ...
$scope.queryUrl["Ebay"] = "http://svcs.sandbox.ebay.com/services/search/FindingService/v1?OPERATION-NAME=findItemsByKeywords&SERVICE-NAME=FindingService&SERVICE-VERSION=1.0.0&GLOBAL-ID=EBAY-US&SECURITY-APPNAME="+dataAuth.EbayKeyApi+"&RESPONSE-DATA-FORMAT=XML&keywords="+$scope.qtext ;
};
$scope.sendRequest = function () {
$scope.getQueryUrl(); // Gets the query url after adding all the parameters
alert($scope.queryUrl.Ebay);
$http.get($scope.queryUrl["Ebay"]).then(
function(response){
alert("success" + response.data );
},
function(response){
alert("error" + response.statusCode );
});
};
How this code should work :
It should create a formated Ebay query url, send it through HTTP GET request and sending back the response .
Note : $scope.qtext & dataAuth.EbayKeyApi are already assigned with their respective values .
What's the problem:
The problem is that using this Angularjs script, the code doesn't work, the alert "Error" is shown, and the response.statusCode is undefined .
But when I copy the formatted Ebay query link in Firefox it works perfectly and the XML response is shown .
The formatted Ebay query was generated using the script provided .
I think it's a header related problem .
$http has some default headers defined. $http sends Json payload and accepts Json as the response by default. Since you are dealing with XML you have to explicitly specify the accepted response type as XML using the header:
Accept: application/xml
Please use the following function with appropriate headers and you should get the response. Also, please look into any Cross Origin Resource Sharing (CORS) restrictions on the ebay API.
function getRequest(url) {
$http({
method: "GET",
url: url,
headers: {
'Content-Type': 'application/xml, text/xml',
'Accept': 'application/xml, text/plain, * / *'
}
})
.then(function (response) {
alert(response.data);
},
function (error) {
alert (error);
});
}
Thank you,
Soma.
Hi I'm trying to interact with an HTTP api and I am required to send an HTTP post request some form data. And then in the HTTP-BODY element I am required to supply some more data (can be either csv or xml). So my challenge is in understanding how to do this via the Needle API as its unclear via just the documentation.
I have been able to successfully do :
needle.post( host, formdata, function( err, resp, body){ });
The above works for the situations where I don't require posting the HTTP-BODY content.
I have tried the following but receive an error :
options = {
headers : { 'Content-Type': 'text/tab-separated-values; charset=iso-8859-1'},
body : text
}
needle.post( host, formdata, options, function( err, resp, body){ });
Could any one guide me to the correct way to do this somewhat basic task.
Thanks.
I used the Restler package and ditched Needle. Needle looks great and I'm sure there is a way to do this, but I couldn't figure it out in time.
https://github.com/danwrong/restler
I am using Angularjs and nodejs in the project and working on file uploads. I want to send the post request to the url endpoint in a secure way as I need to attach accesstoken with the request. So the way I did this was, I added the directive to choose the file from UI and once it gets the file, I append it using FormData() like this in the controller
var fd = new FormData();
fd.append('file',myFile);
and sending this formdata object to the nodejs server like mentioned here http://uncorkedstudios.com/blog/multipartformdata-file-upload-with-angularjs
expect this request will be going to my nodejs server url from there I will be making another post request to external web service
$http.post('api/collections/upload',fd, {
transformRequest: angular.identity,
headers: {
'Content-type': undefined
}
});
So it will attach the right content-type and boundaries in the request. I am getting the file on server side nodejs when I do
function(req,res){
console.log(req.files); //I am able to see the file content
}
It is uploaded on the nodejs server side.
Now I want to make a post request using the req.files to a different endpoint along with proper accessToken and headers. Tried many things but not able to make the request go thru. Not sure how can I attach the imagedata/ req.files along with the request. I tried these two things mentioned in request npm module https://www.npmjs.org/package/request
1)
request.post({
url: 'https://www.example.com/uploadImage',
headers: {
'Authorization': <accessToken>,
'Content-type': 'multipart/form-data'
},
body: req.files
});
Don't know how can I attach and binary data with this request and how can I put boundary. Is boundary needed when you want to send the uploaded image with this request?
2)
fs.createReadStream(req.files.file.path, {encoding: base64}).pipe(request.post({
url: 'https://www.example.com/uploadImage',
headers: {
'Content-type': 'multipart/form-data'
}
}));
Can someone please suggest some ideas on how can I send this post request using request npm module? Thanks.
Documentation here has lots of examples of doing exactly what you describe:
https://github.com/mikeal/request#streaming
As can be seen in that link, the request library adds a .pipe() method to your http's req object, which you should be able to use like the examples in the link:
function(req, res) {
req.pipe(request.post('https://www.example.com/uploadImage');
}
Or something similar.
You were nearly there with your #2 try, but that would only work if you have previously written the file out to disk and were reading it in with fs.createReadStream()
your suggestion helped me to atleast know what I was trying was right. Another article that solved my problem was this http://aguacatelang.wordpress.com/2013/01/05/post-photo-from-node-js-to-facebook/ .Basically, here is what I did and it worked. Thanks for your suggestion.
var form = new FormData();
form.append('file', fs.createReadStream(__dirname + '/image.jpg'));
var options = {
url: 'https://www.example.com/uploadImage?access_token='+ <accesstoken>,
headers: form.getHeaders()
};
form.pipe(request.post(options,function(err,res){
if(err){
log.debug(err);
}
else {
log.debug(res);
}
}));
I try to send some data to my node.js / node-static app and store it in a file.
Right now i use a basic html form and capture the pathfile:
Server side:
var daten = url_parts.query;
if (pathfile== '/get.htm'){ // get.htm just redirects back to form
console.log('Device 1:', daten);
fs.writeFile("devices.txt", devices(daten), function(err) {
if(err) {console.log(err);} else {console.log("The file was saved!");}
});
}
I feel dirty using this but i am a bit overwhelmed by the possibilities to perform such a simple task.
Using jQuery seems the way to go.
My problem: How can I send the data without a page refresh or redirecting to another page? What is the elegant way? Where do I perform the 'magic'?
Client or Server side?
I ended up restructuring server and client side. Maybe someone finds this useful or can improve this suggestion:
Server:
if (pathfile== '/get.json'){
fs.writeFile("devices.txt", devices(daten), function(err) {
if(err) {console.log(err);} else {console.log("The file was saved!");}
});
response.writeHead(200, { "Content-type": "application/json" });
response.end();
return;
}
Client:
$('#enter').click(function(){
var data = $('form').serialize() ;
$.ajax({
type: "get",
url: "get.json",
data: data,
success: function(){alert('data was send!');},
error: function(){alert('server not responding!');}
});
return false;
});