NodeJS + ExpressJS Compression doing nothing? - javascript

I have read that I should be compressing requests to my Node server, so I used npm to install the compression module, added it with require() to my server.js, then passed it in as a function to app.use.
Then I looked at the network tab after, and I wanted to see how much the compression had saved me in kb. So I took the compression off, restarted my server, and it was the same amount of kb as with compression turned on?
Here is my server.js
var express = require('express'),
app = express(),
path = require('path'),
apiRouter = require('./app/routes/api'),
mongoose = require('mongoose'),
compression = require('compression');
app.use(compression());
app.use(express.static('public'));
app.use('/api', apiRouter);
app.use('*', function(req, res) {
res.sendFile(path.join(__dirname + '/public/index.html'));
});
mongoose.connect('mongodb://localhost/triviaattack');
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {
//Connected to DB successfully.
});
app.listen(1337);

Compression does not work unless the client sends a "Accept-Encoding:gzip" request header. You can test the compression here.

When turning compression on and off for testing make sure that you're doing a hard reload in chrome dev tools or otherwise you'll get a 'not-modified' response that is not compressed.
Your example code is working for me!

The compression middleware is a demanding horseman.
You need to set two things for it to work:
On the server, you need to specify a threshold size in bytes (default is 1024 (1kb) ) beyond which the response will be compressed. Start with 0:
app.use(compression({
threshold: 0
}))
On the client, you need to set a header called Accept-Encoding to 'gzip' .
GET /mimic HTTP/1.1
Accept-Encoding: gzip
Host: localhost:3005
To decompress a response from the client side, you can do sth like this:
const response = await axios.post(
'http://localhost:3005/mimic', { headers: { "Accept-Encoding": "gzip" }, responseType: 'arraybuffer' })
const xmlData = zlib.gunzipSync(response.data).toString()
In the above code, I am using the zlib and axios modules. zlib is a nodejs builtin module. There are other approaches if you're doing it from the browser.
Notice that response.data is an array buffer.

Related

Server not receiving HTTP requests from Client

First a quick preface I think may be helpful: I am new to splitting my client and server into separate web apps. My previous apps have all had my server.js at the root level in my directory and the client (typically a create-react-app) in a /client folder.
What I wanted to do: Host a single express.js server on the web which multiple other web applications could make requests to.
I did the first part using an express server and aws elastic beanstalk.
server.js
require('dotenv').config()
const express = require('express');
const app = express();
const cors = require('cors');
const PORT = process.env.PORT || 5000;
const Mongoose = require('mongoose');
const { Sequelize } = require("sequelize");
//ROUTES
const APIUser = require('./routes/api/mongo/api-user');
more routes...
//INITIATE DATA MAPPING CONNECTIONS START
Mongoose.connect(
process.env.MONGO_URI,
{ useNewUrlParser: true, useUnifiedTopology: true },
console.log("connected to MongoDB")
);
const Postgres = new Sequelize(process.env.PG_CONN_STRING);
try {
Postgres.authenticate()
.then(console.log("connected to Postgres"));
} catch {
console.log("Postgres connection failed")
}
//INITIATE DATA MAPPING CONNECTIONS END
//middleware
app.use(cors())
more middleware...
//home route
app.get('/api', (req, res) => {
console.log('RECEIVED REQ to [production]/api/')
res.status(200).send('models api home').end();
})
//all other routes
app.use('/api/user', APIUser);
more route definitions...
//launch
app.listen(PORT, () => console.log(`listening on port ${PORT}`));
The log file for successful boot up on aws: https://imgur.com/vLgdaxK
At first glance it seemed to work as my postman requests were working. Status 200 with appropriate response: https://imgur.com/VH4eHzH
Next I tested this from one of my actual clients in localhost. Here is one of my react client's api util files where axios calls are made to the backend:
import { PROXY_URL } from '../../config';
import { axiosInstance } from '../util';
const axiosProxy = axios.create({baseURL: PROXY_URL}); //this was the most reliable way I found to proxy requests to the server
export const setAuthToken = () => {
const route = "/api/authorization/new-token";
console.log("SENDING REQ TO: " + PROXY_URL + route)
return axiosProxy.post(route)
}
export const authorize = clientsecret => {
const route = "/api/authorization/authorize-survey-analytics-client";
console.log("SENDING REQ TO: " + PROXY_URL + route)
return axiosProxy.post(route, { clientsecret })
}
Once again it worked... or rather I eventually got it to work: https://imgur.com/c2rPuoc
So I figured all was well now but when I tried using the live version of the client the request failed somewhere.
in summary the live api works when requests are made from postman or localhost but doesn't respond when requests are made from a live client https://imgur.com/kOk2RWf
I have confirmed that the requests made from the live client do not make it to the live server (by making requests from a live client and then checking the aws live server logs).
I am not receiving any Cors or Cross-Origin-Access errors and the requests from the live client to the live server don't throw any loud errors, I just eventually get a net::ERR_CONNECTION_TIMED_OUT. Any ideas where I can look for issues or is there more code I could share?
Thank you!
Add a console.log(PROXY_URL) in your client application and check your browser console if that's logged correctly.
If it works on your local client build, and through POSTMAN, then your backend api is probably good. I highly suspect that your env variables are not being set. If PROXY_URL is an emplty string, your axios requests will be routed back to the origin of your client. But I assume they have different origins since you mention that they're separate apps.
Remember environment variables need to prefixed with REACT_APP_ and in a production build they have to be available at build time (wherever you perform npm run build)

Express: PayloadTooLargeError: request entity too large

I'm getting this error when I try to send a Base64 string in a POST request.
POST /saveImage 413 10.564 ms - 1459
PayloadTooLargeError: request entity too large
Already tried
--> app.use(bodyParser.urlencoded({ limit: "50mb", extended: true, parameterLimit: 50000 }))
--> app.use(bodyParser.urlencoded({limit: '50mb'}));
--> app.use(bodyParser({limit: '50mb'}));
Here's my code (api.js class)
const express = require('express');
var app = express();
const router = express.Router();
var Connection = require('tedious').Connection
var Request = require('tedious').Request
var TYPES = require('tedious').TYPES
var multer = require('multer');
....
....
....
router.post('/saveImage', (req, res) => {
request=new Request('SAVE_IMAGE',(err, rowCount, rows)=>{
if(err){
console.log(err);
}
});
request.addParameter("Base64Image", TYPES.Text, req.body.IMG)
connection.callProcedure(request);
});
API CALL (Image class contains a Base64 format image and other fields, but I guess the problem occurs because of the Base64 string length. Small images don't cause any trouble)
create(image: Image) {
return this._http.post('/saveImage', image)
.map(data => data.json()).toPromise()
}
I was having the same error. I tried what you tried it did not work.
I guess you are uploading a file. The simple way to solve this is to not set a Content-Type.
my problem was that I was setting on my headers: Content-Type: application/json and I am [was] using multer (expressjs middle for uploading files).
I have the error whenever I try uploading a file.
So when using postman or making such requests using any tools or libraries like axiosjs or fetch() API do not set content-type.
Once you remove the Content-type it will work. That is what I did
on my code, I have:
const express = require('express');
...
const app = express();
app.use(express.json());
...
...And it is working because I removed Content-Type on my postman headers.
Make sure you are not using Content-Type on the headers.
I would recommend you to use express instead of body-parser, as body-parser got merged back in express a long ago.
I am using this code and it seems to work fine, setting the limit option to 200mb, of both express.json and express.urlencoded
app.use(express.json({ limit: "200mb" }));
app.use(express.urlencoded({ extended: true, limit: "200mb" }));
Source: express.json vs bodyparser.json

Request body in express is undefined?

I am making a simple POST request using Alamofire (in iOS) and handling it in node using express.
My code in iOS:
let boop: [String: AnyObject] = ["username":"fakeuser"];
Alamofire.request(.POST,"http://localhost:3000/test", parameters: boop, encoding: .JSON)
And this is my code in node:
var app = require('express')();
var http = require('http').Server(app);
app.post('/test', function(req, res){
console.log("THE SERVER HAS RECEIVED THE POST! \n")
console.log(req.body);
});
http.listen(PORT, function(){
console.log('listening on *:3000');
});
My terminal console prints out "the server has received the post" , so I know that the post is actually triggered. The issue is that instead of logging the req.body, it instead prints out "undefined". I've looked around and it seems like a "body parser" thing needs to be configured but apparently that is obsolete with the new version of express. So I am lost as to what to do.
Any advice?
I'm pretty sure you need to add the body-parser to your express app to parse the JSON.
const bodyParser = require('body-parser');
app.use(bodyParser.json());
See http://expressjs.com/de/api.html#req.body.

Error: request entity too large

I'm receiving the following error with express:
Error: request entity too large
at module.exports (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/node_modules/raw-body/index.js:16:15)
at json (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/middleware/json.js:60:5)
at Object.bodyParser [as handle] (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/middleware/bodyParser.js:53:5)
at next (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/proto.js:193:15)
at Object.cookieParser [as handle] (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/middleware/cookieParser.js:60:5)
at next (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/proto.js:193:15)
at Object.logger (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/middleware/logger.js:158:5)
at next (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/proto.js:193:15)
at Object.staticMiddleware [as handle] (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/middleware/static.js:55:61)
at next (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/proto.js:193:15)
TypeError: /Users/michaeljames/Documents/Projects/Proj/mean/app/views/includes/foot.jade:31
29| script(type="text/javascript", src="/js/socketio/connect.js")
30|
> 31| if (req.host='localhost')
32| //Livereload script rendered
33| script(type='text/javascript', src='http://localhost:35729/livereload.js')
34|
Cannot set property 'host' of undefined
at eval (eval at <anonymous> (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/jade/lib/jade.js:152:8), <anonymous>:273:15)
at /Users/michaeljames/Documents/Projects/Proj/mean/node_modules/jade/lib/jade.js:153:35
at Object.exports.render (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/jade/lib/jade.js:197:10)
at Object.exports.renderFile (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/jade/lib/jade.js:233:18)
at View.exports.renderFile [as engine] (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/jade/lib/jade.js:218:21)
at View.render (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/lib/view.js:76:8)
at Function.app.render (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/lib/application.js:504:10)
at ServerResponse.res.render (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/lib/response.js:801:7)
at Object.handle (/Users/michaeljames/Documents/Projects/Proj/mean/config/express.js:82:29)
at next (/Users/michaeljames/Documents/Projects/Proj/mean/node_modules/express/node_modules/connect/lib/proto.js:188:17)
POST /api/0.1/people 500 618ms
I am using meanstack. I have the following use statements in my express.js
//Set Request Size Limit
app.use(express.limit(100000000));
Within fiddler I can see the content-length header with a value of: 1078702
I believe this is in octets, this is 1.0787 megabytes.
I have no idea why express is not letting me post the json array I was posting previously in another express project that was not using the mean stack project structure.
I had the same error recently, and all the solutions I've found did not work.
After some digging, I found that setting app.use(express.bodyParser({limit: '50mb'})); did set the limit correctly.
When adding a console.log('Limit file size: '+limit); in node_modules/express/node_modules/connect/lib/middleware/json.js:46 and restarting node, I get this output in the console:
Limit file size: 1048576
connect.multipart() will be removed in connect 3.0
visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives
connect.limit() will be removed in connect 3.0
Limit file size: 52428800
Express server listening on port 3002
We can see that at first, when loading the connect module, the limit is set to 1mb (1048576 bytes). Then when I set the limit, the console.log is called again and this time the limit is 52428800 (50mb). However, I still get a 413 Request entity too large.
Then I added console.log('Limit file size: '+limit); in node_modules/express/node_modules/connect/node_modules/raw-body/index.js:10 and saw another line in the console when calling the route with a big request (before the error output) :
Limit file size: 1048576
This means that somehow, somewhere, connect resets the limit parameter and ignores what we specified. I tried specifying the bodyParser parameters in the route definition individually, but no luck either.
While I did not find any proper way to set it permanently, you can "patch" it in the module directly. If you are using Express 3.4.4, add this at line 46 of node_modules/express/node_modules/connect/lib/middleware/json.js :
limit = 52428800; // for 50mb, this corresponds to the size in bytes
The line number might differ if you don't run the same version of Express.
Please note that this is bad practice and it will be overwritten if you update your module.
So this temporary solution works for now, but as soon as a solution is found (or the module fixed, in case it's a module problem) you should update your code accordingly.
I have opened an issue on their GitHub about this problem.
[edit - found the solution]
After some research and testing, I found that when debugging, I added app.use(express.bodyParser({limit: '50mb'}));, but after app.use(express.json());. Express would then set the global limit to 1mb because the first parser he encountered when running the script was express.json(). Moving bodyParser above it did the trick.
That said, the bodyParser() method will be deprecated in Connect 3.0 and should not be used. Instead, you should declare your parsers explicitly, like so :
app.use(express.json({limit: '50mb'}));
app.use(express.urlencoded({limit: '50mb'}));
In case you need multipart (for file uploads) see this post.
[second edit]
Note that in Express 4, instead of express.json() and express.urlencoded(), you must require the body-parser module and use its json() and urlencoded() methods, like so:
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
If the extended option is not explicitly defined for bodyParser.urlencoded(), it will throw a warning (body-parser deprecated undefined extended: provide extended option). This is because this option will be required in the next version and will not be optional anymore. For more info on the extended option, you can refer to the readme of body-parser.
[third edit]
It seems that in Express v4.16.0 onwards, we can go back to the initial way of doing this (thanks to #GBMan for the tip):
app.use(express.json({limit: '50mb'}));
app.use(express.urlencoded({limit: '50mb'}));
In my case it was not enough to add these lines :
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
I tried adding the parameterLimit option on urlencoded function as the documentation says and error no longer appears.
The parameterLimit option controls the maximum number of parameters
that are allowed in the URL-encoded data. If a request contains more
parameters than this value, a 413 will be returned to the client.
Defaults to 1000.
Try with this code:
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: "50mb"}));
app.use(bodyParser.urlencoded({limit: "50mb", extended: true, parameterLimit:50000}));
If someone tried all the answers, but hadn't had any success yet and uses NGINX to host the site add this line to /etc/nginx/sites-available
client_max_body_size 100M; #100mb
I don't think this is the express global size limit, but specifically the connect.json middleware limit. This is 1MB by default when you use express.bodyParser() and don't provide a limit option.
Try:
app.post('/api/0.1/people', express.bodyParser({limit: '5mb'}), yourHandler);
For express ~4.16.0, express.json with limit works directly
app.use(express.json({limit: '50mb'}));
in my case .. setting parameterLimit:50000 fixed the problem
app.use( bodyParser.json({limit: '50mb'}) );
app.use(bodyParser.urlencoded({
limit: '50mb',
extended: true,
parameterLimit:50000
}));
The following worked for me... Just use
app.use(bodyParser({limit: '50mb'}));
that's it.
Tried all above and none worked. Found that even though we use like the following,
app.use(bodyParser());
app.use(bodyParser({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb'}));
only the 1st app.use(bodyParser()); one gets defined and the latter two lines were ignored.
Refer: https://github.com/expressjs/body-parser/issues/176 >> see 'dougwilson commented on Jun 17, 2016'
2016, none of the above worked for me until i explicity set the 'type' in addition to the 'limit' for bodyparser, example:
var app = express();
var jsonParser = bodyParser.json({limit:1024*1024*20, type:'application/json'});
var urlencodedParser = bodyParser.urlencoded({ extended:true,limit:1024*1024*20,type:'application/x-www-form-urlencoded' })
app.use(jsonParser);
app.use(urlencodedParser);
The setting below has worked for me
Express 4.16.1
app.use(bodyParser.json({ limit: '50mb' }))
app.use(bodyParser.urlencoded({
limit: '50mb',
extended: false,
}))
Nginx
client_max_body_size 50m;
client_body_temp_path /data/temp;
In my case the problem was on Nginx configuration. To solve it I have to edit the file: /etc/nginx/nginx.conf and add this line inside server block:
client_max_body_size 5M;
Restart Nginx and the problems its gone
sudo systemctl restart nginx
After דo many tries I got my solution
I have commented this line
app.use(bodyParser.json());
and I put
app.use(bodyParser.json({limit: '50mb'}))
Then it works
A slightly different approach - the payload is too BIG
All the helpful answers so far deal with increasing the payload limit. But it might also be the case that the payload is indeed too big but for no good reason. If there's no valid reason for it to be, consider looking into why it's so bloated in the first place.
Our own experience
For example, in our case, an Angular app was greedily sending an entire object in the payload. When one bloated and redundant property was removed, the payload size was reduced by a factor of a 100. This significantly improved performance and resolved the 413 error.
Pass the below configs to your server to increase your request size.
app.use(express.json({ extended: false, limit: '50mb' }))
app.use(express.urlencoded({ limit: '50mb', extended: false, parameterLimit: 50000 }))
Little old post but I had the same problem
Using express 4.+
my code looks like this and it works great after two days of extensive testing.
var url = require('url'),
homePath = __dirname + '/../',
apiV1 = require(homePath + 'api/v1/start'),
bodyParser = require('body-parser').json({limit:'100mb'});
module.exports = function(app){
app.get('/', function (req, res) {
res.render( homePath + 'public/template/index');
});
app.get('/api/v1/', function (req, res) {
var query = url.parse(req.url).query;
if ( !query ) {
res.redirect('/');
}
apiV1( 'GET', query, function (response) {
res.json(response);
});
});
app.get('*', function (req,res) {
res.redirect('/');
});
app.post('/api/v1/', bodyParser, function (req, res) {
if ( !req.body ) {
res.json({
status: 'error',
response: 'No data to parse'
});
}
apiV1( 'POST', req.body, function (response) {
res.json(response);
});
});
};
I've used another practice for this problem with multer dependancie.
Example:
multer = require('multer');
var uploading = multer({
limits: {fileSize: 1000000, files:1},
});
exports.uploadpictureone = function(req, res) {
cloudinary.uploader.upload(req.body.url, function(result) {
res.send(result);
});
};
module.exports = function(app) {
app.route('/api/upload', uploading).all(uploadPolicy.isAllowed)
.post(upload.uploadpictureone);
};
If you are using express.json() and bodyParser together it will give error as express sets its own limit.
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
remove above code and just add below code
app.use(bodyParser.json({ limit: "200mb" }));
app.use(bodyParser.urlencoded({ limit: "200mb", extended: true, parameterLimit: 1000000 }));
After trying everything in this post, i was unsuccessful. But I found a solution that worked for me.
I was able to solve it without using the body-parser and only with the express.
It looked like this:
const express = require('express');
const app = express();
app.use(express.json({limit: '25mb'}));
app.use(express.urlencoded({limit: '25mb', extended: true}));
Don't forget to use extended: true to remove the deprecated message from the console.
Just adding this one line must solve it actually
app.use(express.json({limit: '50mb'}));
Also recommend you guys to send the whole image to the backend then convert it rather then sending the data from the frontend
for me following snippet solved the problem.
var bodyParser = require('body-parser');
app.use(bodyParser.json({limit: '50mb'}));
In my case removing Content-type from the request headers worked.
I too faced that issue, I was making a silly mistake by repeating the app.use(bodyParser.json()) like below:
app.use(bodyParser.json())
app.use(bodyParser.json({ limit: '50mb' }))
by removing app.use(bodyParser.json()), solved the problem.
I faced the same issue recently and bellow solution workes for me.
Dependency :
express >> version : 4.17.1
body-parser >> version": 1.19.0
const express = require('express');
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json({limit: '50mb'}));
app.use(bodyParser.urlencoded({limit: '50mb', extended: true}));
For understanding :
HTTP 431
The HTTP 413 Payload Too Large response status code indicates that the
request entity is larger than limits defined by server; the server
might close the connection or return a Retry-After header field.
Work for me:
Config nginx max file zise
[https://patriciahillebrandt.com/nginx-413-request-entity-too-large/][1]
and
app.use(bodyParser.json({ limit: "200mb" }));
app.use(bodyParser.urlencoded({ limit: "200mb", extended: true, parameterLimit: 1000000 }));
To add to Alexander's answer.
By default, NGINX has an upload limit of 1 MB per file. By limiting the file size of uploads, you can prevent some types of Denial-of-service (DOS) attacks and many other issues.
So when you try to upload a file above the 1MB limit you will run into a 413 error.
By editing client_max_body_size, you can adjust the file upload size. Use the http, server, or location block to edit client_max_body_size.
server {
server_name example.com;
location / {
proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:8080;
client_max_body_size 20M;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/infohob.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/infohob.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Reference: Limit File Upload Size in NGINX
The better use you can specify the limit of your file size as it is shown in the given lines:
app.use(bodyParser.json({limit: '10mb', extended: true}))
app.use(bodyParser.urlencoded({limit: '10mb', extended: true}))
You can also change the default setting in node-modules body-parser then in the lib folder, there are JSON and text file. Then change limit here. Actually, this condition pass if you don't pass the limit parameter in the given line
app.use(bodyParser.json({limit: '10mb', extended: true})).
This issue happens in two cases:
1- request body is too large and server cannot process this large data. this will serve it
app.use(express.json({limit: '50mb'}));
2- req.cookies is too large. When testing different next.js applications on the same browser, each time each app was starting on a different port if there were running some apps. Same app might end up starting at port 3000-3005 range. That means if your app saves cookie, that cookie will be saved for each port. Let's say you started 5 different apps at localhost:3000, and each one saved a cookie. If you make a request, all the cookies will be attached to the request object, in this case you will not able to process even small size of post.body. Solution is you have to delete all the cookies
Express 4.17.1
app.use( express.urlencoded( {
extended: true,
limit: '50mb'
} ) )
Demo csb
Following code resolved my issue:
var bodyParser = require('body-parser');
var urlencodedParser = bodyParser.urlencoded({ extended: false, limit: '5mb' });
For me the main trick is
app.use(bodyParser.json({
limit: '20mb'
}));
app.use(bodyParser.urlencoded({
limit: '20mb',
parameterLimit: 100000,
extended: true
}));
bodyParse.json first
bodyParse.urlencoded second
For those who start the NodeJS app in Azure under IIS, do not forget to modify web.config as explained here Azure App Service IIS "maxRequestLength" setting

How to access the request body when POSTing using Node.js and Express?

I have the following Node.js code:
var express = require('express');
var app = express.createServer(express.logger());
app.use(express.bodyParser());
app.post('/', function(request, response) {
response.write(request.body.user);
response.end();
});
Now if I POST something like:
curl -d user=Someone -H Accept:application/json --url http://localhost:5000
I get Someone as expected. Now, what if I want to get the full request body? I tried doing response.write(request.body) but Node.js throws an exception saying "first argument must be a string or Buffer" then goes to an "infinite loop" with an exception that says "Can't set headers after they are sent."; this also true even if I did var reqBody = request.body; and then writing response.write(reqBody).
What's the issue here?
Also, can I just get the raw request without using express.bodyParser()?
Starting from express v4.16 there is no need to require any additional modules, just use the built-in JSON middleware:
app.use(express.json())
Like this:
const express = require('express')
app.use(express.json()) // <==== parse request body as JSON
app.listen(8080)
app.post('/test', (req, res) => {
res.json({requestBody: req.body}) // <==== req.body will be a parsed JSON object
})
Note - body-parser, on which this depends, is already included with express.
Also don't forget to send the header Content-Type: application/json
Express 4.0 and above:
$ npm install --save body-parser
And then in your node app:
const bodyParser = require('body-parser');
app.use(bodyParser);
Express 3.0 and below:
Try passing this in your cURL call:
--header "Content-Type: application/json"
and making sure your data is in JSON format:
{"user":"someone"}
Also, you can use console.dir in your node.js code to see the data inside the object as in the following example:
var express = require('express');
var app = express.createServer();
app.use(express.bodyParser());
app.post('/', function(req, res){
console.dir(req.body);
res.send("test");
});
app.listen(3000);
This other question might also help: How to receive JSON in express node.js POST request?
If you don't want to use the bodyParser check out this other question: https://stackoverflow.com/a/9920700/446681
As of Express 4, the following code appears to do the trick.
Note that you'll need to install body-parser using npm.
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.urlencoded({ extended: false }));
app.listen(8888);
app.post('/update', function(req, res) {
console.log(req.body); // the posted data
});
For 2019, you don't need to install body-parser.
You can use:
var express = require('express');
var app = express();
app.use(express.json())
app.use(express.urlencoded({extended: true}))
app.listen(8888);
app.post('/update', function(req, res) {
console.log(req.body); // the posted data
});
You should not use body-parser it is deprecated. Try this instead
const express = require('express')
const app = express()
app.use(express.json()) //Notice express.json middleware
The app.use() function is used to mount the specified middleware function(s) at the path which is being specified. It is mostly used to set up middleware for your application.
Now to access the body just do the following
app.post('/', (req, res) => {
console.log(req.body)
})
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json())
var port = 9000;
app.post('/post/data', function(req, res) {
console.log('receiving data...');
console.log('body is ',req.body);
res.send(req.body);
});
// start the server
app.listen(port);
console.log('Server started! At http://localhost:' + port);
This will help you. I assume you are sending body in json.
This can be achieved without body-parser dependency as well, listen to request:data and request:end and return the response on end of request, refer below code sample. ref:https://nodejs.org/en/docs/guides/anatomy-of-an-http-transaction/#request-body
var express = require('express');
var app = express.createServer(express.logger());
app.post('/', function(request, response) {
// push the data to body
var body = [];
request.on('data', (chunk) => {
body.push(chunk);
}).on('end', () => {
// on end of data, perform necessary action
body = Buffer.concat(body).toString();
response.write(request.body.user);
response.end();
});
});
In my case, I was missing to set the header:
"Content-Type: application/json"
Try this:
response.write(JSON.stringify(request.body));
That will take the object which bodyParser has created for you and turn it back into a string and write it to the response. If you want the exact request body (with the same whitespace, etc), you will need data and end listeners attached to the request before and build up the string chunk by chunk as you can see in the json parsing source code from connect.
The accepted answer only works for a body that is compatible with the JSON format. In general, the body can be accessed using
app.use(
Express.raw({
inflate: true,
limit: '50mb',
type: () => true, // this matches all content types
})
);
like posted here. The req.body has a Buffer type and can be converted into the desired format.
For example into a string via:
let body = req.body.toString()
Or into JSON via:
let body = req.body.toJSON();
If you're lazy enough to read chunks of post data.
you could simply paste below lines
to read json.
Below is for TypeScript similar can be done for JS as well.
app.ts
import bodyParser from "body-parser";
// support application/json type post data
this.app.use(bodyParser.json());
// support application/x-www-form-urlencoded post data
this.app.use(bodyParser.urlencoded({ extended: false }));
In one of your any controller which receives POST call use as shown below
userController.ts
public async POSTUser(_req: Request, _res: Response) {
try {
const onRecord = <UserModel>_req.body;
/* Your business logic */
_res.status(201).send("User Created");
}
else{
_res.status(500).send("Server error");
}
};
_req.body should be parsing you json data into your TS Model.
I'm absolutely new to JS and ES, but what seems to work for me is just this:
JSON.stringify(req.body)
Let me know if there's anything wrong with it!
Install Body Parser by below command
$ npm install --save body-parser
Configure Body Parser
const bodyParser = require('body-parser');
app.use(bodyParser);
app.use(bodyParser.json()); //Make sure u have added this line
app.use(bodyParser.urlencoded({ extended: false }));
What you claim to have "tried doing" is exactly what you wrote in the code that works "as expected" when you invoke it with curl.
The error you're getting doesn't appear to be related to any of the code you've shown us.
If you want to get the raw request, set handlers on request for the data and end events (and, of course, remove any invocations of express.bodyParser()). Note that the data events will occur in chunks, and that unless you set an encoding for the data event those chunks will be buffers, not strings.
You use the following code to log post data:
router.post("/users",function(req,res){
res.send(JSON.stringify(req.body, null, 4));
});

Categories

Resources