mongoose post.save failing on heroku, works on localhost - javascript

I'm trying to simply post to my MongoDB Atlas db via node,express,mongoose and heroku. A Postman POST request, Raw JSON with body:
{
"title": "heroku post",
"description": "post me plsssss"
}
works on localhost with this exact code, but when uploaded via heroku the try/catch block fails at post.save() as the response is the error.
{
"error": "there's an error",
"message": {}
}
But the error is empty and I'm not sure how to debug it. I've put in mongoose.set('debug', true); in app.js and i've modified my package.json: "start": "node app.js DEBUG=mquery", but those have made no extra output that I am seeing. Is there any other way to know why the post.save() is throwing an error, some logs that I am not utilising.. and how can I see those logs? Or if you just know what the issue is?
App.js
//use dotenv to store secret keys
require("dotenv").config();
const express = require('express');
const mongoose = require('mongoose');
const app = express();
const cors = require('cors');
//connect to DB
mongoose.connect("mongodb+srv://grushevskiy:intercom#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {
console.log('connected to DB!')
})
mongoose.set('debug', true);
const db = mongoose.connection;
db.once('open', () => {
console.log('connection opened')
});
//import routes for middleware
const postsRoute = require('./routes/posts');
//midleware routes
app.use('/posts', postsRoute)
//ROUTES
app.get('/', (req,res) => {
res.send('we are on home')
})
//MIDDLEWARES
//cors
app.use(cors());
//decode url special characters
app.use(express.urlencoded({ extended: true }));
//parse json POSTs
app.use(express.json());
//How do we start listening to the server
app.listen( process.env.PORT || 3000);
posts.js
const express = require ('express')
const router = express.Router();
const Post = require('../models/Post')
//SUBMIT A POST
router.post('/', async (req,res) => {
const post = new Post({
title: req.body.title,
description: req.body.description
});
console.log(post)
try {
const savedPost = await post.save();
res.json(savedPost);
console.log(savedPost)
} catch (err) {
res.json({ error: "there's an error", message: err, })
}
})
module.exports = router;
Post.js Model
const mongoose = require('mongoose')
const PostSchema = mongoose.Schema({
title: {
type: String,
required: true
},
description: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
})
module.exports = mongoose.model('Post', PostSchema)
When I type heroku logs --tail there are no errors, also initially, the 'connected to DB!' message comes in a bit late.. I'm wondering if maybe this is an issue with async/await? My package.json:
{
"name": "22-npmexpressrestapi",
"version": "1.0.0",
"engines": {
"node": "14.15.3",
"npm": "6.14.9"
},
"description": "",
"main": "index.js",
"scripts": {
"start": "node app.js DEBUG=mquery",
"start:dev": "node app.js"
},
"author": "",
"license": "ISC",
"devDependencies": {
"express": "^4.17.1",
"nodemon": "^2.0.7"
},
"dependencies": {
"dotenv": "^8.2.0",
"express": "4.17.1",
"cors": "^2.8.5",
"mongoose": "^5.11.11"
}
}

After reading your question carefully I have a couple of suggestions that I think you might want to try. First, let me explain myself:
In my experience, when one learns how to build REST APIs with Node.js, using mongoose to communicate with a MongoDB cluster, this is what the sequence of events of their index.js file looks like:
Execute a function that uses environment variables to establish a connection with the database cluster. This function runs only once and fills de mongoose instance that the app is going to use with whatever models that have been designed for it.
Set up the app by requiring the appropriate package, defining whatever middleware it's going to require, and calling to all the routes that have been defined for it.
After the app has incorporated everything it's going to need in order to run properly, app.listen() is invoked, a valid port is provided and... voila! You've got your server running.
When the app and the databsae are simple enough, this works like a charm. I have built several services using these very same steps and brought them to production without noticing any miscommunication between mongoose and the app. But, if you check the App.js code you provided, you'll realize that there is a difference in your case: you only connect to your Mongo cluster after you set up your app, its middleware, and its routes. If any of those depend on the mongoose instance or the models to run, there is a good chance that by the time Heroku's compiler gets to the point where your routes need to connect to your database, it simply hasn't implemented that connection (meaning hasn't run the mongoose.connect() part) jet.
Simple solution
I think that Heroku is simply taking a little bit longer to run your whole code than your local machine. Also, if your local version is connecting to a locally stored MongoDB, there is a good chance things run quite quicker than in a cloud service. Your statement
the 'connected to DB!' message comes in a bit late
gives me hope that this might be the case. If that were so, then I guess that moving
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {console.log('connected to DB!')});
to the second line of App.js, right after you call Dotenv, would do the trick. Your mongoose instance would go into App already connected with the database and models that App is going to use.
Asynchronous solution
Even if what I previously wrote fixed your problem, I would like to expand the explanation because, in my experience, that version of the solution wouldn't be right either. The way of doing things that I exposed at the beginning of this answer is ok... As long as you keep your MongoDB cluster and your mongoose instance simple. Not very long ago I had to implement a service that involved a much more complex database pattern, with several different databases being used by the same App, and many of its routes depending directly on those databases' models.
When I tried to use the logic I described before with such a problem, involving heavily populated databases and several models contained in the mongoose instance, I realized that it took Node far less building the App, its middleware, and its routes than connecting to mongoose and loading all the models that that very same App required for it to run. Meaning that I got my Server connected to PORT **** message before the database was actually connected. And that caused trouble.
That made me realize that the only way to do things properly was to ensure asynchronicity, forcing App to run only after all the databases and the models were loaded into the mongoose instance, and feeding it that very same instance. This way, I ended up having an index.js that looked like this:
const Dotenv = require("dotenv").config();
const { connectMongoose } = require("./server/db/mongoose");
const wrappedApp = require("./app");
connectMongoose().then((mongoose) => {
const App = wrappedApp(mongoose);
App.listen(process.env.PORT, () => {
console.log(
`Hello, I'm your server and I'm working on port ${process.env.PORT}`
);
});
});
A file with several functions govering the connections to my databases and modeles called mongoose.js. The function that implements all the connections in the right order looks is:
const connectMongoose = async (mongoose = require("mongoose")) => {
try {
// Close any previous connections
await mongoose.disconnect();
// Connect the Main database
console.log(
"--> Loading databases and models from MongoDB. Please don't use our server jet. <--"
);
mongoose = await connectMother(mongoose);
// Connect all the client databases
mongoose = await connectChildren(mongoose);
console.log(
"--> All databases and models loaded correctly. You can work now. <--"
);
return mongoose;
} catch (e) {
console.error(e);
}
};
module.exports = { connectMongoose };
With this, the file app.js returns a function that needs to be fed a mongoose instance and sets up all the Express environment:
module.exports = (Mongoose) => {
const Express = require("express"); //Enrouting
// Other required packages go here
// EXPRESS SETTINGS
const App = Express();
App.set("port", process.env.PORT || 3000); //Use port from cloud server
// All the middleware App needs
App.use(Flash());
// Routing: load your routes
// Return fully formed App
return App;
};
Only using this pattern I made sure that the chain of events made sense:
Connect all the databases and load their models.
Only after the last database has been connected and its last model, loaded, set up the App, its middleware, and your routes.
Use App.listen() to serve the App.
If your database isn't very large, then the first part of the answer might do the trick. But it's always useful to know the whole story.

I am able to reach an empty error message with your code when i add wrong connection string.
console.log in this callback is incorrect:
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {
console.log('connected to DB!')
})
if you wanna know if you are connected to db use this:
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" });
const db = mongoose.connection;
db.once('open', () => {
console.log('connection opened')
});
I noticed you are using dotenv maybe you are not adding variables to your heroku host.

Related

Server not receiving HTTP requests from Client

First a quick preface I think may be helpful: I am new to splitting my client and server into separate web apps. My previous apps have all had my server.js at the root level in my directory and the client (typically a create-react-app) in a /client folder.
What I wanted to do: Host a single express.js server on the web which multiple other web applications could make requests to.
I did the first part using an express server and aws elastic beanstalk.
server.js
require('dotenv').config()
const express = require('express');
const app = express();
const cors = require('cors');
const PORT = process.env.PORT || 5000;
const Mongoose = require('mongoose');
const { Sequelize } = require("sequelize");
//ROUTES
const APIUser = require('./routes/api/mongo/api-user');
more routes...
//INITIATE DATA MAPPING CONNECTIONS START
Mongoose.connect(
process.env.MONGO_URI,
{ useNewUrlParser: true, useUnifiedTopology: true },
console.log("connected to MongoDB")
);
const Postgres = new Sequelize(process.env.PG_CONN_STRING);
try {
Postgres.authenticate()
.then(console.log("connected to Postgres"));
} catch {
console.log("Postgres connection failed")
}
//INITIATE DATA MAPPING CONNECTIONS END
//middleware
app.use(cors())
more middleware...
//home route
app.get('/api', (req, res) => {
console.log('RECEIVED REQ to [production]/api/')
res.status(200).send('models api home').end();
})
//all other routes
app.use('/api/user', APIUser);
more route definitions...
//launch
app.listen(PORT, () => console.log(`listening on port ${PORT}`));
The log file for successful boot up on aws: https://imgur.com/vLgdaxK
At first glance it seemed to work as my postman requests were working. Status 200 with appropriate response: https://imgur.com/VH4eHzH
Next I tested this from one of my actual clients in localhost. Here is one of my react client's api util files where axios calls are made to the backend:
import { PROXY_URL } from '../../config';
import { axiosInstance } from '../util';
const axiosProxy = axios.create({baseURL: PROXY_URL}); //this was the most reliable way I found to proxy requests to the server
export const setAuthToken = () => {
const route = "/api/authorization/new-token";
console.log("SENDING REQ TO: " + PROXY_URL + route)
return axiosProxy.post(route)
}
export const authorize = clientsecret => {
const route = "/api/authorization/authorize-survey-analytics-client";
console.log("SENDING REQ TO: " + PROXY_URL + route)
return axiosProxy.post(route, { clientsecret })
}
Once again it worked... or rather I eventually got it to work: https://imgur.com/c2rPuoc
So I figured all was well now but when I tried using the live version of the client the request failed somewhere.
in summary the live api works when requests are made from postman or localhost but doesn't respond when requests are made from a live client https://imgur.com/kOk2RWf
I have confirmed that the requests made from the live client do not make it to the live server (by making requests from a live client and then checking the aws live server logs).
I am not receiving any Cors or Cross-Origin-Access errors and the requests from the live client to the live server don't throw any loud errors, I just eventually get a net::ERR_CONNECTION_TIMED_OUT. Any ideas where I can look for issues or is there more code I could share?
Thank you!
Add a console.log(PROXY_URL) in your client application and check your browser console if that's logged correctly.
If it works on your local client build, and through POSTMAN, then your backend api is probably good. I highly suspect that your env variables are not being set. If PROXY_URL is an emplty string, your axios requests will be routed back to the origin of your client. But I assume they have different origins since you mention that they're separate apps.
Remember environment variables need to prefixed with REACT_APP_ and in a production build they have to be available at build time (wherever you perform npm run build)

res.send() is not giving response on POST request in POSTMAN

hy, I'm learning nodeJS but when do post using postman data is saving in db but not displaying response in POSTMAN. On postman just displaying sending request... .
const express = require("express")
const app = express()
// dbConnection
require('./mongo')
// Models
require('./model/Post')
// MIDDLEWARE
app.use(express.urlencoded({extended: true}));
app.use(express.json())
const mongoose = require('mongoose')
const Post = mongoose.model("Post")
// POST REQUEST
app.post('/posts', async (req, res)=>{
// res.send(req.body)
try{
const post = new Post()
post.title = req.body.title
post.content = req.body.content
data = await post.save()
res.json(data)
}catch(error){
res.status(500)
}
})
app.listen(8000, ()=>{
console.log('Server is running on port:8000')
})
I don't think you're even running on a port, it says here
console.log('Server is running on port:8000')
})
All you do is console.log Server is running on Port 8000 with no back tick, therefore your not even running your server. This is why I think your Code is not working, test it out and see, if you get an error then you can debug from there. At least put some effort into debugging rather than immediately going on stack overflow. replace what you done with the port with this
// Create a variable called port and set it to your desired port
const port = 8000;
// Then hook it up to express.
console.log(`Server is running on port: ${port}`)
})
If the problem is still there then I think I have the solution to it
Check if you have mongoose and express installed
(it's npm i mongoose express)
I don't think you're even connected to your mongoose server, try doing this
const express = require("express")
const app = express()
// dbConnection
require('./mongo')
// Models
require('./model/Post')
// MIDDLEWARE
app.use(express.urlencoded({extended: true}));
app.use(express.json())
const mongoose = require('mongoose')
const Post = mongoose.model("Post")
// Hook it up to res
const port = 8000
// POST REQUEST
app.post('/posts', async (req, res)=>{
// res.send(req.body)
try{
const post = new Post()
post.title = req.body.title
post.content = req.body.content
data = await post.save()
res.json(data)
}catch(error){
res.status(500)
}
})
// Mongoose Connection
mongoose
.connect("your connection (it should be connection to application on mongo)", {
useNewUrlParser: true,
useUnifiedTopology: true,
useFindAndModify: true
})
.then(() => {
console.log("Connected to the database");
})
.catch((err) => {
console.log(err);
});
app.listen(8000, ()=>{
console.log(`Server is running on port: ${port}`)
})
Then once you've finished that, you established a connection to the mongodb server and should send the request to post
I think the catch block is executed. In this block, you only set the status of the response to 500 but you don't actually send the response to the client. That's why the Postman screen keeps blocking.
So, there are 2 things:
you need to send something to the client
you need to log the error to debug.
app.post('/posts', async (req, res)=>{
// res.send(req.body)
try{
const post = new Post()
post.title = req.body.title
post.content = req.body.content
data = await post.save()
res.json(data)
}catch(error){
console.log(error);// for debugging
res.status(500).send("ERROR_SERVER"); // send something to client
}
})
I have found the answer for your error, as I said in my old answer, running your tests would've worked, and showed you the error, however I have found the answer, I am assuming you have already found the solution (which is probably the same solution as mine) but if you haven't here's the problem.
The problem
It's very simple, you're creating a variable for mongoose after you required mongoose require('./mongo'); const mongoose = require('mongoose') This is wrong as JavaScript and most programming languages read code line by line (if not then all) so change this up to be instead the following:
Solution
const mongoose = require('mongoose');
require('./mongo');
Information
Create the variable before you require the package like so (in your code example):
const mongoose = require('mongoose');
require('./mongo');
If you have more problems
If you do have more problems then try to reinstall/update the package dependency for mongoose as following:
yarn add mongoose
or
npm install mongoose
If you do still have problems after the only think I can ask you to do is to change the line of code when it says
require('./mongo');
to either
require('./{filename}'); // Whatever the actual filename is.
or:
require('./mongoose');
Tips to improving your question
Even if my question doesn't work for your code make sure to paste the error message or the important parts of the error message into the question, otherwise this makes it hard to pinpoint what your error is. This makes it easier to find the solution for your code.

Trying to GET json data, getting "not a function" error

I'm creating a programmer job board app and I'm trying to display json data on my main page. At some point I'll render it, but for now I'm just trying to get it to show up in json form so that I know it works.
I'm able to connect to the server, but when I load the page I get a TypeError (Job.showAllJobs is not a function).
I'm using a crud app I made the other week as a reference, but there are a few differences between it and this project that are throwing me off.
Here's my project's file structure:
job-board
database
connection.js
schema.sql
models
Job.js
User.js
views
index.ejs
login.ejs
server.js
Unlike my previous crud app, this project is using a connection.js file that gave some trouble earlier. At first I thought I was out of the woods, but I think it might be responsible for my current problem.
Not getting GET to work might seem like a minor error, but it's really bugging me and I haven't been able to keep working because of it.
I populated my table (jobs) with a sample listing as a test, but in the very near future I plan on connecting the app to the GitHub jobs api.
server.js:
const express = require('express');
const app = express();
const PORT = 3000;
const bodyParser = require('body-parser');
const methodOverride = require('method-override');
const Job = require('./models/Job');
const User = require('./models/User');
const connection = require('./database/connection')
app.use(bodyParser.json())
app.use(methodOverride('_method'));
const urlencodedParser = bodyParser.urlencoded({ extended: false })
app.set("view engine", "ejs");
///// GET /////
// GET INDEX
app.get('/', (request, response) => {
Job.showAllJobs().then(everyJob => {
response.json('index');
// response.render('index', { jobs: everyJob });
});
});
Job.js
const Job = {};
const db = require('../database/connection');
///// JOBS /////
/// INDEX ///
Job.showAllJobs = () => {
return db.any('SELECT * FROM jobs');
};
module.exports = Job;
module.exports = db;
connection.js
// require database setup to use pg-Promise
const pgp = require('pg-promise')({});
// connection url
const connectionURL = "postgres://localhost:5432/job_board";
// new database connection
const db = pgp(connectionURL);
// module.exports = db;
You have a couple of problems here.
Make sure you're passing the jobs into res.json instead of the string 'index'
Make sure you're exporting db from connection.js
You're exporting both Job and db from Job.js. Since you're exporting db second, it's overriding the export of Job.

Router instances in Node.js with express with a database

As far as I can gather from the Express documentation, when you declare an express.Router(), it creates a single instance of a router that you can then assign a routing path and execute logic with. The documentation says to think of a router like a mini-app for a specific route or routes, which makes sense.
I'm trying to strategize what to wrap my database connection around (using mongodb, let's say via mongoose, but it shouldn't matter). I of course don't want to open a new database connection on every route call, and I assume that if I wrap it around a router only one Router() instance will only be created. In other words, if I went to /routes/index.js, defined a Router() and then opened a database connection, then within it did router.get (or router.post, etc.), I would be opening one database connection when launching the app, not one per hit on that route.
Yet there might be other routes beyond index.js where I want access to the database. So alternatively, could I wrap the database connection around the app.use route handlers and other middleware within the main app.js file, then require('../app') in /routes files and add the database connection to module.exports in app.js, and finally define route logic in other files like /routes/index.js?
I'm a little confused on how to structure everything. Any guidance would be much appreciated.
If you are using mongoose, you can just connect once with some code like this:
mongoose.connect("mongodb://127.0.0.1:27017/test");
mongoose.connection.on('error', console.error.bind(console, 'Db connection error:'));
// Start the server once we have connected to the database.
mongoose.connection.once('open', () => {
console.log('Db connection open.');
app.listen(3000, function () {
console.log('Listening on port 3000');
});
});
And then, if you have a mongoose model named Foo set up like
const Foo = mongoose.model('Foo', fooSchema); // fooSchema is a mongoose.Schema
Then in your route you can use it like so:
const router = express.Router();
const Foo = require('./models/foo');
router
.route('/foos/:foo_id')
.get((req, res)=> {
Foo.findById(req.params.foo_id, (err, foo) => {
if (err) return res.sendStatus(500);
if (!foo) return res.status(404).send('Foo not found.');
res.status(200).json(foo);
});
});
This kind of setup lets mongoose handle connection pooling.

Unit Testing Express app in Jenkins

Ok, I'm searching a couple of hours for a solution to run my Mocha unit tests inside Jenkins, for my Express JS app.
Writing tests is quite easy, but connecting the tests with my app is a bit harder. An example of my current test:
/tests/backend/route.test.js
var should = require('should'),
assert = require('assert'),
request = require('supertest'),
express = require('express');
describe('Routing', function() {
var app = 'http://someurl.com';
describe('Account', function() {
it('should return error trying to save duplicate username', function(done) {
var profile = {
username: 'vgheri',
password: 'test',
firstName: 'Valerio',
lastName: 'Gheri'
};
// once we have specified the info we want to send to the server via POST verb,
// we need to actually perform the action on the resource, in this case we want to
// POST on /api/profiles and we want to send some info
// We do this using the request object, requiring supertest!
request(app)
.post('/api/profiles')
.send(profile)
// end handles the response
.end(function(err, res) {
if (err) {
throw err;
}
// this is should.js syntax, very clear
res.should.have.status(400);
done();
});
});
});
In above example I connect to a running app (see ```var app = 'http://someurl.com'``). Obviously this is not working inside Jenkins, except if I can tell Jenkins to first run the app and then check for a localhost url. But how to do this?
Now if I take a look at https://www.npmjs.org/package/supertest this should be enough to test my express app:
var request = require('supertest')
, express = require('express');
var app = express();
But it's not. I receive 404 errors on all urls which I would like to test.
Anyone who knows how to test my app inside Jenkins?
Is the app you referenced at http://someurl.com the app you're trying to write this test for? If so...
Supertest should allow you to run tests without needing an external server running. The way you accomplish this is by separating out the server routing Express code from the code you'll run that actually starts the server, and use Supertest on the server routing code.
Here is an example:
Given this directory structure:
./package.json
./server/index.js
./app.js
./test/server-test.js
Here are my files:
./package.json
All of these dependencies aren't required for this example, I just pulled this from my package.json I'm using in my project.
{
"name": "StackOverflowExample",
"version": "1.0.0",
"description": "Example for stackoverflow",
"main": "app.js",
"scripts": {
"test": "mocha"
},
"author": "Mark",
"license": "ISC",
"dependencies": {
"body-parser": "^1.15.2",
"ejs": "^2.5.2",
"express": "^4.14.0",
"express-validator": "^2.21.0",
},
"devDependencies": {
"chai": "^3.5.0",
"mocha": "^3.1.2",
"supertest": "^2.0.1"
}
}
./server/index.js
var express = require('express');
var app = express();
app.get('/', function(req, res) {
res.send('Hello World');
});
module.exports = app;
./app.js
var app = require('./server');
var server = app.listen(8000, function() {
console.log('Listening on port 8000');
});
./tests/server-test.js
var request = require('supertest');
var app = require('../server');
var assert = require('assert'); //mocha
var chai = require('chai');
var expect = null;
chai.should();
expect = chai.expect;
describe('GET /', function() {
it('should return a 200', function(done) {
request(app).get('/').expect(200, done);
});
});
I'm using Mocha and Chai here (didn't have time to refactor and make the example simpler, sorry). The main point is that within the it statement you can run your tests.
Doing it this way does not require a separate server running. Supertest magically starts one up for you and runs it based on the express server you exported from the server module. I think this approach should allow you to run your tests seamlessly through Jenkins.
In ./server/index.js, an Express object is created. That object is exported, and imported via require in both ./app.js and ./test/server-test.js. In ./app.js it's used to start a server via listen(). In the test file, Supetest uses it to start a server for testing internally.
When you are running the server in this example, you would run it via ./app.js. That would set the port you choose (8000 in this example) and start listening for connections on that port.
Hope this helps! Good luck!

Categories

Resources