Develop Reference

NodeJS HTTPS server serving over HTTP instead of HTTPS when running inside Docker container

I'm creating a website using NextJS and Docker so that I can easily deploy it. I used npx-create-next-app to initialize it and used this Dockerfile (slightly modified) to containerize it. Since I wanted to use SSL with my server without going through the hassle of setting up a proxy, I followed this article, and setup the custom server.
This worked fine when I ran it outside of a docker container, and performed as expected, serving over HTTPS. However when I containerized it, and tried to open the webpage over HTTPS, I came up with SSL_ERROR_RX_RECORD_TOO_LONG, but I could open the page using just HTTP (which I could not do when running outside of a container). Some googling led me to this question, from which I concluded that when running outside of a docker container, the custom server runs the server over HTTPS, as expected, however when I containerize it, it starts running HTTP, even though no code has been changed.
I'd expect the behavior to be the same when running locally or containerized.
At first I assumed this was due to invalid key and cert values in httpsOptions however I wasn't able to find anything that would make them invalid, and I don't see how that would cause this strange behavior. I tried changing the Docker run environment from node:alpine-16 to just node:latest to see if it had something to do with the parent image, but that was fruitless.
One other minor issue I had is that console.log does not seem to output to the container's log for some reason, I tried googling this but didn't find much of anything pertaining to it. This has made debugging much harder as I can't really output any debug data. The only log I get when running inside of a container is Listening on port 3000 url: http://localhost:3000, which I assume is output by some library/package as it isn't anywhere in my code.
Here is my custom server code in case it would be helpful:
const https = require('https');
const fs = require('fs');
const { parse } = require('url');
const next = require('next');
const dev = process.env.NODE_ENV !== 'production';
const hostname = "127.0.0.1";
const port = process.env.PORT || 3000
const app = next({ dev, hostname, port })
const handle = app.getRequestHandler()
const httpsOptions = {
key: fs.readFileSync('./cert/privkey.pem'),
cert: fs.readFileSync('./cert/fullchain.pem')
};
app.prepare().then(() => {
https.createServer(httpsOptions, async (req, res) => { // When running on docker this creates an HTTP server instead of HTTPS
const parsedUrl = parse(req.url, true)
const { pathname, query } = parsedUrl
await handle(req, res, parsedUrl)
}).listen(port, (err) => {
if(err) throw err
console.log(`Ready on https://localhost:${port}`)
})
})
Link to a reproducible example here.

The thing is, based on your sample repo, that your server.js file that is in the root of your repo gets overwritten in the image because of this line in the Dockerfile:
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
So the actual server.js that is running in the container is the server.js that is created by the yarn build command and it looks like this (you can exec into the container and see it for yourself):
const NextServer = require('next/dist/server/next-server').default
const http = require('http')
const path = require('path')
process.env.NODE_ENV = 'production'
process.chdir(__dirname)
// Make sure commands gracefully respect termination signals (e.g. from Docker)
// Allow the graceful termination to be manually configurable
if (!process.env.NEXT_MANUAL_SIG_HANDLE) {
process.on('SIGTERM', () => process.exit(0))
process.on('SIGINT', () => process.exit(0))
}
let handler
const server = http.createServer(async (req, res) => {
try {
await handler(req, res)
} catch (err) {
console.error(err);
res.statusCode = 500
res.end('internal server error')
}
})
const currentPort = parseInt(process.env.PORT, 10) || 3000
server.listen(currentPort, (err) => {
if (err) {
console.error("Failed to start server", err)
process.exit(1)
}
const nextServer = new NextServer({
hostname: 'localhost',
port: currentPort,
dir: path.join(__dirname),
dev: false,
customServer: false,
conf: {"env":{},"webpack":null,"webpackDevMiddleware":null,"eslint":{"ignoreDuringBuilds":false},"typescript":{"ignoreBuildErrors":false,"tsconfigPath":"tsconfig.json"},"distDir":"./.next","cleanDistDir":true,"assetPrefix":"","configOrigin":"next.config.js","useFileSystemPublicRoutes":true,"generateEtags":true,"pageExtensions":["tsx","ts","jsx","js"],"target":"server","poweredByHeader":true,"compress":true,"analyticsId":"","images":{"deviceSizes":[640,750,828,1080,1200,1920,2048,3840],"imageSizes":[16,32,48,64,96,128,256,384],"path":"/_next/image","loader":"default","loaderFile":"","domains":[],"disableStaticImages":false,"minimumCacheTTL":60,"formats":["image/webp"],"dangerouslyAllowSVG":false,"contentSecurityPolicy":"script-src 'none'; frame-src 'none'; sandbox;","remotePatterns":[],"unoptimized":false},"devIndicators":{"buildActivity":true,"buildActivityPosition":"bottom-right"},"onDemandEntries":{"maxInactiveAge":15000,"pagesBufferLength":2},"amp":{"canonicalBase":""},"basePath":"","sassOptions":{},"trailingSlash":false,"i18n":{"locales":["en"],"defaultLocale":"en"},"productionBrowserSourceMaps":false,"optimizeFonts":true,"excludeDefaultMomentLocales":true,"serverRuntimeConfig":{},"publicRuntimeConfig":{},"reactStrictMode":true,"httpAgentOptions":{"keepAlive":true},"outputFileTracing":true,"staticPageGenerationTimeout":60,"swcMinify":true,"output":"standalone","experimental":{"middlewarePrefetch":"flexible","optimisticClientCache":true,"manualClientBasePath":false,"legacyBrowsers":false,"newNextLinkBehavior":true,"cpus":7,"sharedPool":true,"profiling":false,"isrFlushToDisk":true,"workerThreads":false,"pageEnv":false,"optimizeCss":false,"nextScriptWorkers":false,"scrollRestoration":false,"externalDir":false,"disableOptimizedLoading":false,"gzipSize":true,"swcFileReading":true,"craCompat":false,"esmExternals":true,"appDir":false,"isrMemoryCacheSize":52428800,"fullySpecified":false,"outputFileTracingRoot":"","swcTraceProfiling":false,"forceSwcTransforms":false,"largePageDataBytes":128000,"enableUndici":false,"adjustFontFallbacks":false,"adjustFontFallbacksWithSizeAdjust":false,"trustHostHeader":false},"configFileName":"next.config.js"},
})
handler = nextServer.getRequestHandler()
console.log(
'Listening on port',
currentPort,
'url: http://localhost:' + currentPort
)
})
And as you see it starts a http server not a https. Also this is why the console.log("lksdfjls"); in your own server.js will not get executed.
What I would suggest is to leave node as it is, running on http://localhost:3000 and set up a reverse proxy that would forward incoming requests to this node backend that is accessible only from the reverse proxy. And of course reverse proxy would handle TLS termination. A docker compose setup would be more convenient for this so you could put the reverse proxy container (nginx for example) in the compose project too and map a directory from the docker host where your cert files are stored into the reverse proxy container at runtime - DO NOT BAKE CERTS OR ANY OTHER SECRETS INTO ANY IMAGE, not even if it is an internally used image only because it could leak out accidentally any time.
Also you could just manually run the two container with docker run but compose would make life easier it has a lot of capabilities for example you could scale compose services up and down so your backend service would run not in one but many containers. But if this would be a high load and/or business critical production stuff then you are better off with a better (real) container orchestrator like kubernetes, docker swarm, nomad etc but today as I see it the de facto container orchestrator is kubernetes.

Localhost refuses to connect when I make setupProxy.js

I made my setupProxy.js to deal with a CORS issue:
const proxy = require('http-proxy-middleware');
module.exports = function (app) {
app.use(
proxy('/myUrl', {
target: 'http://localhost:8090',
changeOrigin: true
})
)
};
After I made it and run 'npm start' again, my browser says that localhost refuses to connect. If I delete setupProxy.js, I can connect to localhost, but the CORS policy blocks to connect with backend server.
Do you have any idea to connect to frontend, still using the setupProxy.js?

You can try this: createProxyMiddleware
const { createProxyMiddleware } = require('http-proxy-middleware');
module.exports = function(app) {
app.use(
createProxyMiddleware('/myUrl', {
target: 'http://localhost:8090',
changeOrigin: true,
})
);
};
https://stackoverflow.com/a/60354374/11652543

See that you have spelled the name of setupProxy.js well that solved the problem for me. If that does not work. Copy and pase the fb's code the way it is into the setupProxy.js Code can be found here

Nuxt - server middleware not working on production

I have builded an app using Nuxt and I have created simply server middleware for handling email sending. Everything is working on dev but on production I have 404 error from that endpoint. Does anybody know how to include server-middleware into build files or any other way?
server-middleware:
const bodyParser = require('body-parser')
const app = require('express')()
app.use(bodyParser.json())
app.post('/', (req, res) => {
// Some code here
})
module.exports = app
Nuxt.config.js
serverMiddleware: [
{ path: '/contact/send', handler: '~/server-middleware/email.js' }
],
Response here:

I found an answer on github issue nuxt repo:
This is correct - server middleware aren't compiled or part of your
webpack build and so you should make sure to copy them separately into
production. (As well as making sure that any dependencies they have
are installed in production.)
(Note that this will change with Nuxt 3 - you'll be able to have a
single built server that includes your server middleware.)
Issue here:
https://github.com/nuxt/nuxt.js/issues/9158#issuecomment-820676790

Here is an in-depth answer regarding the whole setup of a serverMiddleware.
This solutions seems to work: https://github.com/nuxt/nuxt.js/issues/1486#issuecomment-325181524
// nuxt.config.js
serverMiddleware: [
'~/api/index.js',
]
// api/index.js
const app = require('express')()
module.exports = { path: '/api', handler: app }
app.get('/say/:word', (req, res) => {
res.json(req.params)
})

How to Set up Vue CORS for different domain API?

I' trying to deploy an Vue app which has a separate backend and which will be hosted in different domain. For example:
meow.cat.xyz (App)
api.meow.cat.xyz (API)
Now after npm run build I tried to preview it locally by running serve -s dist and the application is severing at localhost:5000. However the problem is it not sending API request at the current end point (which is localhost:8000 at local and api.meow.cat.xyz at server). I tried config CORS as following
vue.config.js
module.exports = {
devServer: {
port: process.env.VUE_APP_DEV_PORT,
proxy: process.env.VUE_APP_API_ROOT_PATH,
},
};
.env.development
VUE_APP_API_ROOT_PATH = 'http://localhost:8000/api'
VUE_APP_DEV_PORT = 3000
Note that I'm using axiox. Here is my axios setup.
API.js
import axios from "axios";
const injectAccessToken = (config) => {
const accessToken = localStorage.getItem("access_token");
if (accessToken)
config.headers.common["Authorization"] = `Bearer ${accessToken}`;
return config;
};
const config = {
baseURL: process.env.VUE_APP_API_ROOT_PATH,
};
const API = axios.create(config);
API.interceptors.request.use(injectAccessToken);
export default API;
and Using it as following
Login.vue
import API from "#/api/Api";
<script>
const res= await API.post('login')
</script>
This solution is not working yet. Its sending request at http://localhost:5000. What's the point ? Note that I'm using axios. thanks in advance.

Allow CORS requests from the server
With the Access-Control-Allow-Origin header, you can specify what origins can use your API.
app.get('/api', (req, res) => {
res.set('Access-Control-Allow-Origin', 'http://localhost:3000');
res.send({
api: "your request."
});
})

Allow CORS from the app's origin on the server (api).
This has nothing to do with with the client (app)

mongoose post.save failing on heroku, works on localhost

I'm trying to simply post to my MongoDB Atlas db via node,express,mongoose and heroku. A Postman POST request, Raw JSON with body:
{
"title": "heroku post",
"description": "post me plsssss"
}
works on localhost with this exact code, but when uploaded via heroku the try/catch block fails at post.save() as the response is the error.
{
"error": "there's an error",
"message": {}
}
But the error is empty and I'm not sure how to debug it. I've put in mongoose.set('debug', true); in app.js and i've modified my package.json: "start": "node app.js DEBUG=mquery", but those have made no extra output that I am seeing. Is there any other way to know why the post.save() is throwing an error, some logs that I am not utilising.. and how can I see those logs? Or if you just know what the issue is?
App.js
//use dotenv to store secret keys
require("dotenv").config();
const express = require('express');
const mongoose = require('mongoose');
const app = express();
const cors = require('cors');
//connect to DB
mongoose.connect("mongodb+srv://grushevskiy:intercom#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {
console.log('connected to DB!')
})
mongoose.set('debug', true);
const db = mongoose.connection;
db.once('open', () => {
console.log('connection opened')
});
//import routes for middleware
const postsRoute = require('./routes/posts');
//midleware routes
app.use('/posts', postsRoute)
//ROUTES
app.get('/', (req,res) => {
res.send('we are on home')
})
//MIDDLEWARES
//cors
app.use(cors());
//decode url special characters
app.use(express.urlencoded({ extended: true }));
//parse json POSTs
app.use(express.json());
//How do we start listening to the server
app.listen( process.env.PORT || 3000);
posts.js
const express = require ('express')
const router = express.Router();
const Post = require('../models/Post')
//SUBMIT A POST
router.post('/', async (req,res) => {
const post = new Post({
title: req.body.title,
description: req.body.description
});
console.log(post)
try {
const savedPost = await post.save();
res.json(savedPost);
console.log(savedPost)
} catch (err) {
res.json({ error: "there's an error", message: err, })
}
})
module.exports = router;
Post.js Model
const mongoose = require('mongoose')
const PostSchema = mongoose.Schema({
title: {
type: String,
required: true
},
description: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
})
module.exports = mongoose.model('Post', PostSchema)
When I type heroku logs --tail there are no errors, also initially, the 'connected to DB!' message comes in a bit late.. I'm wondering if maybe this is an issue with async/await? My package.json:
{
"name": "22-npmexpressrestapi",
"version": "1.0.0",
"engines": {
"node": "14.15.3",
"npm": "6.14.9"
},
"description": "",
"main": "index.js",
"scripts": {
"start": "node app.js DEBUG=mquery",
"start:dev": "node app.js"
},
"author": "",
"license": "ISC",
"devDependencies": {
"express": "^4.17.1",
"nodemon": "^2.0.7"
},
"dependencies": {
"dotenv": "^8.2.0",
"express": "4.17.1",
"cors": "^2.8.5",
"mongoose": "^5.11.11"
}
}

After reading your question carefully I have a couple of suggestions that I think you might want to try. First, let me explain myself:
In my experience, when one learns how to build REST APIs with Node.js, using mongoose to communicate with a MongoDB cluster, this is what the sequence of events of their index.js file looks like:
Execute a function that uses environment variables to establish a connection with the database cluster. This function runs only once and fills de mongoose instance that the app is going to use with whatever models that have been designed for it.
Set up the app by requiring the appropriate package, defining whatever middleware it's going to require, and calling to all the routes that have been defined for it.
After the app has incorporated everything it's going to need in order to run properly, app.listen() is invoked, a valid port is provided and... voila! You've got your server running.
When the app and the databsae are simple enough, this works like a charm. I have built several services using these very same steps and brought them to production without noticing any miscommunication between mongoose and the app. But, if you check the App.js code you provided, you'll realize that there is a difference in your case: you only connect to your Mongo cluster after you set up your app, its middleware, and its routes. If any of those depend on the mongoose instance or the models to run, there is a good chance that by the time Heroku's compiler gets to the point where your routes need to connect to your database, it simply hasn't implemented that connection (meaning hasn't run the mongoose.connect() part) jet.
Simple solution
I think that Heroku is simply taking a little bit longer to run your whole code than your local machine. Also, if your local version is connecting to a locally stored MongoDB, there is a good chance things run quite quicker than in a cloud service. Your statement
the 'connected to DB!' message comes in a bit late
gives me hope that this might be the case. If that were so, then I guess that moving
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {console.log('connected to DB!')});
to the second line of App.js, right after you call Dotenv, would do the trick. Your mongoose instance would go into App already connected with the database and models that App is going to use.
Asynchronous solution
Even if what I previously wrote fixed your problem, I would like to expand the explanation because, in my experience, that version of the solution wouldn't be right either. The way of doing things that I exposed at the beginning of this answer is ok... As long as you keep your MongoDB cluster and your mongoose instance simple. Not very long ago I had to implement a service that involved a much more complex database pattern, with several different databases being used by the same App, and many of its routes depending directly on those databases' models.
When I tried to use the logic I described before with such a problem, involving heavily populated databases and several models contained in the mongoose instance, I realized that it took Node far less building the App, its middleware, and its routes than connecting to mongoose and loading all the models that that very same App required for it to run. Meaning that I got my Server connected to PORT **** message before the database was actually connected. And that caused trouble.
That made me realize that the only way to do things properly was to ensure asynchronicity, forcing App to run only after all the databases and the models were loaded into the mongoose instance, and feeding it that very same instance. This way, I ended up having an index.js that looked like this:
const Dotenv = require("dotenv").config();
const { connectMongoose } = require("./server/db/mongoose");
const wrappedApp = require("./app");
connectMongoose().then((mongoose) => {
const App = wrappedApp(mongoose);
App.listen(process.env.PORT, () => {
console.log(
`Hello, I'm your server and I'm working on port ${process.env.PORT}`
);
});
});
A file with several functions govering the connections to my databases and modeles called mongoose.js. The function that implements all the connections in the right order looks is:
const connectMongoose = async (mongoose = require("mongoose")) => {
try {
// Close any previous connections
await mongoose.disconnect();
// Connect the Main database
console.log(
"--> Loading databases and models from MongoDB. Please don't use our server jet. <--"
);
mongoose = await connectMother(mongoose);
// Connect all the client databases
mongoose = await connectChildren(mongoose);
console.log(
"--> All databases and models loaded correctly. You can work now. <--"
);
return mongoose;
} catch (e) {
console.error(e);
}
};
module.exports = { connectMongoose };
With this, the file app.js returns a function that needs to be fed a mongoose instance and sets up all the Express environment:
module.exports = (Mongoose) => {
const Express = require("express"); //Enrouting
// Other required packages go here
// EXPRESS SETTINGS
const App = Express();
App.set("port", process.env.PORT || 3000); //Use port from cloud server
// All the middleware App needs
App.use(Flash());
// Routing: load your routes
// Return fully formed App
return App;
};
Only using this pattern I made sure that the chain of events made sense:
Connect all the databases and load their models.
Only after the last database has been connected and its last model, loaded, set up the App, its middleware, and your routes.
Use App.listen() to serve the App.
If your database isn't very large, then the first part of the answer might do the trick. But it's always useful to know the whole story.

I am able to reach an empty error message with your code when i add wrong connection string.
console.log in this callback is incorrect:
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" }, () => {
console.log('connected to DB!')
})
if you wanna know if you are connected to db use this:
mongoose.connect("mongodb+srv://xxxxx:xxxxx#cluster-rest.4luv0.mongodb.net/cluster-rest?retryWrites=true&w=majority", { useNewUrlParser: true, useUnifiedTopology: true, dbName: "cluster-rest" });
const db = mongoose.connection;
db.once('open', () => {
console.log('connection opened')
});
I noticed you are using dotenv maybe you are not adding variables to your heroku host.