I'm building an application which uses Node, redis and mongo. I finished the development, and I want to containerize it with docker.
Here's my Dockerfile:
FROM node:13.8.0-alpine3.11
RUN npm install -g pm2
WORKDIR /user/src/app
COPY package*.json ./
RUN npm install --production
COPY . .
EXPOSE 3000
And here my docker-compose.yml
version: '3'
services:
redis-server:
container_name: scrapr-redis
image: 'redis:6.0-rc1-alpine'
ports:
- '6379:6379'
mongo-db:
container_name: scrapr-mongo
image: mongo
ports:
- '27017:27017'
command: --auth
environment:
- MONGO_INITDB_ROOT_USERNAME=user
- MONGO_INITDB_ROOT_PASSWORD=pass
- MONGO_INITDB_DATABASE=db
app:
container_name: scrapr-node
restart: always
build: .
ports:
- '3000:3000'
- '3001:3001'
links:
- mongo-db
depends_on:
- redis-server
environment:
- DB_USER=user
- DB_PWD=pass
- DB_NAME=db
- REDIS_HOST=redis-server
command: 'node index.mjs'
I can start the service successfully, but when node starts, it generates the following error:
Error Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
When I do docker ps -a, I can see that all containers are running:
Why can't it connect with redis? What did I miss?
127.0.0.1 does not look right to me at all. Let me get the quick checks out of the way first, are you sure you are using the REDIS_HOST env variable in the node app correctly? I would add some console logging to your node app to echo out the env variables to check what they are.
Secondly try attach to the running scrapr-node with docker container exec -it scrapr-node sh or /bin/bash if sh does not work.
Then run nslookup scrapr-redis from the shell, this will give you the ip address of the redis container. if ping scraper-redis returns then you know its an issue with your node app not the docker network.
you can also exec into the redis node and run hostname -I which should show the same ip address as you saw from the other container.
This should help you to debug the issue.
EDIT:
Ensure that you are correctly getting the value from your environment into your node app using process.env.REDIS_HOST and then correctly using that value when connecting to redis something like:
const redisClient = redis.createClient({
host: process.env.REDIS_HOST,
port: 6379
});
I would not try and force 127.0.0.1 on the docker network (if that is even possible) it is reserved as the loopback address.
Related
I'm trying to allow communication between my nodeJs docker image with my redis docker image (Mac OS X environment):
nodeJs Dockerfile:
FROM node:4.7.0-slim
EXPOSE 8100
COPY . /nodeExpressDB
CMD ["node", "nodeExpressDB/bin/www"]
redis Dockerfile:
FROM ubuntu:14.04.3
EXPOSE 6379
RUN apt-get update && apt-get install -y redis-server
nodeJs code which is trying to connect to redis is:
var redis = require('redis');
var client = redis.createClient();
docker build steps:
docker build -t redis-docker .
docker build -t node-docker .
docker run images steps flow:
docker run -p 6379:6379 redis-docker
docker run -p 8100:8100 node-docker
ERROR:
Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at Object.exports._errnoException (util.js:907:11)
at exports._exceptionWithHostPort (util.js:930:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1078:14)
What should I do inorder to connect to Redis from node-docker?
Redis runs in a seperate container which has seperate virtual ethernet adapter and IP address to the container your node application is running in. You need to link the two containers or create a user defined network for them
docker network create redis
docker run -d --net "redis" --name redis redis
docker run -d -p 8100:8100 --net "redis" --name node redis-node
Then specify the host redis when connecting in node so the redis client attempts to connect to the redis container rather than the default of localhost
const redis = require('redis')
const client = redis.createClient(6379, 'redis')
client.on('connect', () => console.log('Connected to Redis') )
Docker Compose can help with the definition of multi container setups.
version: '2'
services:
node:
build: .
ports:
- "8100:8100"
networks:
- redis
redis:
image: redis
networks:
- redis
networks:
redis:
driver: bridge
Solution
const client = redis.createClient({
host: 'redis-server',
port: 6379
})
Then rebuild ur docker with => docker-compose up --build
When connecting in node, use redis-docker in the function argument you pass the server IP.
If you can create a new redis docker instance, try mapping the container port to host:
docker run --name some-redis -p 6379:6379 -d redis
docker container start some-redis
Now, you can start the container and connect with host 127.0.0.1
const redis = require("redis");
const client = redis.createClient({
host: '127.0.0.1',
port: '6379'
});
if redis installed then run command
sudo apt-get install redis-server
Then you will get your site running.
You can also change your /etc/hosts file to update the dockerip of the redis container.
Find the docker ip by using docker inspect
Download the redis server.
run the redis server.
and then run your project.
It should work just fine. Here's the download link:
Github - Redis Download Packages
I hope it works.
I have docker-compose.ymllike following
version: '3'
services:
api-server:
build: ./api
links:
- 'db'
ports:
- '3000:3000'
volumes:
- ./api:/src
- ./src/node_modules
tty: true
container_name: api-server
db:
build:
context: .
dockerfile: ./db/Dockerfile
restart: always
hostname: db
environment:
MYSQL_ROOT_PASSWORD: test
MYSQL_USER: test
MYSQL_PASSWORD: test
MYSQL_DATABASE: test
volumes:
- './db:/config'
ports:
- 3306:3306
container_name: db
And then I tried
docker-compose build
docker-compose up -d
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
567e1e7463d api-server "docker-entrypoint.s…" 21 hours ago Up 23 minutes 0.0.0.0:3000->3000/tcp api-server
e85e746d699 db "docker-entrypoint.s…" 3 days ago Up 21 hours 0.0.0.0:3306->3306/tcp, 33060/tcp db
And then I'd like to test Post method to api-server but some error returned.
{
"statusCode": 500,
"message": "Internal server error"
}
Therefore I tried to know the cause of this to enter api-server in docker
docker exec -it api-server sh
And I launch api-server internally
npm run start
And then tried to POST to api-server
But the following error displayed in console.
Error: listen EADDRINUSE: address already in use :::3000
What is the cause of this?
The port 3000 was already used when `docker-compose up -d' ?
How to avoid this error?
If someone has opinion,please let me know
Thanks
There's already an instance of your Node application running (as the main container process) so when you npm run start a second one in a debugging shell you'll get that "address already in use" error. You'll get the same error without Docker if you try to start the server twice in two separate terminal windows.
If you're trying to actively debug the program, I'd suggest using a host Node installation, which will generally be more convenient. Or if you really need two copies of the server, run them in two separate containers; you can use docker compose run api-server ... to get a new container with largely the same setup as the existing container (notably, without published ports) but a different command.
I use Docker to contain my Adonis app. The build was success but when I access the app, I got ERR_SOCKET_NOT_CONNECTED or ERR_CONNECTION_RESET.
My docker compose contains adonis and database. Previously, I use the setup similar with this for my expressjs app, and it has no problem.
The adonis .env is remain standard, modification.
This is my setup:
# docker-compose.yml
version: '3'
services:
adonis:
build: ./adonis
volumes:
- ./adonis/app:/usr/src/app
networks:
- backend
links:
- database
ports:
- "3333:3333"
database:
image: mysql:5.7
ports:
- 33060:3306
networks:
- backend
environment:
MYSQL_USER: "user"
MYSQL_PASSWORD: "root"
MYSQL_ROOT_PASSWORD: "root"
networks:
backend:
driver: bridge
# adonis/Dockerfile
FROM node:12-alpine
RUN npm i -g #adonisjs/cli
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY ./app/. .
RUN npm install
EXPOSE 3333
CMD ["adonis", "serve", "--dev"]
I couldn't spot anything wrong with my setup.
The serve command starts the HTTP server on the port defined inside the .env file in the project root.
You should have something like this(note that HOST has to be set to 0.0.0.0 instead of localhost to accept connections from the outside):
HOST=0.0.0.0
PORT=3333
APP_URL=http://${HOST}:${PORT}
I've installed MongoDB v4.0 for the most amazing feature of it Transaction in Nodejs with mongodb 3.1 as a driver.
When I try to use a transaction session I've faced this error:
MongoError: Transaction numbers are only allowed on a replica set member or mongos.
What's that and how can I get rid of it?
Transactions are undoubtedly the most exciting new feature in MongoDB 4.0. But unfortunately, most tools for installing and running MongoDB start a standalone server as opposed to a replica set. If you try to start a session on a standalone server, you'll get this error.
In order to use transactions, you need a MongoDB replica set, and starting a replica set locally for development is an involved process. The new run-rs npm module makes starting replica sets easy. Running run-rs is all you need to start a replica set, run-rs will even install the correct version of MongoDB for you.
Run-rs has no outside dependencies except Node.js and npm. You do not need to have Docker, homebrew, APT, Python, or even MongoDB installed.
Install run-rs globally with npm's -g flag. You can also list run-rs in your package.json file's devDependencies.
npm install run-rs -g
Next, run run-rs with the --version flag. Run-rs will download MongoDB v4.0.0 for you. Don't worry, it won't overwrite your existing MongoDB install.
run-rs -v 4.0.0 --shell
Then use replicaSet=rs in your connection string.
You find more details about it here.
I got the solution, and it's just three lines configuration inside the MongoDB config file.
After switching from MongoDB atlas and installing MongoDB v 4.4.0 on my CentOS 7 VPS with WHM, I faced that issue also.
the run-rs solution does not work for me, but I managed to solve this issue without any third-party tool, following these steps:
1. turn off mongod.
the most efficient way is by entering the MongoDB shell with the command mongo
checkout the method
db.shutdownServer()
You will be no ability to use the MongoDB server.
For me, the shutdown process took too long, and then I killed the process with the command:
systemctl stop -f mongod
if you killed the mongod process,s probably you will need to run
mongod --dbpath /var/db --repair
The var/db should point to your database directory.
2. setting replicaSet configuration.
for the replicaSet settings step, check out the /etc/mongod.conf file,
look for the replication value line, and you should add the following lines as below:
replication:
oplogSizeMB: <int>
replSetName: <string>
enableMajorityReadConcern: <boolean>
use the replSetName value on the next step.
an example of those settings:
oplogSizeMB: 2000
replSetName: rs0
enableMajorityReadConcern: false
3. add your connection string URL.
add the value of replSetName to your connection URL &replicaSet=--YourReplicationSetName--
if you used the name rs0 from our example, then you should add to your DB connection URL query replicaSet=rs0
4. turn on mongod again
enter the command: systemctl start mongod
5. Access your replicaSet database
enter MongoDB shell with the command mongo, enter the command rs.initiate()
now you should be in your replicaSet database.
Possible solution for local development using docker
Create Dockerfile
FROM mongo:4.4.7
RUN echo "rs.initiate();" > /docker-entrypoint-initdb.d/replica-init.js
CMD [ "--replSet", "rs" ]
Build this Dockerfile
docker build ./ -t mongodb:4.7-replset
Run this created image
docker run --name mongodb-replset -p 27017:27017 -d mongodb:4.7-replset
Connect to database using this URI
mongodb://localhost:27017/myDB
For those who wants to develop against of the dockerized MongoDB instance, here is the single-file docker-compose.yaml solution based on the official MongoDB docker image:
version: '3.9'
services:
mongodb:
image: mongo:5
command: --replSet rs0
ports:
- "28017:27017"
environment:
MONGO_INITDB_DATABASE: attachment-api-local-dev
healthcheck:
test: echo 'db.runCommand("ping").ok' | mongo localhost:27017/admin --quiet
interval: 2s
timeout: 3s
retries: 5
mongo-init:
image: mongo:5
restart: "no"
depends_on:
mongodb:
condition: service_healthy
command: >
mongo --host mongodb:27017 --eval
'
rs.initiate( {
_id : "rs0",
members: [
{ _id: 0, host: "localhost:27017" }
]
})
'
A much easier solution is to just use Bitnami MongoDB image:
services:
mongodb:
image: bitnami/mongodb:5.0
ports:
- "27017:27017"
environment:
MONGODB_REPLICA_SET_MODE: primary
ALLOW_EMPTY_PASSWORD: 'yes'
I faced the same issue recently. In my case it's because I'm connecting to a remote Mongo server with a different version than my local development environment.
To quickly solve the issue, I added the following param to my connection string:
?retryWrites=false
In order to use transactions, you need a MongoDB replica set, and starting a replica set locally for development is an involved process.
You can use the run-rs npm module. Zero-config MongoDB runner. Starts a replica set with no non-Node dependencies, not even MongoDB.
Or you can simply create an account in MongoDB Atlas which gives you a limited resource MongoDB cluster and so you can run/test your application.
MongoDB Atlas
When running MongoDB on a Linux Machine, you can simply use replication by updating connection string via editing service file
/usr/lib/mongod.service or /lib/systemd/system/mongod.service
and update it with following
ExecStart=/usr/bin/mongod --config "/etc/mongod.conf" --replSet rs0
where --config "/etc/mongod.conf" is pointing to your MongoDB Configuration file and --replSet rs0 is telling it to use replication with the name of rs0
and then restart
sudo systemctl daemon-reload //<--To reload service units
sudo systemctl restart mongod //<--To Restart MongoDB Server
and then initiate replication through your mongod instance in terminal
$ mongosh
$ rs.initiate()
I've been fighting against this issue for weeks. I let you my conclusion.
In order to be able to use transactions on a sharded cluster, you need to run at least MongoDB 4.2 on your cluster. If the cluster is not sharded, from 4.0.
I was using a library that has as a sub-dependency mongodb NodeJS driver. This driver from version 3.3.x fails against the sharded MongoDB cluster with version 4.0.4.
The solution for me was to update my cluster to 4.2 version.
src: https://www.bmc.com/blogs/mongodb-transactions/
Works for mongo:5.0.5-focal image.
Dockerfile:
FROM mongo:5.0.5-focal AS rs-mongo
# Make MongoDB a replica set to support transactions. Based on https://stackoverflow.com/a/68621185/1952977
RUN apt-get update && apt-get install patch
# How to create scripts/docker-entrypoint.sh.patch
# 1. Download the original file:
# wget https://github.com/docker-library/mongo/raw/master/5.0/docker-entrypoint.sh
# ( wget https://github.com/docker-library/mongo/raw/b5c0cd58cb5626fee4d963ce05ba4d9026deb265/5.0/docker-entrypoint.sh )
# 2. Make a copy of it:
# cp docker-entrypoint.sh docker-entrypoint-patched.sh
# 3. Add required modifications to docker-entrypoint-patched.sh
# 4. Create patch:
# diff -u docker-entrypoint.sh docker-entrypoint-patched.sh > scripts/docker-entrypoint.sh.patch
# 5. Clean up:
# rm docker-entrypoint.sh docker-entrypoint-patched.sh
COPY scripts/docker-entrypoint.sh.patch .
RUN patch /usr/local/bin/docker-entrypoint.sh docker-entrypoint.sh.patch
RUN mkdir -p /etc/mongo-key && chown mongodb:mongodb /etc/mongo-key
CMD ["--replSet", "rs", "--keyFile", "/etc/mongo-key/mongodb.key"]
scripts/docker-entrypoint.sh.patch:
--- docker-entrypoint.sh 2022-01-04 15:35:19.594435819 +0300
+++ docker-entrypoint-patched.sh 2022-01-06 10:16:26.285394681 +0300
## -288,6 +288,10 ##
fi
if [ -n "$shouldPerformInitdb" ]; then
+
+ openssl rand -base64 756 > /etc/mongo-key/mongodb.key
+ chmod 400 /etc/mongo-key/mongodb.key
+
mongodHackedArgs=( "$#" )
if _parse_config "$#"; then
_mongod_hack_ensure_arg_val --config "$tempConfigFile" "${mongodHackedArgs[#]}"
## -408,7 +412,14 ##
set -- "$#" --bind_ip_all
fi
- unset "${!MONGO_INITDB_#}"
+ echo 'Initiating replica set'
+ "$#" --logpath "/proc/$$/fd/1" --fork
+ echo 'rs.initiate({"_id":"rs","members":[{"_id":0,"host":"127.0.0.1:27017"}]});' | mongosh -u "$MONGO_INITDB_ROOT_USERNAME" -p "$MONGO_INITDB_ROOT_PASSWORD"
+ "$#" --logpath "/proc/$$/fd/1" --shutdown
+ echo 'Done initiating replica set'
+
+ unset "${!MONGO_INITDB_#}"
+
fi
rm -f "$jsonConfigFile" "$tempConfigFile"
docker-compose.yml:
version: '3.9'
services:
mongo:
image: rs-mongo:current
restart: always
env_file:
- .env
ports:
- 127.0.0.1:27017:27017
volumes:
- mongo-db:/data/db
- mongo-configdb:/data/configdb
- mongo-key:/etc/mongo-key
volumes:
mongo-db:
driver: local
mongo-configdb:
driver: local
mongo-key:
driver: local
UPDATED: 6th of Jan, 2022
The error is because you are using MongoDB sessions and it is not configured on your system.
run this to install run-rs :-
npm install run-rs -g
run:-
run-rs -v 4.0.0 --shell
You should see the below output. Please be patient since MongoDB 4.0.0 is about 70MB.
$ run-rs -v 4.0.0 --shell
Downloading MongoDB 4.0.0
Copied MongoDB 4.0.0 to '/home/node/lib/node_modules/run-rs/4.0.0'
Purging database...
Running '/home/node/lib/node_modules/run-rs/4.0.0/mongod'
Starting replica set...
Started replica set on "mongodb://localhost:27017,localhost:27018,localhost:27019"
Running mongo shell: /home/node/lib/node_modules/run-rs/4.0.0/mongo
rs:PRIMARY>
You now have a replica set running MongoDB 4.0.0 locally. Run rs.status() to verify the replica set is running.
NOTE:- Your nodejs version should be $gte v3.1.0
I working with node.js by expressjs I try to store an account to session. So, i try to test to use session with code in expressjs
var RedisStore = require('connect-redis')(express);
app.use(express.bodyParser());
app.use(express.cookieParser());
app.use(express.session({ secret: "keyboard cat", store: new RedisStore }));
but I got error Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED. Please help me resolve this problem
After you install redis, type from terminal:
redis-server
and you'll have redis running
I solve this problem in next way:
sudo apt-get install redis-server
then run command to confirm that everything ok:
sudo service redis-server status
And the output will be: redis-server is running - that means that the problem is solved.
Install redis on your system first -
brew install redis
then start the redis server -
redis-server
I'm on windows, and had to install Redis from here and then run redis-server.exe.
From the top of this SO question.
For those of you who are using docker with docker-compose and Typescript my solution was
import { RedisClient } from 'redis';
const pubClient = new RedisClient({ url: 'redis://redis:6379' });
to
import { createClient } from 'redis';
const pubClient = createClient({ url: 'redis://redis:6379' });
docker-compose.yml
version: '3.9'
services:
main:
build:
context: .
target: development
ports:
- ${PORT}:${PORT}
volumes:
- ./src:/usr/src/app/src
- /app/node_modules
env_file:
- .env
command: npm run start:dev
depends_on:
- mongo
- redis
mongo:
image: mongo:5.0.2-focal
volumes:
- mongo-data:/data/db
mongo-express:
image: mongo-express:0.54.0
ports:
- 8081:8081
depends_on:
- mongo
redis:
image: redis:6.2.5-alpine
volumes:
mongo-data:
Simple solution:
only hit below commend once and restart your server again
redis-server
Using Windows 10?
Go here: https://learn.microsoft.com/en-us/windows/wsl/wsl2-install
Then run...
$ wget https://github.com/antirez/redis/archive/5.0.5.tar.gz <- change this to whatever Redis version you want (https://github.com/antirez/redis/releases)
$ tar xzf redis-5.0.5.tar.gz
$ cd redis-5.0.5
$ make
for Windows users, you can use chocolatey to install Redis
choco install redis-64
then run server from
C:\ProgramData\chocolatey\lib\redis-64\redis-server.exe
I also have the same problem, first I tried to restart redis-server by sudo service restart but the problem still remained. Then I removed redis-server by sudo apt-get purge redis-server and install it again by sudo apt-get install redis-server and then the redis was working again. It also worth to have a look at redis log which located in here /var/log/redis/redis-server.log
I used ubuntu 12.04
I solved that problem by installing redis-server
redis-server installation for ubuntu 12.04
some configuration will new root permission
Also listed manuals for other OS
Thanks
For me I had this issue on Ubuntu 18.x, but my problem was that my redis-server was running on 127.0.0.1 but I found out I needed to run it on my IP address xxx.xx.xx.xx
I went into my Ubuntu machine and did the following.
cd /etc/redis/
sudo vim redis.conf
Then I edited this part.
################################## NETWORK #####################################
# By default, if no "bind" configuration directive is specified, Redis listens
# for connections from all the network interfaces available on the server.
# It is possible to listen to just one or multiple selected interfaces using
# the "bind" configuration directive, followed by one or more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1 ::1
#
# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the
# internet, binding to all the interfaces is dangerous and will expose the
# instance to everybody on the internet. So by default we uncomment the
# following bind directive, that will force Redis to listen only into
# the IPv4 loopback interface address (this means Redis will be able to
# accept connections only from clients running into the same computer it
# is running).le to listen to just one or multiple selected interfaces using
#
# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES
# JUST COMMENT THE FOLLOWING LINE.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# bind 127.0.0.1 ::1 10.0.0.1
bind 127.0.0.1 ::1 # <<-------- change this to what your iP address is something like (bind 192.168.2.2)
Save that, and then restart redis-server.
sudo service redis-server restart or simply run redis-server
For windows platform, You must check if redis-server is running on given ip:port. you can find redis configuration at installation directory /conf/redis.conf. by default client accept 127.0.0.1:6379.
I'm on MBP , and install redis detail my problem was resolved .Fixed the
Download, extract and compile Redis with:
$ wget http://download.redis.io/releases/redis-3.0.2.tar.gz
$ tar xzf redis-3.0.2.tar.gz
$ cd redis-3.0.2
$ make
The binaries that are now compiled are available in the src directory.
Run Redis with:
$ src/redis-server
I think maybe you installed redis by source code.If that you need locate to redis-source-code-path/utils and run sudo install_server.sh command.
After that, make sure redis-server has been running as a service for your system
sudo service redis-server status
PS: based on Debian/Ubuntu
In case of ubuntu, the error is due to redis-server not being set up.
Install the redis-server again and then check for the status.
If there is no error, then a message like this would be displayed :-
● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2018-01-17 20:07:27 IST; 16s ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Main PID: 4327 (redis-server)
CGroup: /system.slice/redis-server.service
└─4327 /usr/bin/redis-server 127.0.0.1:6379
You have to install redis server first;
You can install redis server on mac by following step -
$ curl -O http://download.redis.io/redis-stable.tar.gz
$ tar xzvf redis-stable.tar.gz
$ cd redis-stable
$ make
$ make test
$ sudo make install
$ redis-server
Good luck.
Your connection to redis is failing. Try restarting your redis server, then starting up your client again by running these 3 commands:
sudo service redis-server restart
redis-server
redis-cli
For Windows I solved this by...
using...
let redisClient = createClient({
legacyMode: true ,
url: 'redis://redis:6379',
});
Its for redis version > 4.0
You can refer to the image below.
Try upgrading your node to latest version.
sudo npm cache clean -f
sudo npm install -g n
sudo n stable
version 0.4 may not work properly.