Replication Setup in Postgres Using Docker

Setting Up PostgreSQL Master-Replica Architecture with Docker

Here's a comprehensive step-by-step guide to set up a PostgreSQL master-replica cluster with one master and three replica nodes. We will use postgres asynchronous replication for master and replica

Project Structure Setup

First, ensure your project directory follows this structure:

postgres-replica-demo/
├── docker-compose.yml
├── master/
│   ├── Dockerfile
│   └── init-master.sh
└── replica/
    ├── Dockerfile
    └── docker-entrypoint.sh

Step 1: Complete the Master Configuration Files

Master Dockerfile

Dockerfile for master database docker instance , it will create image and added sh script as entry point to mark it for master for replication.

./master/Dockerfile:

FROM postgres:16

COPY init-master.sh /docker-entrypoint-initdb.d/init-master.sh

RUN chmod +x /docker-entrypoint-initdb.d/init-master.sh

Master Initialization Script

Here is the bash script that will set replication in master database instance

./master/init-master.sh

#!/bin/bash

set -e

echo "host replication replica_user 0.0.0.0/0 md5" >> "$PGDATA/pg_hba.conf"

cat >> "$PGDATA/postgresql.conf" << EOF
# Replication settings
wal_level = replica
max_wal_senders = 10
max_replication_slots = 10
synchronous_commit = off

# Performance settings
shared_buffers = 256MB
checkpoint_completion_target = 0.7
wal_buffers = 16MB
default_statistics_target = 100
random_page_cost = 1.1
effective_cache_size = 512MB
EOF

# Create replication user
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
    CREATE USER replica_user REPLICATION LOGIN ENCRYPTED PASSWORD 'root';
EOSQL

Key Configuration Explanations:

wal_level = replica: Enables write-ahead logging for replication
max_wal_senders = 10: Allows up to 10 concurrent replica connections
max_replication_slots = 10: Creates slots for tracking replication progress
synchronous_commit = off: Improves performance by not waiting for replica confirmation

Step 2: Complete the Replica Configuration Files

Replica Dockerfile

Replica container docker file, will create postgres container and run script to mark it as replica.

./replica/Dockerfile:

FROM postgres:16

RUN apt-get update && \
    apt-get install -y gosu && \
    rm -rf /var/lib/apt/lists/*

COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh

RUN chmod +x /usr/local/bin/docker-entrypoint.sh

ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

Replica Entrypoint Script

Replication script that handles the replica initialization:

./replica/docker-entrypoint.sh

#!/bin/bash

set -e

# Ensure proper ownership
chown -R postgres:postgres "$PGDATA"
chmod 700 "$PGDATA"

# Wait until master is ready
echo "Waiting for master to be ready..."
until pg_isready -h master -p 5432 -U replica_user; do
    echo "Still waiting for master..."
    sleep 2
done

# Initialize replica if needed
if [ ! -s "$PGDATA/PG_VERSION" ]; then
    echo "Running base backup..."
    gosu postgres pg_basebackup -h master -U replica_user -D "$PGDATA" -Fp -Xs -P -R
    chmod 700 "$PGDATA"
fi

# Start postgres as the correct user
exec gosu postgres postgres

Critical Script Components:

pg_isready check: Ensures master is accepting connections before proceeding
pg_basebackup: Creates a physical copy of the master database
-R flag: Automatically configures replication settings
gosu: Ensures PostgreSQL runs with correct user permissions

Step 3: Docker -compose file

The docker-compose file will be responsible for setting infra.

creates master postgres instance running on port 6000
creates 3 replica postgres instance running on port 6001, 6002, 6003
Every container has volume mount so that data is not lost in case of container shut down.

version: '3.8'

services:
  master:
    build: ./master
    container_name: master
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: root
    ports:
      - "6000:5432"
    volumes:
      - master-data:/var/lib/postgresql/data
    networks:
      - pgnet

  replica1:
    build: ./replica
    container_name: replica1
    depends_on:
      - master
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: root      # required by postgres image but not used for init
      PGPASSWORD: root             # for pg_basebackup auth replication user
      REPLICA_NAME: replica1
    ports:
      - "6001:5432"
    volumes:
      - replica1-data:/var/lib/postgresql/data
    networks:
      - pgnet

  replica2:
    build: ./replica
    container_name: replica2
    depends_on:
      - master
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: root
      PGPASSWORD: root
      REPLICA_NAME: replica2
    ports:
      - "6002:5432"
    volumes:
      - replica2-data:/var/lib/postgresql/data
    networks:
      - pgnet

  replica3:
    build: ./replica
    container_name: replica3
    depends_on:
      - master
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: root
      PGPASSWORD: root
      REPLICA_NAME: replica3
    ports:
      - "6003:5432"
    volumes:
      - replica3-data:/var/lib/postgresql/data
    networks:
      - pgnet

volumes:
  master-data:
  replica1-data:
  replica2-data:
  replica3-data:

networks:
  pgnet:

Step 4: Deploy the Cluster

Build and Start Services

# Navigate to your project directory
cd postgres-replica-demo

# Build and start all services
docker-compose up --build -d

# Monitor the startup process
docker-compose logs -f

Verify Container Status

# Check all containers are running
docker-compose ps

# Expected output:
# NAME       COMMAND                  SERVICE    STATUS         PORTS
# master     "docker-entrypoint.s…"   master     Up 2 minutes   0.0.0.0:6000->5432/tcp
# replica1   "/usr/local/bin/dock…"   replica1   Up 2 minutes   0.0.0.0:6001->5432/tcp
# replica2   "/usr/local/bin/dock…"   replica2   Up 2 minutes   0.0.0.0:6002->5432/tcp
# replica3   "/usr/local/bin/dock…"   replica3   Up 2 minutes   0.0.0.0:6003->5432/tcp

Step 5: Verification and Testing

Test Master Database Connection

# Connect to master
docker exec -it master psql -U postgres

# Create a test database and table
CREATE DATABASE testdb;
\c testdb;
CREATE TABLE users (id SERIAL PRIMARY KEY, name VARCHAR(50), email VARCHAR(100));
INSERT INTO users (name, email) VALUES ('John Doe', 'john@example.com');

Verify Replication Status on Master

sql-- Check replication status
SELECT client_addr, state, sync_state FROM pg_stat_replication;

-- Expected output showing 3 connected replicas:
-- client_addr | state     | sync_state 
-- 172.x.x.x   | streaming | async
-- 172.x.x.x   | streaming | async  
-- 172.x.x.x   | streaming | async

Test Read Operations on Replicas

# Connect to replica1
docker exec -it replica1 psql -U postgres -d testdb

# Verify data replication
SELECT * FROM users;

# Should show the inserted data from master

Test Replication Lag

# On master, check current WAL location
docker exec -it master psql -U postgres -c "SELECT pg_current_wal_lsn();"

# On replica, check received WAL location  
docker exec -it replica1 psql -U postgres -c "SELECT pg_last_wal_receive_lsn();"

Step 6: Performance Monitoring and Health Checks

Create Monitoring Scripts

Create a monitor.sh script for ongoing health checks:

#!/bin/bash

echo "=== Master Status ==="
docker exec master psql -U postgres -c "SELECT client_addr, application_name, state, sync_state FROM pg_stat_replication;"

echo -e "\n=== Replica Status ==="
for replica in replica1 replica2 replica3; do
    echo "--- $replica ---"
    docker exec $replica psql -U postgres -c "SELECT pg_is_in_recovery(), pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn();"
done

echo -e "\n=== Container Resources ==="
docker stats --no-stream master replica1 replica2 replica3

Troubleshooting Common Issues

Replica Startup Failures

# Check replica logs for connection issues
docker-compose logs replica1

# Common solutions:
# 1. Ensure master is fully started
# 2. Verify network connectivity
# 3. Check replication user permissions

Replication Lag Issues

# Monitor WAL sender processes
docker exec master psql -U postgres -c "SELECT * FROM pg_stat_replication;"

# Check for slow replicas
docker exec master psql -U postgres -c "SELECT client_addr, state, sent_lsn, write_lsn, flush_lsn, replay_lsn FROM pg_stat_replication;"

Data Directory Persistence

Your configuration uses named volumes for data persistence:

master-data: Stores master database files
replica1-data, replica2-data, replica3-data: Store replica-specific files

To reset the entire cluster:

# Stop and remove containers
docker-compose down

# Remove all volumes (WARNING: This deletes all data)
docker-compose down -v

# Restart fresh
docker-compose up --build -d

This setup provides a robust PostgreSQL master-replica architecture that's perfect for Spring Boot read replica demo, with proper replication monitoring, health checks, and scalability options.

PreviousDatabase Replication Types NextSpring Boot Setup And Demo

Last updated 12 days ago