Consul and Vault Configuration using Docker Compose

consul-vault

This article demonstrates how to deploy Consul and Vault using Docker Compose. These tools are widely used in modern DevOps environments for service discovery and secrets management.

What is Consul?

Consul is a Service Mesh tool that provides:

• Service discovery
• Health checks
• Key-Value store
• Node and service management via UI and CLI

You can use both Consul and Vault to store key-value pairs, but the key difference lies in how they store and manage those values.

What is Vault?

Vault is a secrets management system designed to:

• Securely store sensitive information such as passwords, tokens, and API keys
• Manage access policies for secrets
• Encrypt secrets at rest and in transit

It supports various storage backends, including:

• Consul
• Filesystem
• Amazon S3

🔸 Feature Comparison: Vault vs. Consul

🔸 Feature Comparison: Vault vs. Consul

Use Cases

Use Cases

Consul Setup using Docker Compose:

version: '3.8'
services:
  consul:
    image: hashicorp/consul:latest
    container_name: consul
    restart: always
    volumes:
      - ./consul_data:/consul/data
      - ./consul_config:/consul/config
    ports:
      - "8500:8500"        # UI
      - "8600:8600/udp"    # DNS (UDP)
      - "8600:8600/tcp"    # DNS (TCP)
      - "8300:8300"        # Server RPC
      - "8301:8301"        # Serf LAN
      - "8301:8301/udp"
      - "8302:8302"
      - "8302:8302/udp"
    command: agent -server -bootstrap -ui -client=0.0.0.0 -data-dir=/consul/data -config-dir=/consul/config
    environment:
      - CONSUL_LOCAL_CONFIG={"acl":{"enabled":true, "default_policy":"deny", "down_policy":"extend-cache"}}
      - CONSUL_RETRY_JOIN_ADDRESS=178.18.63.42
      - CONSUL_GOSSIP_ENCRYPTION=enable
      - CONSUL_GOSSIP_ENCRYPTION_KEY=Mmqfj/p23iDGvBi1U6gE+WCr+dT+Fzm8+S2HcoVTB6c=
    networks:
      - consul-network

networks:
  consul-network:
    driver: bridge

volumes:
  consul_data:
  consul_config:

Notes:

• UI Access: http://127.0.0.1:8500/ui
• CONSUL_GOSSIP_ENCRYPTION_KEY must be a unique value for secure clustering.
• Ports:
• 8500: Web UI
• 8600: DNS Interface (UDP & TCP)
• 8300: Server RPC
• 8301: Serf LAN (Gossip protocol)

ACL System in Consul

When ACL is enabled (acl.enabled=true), all access is denied by default. You'll need to create a bootstrap token:

docker exec -it consul consul acl bootstrap

Example output:

AccessorID:       a3107a880712-0241
SecretID:         a92eb736-c4b7-940f
Description:      Bootstrap Token (Global Management)
Create Time:      2025-07-19T10:46:45Z
Policies:         global-management

⚠️ Save the SecretID value — you’ll use it to log into the Consul UI.

Creating ACL Policies & Tokens

To give a user edit access, you can define a policy like this:

node_prefix "" {
  policy = "read"
}
service_prefix "" {
  policy = "write"
}
key_prefix "" {
  policy = "write"
}

Then assign this policy to a token.

Vault Setup using Docker Compose

version: '3.8'
services:
  vault:
    image: hashicorp/vault:latest
    container_name: vault-server
    ports:
      - "8200:8200"
    volumes:
      - ./vault-data:/vault/file
    cap_add:
      - IPC_LOCK
    environment:
      VAULT_LOCAL_CONFIG: |
        {
          "storage": {
            "file": {
              "path": "/vault/file"
            }
          },
          "listener": [
            {
              "tcp": {
                "address": "0.0.0.0:8200",
                "tls_disable": true
              }
            }
          ],
          "default_lease_ttl": "168h",
          "max_lease_ttl": "720h",
          "ui": true
        }
    command: server
    restart: unless-stopped

Notes:

• UI Access: http://127.0.0.1:8200
• Vault data is stored in the local filesystem: /vault/file

In production, replace file storage with Consul backend for high availability and clustering.

Vault Initialization & Policy Management

1. When Vault starts for the first time, you’ll be asked how many unseal keys to generate.
2. Save them securely — you’ll need them to unseal the Vault.

Creating a Policy in Vault

1. Create an HCL policy file named editor.hcl:

path "secret/data/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

2. Copy it to the container:

docker cp editor.hcl vault-server:/tmp/editor.hcl

3. Enter the container:

docker exec -it vault-server /bin/sh

4. Set Vault address:

export VAULT_ADDR='http://127.0.0.1:8200'

5. Login with your root token:

vault login <your-token>

6. Write the policy:

vault policy write editor /tmp/editor.hcl

7. Create a token with the policy:

vault token create -policy=editor -ttl=1h

Conclusion

This guide explained how to set up Consul and Vault using Docker Compose, configure ACL and encryption, and create custom policies. These tools are essential for secure infrastructure management and service discovery in distributed systems.