Day 03: Docker Compose & Profiles | EmpowerNow Developer Academy

🎯 Learning Objectives

By the end of today's session, you will be able to:

Understand the Docker Compose file structure and profile system
Select appropriate profiles for different development scenarios
Optimize resource usage by running only necessary services
Troubleshoot common Docker issues with EmpowerNow services
Build services from source for local development
Use pgAdmin to inspect and query PostgreSQL databases

🐳 Understanding Docker Compose Profiles

Why Profiles?

The EmpowerNow platform consists of 30+ services. Running all of them simultaneously requires significant resources and isn't necessary for most development tasks. Docker Compose profiles let you:

Selective Startup: Run only the services relevant to your current task
Resource Efficiency: Save memory and CPU by not running unnecessary services
Faster Iteration: Quicker startup times when working on specific features
Environment Flexibility: Different profiles for development, testing, and full stack

💡

How Profiles Work

Services in docker-compose.yml are tagged with profile names. When you run docker compose --profile NAME up, only services with that profile (plus services with no profile) start.

📦 Available Profiles

EmpowerNow organizes services into logical profiles. Here's what each profile includes:

🔧 Interactive Command Builder

🔧 Profile selection is automated using your organization’s deployment procedures and approved automation.

Select the profiles you need, and we'll generate the exact command for you:

🛠️ Build Your Docker Compose Command

docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build

🔐

Before running docker compose: Authenticate to Azure Container Registry (when using pre-built images)

If you run without --build, Docker pulls images from eidci.azurecr.io. Without ACR authentication, pulls fail with "authentication required".

Using --build? When you use docker-compose-build.yml with --build, services are built locally from source and tagged as :local. ACR login is not required for those overridden services.

# If you have Azure CLI installed and access to the eidci registry:
az acr login --name eidci
                        

No Azure CLI or no ACR access yet? Get credentials from Azure Portal: eidci → Settings → Access keys. Then run:

docker login eidci.azurecr.io
# Use the Login server as registry, Username and Password from Access keys
                        

⚠️

Before running docker compose: Grant Docker access to config_secrets

If Docker Desktop keeps restarting or cannot access secrets, run this once. Replace <YourRepoRoot> with your local clone path (e.g. D:\EmpowerNow or E:\Source\Repos\EmpowerNow on Windows; /Users/you/EmpowerNow on macOS/Linux):

# Run in PowerShell as Administrator
icacls "<YourRepoRoot>\ServiceConfigs\Deployment\config_secrets" /grant Everyone:F
                        

Path must end with ...\ServiceConfigs\Deployment\config_secrets (Windows) or .../ServiceConfigs/Deployment/config_secrets (macOS/Linux).

macOS/Linux: Ensure config_secrets is readable (e.g. chmod -R u+r config_secrets from ServiceConfigs/Deployment).

📄

Before running docker compose: Create idp-bootstrap-secrets.env

The compose file references env/services/idp-bootstrap-secrets.env for the IdP bootstrap init container. If this file is missing or empty, create it under ServiceConfigs/Deployment/env/services/:

# IdP Bootstrap Secrets
# Used by idp-bootstrap-secrets init container to set admin password
# and service-client secret in the IdP database after migrations.

IDP_ADMIN_PASSWORD=<your-secure-password>
IDP_SERVICE_CLIENT_SECRET=<your-secure-secret>
                        

Generate strong values: openssl rand -base64 24 for password, openssl rand -base64 32 for secret. If both are unset, the init container skips (stack still comes up, but BFF DCR may fail). Do not commit real values to git.

🔧 DCR script and stack restart are automated using your organization’s deployment procedures and approved automation.

🚨

First-Time Setup: BFF Registration Required!

On first startup, the BFF needs to register with IdP. If BFF keeps restarting after docker compose up, run this script from ServiceConfigs\Deployment:

# Run from ServiceConfigs\Deployment directory
.\scripts\create-idp-admin-jwt-secret.ps1

# Then restart services
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build
                        

This generates a JWT admin token for BFF's Dynamic Client Registration (DCR) with IdP. Only needed once!

📋 Common Usage Patterns

Pattern 1: Minimal Development

🔧 Starting the stack is automated using your organization’s deployment procedures and approved automation.

For working on a single service with minimal dependencies. Always run from ServiceConfigs/Deployment (do not run compose from CRUDService). On Linux use exact casing: serviceconfigs/Deployment.

PowerShell

# Start core services (from ServiceConfigs/Deployment)
cd <YourRepoRoot>/ServiceConfigs/Deployment
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build
                

🔧 Service verification is automated using your organization’s deployment procedures and approved automation.

🧪 Try It: Verify Core Services Started

After running docker compose with core profile, verify services are healthy:

Pattern 2: Full Development Stack

For comprehensive development with monitoring and security:

PowerShell

# Full development stack (from ServiceConfigs/Deployment)
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core --profile security --profile monitoring up -d --build
                

Pattern 3: Everything

For integration testing or production-like environment:

PowerShell

# All profiles - requires significant resources! (from ServiceConfigs/Deployment)
docker compose -f docker-compose.yml -f docker-compose-build.yml \
  --profile core \
  --profile security \
  --profile monitoring \
  --profile analytics \
  --profile ldap \
  --profile datacollector \
  up -d --build
                

⚠️

Resource Warning

Running all profiles requires at least 16GB RAM allocated to Docker. Monitor your system resources and adjust profiles based on your hardware capabilities.

💾 Resource Estimation

Understanding resource usage helps you choose appropriate profiles:

Estimated Resource Usage by Profile Combination

~4GB

Core Only

Essential services

~7GB

Core + Monitoring

With observability

~10GB

Full Dev Stack

Core + Security + Monitoring

~14GB

All Profiles

Complete platform

📁 Deployment File Structure

Understanding the file structure helps you customize configurations:

ServiceConfigs/Deployment Directory Structure

📁 Deployment/

📄 docker-compose.yml ← Main compose file (images)

📄 docker-compose-build.yml ← Build from source

📄 .env ← Root environment variables

📁 env/

📁 services/ ← Service-specific env files

📄 idp.env

📄 pdp.env

📄 bff.env

📁 config_secrets/ ← Docker secrets

📁 scripts/ ← Helper scripts

🔨 Building from Source

When you need to test local changes, use the build compose file:

PowerShell

# Build all services from source
docker compose -f docker-compose.yml -f docker-compose-build.yml \
  --profile core up -d --build

# Build specific service only
docker compose -f docker-compose.yml -f docker-compose-build.yml \
  --profile core up -d --build --no-deps idp

# Force rebuild without cache
docker compose -f docker-compose.yml -f docker-compose-build.yml \
  --profile core build --no-cache idp
                

💜

Development Workflow

Use --no-deps when rebuilding a single service to avoid restarting dependent services. Use --build to ensure Docker picks up your latest changes.

🚀 First Run: BFF DCR Setup

The first time you start the platform, you need to configure the BFF's Dynamic Client Registration (DCR):

PowerShell

❯ cd <YourRepoRoot>/ServiceConfigs/Deployment

Replace <YourRepoRoot> with your project root (e.g. EmpowerNow or full path). Windows: use backslashes.

❯ docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build

# BFF will start and crash initially - this is expected!

❯ ./scripts/create-idp-admin-jwt-secret.ps1

✓ Created admin JWT secret for BFF DCR

❯ docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build

# BFF will now start successfully

⚠️

Important: First Run Only

This DCR setup is only needed on the first run. The secret is persisted and reused on subsequent starts. If you reset volumes, you'll need to run this again.

🗄️ Using pgAdmin (Tools Profile)

pgAdmin is a powerful PostgreSQL administration tool included in the tools profile. It provides a web-based GUI for inspecting databases, running queries, and managing schemas.

Starting pgAdmin

PowerShell

# Start core services with tools profile
docker compose --profile core --profile tools up -d

# Or add tools to existing running services
docker compose --profile tools up -d
                

Accessing pgAdmin

🌐

Access URLs

Via Traefik: https://pgadmin.self.empowernow.ai
Direct Port: http://localhost:5050

Default Credentials

pgAdmin Login

Email:    admin@empowernow.ai
Password: EmpowerNow2024!
                

Configuring Database Connection

After logging in, you need to add a server connection to access the PostgreSQL database:

Right-click Servers in the left panel → Register → Server...
On the General tab:
- Name: EmpowerNow DB (any friendly name)
On the Connection tab:
- Host name/address: db (Docker service name)
- Port: 5432
- Maintenance database: postgres
- Username: postgres
- Password: empowernow (from Docker secrets)
Check Save password and click Save

💜

Docker Networking

Use db (the Docker service name) as the hostname, not localhost. pgAdmin runs inside Docker and needs to connect to the database via Docker's internal network.

Key Databases

Once connected, you'll see these main databases:

PostgreSQL Databases

idp_db         → Identity Provider (users, clients, tokens)
pdp_db         → Policy Decision Point (policies, scopes)  
crud_db        → CrudService (workflows, schemas, data)
membership_db  → Membership service (roles, assignments)
                

Useful pgAdmin Tasks

Query IdP Users

SQL

-- List all users
SELECT username, email, is_active, created_at 
FROM idp.users ORDER BY created_at DESC;

-- Find a specific user
SELECT * FROM idp.users WHERE username = 'test';
                

Query OAuth Clients

SQL

-- List all registered OAuth clients
SELECT client_id, client_name, client_type, grant_types, scopes
FROM idp.oauth_clients ORDER BY created_at DESC;

-- Check client configuration details
SELECT c.client_id, c.client_name, e.config
FROM idp.oauth_clients c
LEFT JOIN idp.oauth_client_ext e ON c.client_id = e.client_id;
                

✅

Best Practice: Read-Only Queries

Use pgAdmin primarily for inspection and debugging. Avoid manual INSERT/UPDATE/DELETE on production data—use the service APIs instead for data modifications.

📖 Essential Commands Reference

Starting Services

Commands

# Start with profile(s) - from ServiceConfigs/Deployment
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build

# Start and follow logs
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up

# Start specific service
docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d idp
                

Stopping Services

Commands

# Stop all services (preserves data)
docker compose --profile core down

# Stop and remove volumes (DESTRUCTIVE!)
docker compose --profile core down -v

# Stop specific service
docker compose stop idp
                

Viewing Logs

Commands

# All logs
docker compose logs

# Follow logs in real-time
docker compose logs -f

# Specific service logs
docker compose logs -f idp

# Last 100 lines
docker compose logs --tail 100 idp
                

Service Status

Commands

# List running services
docker compose ps

# Detailed status
docker compose ps -a

# Resource usage
docker stats
                

🛠️ Hands-on Exercises

Exercise 1: Start Core Services

Navigate to ServiceConfigs/Deployment
Start the core profile: docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core up -d --build
Run the DCR script if first time
Verify all services are healthy: docker compose ps

Exercise 2: Add Monitoring

Add the monitoring profile: docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core --profile monitoring up -d --build
Access Grafana at http://grafana.self.empowernow.ai
Check Prometheus targets at http://prometheus.self.empowernow.ai/targets

Exercise 3: Resource Monitoring

Run docker stats in a terminal
Identify which services use the most memory
Note CPU usage patterns

Exercise 4: Explore with pgAdmin

Start the tools profile: docker compose -f docker-compose.yml -f docker-compose-build.yml --profile core --profile tools up -d --build
Access pgAdmin at http://localhost:5050
Login with admin@empowernow.ai / EmpowerNow2024!
Register a new server connection to the db container
Navigate to idp_db → Schemas → idp → Tables
Right-click the users table → View/Edit Data → All Rows
Try running this query in the Query Tool: SELECT * FROM idp.oauth_clients;

📋 Day 3 Summary

✅

Key Achievements

You now understand how to efficiently manage the EmpowerNow Docker environment using profiles, optimize resource usage, and build services from source for development.

Key Takeaways:

Profiles let you run only needed services
Core profile is your minimum viable setup
Tools profile adds pgAdmin for database inspection
docker-compose-build.yml enables building from source
First run requires DCR secret configuration
Monitor resources to avoid system overload
pgAdmin connects via Docker service name (db), not localhost

Tomorrow's Preview:

Day 4 covers Configuration & Secrets Management - understanding how services are configured, environment files, and OpenBao vault integration.

Docker Compose & Profiles

🎯 Learning Objectives

🐳 Understanding Docker Compose Profiles

Why Profiles?

How Profiles Work

📦 Available Profiles

Services Included

Services Included

Services Included

Services Included

Services Included

Services Included

Services Included

Services Included

🔧 Interactive Command Builder

Before running docker compose: Authenticate to Azure Container Registry (when using pre-built images)

Before running docker compose: Grant Docker access to config_secrets

Before running docker compose: Create idp-bootstrap-secrets.env

First-Time Setup: BFF Registration Required!

📋 Common Usage Patterns

Pattern 1: Minimal Development

🧪 Try It: Verify Core Services Started

Pattern 2: Full Development Stack

Pattern 3: Everything

Resource Warning

💾 Resource Estimation

Estimated Resource Usage by Profile Combination

📁 Deployment File Structure

🔨 Building from Source

Development Workflow

🚀 First Run: BFF DCR Setup

Important: First Run Only

🗄️ Using pgAdmin (Tools Profile)

Starting pgAdmin

Accessing pgAdmin

Access URLs

Default Credentials

Configuring Database Connection

Docker Networking

Key Databases

Useful pgAdmin Tasks

Query IdP Users

Query OAuth Clients

Best Practice: Read-Only Queries

📖 Essential Commands Reference

Starting Services

Stopping Services

Viewing Logs

Service Status

🛠️ Hands-on Exercises

Exercise 1: Start Core Services

Exercise 2: Add Monitoring

Exercise 3: Resource Monitoring

Exercise 4: Explore with pgAdmin

📋 Day 3 Summary

Key Achievements

Key Takeaways:

Tomorrow's Preview: