Skip to main content

Redis Database Allocation

This document describes how Redis databases are allocated across different concerns in the ODAPI application.

Database Allocation Strategy​

DatabasePurposeComponentsEnvironment
0Sidekiq queues and job dataSidekiq workers, job queues, cron jobsAll
1Rails application cacheRails.cache, Setting model cacheAll
2Redis Objects and general dataRedis::Objects, general Redis dataAll
3Test environmentAll Redis data during testingTest only

Configuration Files​

Sidekiq Configuration​

  • File: config/initializers/sidekiq.rb
  • Database: 0
  • Purpose: Job queues, scheduled jobs, job status tracking

Rails Cache Configuration​

  • Files:
    • config/environments/production.rb
    • config/environments/development.rb
  • Database: 1
  • Purpose: Application-level caching, Settings model cache

Redis Objects Configuration​

  • File: config/initializers/redis.rb
  • Database: 2
  • Purpose: Redis::Objects data, general Redis usage

Test Configuration​

  • Database: 3
  • Purpose: Isolated testing environment

Benefits of Database Separation​

1. Clear Separation of Concerns​

  • Sidekiq data is isolated from application cache
  • Cache data doesn't interfere with job processing
  • Easy to monitor and debug specific subsystems

2. Selective Data Management​

  • Clear specific databases without affecting others
  • Different retention policies per database
  • Independent monitoring and alerting

3. Performance Optimization​

  • Different memory allocation strategies per database
  • Optimized eviction policies per use case
  • Reduced key collision and namespace conflicts

4. Operational Benefits​

  • Easier troubleshooting and debugging
  • Selective backup and restore capabilities
  • Clear data ownership and responsibility

Environment Variables​

The configuration automatically uses the appropriate Redis database based on the environment:

Production​

  • Base URL: REDIS_URL environment variable
  • Password: BR_REDIS_PASSWORD environment variable
  • Example: redis://:password@redis:6379/0 → redis://:password@redis:6379/{database}

Development​

  • Base URL: BR_REDIS_HOST environment variable
  • Password: BR_REDIS_PASSWORD environment variable (optional)
  • Example: redis://localhost:6379/0 → redis://localhost:6379/{database}

Test​

  • Fixed URL: redis://localhost:6379/3
  • No password required

Monitoring and Debugging​

Check Database Usage​

# Connect to Redis CLI
redis-cli

# Switch to specific database
SELECT 0  # Sidekiq data
SELECT 1  # Rails cache
SELECT 2  # Redis Objects
SELECT 3  # Test data

# List keys in current database
KEYS *

# Get database info
INFO keyspace

Database-Specific Operations​

# Clear Sidekiq queues only
redis-cli -n 0 FLUSHDB

# Clear Rails cache only
redis-cli -n 1 FLUSHDB

# Clear Redis Objects only
redis-cli -n 2 FLUSHDB

# Clear test data only
redis-cli -n 3 FLUSHDB

Migration Notes​

When deploying this change:

  1. No data loss: Existing data in database 0 will continue to work
  2. Gradual migration: Rails cache will start using database 1, old cache data in database 0 will expire naturally
  3. No downtime required: Changes take effect on application restart
  4. Backwards compatible: If environment variables are not properly configured, defaults will maintain functionality

Troubleshooting​

Issue: Cache not syncing between Rails and Sidekiq​

Check: Ensure both services use the same Redis base URL but different databases (Rails: 1, Sidekiq: 0)

Issue: Jobs not processing​

Check: Verify Sidekiq connects to database 0 correctly

Issue: Redis Objects not working​

Check: Confirm Redis::Objects uses database 2

Check: Tests should use database 3 exclusively