Cache Management System

This Rails application uses a Redis-based caching system for reference data models. The system is built around the Cacheable concern which provides standardized caching functionality.

Models Using Cacheable

The following models implement the Cacheable concern:

Location::City - Cities and postal code data
Location::Odonym - Street names and address components
Location::Region - Administrative regions/departments
Reference::Country - Country reference data
Reference::LegalStatus - Legal status types
Reference::Naf - NAF (business activity) codes
Reference::Dictionary - Dictionary entries (uses custom cache logic)

Cache Management

Using Rake Tasks

# Set cache for all models implementing Cacheable
rake cache:set_all

# Clear cache for all models
rake cache:clear_all

# Refresh cache (clear + set)
rake cache:refresh_all

# Show cache status
rake cache:status

# Set cache for specific models
rake cache:set_models[Location::City,Reference::Country]

# List all models implementing Cacheable
rake cache:list_models

Using Ruby Code

# Set cache for all models
CacheManager.set_cache_for_all

# Clear cache for all models
CacheManager.clear_cache_for_all

# Get status of all cacheable models
CacheManager.status

# Set cache for specific models
CacheManager.set_cache_for_models('Location::City', 'Reference::Country')

# Get all models implementing Cacheable
Cacheable.models
CacheManager.cacheable_models

# Work with individual models
Location::City.set_cache
Location::City.clear_cache
Location::City.find_cached(123)

# Work with individual instances
city = Location::City.first
city.cache_attributes  # Get cacheable attributes for this record
city.set_cache         # Set cache for the entire model (same as Location::City.set_cache)

Using the Initializer

The cache system includes an initializer at config/initializers/cache_setup.rb that provides utilities for automatic cache setup during application startup.

# In an initializer or application configuration
Rails.application.config.after_initialize do
  if Rails.env.development?
    CacheSetup.setup_all_caches
  end
end

Cache Structure

Standardized Key Hierarchy

All cache keys follow a consistent hierarchical pattern:

{Model}:{Purpose}:{Identifier}:{SubKey}

Purposes:

records: Standard Cacheable individual/batch records (default)
index: Specialized lookup indices for fast queries
current: Temporal or state-based cache

Individual Record Caching

Each model stores individual records as Redis hashes:

Location:City:records:123 -> { field1: value1, field2: value2 }

Batch Caching

Some models also support batch caching for performance:

Location:City:records:all -> { "1": json_data, "2": json_data }

Specialized Index Caching

Models with specialized lookup requirements use index caches:

Location:City:index:com:all -> { "75001": 123, "75002": 456 }
Location:City:index:ncc:all -> { "PARIS": 123, "LYON": 456 }
ActivityDescription:index:descriptions:all -> { "VENTE": 1, "ACHAT": 2 }

Temporal Caching

Time-based caches use the current purpose:

Stat:current:2025-10-22 -> { companies_count: 1000, ... }

Adding Cacheable to New Models

To add caching to a new model:

Include the concern:

class YourModel < ApplicationRecord
  include Cacheable
end

(Optional) Configure excluded attributes:

class YourModel < ApplicationRecord
  include Cacheable

  # Exclude specific attributes from caching
  self.cache_excluded_attributes = %w[sensitive_field internal_field]
end

(Optional) Define aggregation name for search:

def agg_name
  "#{code} - #{name}"
end

(Optional) Create specialized index caches:

class YourModel < ApplicationRecord
  include Cacheable

  def self.set_specialized_cache
    # Build your specialized index
    lookup_hash = all.map { |record| [record.code, record.id] }.to_h

    # Use the standardized helper to build cache key
    cache_key = specialized_cache_key("index", "code", "all")
    Redis::Objects.redis.mapped_hmset(cache_key, lookup_hash)
  end

  def self.find_by_code_cached(code)
    cache_key = specialized_cache_key("index", "code", "all")
    cached_id = Redis::Objects.redis.hget(cache_key, code)
    cached_id.to_i if cached_id.present?
  end

  # Remember to clear specialized caches
  def self.clear_cache
    super
    Redis::Objects.redis.del(specialized_cache_key("index", "code", "all"))
  end
end

Memory Management

The caching system handles large datasets efficiently:

Uses find_each with batching for memory efficiency
Falls back to manual batching if find_each fails
Provides progress indicators for large operations
Includes error handling and debugging output

Error Handling

The cache system includes comprehensive error handling:

Skips invalid records that can't be cached
Provides detailed error messages
Continues processing when individual records fail
Logs all operations with colored output

Performance Considerations

Large datasets (40k+ records) are processed in batches
Cache operations show progress indicators
Memory usage is optimized through batching
Redis operations are atomic per record

Debugging

Check Cache Status

Use the status command to check cache health:

rake cache:status

This shows:

Record count vs cache count
Cache types breakdown (records, index, current)
Cache coverage percentage
Sample cache keys
Error states

Inspect Cache Keys in Redis

With the standardized key hierarchy, you can easily inspect caches by purpose:

# View all cache keys for a model
redis-cli KEYS "Location:City:*"

# View only record caches
redis-cli KEYS "Location:City:records:*"

# View only index caches
redis-cli KEYS "Location:City:index:*"

# View specific index type
redis-cli KEYS "Location:City:index:com:*"

# Count cache keys by type
redis-cli KEYS "Location:City:records:*" | wc -l
redis-cli KEYS "Location:City:index:*" | wc -l

Cache Key Examples

Standard patterns you'll see:

# Standard record caches (Cacheable concern default)
Location:City:records:123
Location:City:records:all
Reference:Naf:records:456
Reference:Naf:records:all

# Specialized index caches
Location:City:index:com:all
Location:City:index:ncc:all
Location:City:index:comparent:all
Location:City:index:comparent_ncc:all
Location:Odonym:index:values:all
ActivityDescription:index:descriptions:all
Reference:Dictionary:index:TOPIC_ID:key_name

# Temporal caches
Stat:current:2025-10-22

Array and Hash Field Support

The caching system properly handles PostgreSQL array columns and JSON/Hash fields:

Array columns (like abbrs in Location::Odonym) are automatically serialized to JSON for Redis storage
Hash/JSON fields (like store_accessor fields) are also serialized properly
Deserialization happens automatically when retrieving from cache
Type preservation ensures cached data maintains the same Ruby types as the original

# Example with array field
odonym = Location::Odonym.first
puts odonym.abbrs.inspect        # ["ALLE", "ALL", "ALLEE"]

# After caching and retrieval
cached = Location::Odonym.find_cached(odonym.id)
puts cached.abbrs.inspect        # ["ALLE", "ALL", "ALLEE"]
puts cached.abbrs.class          # Array (same as original)

Models Using Cacheable​

Cache Management​

Using Rake Tasks​

Using Ruby Code​

Using the Initializer​

Cache Structure​

Standardized Key Hierarchy​

Individual Record Caching​

Batch Caching​

Specialized Index Caching​

Temporal Caching​

Adding Cacheable to New Models​

Memory Management​

Error Handling​

Performance Considerations​

Debugging​

Check Cache Status​

Inspect Cache Keys in Redis​

Cache Key Examples​

Array and Hash Field Support​