Skip to main content

Background Job Parameters Configuration

Overview​

Background jobs can be configured with runtime-adjustable parameters stored in the Setting model under the :jobs_params key. This allows changing job behavior (e.g., batch sizes, limits) without code changes.

Architecture​

Storage: Settings table → Redis cache → Job access via fetch_job_defaults

Pattern:

# In any job inheriting from ApplicationJob
def perform
  params = fetch_job_defaults  # Returns symbolized hash
  batch_size = params[:batch_size]
  # ... use params
end

Setting Structure​

{
  "fetch_companies_data_job" => {
    "batch_size" => 100
  },
  "load_companies_job" => {
    "jobs_count" => 1000
  },
  "update_stats_job" => {
    "batch_size" => 500
  }
}

Key format: Job class name underscored (e.g., FetchCompaniesDataJob → "fetch_companies_data_job")

Usage​

Initialize Settings​

# Create/update job parameters with defaults
rake jobs:initialize_params

# View current parameters
rake jobs:show_params

Update Parameters​

Via rake task:

rake jobs:update_param[fetch_companies_data_job,batch_size,200]

Via Rails console:

params = Setting.get_hash(:jobs_params)
params["fetch_companies_data_job"]["batch_size"] = 200
Setting.set_hash(:jobs_params, params)

Via admin UI (if implemented): Navigate to Settings → Job Parameters → Edit

Access in Jobs​

class FetchCompaniesDataJob < ApplicationJob
  def perform
    params = fetch_job_defaults
    batch_size = params[:batch_size] || 100  # Fallback if not set

    puts "Processing #{batch_size} companies..."
  end
end

Implementation Details​

ApplicationJob#fetch_job_defaults​

def fetch_job_defaults
  setting = Setting.get_hash(:jobs_params)
  job_params = setting[self.class.name.underscore] || {}
  job_params.symbolize_keys
rescue => e
  Sentry.capture_exception(e, extra: {
    setting_key: :jobs_params,
    job_class: self.class.name
  })
  Rails.logger.error "Failed to fetch job defaults for #{self.class.name}: #{e.message}"
  {}  # Return empty hash on error
end

Features:

  • Automatic key normalization (underscored class name)
  • Symbolized keys for consistent access
  • Error handling with Sentry reporting
  • Falls back to empty hash if setting missing

Parameter Types​

Static Parameters (Configure via Settings)​

These rarely change and are good candidates for Settings:

  • batch_size - Number of records to process
  • jobs_count - How many jobs to enqueue
  • timeout - Processing timeout in seconds
  • folder_path - File system paths
  • retry_limit - Maximum retry attempts
  • api_delay - Delay between API calls

Example:

def perform
  params = fetch_job_defaults
  batch_size = params[:batch_size]  # From settings
  timeout = params[:timeout]        # From settings
end

Dynamic Parameters (Pass as Arguments)​

These are unique per job invocation and should be passed as arguments:

  • Record IDs (company_id, user_id)
  • Timestamps (since, until)
  • Request-specific data
  • User-provided filters

Example:

def perform(company_id, options = {})
  params = fetch_job_defaults
  batch_size = params[:batch_size]  # Static from settings
  company = Company.find(company_id)  # Dynamic from args
end

Best Practices​

1. Always Provide Fallbacks​

# ✅ Good - has fallback
batch_size = params[:batch_size] || 100

# ❌ Risky - could be nil
batch_size = params[:batch_size]

2. Validate Parameters​

def perform
  params = fetch_job_defaults
  batch_size = params[:batch_size] || 100

  # Validate
  raise ArgumentError, "batch_size must be positive" if batch_size <= 0
  raise ArgumentError, "batch_size too large" if batch_size > 1000
end

3. Document Expected Parameters​

class FetchCompaniesDataJob < ApplicationJob
  # Expected params:
  # - batch_size: Integer (default: 100) - Number of companies to fetch
  # - api_timeout: Integer (default: 30) - API request timeout in seconds

  def perform
    params = fetch_job_defaults
    # ...
  end
end

4. Log Parameter Changes​

When updating parameters in production, log the changes:

old_params = Setting.get_hash(:jobs_params)["fetch_companies_data_job"]
# ... update ...
new_params = Setting.get_hash(:jobs_params)["fetch_companies_data_job"]

Rails.logger.info "Updated FetchCompaniesDataJob params: #{old_params} → #{new_params}"

Testing​

RSpec Example​

RSpec.describe FetchCompaniesDataJob do
  describe "#perform" do
    context "with custom batch_size" do
      before do
        Setting.set_hash(:jobs_params, {
          "fetch_companies_data_job" => { "batch_size" => 50 }
        })
      end

      it "uses configured batch size" do
        # Test job behavior with batch_size=50
      end
    end

    context "without configured parameters" do
      before do
        Setting.set_hash(:jobs_params, {})
      end

      it "uses fallback values" do
        # Test job uses defaults
      end
    end
  end
end

Monitoring​

Check Current Configuration​

# In Rails console
Setting.get_hash(:jobs_params)
# => {
#   "fetch_companies_data_job" => {"batch_size"=>100},
#   "load_companies_job" => {"jobs_count"=>1000}
# }

Cache Status​

# Check if setting is cached
rails console
> Setting.exists?(:jobs_params)
=> true

Troubleshooting​

Setting Not Found​

Error: ActiveRecord::RecordNotFound: Couldn't find Setting

Solution:

rake jobs:initialize_params

Parameters Not Updating​

Symptom: Job still uses old values after update

Causes:

  1. Cache not invalidated - Setting.set_hash should handle this
  2. Job already enqueued with old params - wait for queue to drain
  3. Setting key typo - verify exact key name

Debug:

# Check cache
Rails.cache.read(Setting.redis_key(:jobs_params))

# Force cache refresh
Setting.set_hash(:jobs_params, Setting.get_hash(:jobs_params))

Type Mismatch​

Symptom: NoMethodError or unexpected type

Cause: Parameters stored as strings instead of integers

Solution:

# When updating, parse types explicitly
params = Setting.get_hash(:jobs_params)
params["my_job"]["batch_size"] = value.to_i  # Not value.to_s
Setting.set_hash(:jobs_params, params)

Migration Guide​

Converting Hardcoded Values​

Before:

class MyJob < ApplicationJob
  BATCH_SIZE = 100  # Hardcoded constant

  def perform
    companies.in_batches(of: BATCH_SIZE) do |batch|
      # ...
    end
  end
end

After:

class MyJob < ApplicationJob
  def perform
    params = fetch_job_defaults
    batch_size = params[:batch_size] || 100  # Fallback to old default

    companies.in_batches(of: batch_size) do |batch|
      # ...
    end
  end
end

# Then run:
# rake jobs:initialize_params
# rake jobs:update_param[my_job,batch_size,100]
  • ApplicationJob: app/jobs/application_job.rb:244-256
  • Setting Model: app/models/setting.rb
  • Rake Tasks: lib/tasks/job_params.rake
  • Jobs Guide: docs/SERVER_JOBS_COMPLETE_GUIDE.md