Files
AzerothCore-RealmMaster/docs/IMPLEMENTATION_MAP.md
uprightbass360 e6231bb4a4 feat: comprehensive module system and database management improvements
This commit introduces major enhancements to the module installation system,
database management, and configuration handling for AzerothCore deployments.

## Module System Improvements

### Module SQL Staging & Installation
- Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure
- Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*)
- Implement conditional module database importing based on enabled modules
- Add support for both cpp-modules and lua-scripts module types
- Handle rsync exit code 23 (permission warnings) gracefully during deployment

### Module Manifest & Automation
- Add automated module manifest generation via GitHub Actions workflow
- Implement Python-based module manifest updater with comprehensive validation
- Add module dependency tracking and SQL file discovery
- Support for blocked modules and module metadata management

## Database Management Enhancements

### Database Import System
- Add db-guard container for continuous database health monitoring and verification
- Implement conditional database import that skips when databases are current
- Add backup restoration and SQL staging coordination
- Support for Playerbots database (4th database) in all import operations
- Add comprehensive database health checking and status reporting

### Database Configuration
- Implement 10 new dbimport.conf settings from environment variables:
  - Database.Reconnect.Seconds/Attempts for connection reliability
  - Updates.AllowedModules for module auto-update control
  - Updates.Redundancy for data integrity checks
  - Worker/Synch thread settings for all three core databases
- Auto-apply dbimport.conf settings via auto-post-install.sh
- Add environment variable injection for db-import and db-guard containers

### Backup & Recovery
- Fix backup scheduler to prevent immediate execution on container startup
- Add backup status monitoring script with detailed reporting
- Implement backup import/export utilities
- Add database verification scripts for SQL update tracking

## User Import Directory

- Add new import/ directory for user-provided database files and configurations
- Support for custom SQL files, configuration overrides, and example templates
- Automatic import of user-provided databases and configs during initialization
- Documentation and examples for custom database imports

## Configuration & Environment

- Eliminate CLIENT_DATA_VERSION warning by adding default value syntax
- Improve CLIENT_DATA_VERSION documentation in .env.template
- Add comprehensive database import settings to .env and .env.template
- Update setup.sh to handle new configuration variables with proper defaults

## Monitoring & Debugging

- Add status dashboard with Go-based terminal UI (statusdash.go)
- Implement JSON status output (statusjson.sh) for programmatic access
- Add comprehensive database health check script
- Add repair-storage-permissions.sh utility for permission issues

## Testing & Documentation

- Add Phase 1 integration test suite for module installation verification
- Add comprehensive documentation for:
  - Database management (DATABASE_MANAGEMENT.md)
  - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md)
  - Implementation mapping (IMPLEMENTATION_MAP.md)
  - SQL staging comparison and path coverage
  - Module assets and DBC file requirements
- Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation
- Update references from database-import/ to import/ directory

## Breaking Changes

- Renamed database-import/ directory to import/ for clarity
- Module SQL files now staged to AzerothCore-compatible paths
- db-guard container now required for proper database lifecycle management

## Bug Fixes

- Fix module SQL staging directory structure for AzerothCore compatibility
- Handle rsync exit code 23 gracefully during deployments
- Prevent backup from running immediately on container startup
- Correct SQL staging paths for proper module installation
2025-11-22 16:56:02 -05:00

27 KiB

Implementation Map: Database & Module Management Improvements

Created: 2025-01-14 Status: Planning Phase Total Improvements: 19 across 6 categories


TOUCHPOINT AUDIT

Core Files by Size and Impact

File Lines Category Impact Level
scripts/bash/backup-merge.sh 1041 Backup Medium
scripts/bash/manage-modules.sh 616 Module Mgmt HIGH
scripts/python/modules.py 546 Module Mgmt HIGH
scripts/bash/rebuild-with-modules.sh 524 Build Low
scripts/bash/backup-import.sh 473 Backup Medium
scripts/bash/migrate-stack.sh 416 Deployment Low
scripts/bash/manage-modules-sql.sh 381 Module SQL CRITICAL
scripts/bash/stage-modules.sh 375 Module Mgmt Medium
scripts/bash/db-import-conditional.sh 340 DB Import CRITICAL
scripts/python/apply-config.py 322 Config Medium
scripts/bash/backup-export.sh 272 Backup Low
scripts/bash/fix-item-import.sh 256 Backup Low
scripts/bash/backup-scheduler.sh 225 Backup Medium
scripts/bash/download-client-data.sh 202 Setup Low
scripts/bash/verify-deployment.sh 196 Deployment Low
scripts/bash/auto-post-install.sh 190 Config HIGH
scripts/bash/configure-server.sh 163 Config Medium
scripts/bash/setup-source.sh 154 Setup Low

CRITICAL FILES (Will be modified in Phase 1):

  1. scripts/bash/manage-modules-sql.sh (381 lines) - Complete refactor
  2. scripts/bash/db-import-conditional.sh (340 lines) - Add verification
  3. scripts/bash/auto-post-install.sh (190 lines) - Playerbots DB integration

HIGH IMPACT FILES (Will be modified in Phase 2-3):

  1. scripts/bash/manage-modules.sh (616 lines) - SQL staging changes
  2. scripts/python/modules.py (546 lines) - Minor updates

DETAILED TOUCHPOINT ANALYSIS

Category A: Module SQL Management

A1: Refactor Module SQL to Use AzerothCore's System

Files to Modify:

  1. scripts/bash/manage-modules-sql.sh (381 lines)

    • Current Function: Manually executes SQL files via mysql_exec
    • Changes Required:
      • Remove run_custom_sql_group() function
      • Remove mysql_exec() wrapper
      • Remove render_sql_file_for_execution() (playerbots template)
      • Remove playerbots_table_exists() check
      • Add SQL staging logic to copy files to AzerothCore structure
      • Add verification via updates table query
    • Lines to Remove: ~250 lines (execution logic)
    • Lines to Add: ~50 lines (staging + verification)
    • Net Change: -200 lines
  2. scripts/bash/manage-modules.sh (616 lines)

    • Current Function: Calls manage-modules-sql.sh for SQL execution
    • Changes Required:
      • Update SQL helper invocation (lines 472-606)
      • Add SQL file staging to proper AzerothCore directory structure
      • Add timestamp-based filename generation
      • Add SQL validation before staging
    • Lines to Change: ~50 lines
    • Lines to Add: ~80 lines (staging logic)
    • Net Change: +30 lines
  3. scripts/python/modules.py (546 lines)

    • Current Function: Module manifest management
    • Changes Required:
      • Add SQL file discovery in module repos
      • Add SQL file metadata to module state
      • Generate SQL staging manifest
    • Lines to Add: ~40 lines
    • Net Change: +40 lines

New Files to Create:

  1. scripts/bash/stage-module-sql.sh (NEW)

    • Purpose: Stage module SQL files to AzerothCore structure
    • Functions:
      • copy_sql_to_acore_structure() - Copy SQL with proper naming
      • validate_sql_file() - Basic SQL syntax check
      • generate_sql_timestamp() - Create YYYYMMDD_HH filename
    • Estimated Lines: ~150 lines
  2. scripts/bash/verify-sql-updates.sh (NEW)

    • Purpose: Verify SQL updates in updates table
    • Functions:
      • check_update_applied() - Query updates table
      • list_module_updates() - Show module SQL status
      • verify_sql_hash() - Check hash matches
    • Estimated Lines: ~100 lines

Docker/Config Files:

  1. docker-compose.yml or relevant compose file
    • Add volume mount for module SQL staging directory
    • Ensure /azerothcore/modules/ is accessible

SQL Directory Structure to Create:

local-storage/source/azerothcore-playerbots/modules/
├── mod-aoe-loot/
│   └── data/
│       └── sql/
│           ├── base/
│           │   └── db_world/
│           └── updates/
│               └── db_world/
│                   └── 20250114_01_aoe_loot_init.sql
├── mod-learn-spells/
│   └── data/
│       └── sql/...
└── [other modules...]

Total Impact:

  • Files Modified: 3
  • Files Created: 2
  • Net Code Change: -130 lines (significant reduction!)
  • Complexity: Medium-High

A2: Add Module SQL Verification

Files to Modify:

  1. scripts/bash/verify-sql-updates.sh (created in A1)

    • Already includes verification logic
  2. scripts/bash/manage-modules.sh

    • Add post-installation verification call
    • Lines to add: ~20 lines

Total Impact:

  • Files Modified: 1
  • Code Change: +20 lines
  • Complexity: Low (builds on A1)

A3: Support Module SQL Rollback

New Files to Create:

  1. scripts/bash/rollback-module-sql.sh (NEW)
    • Purpose: Rollback module SQL changes
    • Functions:
      • create_rollback_sql() - Generate reverse SQL
      • apply_rollback() - Execute rollback
      • track_rollback() - Update rollback state
    • Estimated Lines: ~200 lines

Module Directory Structure:

modules/mod-example/
└── data/
    └── sql/
        ├── updates/
        │   └── db_world/
        │       └── 20250114_01_feature.sql
        └── rollback/
            └── db_world/
                └── 20250114_01_feature_rollback.sql

Total Impact:

  • Files Created: 1
  • Code Change: +200 lines
  • Complexity: Medium

Category B: Database Restoration & Verification

B1: Add Post-Restore Verification

Files to Modify:

  1. scripts/bash/db-import-conditional.sh (340 lines) - CRITICAL
    • Current Function: Restores backups or runs dbimport
    • Changes Required:
      • Add verification step after restore (line ~283-290)
      • Call dbimport with --dry-run to check state
      • Apply missing updates if found
      • Log verification results
    • Location: After restore_backup function
    • Lines to Add: ~60 lines

Code Insertion Point:

# Current code (line ~283):
if restore_backup "$backup_path"; then
    echo "$(date): Backup successfully restored from $backup_path" > "$RESTORE_SUCCESS_MARKER"
    echo "🎉 Backup restoration completed successfully!"
    exit 0
fi

# ADD HERE: Verification step
verify_and_update_databases() {
    # New function to add
}

New Functions to Add:

verify_and_update_databases() {
    echo "🔍 Verifying restored database integrity..."
    cd /azerothcore/env/dist/bin

    # Check what would be applied
    local dry_run_output
    dry_run_output=$(./dbimport --dry-run 2>&1) || true

    # Parse output to see if updates are needed
    if echo "$dry_run_output" | grep -q "would be applied"; then
        warn "Missing updates detected, applying now..."
        ./dbimport || { err "Update verification failed"; return 1; }
    else
        ok "All updates are current"
    fi

    # Verify critical tables exist
    verify_core_tables
}

verify_core_tables() {
    # Check that core tables are present
    local tables=("account" "characters" "creature")
    # ... verification logic
}

Total Impact:

  • Files Modified: 1
  • Code Change: +60 lines
  • Complexity: Medium

B2: Use updates Table for State Tracking

Files to Modify:

  1. scripts/bash/db-import-conditional.sh (340 lines)
    • Changes: Replace marker file checks with SQL queries
    • Lines to Change: ~40 lines
    • Lines to Add: ~30 lines (helper functions)

New Helper Functions:

is_database_initialized() {
    local db_name="$1"
    mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
        "SELECT COUNT(*) FROM ${db_name}.updates WHERE state='RELEASED'" 2>/dev/null || echo 0
}

get_last_update_timestamp() {
    local db_name="$1"
    mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
        "SELECT MAX(timestamp) FROM ${db_name}.updates" 2>/dev/null || echo ""
}

count_module_updates() {
    local db_name="$1"
    mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
        "SELECT COUNT(*) FROM ${db_name}.updates WHERE state='MODULE'" 2>/dev/null || echo 0
}

Replacement Examples:

# OLD:
if [ -f "$RESTORE_SUCCESS_MARKER" ]; then
    echo "✅ Backup restoration completed successfully"
    exit 0
fi

# NEW:
if is_database_initialized "acore_world"; then
    local last_update
    last_update=$(get_last_update_timestamp "acore_world")
    echo "✅ Database initialized (last update: $last_update)"
    exit 0
fi

Total Impact:

  • Files Modified: 1
  • Code Change: +30 lines, -10 lines (marker logic)
  • Complexity: Low-Medium

B3: Add Database Schema Version Checking

New Files to Create:

  1. scripts/bash/check-schema-version.sh (NEW)
    • Purpose: Check and report database schema version
    • Functions:
      • get_schema_version() - Query version from DB
      • compare_versions() - Version comparison logic
      • warn_version_mismatch() - Alert on incompatibility
    • Estimated Lines: ~120 lines

Files to Modify:

  1. scripts/bash/db-import-conditional.sh
    • Add version check before restore
    • Lines to add: ~15 lines

Total Impact:

  • Files Created: 1
  • Files Modified: 1
  • Code Change: +135 lines
  • Complexity: Medium

B4: Implement Database Health Check Script

New Files to Create:

  1. scripts/bash/db-health-check.sh (NEW) - Quick Win!
    • Purpose: Comprehensive database health reporting
    • Functions:
      • check_auth_db() - Auth database status
      • check_world_db() - World database status
      • check_characters_db() - Characters database status
      • check_module_updates() - Module SQL status
      • show_database_sizes() - Storage usage
      • list_pending_updates() - Show pending SQL
      • generate_health_report() - Formatted output
    • Estimated Lines: ~250 lines

Example Output:

🏥 AZEROTHCORE DATABASE HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📊 Database Status
  ✅ Auth DB (acore_auth)
     - Updates: 45 applied
     - Last update: 2025-01-26 14:30:22
     - Size: 12.3 MB

  ✅ World DB (acore_world)
     - Updates: 1,234 applied (15 module)
     - Last update: 2025-01-26 14:32:15
     - Size: 2.1 GB

  ✅ Characters DB (acore_characters)
     - Updates: 89 applied
     - Last update: 2025-01-26 14:31:05
     - Characters: 145 (5 active today)
     - Size: 180.5 MB

📦 Module Updates
  ✅ mod-aoe-loot: 2 updates applied
  ✅ mod-learn-spells: 1 update applied
  ✅ mod-playerbots: 12 updates applied

⚠️  Pending Updates
  - db_world/2025_01_27_00.sql (waiting)
  - db_world/2025_01_27_01.sql (waiting)

💾 Total Storage: 2.29 GB
🔄 Last backup: 2 hours ago

Total Impact:

  • Files Created: 1
  • Code Change: +250 lines
  • Complexity: Low-Medium
  • User Value: HIGH (immediate utility)

Category C: Playerbots Database Integration

C1: Integrate Playerbots into dbimport

Files to Modify:

  1. scripts/bash/db-import-conditional.sh (340 lines)
    • Changes: Update dbimport.conf generation (lines 310-327)
    • Current: Only has Login, World, Character DBs
    • Add: PlayerbotsDatabaseInfo line
    • Update: Updates.EnableDatabases = 15 (was 7)

Code Change:

# OLD (line 310-318):
cat > /azerothcore/env/dist/etc/dbimport.conf <<EOF
LoginDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
Updates.EnableDatabases = 7
Updates.AutoSetup = 1
...
EOF

# NEW:
cat > /azerothcore/env/dist/etc/dbimport.conf <<EOF
LoginDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
PlayerbotsDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_PLAYERBOTS_NAME}"
Updates.EnableDatabases = 15
Updates.AutoSetup = 1
...
EOF
  1. scripts/bash/auto-post-install.sh (190 lines)
    • Changes: Update config file generation
    • Add PlayerbotsDatabaseInfo to worldserver.conf (if not using includes)
    • Lines to change: ~5 lines

Total Impact:

  • Files Modified: 2
  • Code Change: +5 lines
  • Complexity: Low

C2: Remove Custom Playerbots SQL Handling

Files to Modify:

  1. scripts/bash/manage-modules-sql.sh (381 lines)
    • Remove:
      • playerbots_table_exists() function (lines 74-79)
      • render_sql_file_for_execution() playerbots logic (lines 16-46)
      • Playerbots conditional checks in run_custom_sql_group() (lines 93-98)
    • Lines to Remove: ~35 lines

Total Impact:

  • Files Modified: 1
  • Code Change: -35 lines
  • Complexity: Low
  • Depends on: C1 must be completed first

Category D: Configuration Management

D1: Use AzerothCore's Config Include System

Files to Modify:

  1. scripts/bash/auto-post-install.sh (190 lines)
    • Current: Uses sed to modify config files directly
    • Changes:
      • Create conf.d/ directory structure
      • Generate override files instead of modifying base configs
      • Update config references to use includes
    • Lines to Change: ~80 lines (config update section)
    • Lines to Add: ~40 lines (include generation)

New Directory Structure:

storage/config/
├── conf.d/
│   ├── database.conf (generated)
│   ├── environment.conf (generated)
│   └── overrides.conf (user edits)
├── authserver.conf (pristine, includes conf.d/*)
└── worldserver.conf (pristine, includes conf.d/*)

New Functions:

generate_database_config() {
    local conf_dir="/azerothcore/config/conf.d"
    mkdir -p "$conf_dir"

    cat > "$conf_dir/database.conf" <<EOF
# Auto-generated database configuration
# DO NOT EDIT - Generated from environment variables

LoginDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
PlayerbotsDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_PLAYERBOTS_NAME}"
EOF
}

generate_environment_config() {
    # Similar for other environment-specific settings
}

Total Impact:

  • Files Modified: 1
  • Code Change: +40 lines, -20 lines (sed replacements)
  • Complexity: Medium
  • Benefit: Cleaner, more maintainable config management

D2: Environment Variable Based Configuration

New Files to Create:

  1. scripts/bash/generate-config.sh (NEW)
    • Purpose: Generate all config files from environment
    • Functions:
      • template_substitute() - Replace variables in templates
      • validate_config() - Check required values
      • generate_all_configs() - Orchestrate generation
    • Estimated Lines: ~180 lines

Template Files:

config/templates/
├── authserver.conf.template
├── worldserver.conf.template
└── dbimport.conf.template

Total Impact:

  • Files Created: 1 + templates
  • Code Change: +180 lines + templates
  • Complexity: Medium
  • Depends on: D1

Category E: Backup Enhancements

E1: Create Backup Status Dashboard

New Files to Create:

  1. scripts/bash/backup-status.sh (NEW) - Quick Win!
    • Purpose: Display backup system status
    • Functions:
      • show_last_backups() - Recent backup times
      • show_backup_schedule() - Next scheduled backups
      • show_storage_usage() - Backup disk usage
      • show_backup_trends() - Size over time
      • list_available_backups() - All backups with ages
    • Estimated Lines: ~300 lines

Total Impact:

  • Files Created: 1
  • Code Change: +300 lines
  • Complexity: Medium
  • User Value: HIGH

E2: Add Backup Verification Job

Files to Modify:

  1. scripts/bash/backup-scheduler.sh (225 lines)
    • Add verification job after backup creation
    • Lines to add: ~30 lines

New Files:

  1. scripts/bash/verify-backup-integrity.sh (NEW)
    • Test restore to temporary database
    • Verify SQL can be parsed
    • Check for corruption
    • Estimated lines: ~200 lines

Total Impact:

  • Files Created: 1
  • Files Modified: 1
  • Code Change: +230 lines
  • Complexity: Medium-High

E3: Incremental Backup Support

Files to Modify:

  1. scripts/bash/backup-scheduler.sh (225 lines)
    • Add incremental backup mode
    • Binary log management
    • Lines to add: ~150 lines

Total Impact:

  • Files Modified: 1
  • Code Change: +150 lines
  • Complexity: High (requires MySQL binary log setup)

E4: Weekly/Monthly Backup Tiers

Files to Modify:

  1. scripts/bash/backup-scheduler.sh (225 lines)
    • Add weekly/monthly scheduling
    • Extended retention logic
    • Lines to add: ~80 lines

Total Impact:

  • Files Modified: 1
  • Code Change: +80 lines
  • Complexity: Medium

Category F: Documentation & Tooling

F1: Create Database Management Guide

New Files to Create:

  1. docs/DATABASE_MANAGEMENT.md (NEW) - Quick Win!
    • Backup/restore procedures
    • Module SQL installation
    • Troubleshooting guide
    • Migration scenarios
    • Estimated lines: ~500 lines (markdown)

Total Impact:

  • Files Created: 1
  • User Value: HIGH
  • Complexity: Low (documentation)

F2: Add Migration Helper Script

New Files to Create:

  1. scripts/bash/migrate-database.sh (NEW)
    • Schema version upgrades
    • Pre-migration backup
    • Post-migration verification
    • Estimated lines: ~250 lines

Total Impact:

  • Files Created: 1
  • Code Change: +250 lines
  • Complexity: Medium
  • Depends on: B3 (schema version checking)

IMPLEMENTATION PHASES WITH FILE CHANGES

Phase 1: Foundation (Days 1-3)

Goal: Refactor SQL management, add verification, integrate playerbots

Files to Create:

  • scripts/bash/stage-module-sql.sh (150 lines)
  • scripts/bash/verify-sql-updates.sh (100 lines)

Files to Modify:

  • scripts/bash/manage-modules-sql.sh (381 → 181 lines, -200)
  • scripts/bash/manage-modules.sh (616 → 646 lines, +30)
  • scripts/python/modules.py (546 → 586 lines, +40)
  • scripts/bash/db-import-conditional.sh (340 → 405 lines, +65)
  • scripts/bash/auto-post-install.sh (190 → 195 lines, +5)

Total Code Change: +250 new, -200 removed = +50 net Files Created: 2 Files Modified: 5


Phase 2: Verification & Monitoring (Days 4-5)

Goal: Add health checks, state tracking, status dashboard

Files to Create:

  • scripts/bash/db-health-check.sh (250 lines) Quick Win
  • scripts/bash/backup-status.sh (300 lines) Quick Win

Files to Modify:

  • scripts/bash/db-import-conditional.sh (405 → 435 lines, +30)
  • scripts/bash/manage-modules.sh (646 → 666 lines, +20)

Total Code Change: +600 new, +50 modified = +650 net Files Created: 2 Files Modified: 2


Phase 3: Cleanup (Day 6)

Goal: Remove technical debt, simplify config management

Files to Modify:

  • scripts/bash/manage-modules-sql.sh (181 → 146 lines, -35)
  • scripts/bash/auto-post-install.sh (195 → 215 lines, +20)

Total Code Change: -15 net Files Modified: 2


Phase 4: Enhancements (Days 7-9)

Goal: Advanced features, version checking, rollback support

Files to Create:

  • scripts/bash/check-schema-version.sh (120 lines)
  • scripts/bash/rollback-module-sql.sh (200 lines)
  • scripts/bash/verify-backup-integrity.sh (200 lines)
  • docs/DATABASE_MANAGEMENT.md (500 lines markdown) Quick Win

Files to Modify:

  • scripts/bash/db-import-conditional.sh (435 → 450 lines, +15)
  • scripts/bash/backup-scheduler.sh (225 → 255 lines, +30)

Total Code Change: +1065 net Files Created: 4 Files Modified: 2


Phase 5: Advanced (Days 10-12)

Goal: Enterprise features

Files to Create:

  • scripts/bash/migrate-database.sh (250 lines)
  • scripts/bash/generate-config.sh (180 lines)
  • Config templates (3 files, ~200 lines total)

Files to Modify:

  • scripts/bash/backup-scheduler.sh (255 → 485 lines, +230)

Total Code Change: +860 net Files Created: 5 Files Modified: 1


SUMMARY STATISTICS

Code Changes by Phase

Phase New Files Modified Files Lines Added Lines Removed Net Change
1 2 5 250 200 +50
2 2 2 650 0 +650
3 0 2 20 35 -15
4 4 2 1065 0 +1065
5 5 1 860 0 +860
Total 13 12 2845 235 +2610

Impact by File

Most Modified Files:

  1. scripts/bash/db-import-conditional.sh - Modified in 4 phases (+110 lines)
  2. scripts/bash/backup-scheduler.sh - Modified in 3 phases (+260 lines)
  3. scripts/bash/manage-modules-sql.sh - Modified in 2 phases (-235 lines!)
  4. scripts/bash/manage-modules.sh - Modified in 2 phases (+50 lines)
  5. scripts/bash/auto-post-install.sh - Modified in 2 phases (+25 lines)

Largest New Files:

  1. docs/DATABASE_MANAGEMENT.md - 500 lines (documentation)
  2. scripts/bash/backup-status.sh - 300 lines
  3. scripts/bash/db-health-check.sh - 250 lines
  4. scripts/bash/migrate-database.sh - 250 lines
  5. scripts/bash/rollback-module-sql.sh - 200 lines

RISK ASSESSMENT

High Risk Changes

  • manage-modules-sql.sh refactor - Complete rewrite of SQL execution

    • Mitigation: Comprehensive testing, rollback plan
    • Testing: Install 5+ modules, verify all SQL applied
  • dbimport.conf playerbots integration - Could break existing setups

    • Mitigation: Conditional logic, backwards compatibility
    • Testing: Fresh install + migration from existing

Medium Risk Changes

  • Post-restore verification - Could slow down startup

    • Mitigation: Make verification optional via env var
    • Testing: Test with various backup sizes
  • Config include system - Changes config structure

    • Mitigation: Keep old method as fallback
    • Testing: Verify all config values applied correctly

Low Risk Changes

  • Health check script (read-only)
  • Backup status dashboard (read-only)
  • Documentation (no code impact)

TESTING STRATEGY

Phase 1 Testing

  1. Module SQL Refactor:

    • Fresh install with 0 modules
    • Install single module with SQL
    • Install 5+ modules simultaneously
    • Verify SQL in updates table
    • Check for duplicate executions
    • Test module with playerbots SQL
  2. Post-Restore Verification:

    • Restore from fresh backup
    • Restore from 1-week-old backup
    • Restore from 1-month-old backup
    • Test with missing SQL updates
    • Verify auto-update applies correctly
  3. Playerbots Integration:

    • Fresh install with playerbots enabled
    • Migration with existing playerbots DB
    • Verify playerbots updates tracked separately

Phase 2 Testing

  1. Health Check:

    • Run on healthy database
    • Run on database with missing updates
    • Run on database with zero updates
    • Test all output formatting
  2. Backup Status:

    • Check with no backups
    • Check with only hourly backups
    • Check with full backup history
    • Verify size calculations

Integration Testing

  • Complete deployment flow (fresh install)
  • Migration from previous version
  • Module add/remove cycle
  • Backup/restore cycle
  • Performance testing (large databases)

ROLLBACK PROCEDURES

Phase 1 Rollback

If module SQL refactor fails:

  1. Revert manage-modules-sql.sh to original
  2. Revert manage-modules.sh SQL sections
  3. Remove staged SQL files from AzerothCore structure
  4. Restore module SQL to /tmp/scripts/sql/custom/
  5. Re-run module installation

Phase 2 Rollback

If verification causes issues:

  1. Set SKIP_DB_VERIFICATION=1 env var
  2. Revert db-import-conditional.sh changes
  3. Restore original marker file logic

Emergency Rollback (All Phases)

  1. Git revert to tag before changes
  2. Restore database from backup
  3. Re-run deployment without new features
  4. Document failure scenario

SUCCESS CRITERIA

Phase 1 Success

  • All module SQL applied via AzerothCore's updater
  • Zero manual SQL execution in module installation
  • All SQL tracked in updates table with correct hashes
  • Playerbots database in dbimport configuration
  • Post-restore verification catches missing updates
  • No regression in existing functionality
  • Code reduction: -150+ lines

Phase 2 Success

  • Health check script provides accurate status
  • Backup dashboard shows useful information
  • State tracking via database (not files)
  • User value: Quick troubleshooting tools available

Phase 3 Success

  • Playerbots SQL handling simplified
  • Config management cleaner (no sed hacks)
  • Code quality improved
  • Maintenance burden reduced

Overall Success

  • Database management leverages AzerothCore features
  • Less custom code to maintain
  • Better observability and debugging
  • Improved reliability and consistency
  • Clear upgrade path for users
  • Comprehensive documentation

NEXT STEPS

  1. Review this implementation map with stakeholders
  2. Set up test environment for Phase 1
  3. Create feature branch for development
  4. Begin Phase 1 implementation:
    • Start with stage-module-sql.sh (new file, low risk)
    • Then modify manage-modules.sh (add staging calls)
    • Finally refactor manage-modules-sql.sh (high impact)
  5. Test thoroughly before moving to Phase 2
  6. Document changes in CHANGELOG
  7. Create migration guide for existing users

End of Implementation Map