This commit introduces major enhancements to the module installation system, database management, and configuration handling for AzerothCore deployments. ## Module System Improvements ### Module SQL Staging & Installation - Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure - Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*) - Implement conditional module database importing based on enabled modules - Add support for both cpp-modules and lua-scripts module types - Handle rsync exit code 23 (permission warnings) gracefully during deployment ### Module Manifest & Automation - Add automated module manifest generation via GitHub Actions workflow - Implement Python-based module manifest updater with comprehensive validation - Add module dependency tracking and SQL file discovery - Support for blocked modules and module metadata management ## Database Management Enhancements ### Database Import System - Add db-guard container for continuous database health monitoring and verification - Implement conditional database import that skips when databases are current - Add backup restoration and SQL staging coordination - Support for Playerbots database (4th database) in all import operations - Add comprehensive database health checking and status reporting ### Database Configuration - Implement 10 new dbimport.conf settings from environment variables: - Database.Reconnect.Seconds/Attempts for connection reliability - Updates.AllowedModules for module auto-update control - Updates.Redundancy for data integrity checks - Worker/Synch thread settings for all three core databases - Auto-apply dbimport.conf settings via auto-post-install.sh - Add environment variable injection for db-import and db-guard containers ### Backup & Recovery - Fix backup scheduler to prevent immediate execution on container startup - Add backup status monitoring script with detailed reporting - Implement backup import/export utilities - Add database verification scripts for SQL update tracking ## User Import Directory - Add new import/ directory for user-provided database files and configurations - Support for custom SQL files, configuration overrides, and example templates - Automatic import of user-provided databases and configs during initialization - Documentation and examples for custom database imports ## Configuration & Environment - Eliminate CLIENT_DATA_VERSION warning by adding default value syntax - Improve CLIENT_DATA_VERSION documentation in .env.template - Add comprehensive database import settings to .env and .env.template - Update setup.sh to handle new configuration variables with proper defaults ## Monitoring & Debugging - Add status dashboard with Go-based terminal UI (statusdash.go) - Implement JSON status output (statusjson.sh) for programmatic access - Add comprehensive database health check script - Add repair-storage-permissions.sh utility for permission issues ## Testing & Documentation - Add Phase 1 integration test suite for module installation verification - Add comprehensive documentation for: - Database management (DATABASE_MANAGEMENT.md) - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md) - Implementation mapping (IMPLEMENTATION_MAP.md) - SQL staging comparison and path coverage - Module assets and DBC file requirements - Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation - Update references from database-import/ to import/ directory ## Breaking Changes - Renamed database-import/ directory to import/ for clarity - Module SQL files now staged to AzerothCore-compatible paths - db-guard container now required for proper database lifecycle management ## Bug Fixes - Fix module SQL staging directory structure for AzerothCore compatibility - Handle rsync exit code 23 gracefully during deployments - Prevent backup from running immediately on container startup - Correct SQL staging paths for proper module installation
27 KiB
Implementation Map: Database & Module Management Improvements
Created: 2025-01-14 Status: Planning Phase Total Improvements: 19 across 6 categories
TOUCHPOINT AUDIT
Core Files by Size and Impact
| File | Lines | Category | Impact Level |
|---|---|---|---|
scripts/bash/backup-merge.sh |
1041 | Backup | Medium |
scripts/bash/manage-modules.sh |
616 | Module Mgmt | HIGH |
scripts/python/modules.py |
546 | Module Mgmt | HIGH |
scripts/bash/rebuild-with-modules.sh |
524 | Build | Low |
scripts/bash/backup-import.sh |
473 | Backup | Medium |
scripts/bash/migrate-stack.sh |
416 | Deployment | Low |
scripts/bash/manage-modules-sql.sh |
381 | Module SQL | CRITICAL |
scripts/bash/stage-modules.sh |
375 | Module Mgmt | Medium |
scripts/bash/db-import-conditional.sh |
340 | DB Import | CRITICAL |
scripts/python/apply-config.py |
322 | Config | Medium |
scripts/bash/backup-export.sh |
272 | Backup | Low |
scripts/bash/fix-item-import.sh |
256 | Backup | Low |
scripts/bash/backup-scheduler.sh |
225 | Backup | Medium |
scripts/bash/download-client-data.sh |
202 | Setup | Low |
scripts/bash/verify-deployment.sh |
196 | Deployment | Low |
scripts/bash/auto-post-install.sh |
190 | Config | HIGH |
scripts/bash/configure-server.sh |
163 | Config | Medium |
scripts/bash/setup-source.sh |
154 | Setup | Low |
CRITICAL FILES (Will be modified in Phase 1):
scripts/bash/manage-modules-sql.sh(381 lines) - Complete refactorscripts/bash/db-import-conditional.sh(340 lines) - Add verificationscripts/bash/auto-post-install.sh(190 lines) - Playerbots DB integration
HIGH IMPACT FILES (Will be modified in Phase 2-3):
scripts/bash/manage-modules.sh(616 lines) - SQL staging changesscripts/python/modules.py(546 lines) - Minor updates
DETAILED TOUCHPOINT ANALYSIS
Category A: Module SQL Management
A1: Refactor Module SQL to Use AzerothCore's System
Files to Modify:
-
scripts/bash/manage-modules-sql.sh(381 lines)- Current Function: Manually executes SQL files via
mysql_exec - Changes Required:
- Remove
run_custom_sql_group()function - Remove
mysql_exec()wrapper - Remove
render_sql_file_for_execution()(playerbots template) - Remove
playerbots_table_exists()check - Add SQL staging logic to copy files to AzerothCore structure
- Add verification via
updatestable query
- Remove
- Lines to Remove: ~250 lines (execution logic)
- Lines to Add: ~50 lines (staging + verification)
- Net Change: -200 lines
- Current Function: Manually executes SQL files via
-
scripts/bash/manage-modules.sh(616 lines)- Current Function: Calls
manage-modules-sql.shfor SQL execution - Changes Required:
- Update SQL helper invocation (lines 472-606)
- Add SQL file staging to proper AzerothCore directory structure
- Add timestamp-based filename generation
- Add SQL validation before staging
- Lines to Change: ~50 lines
- Lines to Add: ~80 lines (staging logic)
- Net Change: +30 lines
- Current Function: Calls
-
scripts/python/modules.py(546 lines)- Current Function: Module manifest management
- Changes Required:
- Add SQL file discovery in module repos
- Add SQL file metadata to module state
- Generate SQL staging manifest
- Lines to Add: ~40 lines
- Net Change: +40 lines
New Files to Create:
-
scripts/bash/stage-module-sql.sh(NEW)- Purpose: Stage module SQL files to AzerothCore structure
- Functions:
copy_sql_to_acore_structure()- Copy SQL with proper namingvalidate_sql_file()- Basic SQL syntax checkgenerate_sql_timestamp()- Create YYYYMMDD_HH filename
- Estimated Lines: ~150 lines
-
scripts/bash/verify-sql-updates.sh(NEW)- Purpose: Verify SQL updates in
updatestable - Functions:
check_update_applied()- Query updates tablelist_module_updates()- Show module SQL statusverify_sql_hash()- Check hash matches
- Estimated Lines: ~100 lines
- Purpose: Verify SQL updates in
Docker/Config Files:
docker-compose.ymlor relevant compose file- Add volume mount for module SQL staging directory
- Ensure
/azerothcore/modules/is accessible
SQL Directory Structure to Create:
local-storage/source/azerothcore-playerbots/modules/
├── mod-aoe-loot/
│ └── data/
│ └── sql/
│ ├── base/
│ │ └── db_world/
│ └── updates/
│ └── db_world/
│ └── 20250114_01_aoe_loot_init.sql
├── mod-learn-spells/
│ └── data/
│ └── sql/...
└── [other modules...]
Total Impact:
- Files Modified: 3
- Files Created: 2
- Net Code Change: -130 lines (significant reduction!)
- Complexity: Medium-High
A2: Add Module SQL Verification
Files to Modify:
-
scripts/bash/verify-sql-updates.sh(created in A1)- Already includes verification logic
-
scripts/bash/manage-modules.sh- Add post-installation verification call
- Lines to add: ~20 lines
Total Impact:
- Files Modified: 1
- Code Change: +20 lines
- Complexity: Low (builds on A1)
A3: Support Module SQL Rollback
New Files to Create:
scripts/bash/rollback-module-sql.sh(NEW)- Purpose: Rollback module SQL changes
- Functions:
create_rollback_sql()- Generate reverse SQLapply_rollback()- Execute rollbacktrack_rollback()- Update rollback state
- Estimated Lines: ~200 lines
Module Directory Structure:
modules/mod-example/
└── data/
└── sql/
├── updates/
│ └── db_world/
│ └── 20250114_01_feature.sql
└── rollback/
└── db_world/
└── 20250114_01_feature_rollback.sql
Total Impact:
- Files Created: 1
- Code Change: +200 lines
- Complexity: Medium
Category B: Database Restoration & Verification
B1: Add Post-Restore Verification
Files to Modify:
scripts/bash/db-import-conditional.sh(340 lines) - CRITICAL- Current Function: Restores backups or runs dbimport
- Changes Required:
- Add verification step after restore (line ~283-290)
- Call dbimport with --dry-run to check state
- Apply missing updates if found
- Log verification results
- Location: After
restore_backupfunction - Lines to Add: ~60 lines
Code Insertion Point:
# Current code (line ~283):
if restore_backup "$backup_path"; then
echo "$(date): Backup successfully restored from $backup_path" > "$RESTORE_SUCCESS_MARKER"
echo "🎉 Backup restoration completed successfully!"
exit 0
fi
# ADD HERE: Verification step
verify_and_update_databases() {
# New function to add
}
New Functions to Add:
verify_and_update_databases() {
echo "🔍 Verifying restored database integrity..."
cd /azerothcore/env/dist/bin
# Check what would be applied
local dry_run_output
dry_run_output=$(./dbimport --dry-run 2>&1) || true
# Parse output to see if updates are needed
if echo "$dry_run_output" | grep -q "would be applied"; then
warn "Missing updates detected, applying now..."
./dbimport || { err "Update verification failed"; return 1; }
else
ok "All updates are current"
fi
# Verify critical tables exist
verify_core_tables
}
verify_core_tables() {
# Check that core tables are present
local tables=("account" "characters" "creature")
# ... verification logic
}
Total Impact:
- Files Modified: 1
- Code Change: +60 lines
- Complexity: Medium
B2: Use updates Table for State Tracking
Files to Modify:
scripts/bash/db-import-conditional.sh(340 lines)- Changes: Replace marker file checks with SQL queries
- Lines to Change: ~40 lines
- Lines to Add: ~30 lines (helper functions)
New Helper Functions:
is_database_initialized() {
local db_name="$1"
mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
"SELECT COUNT(*) FROM ${db_name}.updates WHERE state='RELEASED'" 2>/dev/null || echo 0
}
get_last_update_timestamp() {
local db_name="$1"
mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
"SELECT MAX(timestamp) FROM ${db_name}.updates" 2>/dev/null || echo ""
}
count_module_updates() {
local db_name="$1"
mysql -h ${CONTAINER_MYSQL} -u${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} -N -e \
"SELECT COUNT(*) FROM ${db_name}.updates WHERE state='MODULE'" 2>/dev/null || echo 0
}
Replacement Examples:
# OLD:
if [ -f "$RESTORE_SUCCESS_MARKER" ]; then
echo "✅ Backup restoration completed successfully"
exit 0
fi
# NEW:
if is_database_initialized "acore_world"; then
local last_update
last_update=$(get_last_update_timestamp "acore_world")
echo "✅ Database initialized (last update: $last_update)"
exit 0
fi
Total Impact:
- Files Modified: 1
- Code Change: +30 lines, -10 lines (marker logic)
- Complexity: Low-Medium
B3: Add Database Schema Version Checking
New Files to Create:
scripts/bash/check-schema-version.sh(NEW)- Purpose: Check and report database schema version
- Functions:
get_schema_version()- Query version from DBcompare_versions()- Version comparison logicwarn_version_mismatch()- Alert on incompatibility
- Estimated Lines: ~120 lines
Files to Modify:
scripts/bash/db-import-conditional.sh- Add version check before restore
- Lines to add: ~15 lines
Total Impact:
- Files Created: 1
- Files Modified: 1
- Code Change: +135 lines
- Complexity: Medium
B4: Implement Database Health Check Script
New Files to Create:
scripts/bash/db-health-check.sh(NEW) - Quick Win!- Purpose: Comprehensive database health reporting
- Functions:
check_auth_db()- Auth database statuscheck_world_db()- World database statuscheck_characters_db()- Characters database statuscheck_module_updates()- Module SQL statusshow_database_sizes()- Storage usagelist_pending_updates()- Show pending SQLgenerate_health_report()- Formatted output
- Estimated Lines: ~250 lines
Example Output:
🏥 AZEROTHCORE DATABASE HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 Database Status
✅ Auth DB (acore_auth)
- Updates: 45 applied
- Last update: 2025-01-26 14:30:22
- Size: 12.3 MB
✅ World DB (acore_world)
- Updates: 1,234 applied (15 module)
- Last update: 2025-01-26 14:32:15
- Size: 2.1 GB
✅ Characters DB (acore_characters)
- Updates: 89 applied
- Last update: 2025-01-26 14:31:05
- Characters: 145 (5 active today)
- Size: 180.5 MB
📦 Module Updates
✅ mod-aoe-loot: 2 updates applied
✅ mod-learn-spells: 1 update applied
✅ mod-playerbots: 12 updates applied
⚠️ Pending Updates
- db_world/2025_01_27_00.sql (waiting)
- db_world/2025_01_27_01.sql (waiting)
💾 Total Storage: 2.29 GB
🔄 Last backup: 2 hours ago
Total Impact:
- Files Created: 1
- Code Change: +250 lines
- Complexity: Low-Medium
- User Value: HIGH (immediate utility)
Category C: Playerbots Database Integration
C1: Integrate Playerbots into dbimport
Files to Modify:
scripts/bash/db-import-conditional.sh(340 lines)- Changes: Update dbimport.conf generation (lines 310-327)
- Current: Only has Login, World, Character DBs
- Add: PlayerbotsDatabaseInfo line
- Update:
Updates.EnableDatabases = 15(was 7)
Code Change:
# OLD (line 310-318):
cat > /azerothcore/env/dist/etc/dbimport.conf <<EOF
LoginDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
Updates.EnableDatabases = 7
Updates.AutoSetup = 1
...
EOF
# NEW:
cat > /azerothcore/env/dist/etc/dbimport.conf <<EOF
LoginDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
PlayerbotsDatabaseInfo = "${CONTAINER_MYSQL};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_PLAYERBOTS_NAME}"
Updates.EnableDatabases = 15
Updates.AutoSetup = 1
...
EOF
scripts/bash/auto-post-install.sh(190 lines)- Changes: Update config file generation
- Add PlayerbotsDatabaseInfo to worldserver.conf (if not using includes)
- Lines to change: ~5 lines
Total Impact:
- Files Modified: 2
- Code Change: +5 lines
- Complexity: Low
C2: Remove Custom Playerbots SQL Handling
Files to Modify:
scripts/bash/manage-modules-sql.sh(381 lines)- Remove:
playerbots_table_exists()function (lines 74-79)render_sql_file_for_execution()playerbots logic (lines 16-46)- Playerbots conditional checks in
run_custom_sql_group()(lines 93-98)
- Lines to Remove: ~35 lines
- Remove:
Total Impact:
- Files Modified: 1
- Code Change: -35 lines
- Complexity: Low
- Depends on: C1 must be completed first
Category D: Configuration Management
D1: Use AzerothCore's Config Include System
Files to Modify:
scripts/bash/auto-post-install.sh(190 lines)- Current: Uses
sedto modify config files directly - Changes:
- Create
conf.d/directory structure - Generate override files instead of modifying base configs
- Update config references to use includes
- Create
- Lines to Change: ~80 lines (config update section)
- Lines to Add: ~40 lines (include generation)
- Current: Uses
New Directory Structure:
storage/config/
├── conf.d/
│ ├── database.conf (generated)
│ ├── environment.conf (generated)
│ └── overrides.conf (user edits)
├── authserver.conf (pristine, includes conf.d/*)
└── worldserver.conf (pristine, includes conf.d/*)
New Functions:
generate_database_config() {
local conf_dir="/azerothcore/config/conf.d"
mkdir -p "$conf_dir"
cat > "$conf_dir/database.conf" <<EOF
# Auto-generated database configuration
# DO NOT EDIT - Generated from environment variables
LoginDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}"
WorldDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}"
CharacterDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}"
PlayerbotsDatabaseInfo = "${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_PLAYERBOTS_NAME}"
EOF
}
generate_environment_config() {
# Similar for other environment-specific settings
}
Total Impact:
- Files Modified: 1
- Code Change: +40 lines, -20 lines (sed replacements)
- Complexity: Medium
- Benefit: Cleaner, more maintainable config management
D2: Environment Variable Based Configuration
New Files to Create:
scripts/bash/generate-config.sh(NEW)- Purpose: Generate all config files from environment
- Functions:
template_substitute()- Replace variables in templatesvalidate_config()- Check required valuesgenerate_all_configs()- Orchestrate generation
- Estimated Lines: ~180 lines
Template Files:
config/templates/
├── authserver.conf.template
├── worldserver.conf.template
└── dbimport.conf.template
Total Impact:
- Files Created: 1 + templates
- Code Change: +180 lines + templates
- Complexity: Medium
- Depends on: D1
Category E: Backup Enhancements
E1: Create Backup Status Dashboard
New Files to Create:
scripts/bash/backup-status.sh(NEW) - Quick Win!- Purpose: Display backup system status
- Functions:
show_last_backups()- Recent backup timesshow_backup_schedule()- Next scheduled backupsshow_storage_usage()- Backup disk usageshow_backup_trends()- Size over timelist_available_backups()- All backups with ages
- Estimated Lines: ~300 lines
Total Impact:
- Files Created: 1
- Code Change: +300 lines
- Complexity: Medium
- User Value: HIGH
E2: Add Backup Verification Job
Files to Modify:
scripts/bash/backup-scheduler.sh(225 lines)- Add verification job after backup creation
- Lines to add: ~30 lines
New Files:
scripts/bash/verify-backup-integrity.sh(NEW)- Test restore to temporary database
- Verify SQL can be parsed
- Check for corruption
- Estimated lines: ~200 lines
Total Impact:
- Files Created: 1
- Files Modified: 1
- Code Change: +230 lines
- Complexity: Medium-High
E3: Incremental Backup Support
Files to Modify:
scripts/bash/backup-scheduler.sh(225 lines)- Add incremental backup mode
- Binary log management
- Lines to add: ~150 lines
Total Impact:
- Files Modified: 1
- Code Change: +150 lines
- Complexity: High (requires MySQL binary log setup)
E4: Weekly/Monthly Backup Tiers
Files to Modify:
scripts/bash/backup-scheduler.sh(225 lines)- Add weekly/monthly scheduling
- Extended retention logic
- Lines to add: ~80 lines
Total Impact:
- Files Modified: 1
- Code Change: +80 lines
- Complexity: Medium
Category F: Documentation & Tooling
F1: Create Database Management Guide
New Files to Create:
docs/DATABASE_MANAGEMENT.md(NEW) - Quick Win!- Backup/restore procedures
- Module SQL installation
- Troubleshooting guide
- Migration scenarios
- Estimated lines: ~500 lines (markdown)
Total Impact:
- Files Created: 1
- User Value: HIGH
- Complexity: Low (documentation)
F2: Add Migration Helper Script
New Files to Create:
scripts/bash/migrate-database.sh(NEW)- Schema version upgrades
- Pre-migration backup
- Post-migration verification
- Estimated lines: ~250 lines
Total Impact:
- Files Created: 1
- Code Change: +250 lines
- Complexity: Medium
- Depends on: B3 (schema version checking)
IMPLEMENTATION PHASES WITH FILE CHANGES
Phase 1: Foundation (Days 1-3)
Goal: Refactor SQL management, add verification, integrate playerbots
Files to Create:
scripts/bash/stage-module-sql.sh(150 lines)scripts/bash/verify-sql-updates.sh(100 lines)
Files to Modify:
scripts/bash/manage-modules-sql.sh(381 → 181 lines, -200)scripts/bash/manage-modules.sh(616 → 646 lines, +30)scripts/python/modules.py(546 → 586 lines, +40)scripts/bash/db-import-conditional.sh(340 → 405 lines, +65)scripts/bash/auto-post-install.sh(190 → 195 lines, +5)
Total Code Change: +250 new, -200 removed = +50 net Files Created: 2 Files Modified: 5
Phase 2: Verification & Monitoring (Days 4-5)
Goal: Add health checks, state tracking, status dashboard
Files to Create:
scripts/bash/db-health-check.sh(250 lines) ✨ Quick Winscripts/bash/backup-status.sh(300 lines) ✨ Quick Win
Files to Modify:
scripts/bash/db-import-conditional.sh(405 → 435 lines, +30)scripts/bash/manage-modules.sh(646 → 666 lines, +20)
Total Code Change: +600 new, +50 modified = +650 net Files Created: 2 Files Modified: 2
Phase 3: Cleanup (Day 6)
Goal: Remove technical debt, simplify config management
Files to Modify:
scripts/bash/manage-modules-sql.sh(181 → 146 lines, -35)scripts/bash/auto-post-install.sh(195 → 215 lines, +20)
Total Code Change: -15 net Files Modified: 2
Phase 4: Enhancements (Days 7-9)
Goal: Advanced features, version checking, rollback support
Files to Create:
scripts/bash/check-schema-version.sh(120 lines)scripts/bash/rollback-module-sql.sh(200 lines)scripts/bash/verify-backup-integrity.sh(200 lines)docs/DATABASE_MANAGEMENT.md(500 lines markdown) ✨ Quick Win
Files to Modify:
scripts/bash/db-import-conditional.sh(435 → 450 lines, +15)scripts/bash/backup-scheduler.sh(225 → 255 lines, +30)
Total Code Change: +1065 net Files Created: 4 Files Modified: 2
Phase 5: Advanced (Days 10-12)
Goal: Enterprise features
Files to Create:
scripts/bash/migrate-database.sh(250 lines)scripts/bash/generate-config.sh(180 lines)- Config templates (3 files, ~200 lines total)
Files to Modify:
scripts/bash/backup-scheduler.sh(255 → 485 lines, +230)
Total Code Change: +860 net Files Created: 5 Files Modified: 1
SUMMARY STATISTICS
Code Changes by Phase
| Phase | New Files | Modified Files | Lines Added | Lines Removed | Net Change |
|---|---|---|---|---|---|
| 1 | 2 | 5 | 250 | 200 | +50 |
| 2 | 2 | 2 | 650 | 0 | +650 |
| 3 | 0 | 2 | 20 | 35 | -15 |
| 4 | 4 | 2 | 1065 | 0 | +1065 |
| 5 | 5 | 1 | 860 | 0 | +860 |
| Total | 13 | 12 | 2845 | 235 | +2610 |
Impact by File
Most Modified Files:
scripts/bash/db-import-conditional.sh- Modified in 4 phases (+110 lines)scripts/bash/backup-scheduler.sh- Modified in 3 phases (+260 lines)scripts/bash/manage-modules-sql.sh- Modified in 2 phases (-235 lines!)scripts/bash/manage-modules.sh- Modified in 2 phases (+50 lines)scripts/bash/auto-post-install.sh- Modified in 2 phases (+25 lines)
Largest New Files:
docs/DATABASE_MANAGEMENT.md- 500 lines (documentation)scripts/bash/backup-status.sh- 300 linesscripts/bash/db-health-check.sh- 250 linesscripts/bash/migrate-database.sh- 250 linesscripts/bash/rollback-module-sql.sh- 200 lines
RISK ASSESSMENT
High Risk Changes
-
manage-modules-sql.shrefactor - Complete rewrite of SQL execution- Mitigation: Comprehensive testing, rollback plan
- Testing: Install 5+ modules, verify all SQL applied
-
dbimport.conf playerbots integration - Could break existing setups
- Mitigation: Conditional logic, backwards compatibility
- Testing: Fresh install + migration from existing
Medium Risk Changes
-
Post-restore verification - Could slow down startup
- Mitigation: Make verification optional via env var
- Testing: Test with various backup sizes
-
Config include system - Changes config structure
- Mitigation: Keep old method as fallback
- Testing: Verify all config values applied correctly
Low Risk Changes
- Health check script (read-only)
- Backup status dashboard (read-only)
- Documentation (no code impact)
TESTING STRATEGY
Phase 1 Testing
-
Module SQL Refactor:
- Fresh install with 0 modules
- Install single module with SQL
- Install 5+ modules simultaneously
- Verify SQL in
updatestable - Check for duplicate executions
- Test module with playerbots SQL
-
Post-Restore Verification:
- Restore from fresh backup
- Restore from 1-week-old backup
- Restore from 1-month-old backup
- Test with missing SQL updates
- Verify auto-update applies correctly
-
Playerbots Integration:
- Fresh install with playerbots enabled
- Migration with existing playerbots DB
- Verify playerbots updates tracked separately
Phase 2 Testing
-
Health Check:
- Run on healthy database
- Run on database with missing updates
- Run on database with zero updates
- Test all output formatting
-
Backup Status:
- Check with no backups
- Check with only hourly backups
- Check with full backup history
- Verify size calculations
Integration Testing
- Complete deployment flow (fresh install)
- Migration from previous version
- Module add/remove cycle
- Backup/restore cycle
- Performance testing (large databases)
ROLLBACK PROCEDURES
Phase 1 Rollback
If module SQL refactor fails:
- Revert
manage-modules-sql.shto original - Revert
manage-modules.shSQL sections - Remove staged SQL files from AzerothCore structure
- Restore module SQL to
/tmp/scripts/sql/custom/ - Re-run module installation
Phase 2 Rollback
If verification causes issues:
- Set
SKIP_DB_VERIFICATION=1env var - Revert db-import-conditional.sh changes
- Restore original marker file logic
Emergency Rollback (All Phases)
- Git revert to tag before changes
- Restore database from backup
- Re-run deployment without new features
- Document failure scenario
SUCCESS CRITERIA
Phase 1 Success
- ✅ All module SQL applied via AzerothCore's updater
- ✅ Zero manual SQL execution in module installation
- ✅ All SQL tracked in
updatestable with correct hashes - ✅ Playerbots database in dbimport configuration
- ✅ Post-restore verification catches missing updates
- ✅ No regression in existing functionality
- ✅ Code reduction: -150+ lines
Phase 2 Success
- ✅ Health check script provides accurate status
- ✅ Backup dashboard shows useful information
- ✅ State tracking via database (not files)
- ✅ User value: Quick troubleshooting tools available
Phase 3 Success
- ✅ Playerbots SQL handling simplified
- ✅ Config management cleaner (no sed hacks)
- ✅ Code quality improved
- ✅ Maintenance burden reduced
Overall Success
- ✅ Database management leverages AzerothCore features
- ✅ Less custom code to maintain
- ✅ Better observability and debugging
- ✅ Improved reliability and consistency
- ✅ Clear upgrade path for users
- ✅ Comprehensive documentation
NEXT STEPS
- Review this implementation map with stakeholders
- Set up test environment for Phase 1
- Create feature branch for development
- Begin Phase 1 implementation:
- Start with
stage-module-sql.sh(new file, low risk) - Then modify
manage-modules.sh(add staging calls) - Finally refactor
manage-modules-sql.sh(high impact)
- Start with
- Test thoroughly before moving to Phase 2
- Document changes in CHANGELOG
- Create migration guide for existing users
End of Implementation Map