feat: comprehensive module system and database management improvements

This commit introduces major enhancements to the module installation system,
database management, and configuration handling for AzerothCore deployments.

## Module System Improvements

### Module SQL Staging & Installation
- Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure
- Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*)
- Implement conditional module database importing based on enabled modules
- Add support for both cpp-modules and lua-scripts module types
- Handle rsync exit code 23 (permission warnings) gracefully during deployment

### Module Manifest & Automation
- Add automated module manifest generation via GitHub Actions workflow
- Implement Python-based module manifest updater with comprehensive validation
- Add module dependency tracking and SQL file discovery
- Support for blocked modules and module metadata management

## Database Management Enhancements

### Database Import System
- Add db-guard container for continuous database health monitoring and verification
- Implement conditional database import that skips when databases are current
- Add backup restoration and SQL staging coordination
- Support for Playerbots database (4th database) in all import operations
- Add comprehensive database health checking and status reporting

### Database Configuration
- Implement 10 new dbimport.conf settings from environment variables:
  - Database.Reconnect.Seconds/Attempts for connection reliability
  - Updates.AllowedModules for module auto-update control
  - Updates.Redundancy for data integrity checks
  - Worker/Synch thread settings for all three core databases
- Auto-apply dbimport.conf settings via auto-post-install.sh
- Add environment variable injection for db-import and db-guard containers

### Backup & Recovery
- Fix backup scheduler to prevent immediate execution on container startup
- Add backup status monitoring script with detailed reporting
- Implement backup import/export utilities
- Add database verification scripts for SQL update tracking

## User Import Directory

- Add new import/ directory for user-provided database files and configurations
- Support for custom SQL files, configuration overrides, and example templates
- Automatic import of user-provided databases and configs during initialization
- Documentation and examples for custom database imports

## Configuration & Environment

- Eliminate CLIENT_DATA_VERSION warning by adding default value syntax
- Improve CLIENT_DATA_VERSION documentation in .env.template
- Add comprehensive database import settings to .env and .env.template
- Update setup.sh to handle new configuration variables with proper defaults

## Monitoring & Debugging

- Add status dashboard with Go-based terminal UI (statusdash.go)
- Implement JSON status output (statusjson.sh) for programmatic access
- Add comprehensive database health check script
- Add repair-storage-permissions.sh utility for permission issues

## Testing & Documentation

- Add Phase 1 integration test suite for module installation verification
- Add comprehensive documentation for:
  - Database management (DATABASE_MANAGEMENT.md)
  - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md)
  - Implementation mapping (IMPLEMENTATION_MAP.md)
  - SQL staging comparison and path coverage
  - Module assets and DBC file requirements
- Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation
- Update references from database-import/ to import/ directory

## Breaking Changes

- Renamed database-import/ directory to import/ for clarity
- Module SQL files now staged to AzerothCore-compatible paths
- db-guard container now required for proper database lifecycle management

## Bug Fixes

- Fix module SQL staging directory structure for AzerothCore compatibility
- Handle rsync exit code 23 gracefully during deployments
- Prevent backup from running immediately on container startup
- Correct SQL staging paths for proper module installation
This commit is contained in:
uprightbass360
2025-11-20 18:26:00 -05:00
committed by Deckard
parent 0d83f01995
commit e6231bb4a4
56 changed files with 11298 additions and 487 deletions

View File

@@ -100,7 +100,14 @@ else
# Skip core config files (already handled)
case "$filename" in
authserver.conf|worldserver.conf|dbimport.conf)
authserver.conf|worldserver.conf)
continue
;;
dbimport.conf)
if [ ! -f "$conffile" ] || grep -q "Updates.ExceptionShutdownDelay" "$conffile"; then
echo " 📝 Creating/refreshing $filename from $(basename "$file")"
cp "$file" "$conffile"
fi
continue
;;
esac
@@ -140,6 +147,28 @@ else
sed -i "s|^LoginDatabaseInfo *=.*|LoginDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}\"|" /azerothcore/config/worldserver.conf || true
sed -i "s|^WorldDatabaseInfo *=.*|WorldDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}\"|" /azerothcore/config/worldserver.conf || true
sed -i "s|^CharacterDatabaseInfo *=.*|CharacterDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}\"|" /azerothcore/config/worldserver.conf || true
if [ -f "/azerothcore/config/dbimport.conf" ]; then
sed -i "s|^LoginDatabaseInfo *=.*|LoginDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_AUTH_NAME}\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^WorldDatabaseInfo *=.*|WorldDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_WORLD_NAME}\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^CharacterDatabaseInfo *=.*|CharacterDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_CHARACTERS_NAME}\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^PlayerbotsDatabaseInfo *=.*|PlayerbotsDatabaseInfo = \"${MYSQL_HOST};${MYSQL_PORT};${MYSQL_USER};${MYSQL_ROOT_PASSWORD};${DB_PLAYERBOTS_NAME}\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^MySQLExecutable *=.*|MySQLExecutable = \"/usr/bin/mysql\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^TempDir *=.*|TempDir = \"/azerothcore/env/dist/temp\"|" /azerothcore/config/dbimport.conf || true
# Database reconnection settings
sed -i "s|^Database\.Reconnect\.Seconds *=.*|Database.Reconnect.Seconds = ${DB_RECONNECT_SECONDS:-5}|" /azerothcore/config/dbimport.conf || true
sed -i "s|^Database\.Reconnect\.Attempts *=.*|Database.Reconnect.Attempts = ${DB_RECONNECT_ATTEMPTS:-5}|" /azerothcore/config/dbimport.conf || true
# Update settings
sed -i "s|^Updates\.AllowedModules *=.*|Updates.AllowedModules = \"${DB_UPDATES_ALLOWED_MODULES:-all}\"|" /azerothcore/config/dbimport.conf || true
sed -i "s|^Updates\.Redundancy *=.*|Updates.Redundancy = ${DB_UPDATES_REDUNDANCY:-1}|" /azerothcore/config/dbimport.conf || true
# Worker thread settings
sed -i "s|^LoginDatabase\.WorkerThreads *=.*|LoginDatabase.WorkerThreads = ${DB_LOGIN_WORKER_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
sed -i "s|^WorldDatabase\.WorkerThreads *=.*|WorldDatabase.WorkerThreads = ${DB_WORLD_WORKER_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
sed -i "s|^CharacterDatabase\.WorkerThreads *=.*|CharacterDatabase.WorkerThreads = ${DB_CHARACTER_WORKER_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
# Synch thread settings
sed -i "s|^LoginDatabase\.SynchThreads *=.*|LoginDatabase.SynchThreads = ${DB_LOGIN_SYNCH_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
sed -i "s|^WorldDatabase\.SynchThreads *=.*|WorldDatabase.SynchThreads = ${DB_WORLD_SYNCH_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
sed -i "s|^CharacterDatabase\.SynchThreads *=.*|CharacterDatabase.SynchThreads = ${DB_CHARACTER_SYNCH_THREADS:-1}|" /azerothcore/config/dbimport.conf || true
fi
update_playerbots_conf /azerothcore/config/modules/playerbots.conf
update_playerbots_conf /azerothcore/config/modules/playerbots.conf.dist