This commit introduces major enhancements to the module installation system, database management, and configuration handling for AzerothCore deployments. ## Module System Improvements ### Module SQL Staging & Installation - Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure - Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*) - Implement conditional module database importing based on enabled modules - Add support for both cpp-modules and lua-scripts module types - Handle rsync exit code 23 (permission warnings) gracefully during deployment ### Module Manifest & Automation - Add automated module manifest generation via GitHub Actions workflow - Implement Python-based module manifest updater with comprehensive validation - Add module dependency tracking and SQL file discovery - Support for blocked modules and module metadata management ## Database Management Enhancements ### Database Import System - Add db-guard container for continuous database health monitoring and verification - Implement conditional database import that skips when databases are current - Add backup restoration and SQL staging coordination - Support for Playerbots database (4th database) in all import operations - Add comprehensive database health checking and status reporting ### Database Configuration - Implement 10 new dbimport.conf settings from environment variables: - Database.Reconnect.Seconds/Attempts for connection reliability - Updates.AllowedModules for module auto-update control - Updates.Redundancy for data integrity checks - Worker/Synch thread settings for all three core databases - Auto-apply dbimport.conf settings via auto-post-install.sh - Add environment variable injection for db-import and db-guard containers ### Backup & Recovery - Fix backup scheduler to prevent immediate execution on container startup - Add backup status monitoring script with detailed reporting - Implement backup import/export utilities - Add database verification scripts for SQL update tracking ## User Import Directory - Add new import/ directory for user-provided database files and configurations - Support for custom SQL files, configuration overrides, and example templates - Automatic import of user-provided databases and configs during initialization - Documentation and examples for custom database imports ## Configuration & Environment - Eliminate CLIENT_DATA_VERSION warning by adding default value syntax - Improve CLIENT_DATA_VERSION documentation in .env.template - Add comprehensive database import settings to .env and .env.template - Update setup.sh to handle new configuration variables with proper defaults ## Monitoring & Debugging - Add status dashboard with Go-based terminal UI (statusdash.go) - Implement JSON status output (statusjson.sh) for programmatic access - Add comprehensive database health check script - Add repair-storage-permissions.sh utility for permission issues ## Testing & Documentation - Add Phase 1 integration test suite for module installation verification - Add comprehensive documentation for: - Database management (DATABASE_MANAGEMENT.md) - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md) - Implementation mapping (IMPLEMENTATION_MAP.md) - SQL staging comparison and path coverage - Module assets and DBC file requirements - Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation - Update references from database-import/ to import/ directory ## Breaking Changes - Renamed database-import/ directory to import/ for clarity - Module SQL files now staged to AzerothCore-compatible paths - db-guard container now required for proper database lifecycle management ## Bug Fixes - Fix module SQL staging directory structure for AzerothCore compatibility - Handle rsync exit code 23 gracefully during deployments - Prevent backup from running immediately on container startup - Correct SQL staging paths for proper module installation
13 KiB
Advanced Configuration
Table of Contents
- Architecture Overview
- Database Hardening
- Compose Overrides
- Storage Structure
- Build System
- Custom Configuration
This document provides technical details for advanced users and system administrators working with AzerothCore RealmMaster. It covers the internal architecture, configuration options, and customization capabilities for specialized deployments and development environments.
Architecture Overview
Container Profiles
flowchart TB
subgraph init["🔧 Initialization"]
direction LR
volinit["ac-volume-init<br/>Docker Volume Permissions"]
storinit["ac-storage-init<br/>Storage Directory Permissions"]
end
subgraph database["💾 Database Layer"]
direction LR
mysql["ac-mysql<br/>MySQL 8.0<br/>:64306"]
dbinit["ac-db-init<br/>Database Creation"]
dbimport["ac-db-import<br/>Schema Import"]
backup["ac-backup<br/>Automated Backups"]
end
subgraph clientdata["📦 Client Data"]
client["ac-client-data<br/>WoW Assets (~15GB)"]
end
subgraph modmgmt["⚙️ Module Management"]
direction LR
modmgr["ac-modules<br/>Downloader & SQL"]
postinst["ac-post-install<br/>Configuration"]
end
subgraph services["🎮 Game Services"]
direction TB
subgraph standard["services-standard"]
direction LR
auth1["ac-authserver<br/>:3784"]
world1["ac-worldserver<br/>:8215, :7778"]
end
subgraph playerbots["services-playerbots"]
direction LR
auth2["ac-authserver-playerbots<br/>:3784"]
world2["ac-worldserver-playerbots<br/>:8215, :7778"]
end
subgraph mods["services-modules"]
direction LR
auth3["ac-authserver-modules<br/>:3784"]
world3["ac-worldserver-modules<br/>:8215, :7778"]
end
end
subgraph tools["🛠️ Management Tools"]
direction LR
pma["ac-phpmyadmin<br/>:8081"]
keira["ac-keira3<br/>:4201"]
end
init -.-> database
database -.-> modmgmt
database -.-> clientdata
modmgmt -.-> services
clientdata -.-> services
database -.-> tools
style init fill:#f5f5f5,stroke:#999,color:#000
style database fill:#e1ffe1,stroke:#4caf50,color:#000
style clientdata fill:#f0e1ff,stroke:#9c27b0,color:#000
style modmgmt fill:#ffe1e1,stroke:#f44336,color:#000
style services fill:#fff4e1,stroke:#ff9800,color:#000
style tools fill:#e1f5ff,stroke:#2196f3,color:#000
style standard fill:#fff9e1,color:#000
style playerbots fill:#fff9e1,color:#000
style mods fill:#fff9e1,color:#000
Profile Notes:
- Initialization: Auto-runs, no profile needed
- Database Layer:
profile: db - Client Data:
profile: client-dataorclient-data-bots - Module Management:
profile: modules - Game Services: ONE of
services-standard,services-playerbots, orservices-modules - Management Tools:
profile: tools
Service Inventory & Ports
| Service / Container | Role | Ports (host → container) | Profile |
|---|---|---|---|
ac-mysql |
MySQL 8.0 database | (optional) 64306 → 3306 (COMPOSE_OVERRIDE_MYSQL_EXPOSE_ENABLED=1) |
db |
ac-db-init |
Database schema initialization | – | db |
ac-db-import |
Database content import | – | db |
ac-backup |
Automated backup system | – | db |
ac-authserver |
Auth server (standard) | 3784 → 3724 |
services-standard |
ac-worldserver |
World server (standard) | 8215 → 8085, 7778 → 7878 |
services-standard |
ac-authserver-playerbots |
Playerbots auth | 3784 → 3724 |
services-playerbots |
ac-worldserver-playerbots |
Playerbots world | 8215 → 8085, 7778 → 7878 |
services-playerbots |
ac-authserver-modules |
Custom build auth | 3784 → 3724 |
services-modules |
ac-worldserver-modules |
Custom build world | 8215 → 8085, 7778 → 7878 |
services-modules |
ac-client-data |
Client data fetcher | – | client-data |
ac-modules |
Module manager | – | modules |
ac-post-install |
Post-installation configuration | – | Auto-start |
ac-phpmyadmin |
Database admin UI | 8081 → 80 |
tools |
ac-keira3 |
Game content editor | 4201 → 8080 |
tools |
Database Hardening
- MySQL port exposure – By default
COMPOSE_OVERRIDE_MYSQL_EXPOSE_ENABLED=0, soac-mysqlis reachable only from the internal Docker network. Set it to1to publish${MYSQL_EXTERNAL_PORT}on the host; RealmMaster scripts automatically includecompose-overrides/mysql-expose.ymlso the override Just Works. If you invoke Compose manually, remember to add-f compose-overrides/mysql-expose.yml. You can follow the sameCOMPOSE_OVERRIDE_<NAME>_ENABLED=1pattern for any custom override files you drop intocompose-overrides/. - Worldserver debug logging – Need extra verbosity temporarily? Flip
COMPOSE_OVERRIDE_WORLDSERVER_DEBUG_LOGGING_ENABLED=1to includecompose-overrides/worldserver-debug-logging.yml, which bumpsAC_LOG_LEVELacross all worldserver profiles. Turn it back off once you're done to avoid noisy logs. - Binary logging toggle –
MYSQL_DISABLE_BINLOG=1appends--skip-log-binvia the MySQL wrapper entrypoint to keep disk churn low (and match Playerbot guidance). Flip the flag to0to re-enable binlogs for debugging or replication. - Drop-in configs – Any
.cnfplaced in${STORAGE_PATH}/config/mysql/conf.d(exposed viaMYSQL_CONFIG_DIR) is mounted into/etc/mysql/conf.d. Use this to add custom tunables or temporarily override the binlog setting without touching the image. - Forcing a fresh database import – MySQL’s persistent files (and the
.restore-*sentinels) now live inside the Docker volumemysql-dataat/var/lib/mysql-persistent. The import workflow still double-checks the live runtime before trusting those markers, loggingRestoration marker found, but databases are empty - forcing re-importif the tmpfs is empty. When you intentionally need to rerun the import, delete the sentinel withdocker run --rm -v mysql-data:/var/lib/mysql-persistent alpine sh -c 'rm -f /var/lib/mysql-persistent/.restore-completed'and then executedocker compose run --rm ac-db-importor./scripts/bash/stage-modules.sh. Leave the sentinel alone during normal operations so the import job doesn’t wipe existing data on every start. - Module-driven SQL migration – Module code is staged through the
ac-modulesservice andscripts/bash/manage-modules.sh, while SQL payloads are copied into the runningac-worldservercontainer byscripts/bash/stage-modules.sh. Every run clears/azerothcore/data/sql/updates/{db_world,db_characters,db_auth}and recopies all enabled module SQL files with deterministic names, letting AzerothCore’s built-in updater decide what to apply. Always trigger module/deploy workflows via these scripts rather than copying repositories manually; this keeps C++ builds, Lua assets, and SQL migrations synchronized with the database state.
Restore-aware module SQL
When a backup successfully restores, the ac-db-import container automatically executes scripts/bash/restore-and-stage.sh, which simply drops storage/modules/.modules-meta/.restore-prestaged. The next ./scripts/bash/stage-modules.sh --yes clears any previously staged files and recopies every enabled module SQL file before the worldserver boots. AzerothCore’s auto-updater then scans /azerothcore/data/sql/updates/*, applies any scripts that aren’t recorded in the updates tables yet, and skips the rest—without ever complaining about missing history files.
Compose Overrides
All helper scripts automatically include any override file found in compose-overrides/ when its matching flag COMPOSE_OVERRIDE_<NAME>_ENABLED is set to 1 in .env. Each override declares its flag at the top with # override-flag: .... This lets you ship opt-in tweaks without editing docker-compose.yml.
Current examples:
compose-overrides/mysql-expose.yml(COMPOSE_OVERRIDE_MYSQL_EXPOSE_ENABLED) – Publishes MySQL to${MYSQL_EXTERNAL_PORT}for external clients.compose-overrides/worldserver-debug-logging.yml(COMPOSE_OVERRIDE_WORLDSERVER_DEBUG_LOGGING_ENABLED) – RaisesAC_LOG_LEVELto3across all worldserver profiles for troubleshooting.
Add your own override by dropping a new .yml file into compose-overrides/, documenting the flag name in a comment, and toggling that flag in .env.
Storage Structure
The project uses a dual-storage approach for optimal performance:
Primary Storage (STORAGE_PATH - default: NFS mount or shared storage)
storage/
├── config/ # Server configuration files (.conf)
│ └── mysql/
│ └── conf.d/ # Drop-in MySQL overrides (mapped to /etc/mysql/conf.d)
├── client-data/ # Unpacked WoW client data & DBC overrides
├── logs/ # Server log files
├── modules/ # Downloaded module source code
├── lua_scripts/ # Eluna Lua scripts (auto-loaded)
├── install-markers/ # Module installation state tracking
└── backups/ # Automated database backups
├── daily/ # Daily backups (retained per BACKUP_RETENTION_DAYS)
└── hourly/ # Hourly backups (retained per BACKUP_RETENTION_HOURS)
storage/client-data is bind-mounted into every world/auth/client-data container. Drop patched dbc, maps, vmaps, or mmaps files directly into that folder (e.g., storage/client-data/dbc/SkillLine.dbc) and the containers will read them immediately—perfect for modules like Individual Progression or mod-worgoblin that need to overwrite Blizzard data.
Need to capture more than the default auth/world/characters schemas? Set BACKUP_EXTRA_DATABASES in .env (comma or space separated) and the ac-backup scheduler will validate each schema before adding it to the hourly/daily dump rotation. The long-standing auto-detection for acore_playerbots still works, and any missing optional schemas are logged and skipped instead of breaking the backup run.
To tweak MySQL settings, place .cnf snippets in storage/config/mysql/conf.d. Files in this directory map straight to /etc/mysql/conf.d inside ac-mysql, so you can re-enable binary logs or tune buffers without rebuilding images.
Local Storage (STORAGE_PATH_LOCAL - default: ./local-storage)
local-storage/
├── client-data-cache/ # Downloaded WoW client data archives
├── source/ # AzerothCore source repository (created during builds)
│ └── azerothcore-playerbots/ # Playerbot fork (when playerbots enabled)
└── images/ # Exported Docker images for remote deployment
Local storage now only hosts build artifacts, cached downloads, and helper images; the database files have moved into a dedicated Docker volume.
Docker Volumes
client-data-cache– Temporary storage for client data downloadsmysql-data– MySQL persistent data +.restore-*sentinels (/var/lib/mysql-persistent)
This separation ensures database and build artifacts stay on fast local storage while configuration, modules, and backups can be shared across hosts via NFS.
Build System
The build system is optimized for development and production deployments with Docker multi-stage builds and caching.
Build Configuration:
- Automated dependency resolution and compilation
- Multi-stage Docker builds for optimal image sizes
- Build artifact caching for faster rebuilds
- Support for custom patches and modifications
Custom Configuration
Advanced customization options for specialized deployments and development environments.
Configuration Options:
- Environment-specific overrides in
docker-compose.override.yml - Custom module loading and configuration
- Database schema modifications and patches
- Network and security configuration
Related Documentation
For additional information, see the following documents:
- Getting Started Guide - Basic setup and deployment
- Module Documentation - Complete module catalog and configurations
- Script Reference - Detailed script documentation and usage
- Main README - Project overview and quick start guide
This documentation is part of the AzerothCore RealmMaster project. For updates and contributions, visit the main project repository.