This commit introduces major enhancements to the module installation system, database management, and configuration handling for AzerothCore deployments. ## Module System Improvements ### Module SQL Staging & Installation - Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure - Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*) - Implement conditional module database importing based on enabled modules - Add support for both cpp-modules and lua-scripts module types - Handle rsync exit code 23 (permission warnings) gracefully during deployment ### Module Manifest & Automation - Add automated module manifest generation via GitHub Actions workflow - Implement Python-based module manifest updater with comprehensive validation - Add module dependency tracking and SQL file discovery - Support for blocked modules and module metadata management ## Database Management Enhancements ### Database Import System - Add db-guard container for continuous database health monitoring and verification - Implement conditional database import that skips when databases are current - Add backup restoration and SQL staging coordination - Support for Playerbots database (4th database) in all import operations - Add comprehensive database health checking and status reporting ### Database Configuration - Implement 10 new dbimport.conf settings from environment variables: - Database.Reconnect.Seconds/Attempts for connection reliability - Updates.AllowedModules for module auto-update control - Updates.Redundancy for data integrity checks - Worker/Synch thread settings for all three core databases - Auto-apply dbimport.conf settings via auto-post-install.sh - Add environment variable injection for db-import and db-guard containers ### Backup & Recovery - Fix backup scheduler to prevent immediate execution on container startup - Add backup status monitoring script with detailed reporting - Implement backup import/export utilities - Add database verification scripts for SQL update tracking ## User Import Directory - Add new import/ directory for user-provided database files and configurations - Support for custom SQL files, configuration overrides, and example templates - Automatic import of user-provided databases and configs during initialization - Documentation and examples for custom database imports ## Configuration & Environment - Eliminate CLIENT_DATA_VERSION warning by adding default value syntax - Improve CLIENT_DATA_VERSION documentation in .env.template - Add comprehensive database import settings to .env and .env.template - Update setup.sh to handle new configuration variables with proper defaults ## Monitoring & Debugging - Add status dashboard with Go-based terminal UI (statusdash.go) - Implement JSON status output (statusjson.sh) for programmatic access - Add comprehensive database health check script - Add repair-storage-permissions.sh utility for permission issues ## Testing & Documentation - Add Phase 1 integration test suite for module installation verification - Add comprehensive documentation for: - Database management (DATABASE_MANAGEMENT.md) - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md) - Implementation mapping (IMPLEMENTATION_MAP.md) - SQL staging comparison and path coverage - Module assets and DBC file requirements - Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation - Update references from database-import/ to import/ directory ## Breaking Changes - Renamed database-import/ directory to import/ for clarity - Module SQL files now staged to AzerothCore-compatible paths - db-guard container now required for proper database lifecycle management ## Bug Fixes - Fix module SQL staging directory structure for AzerothCore compatibility - Handle rsync exit code 23 gracefully during deployments - Prevent backup from running immediately on container startup - Correct SQL staging paths for proper module installation
13 KiB
Installing AzerothCore RealmMaster with Docker Compose
This guide mirrors the community “Installing AzerothCore using Docker” workflow so existing AzerothCore operators feel at home, while pointing to RealmMaster-specific tooling documented in our README. Everything below assumes Linux or WSL2; native Windows Docker Desktop is still unsupported.
Pre-requisites
- Docker Engine + Docker Compose v2 in their latest versions. RealmMaster inherits every requirement from the upstream guide; follow the Quick Start section to install dependencies and clone the repo.
- Root (sudo) access during Docker operations. Just like the upstream warning, we recommend standard Docker with
sudorather than the rootless variant, because several services (MySQL tmpfs, bind mounts) need elevated capabilities. - Hardware: minimally 16 GB RAM and 32 GB free disk, as noted in README → Quick Start. First-run provisioning downloads ~15 GB of client data and compiles modules when enabled.
Compose File Layout
RealmMaster keeps the familiar docker-compose.yml at the repo root. Instead of editing the YAML directly, run ./setup.sh (see README → Getting Started) to generate .env; every setting from storage paths and ports to module toggles lives there. This mirrors the upstream “use docker-compose.override.yml” advice while preserving a single declarative stack.
- Security: databases stay on the internal
azerothcorebridge and never publish MySQL ports unless you explicitly setCOMPOSE_OVERRIDE_MYSQL_EXPOSE_ENABLED=1in.env. Binary logging is disabled viaMYSQL_DISABLE_BINLOG=1, matching the upstream recommendation for playerbots. - Storage: bind mounts map to
storage/andlocal-storage/, ensuring data survives container rebuilds just like the original bind-mount instructions.ac-volume-initandac-storage-initbootstrap ownership so you do not need to chown paths manually. - Networks & profiles: all services share the
azerothcorebridge, and Compose profiles (services-standard,services-playerbots,services-modules,tools) let you enable only what you need, similar to copying multiple override files upstream. - Override toggles: drop-in files under
compose-overrides/(likemysql-expose.ymlfor port exposure orworldserver-debug-logging.ymlfor verbose logs) can be activated by settingCOMPOSE_OVERRIDE_<NAME>_ENABLED=1in.env, so you can extend the stack without editing the main compose file. - Module manifest: all module metadata lives in
config/module-manifest.json; presets surfaced insetup.shcome fromconfig/module-profiles/*.json, so you can adapt the same workflow the upstream document used by editing those files.
Override Examples
RealmMaster ships with two opt-in overrides to demonstrate the pattern:
compose-overrides/mysql-expose.yml(COMPOSE_OVERRIDE_MYSQL_EXPOSE_ENABLED=1) publishes MySQL on${MYSQL_EXTERNAL_PORT}for IDEs or external tooling.compose-overrides/worldserver-debug-logging.yml(COMPOSE_OVERRIDE_WORLDSERVER_DEBUG_LOGGING_ENABLED=1) bumpsAC_LOG_LEVELto3across every worldserver profile for troubleshooting.
Add your own overrides by dropping a .yml file into compose-overrides/ with a # override-flag: ... header and toggling the matching env flag. All project scripts automatically include enabled overrides, so the workflow mirrors the upstream “override file” approach without manual compose arguments.
Module Layout
- Manifest:
config/module-manifest.jsontracks every supported module (type, repo, dependencies). Edit this if you need to add or update modules—scripts/python/modules.pyand all container helpers consume it automatically. - Presets:
config/module-profiles/*.jsonreplaces the oldprofiles/*.json. Each preset defines amoduleslist plus optionallabel/description/order, andsetup.shsurfaces them in the module-selection menu or via--module-config <name>.
Because the manifest/preset locations mirror the upstream structure conceptually, experienced users can jump straight into editing those files without re-learning the workflow.
Example excerpt (trimmed for clarity):
services:
ac-mysql:
image: ${MYSQL_IMAGE}
container_name: ac-mysql
volumes:
- mysql-data:/var/lib/mysql-persistent
- ${HOST_ZONEINFO_PATH}:/usr/share/zoneinfo:ro
command:
- mysqld
- --character-set-server=${MYSQL_CHARACTER_SET}
- --collation-server=${MYSQL_COLLATION}
- --innodb-buffer-pool-size=${MYSQL_INNODB_BUFFER_POOL_SIZE}
healthcheck:
test: ["CMD","sh","-c","mysqladmin ping -h localhost -u ${MYSQL_USER} -p${MYSQL_ROOT_PASSWORD} --silent || exit 1"]
networks: [azerothcore]
ac-db-import:
image: ${AC_DB_IMPORT_IMAGE}
depends_on:
ac-mysql:
condition: service_healthy
ac-storage-init:
condition: service_completed_successfully
volumes:
- ${STORAGE_PATH}/config:/azerothcore/env/dist/etc
- ${STORAGE_PATH}/logs:/azerothcore/logs
- mysql-data:/var/lib/mysql-persistent
Tip: Need custom bind mounts for DBC overrides like in the upstream doc? Add them to
${STORAGE_PATH}/client-dataor mount extra read-only paths under theac-worldserver-*service. RealmMaster already downloadsdata.zipviaac-client-data-*containers, so you can drop additional files beside the cached dataset.
Service Roles (parallels to the original guide)
| Upstream Concept | RealmMaster Equivalent | Notes |
|---|---|---|
| MySQL container with bind-mounted storage | ac-mysql + ac-storage-init |
Bind mounts live under storage/ and local-storage/; tmpfs keeps runtime data fast and is checkpointed to disk automatically. |
| Manual DB import container | ac-db-import & ac-db-init |
Automatically imports schemas or restores from backups; disable by skipping the db profile if you truly want manual control. |
| World/Auth servers with optional DBC overrides | ac-authserver-* / ac-worldserver-* |
Profile-based builds cover vanilla, playerbots, and custom module binaries. DBC overrides go into the shared client data mount just like upstream. |
| Client data bind mounts | ac-client-data-standard (or -playerbots) |
Runs scripts/bash/download-client-data.sh, caches releases, and mounts them read-only into the worldserver. |
| Optional helpers (phpMyAdmin, scripts) | ac-phpmyadmin, ac-keira3, scripts/bash/*.sh |
Enable via --profile tools. Credentials still come from MYSQL_ROOT_PASSWORD, identical to upstream instructions. |
For a full architecture diagram, cross-reference README → Architecture Overview.
Storage / Bind Mount Map
| Host Path | Mounted In | Purpose / Notes |
|---|---|---|
${STORAGE_PATH}/config |
ac-authserver-*, ac-worldserver-*, ac-db-import, ac-db-guard, ac-post-install |
Holds authserver.conf, worldserver.conf, dbimport.conf, and module configs. Generated from the .dist templates during setup.sh / auto-post-install.sh. |
${STORAGE_PATH}/logs |
ac-worldserver-*, ac-authserver-*, ac-db-import, ac-db-guard |
Persistent server logs (mirrors upstream logs/ bind mount). |
${STORAGE_PATH}/modules |
ac-worldserver-*, ac-db-import, ac-db-guard, ac-modules |
Cloned module repositories live here. ac-modules / stage-modules.sh sync this tree. |
${STORAGE_PATH}/lua_scripts |
ac-worldserver-* |
Custom Lua scripts (same structure as upstream lua_scripts). |
${STORAGE_PATH}/backups |
ac-db-import, ac-backup, ac-mysql (via mysql-data volume) |
Automatic hourly/daily SQL dumps. ac-db-import restores from here on cold start. |
${STORAGE_PATH}/client-data |
ac-client-data-*, ac-worldserver-*, ac-authserver-* |
Cached Data.zip plus optional DBC/maps/vmaps overrides. Equivalent to mounting data in the original instructions. |
${STORAGE_PATH}/module-sql-updates (host literal path only used when you override the default) |
(legacy, see below) | Prior to this update, this path stayed under storage/. It now defaults to ${STORAGE_PATH_LOCAL}/module-sql-updates so it can sit on a writable share even if storage/ is NFS read-only. |
${STORAGE_PATH_LOCAL}/module-sql-updates |
ac-db-import, ac-db-guard (mounted as /modules-sql) |
New: stage-modules.sh copies every staged MODULE_*.sql into this directory. The guard and importer copy from /modules-sql into /azerothcore/data/sql/updates/* before running dbimport, so historical module SQL is preserved across container rebuilds. |
${STORAGE_PATH_LOCAL}/client-data-cache |
ac-client-data-* |
Download cache for Data.zip. Keeps the upstream client-data instructions intact. |
${STORAGE_PATH_LOCAL}/source/azerothcore-playerbots/data/sql |
ac-db-import, ac-db-guard |
Mounted read-only so dbimport always sees the checked-out SQL (matches the upstream “mount the source tree” advice). |
mysql-data (named volume) |
ac-mysql, ac-db-import, ac-db-init, ac-backup |
Stores the persistent InnoDB files. Runtime tmpfs lives inside the container, just like the original guide’s “tmpfs + bind mount” pattern. |
Hosting storage over NFS/SMB? Point
STORAGE_PATHat your read-only export and keepSTORAGE_PATH_LOCALon a writable tier for caches (client-data-cache,module-sql-updates, etc.).stage-modules.shandrepair-storage-permissions.shrespect those split paths.
Familiar Workflow Using RealmMaster Commands
The upstream document introduced up.sh, down.sh, and boot.sh. RealmMaster provides higher-level wrappers while keeping the same mental model:
- Configure –
./setup.sh(interactive.envgenerator). Mirrors creatingdocker-compose.override.ymlwithout editing YAML. - Build (optional) –
./build.shcompiles images when playerbots or C++ modules are enabled, as described in README → Getting Started → Step 2. Skip if you only need vanilla binaries. - Deploy –
./deploy.shchooses the right profile and runsdocker compose up -d --build, equivalent to the upstreamup.sh. - Stop –
./scripts/bash/stop-containers.shordocker compose down(from the README Management Commands), matching the upstreamdown.sh. - Reboot – run
./scripts/bash/stop-containers.sh && ./scripts/bash/start-containers.sh, similar to theirboot.sh. - Status & Logs –
./status.shsummarizes container health and exposed ports (see README → Management & Operations → Common Workflows).
If you still prefer tiny wrappers, feel free to recreate the original scripts pointing at our compose file:
#!/usr/bin/env bash
set -euo pipefail
PROJECT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)"
if [ -f "${PROJECT_DIR}/.env" ]; then
set -a
source "${PROJECT_DIR}/.env"
set +a
else
echo "Missing ${PROJECT_DIR}/.env" >&2
exit 1
fi
docker compose -f "${PROJECT_DIR}/docker-compose.yml" \
--profile services-standard \
-p "${COMPOSE_PROJECT_NAME}" up -d --build
Optional Make Targets
The original doc used make start/stop/boot. You can mirror that by wiring our scripts:
start:
@./deploy.sh
stop:
@./scripts/bash/stop-containers.sh
boot:
@./scripts/bash/stop-containers.sh && ./deploy.sh
boot.log:
@./deploy.sh && docker logs -f ac-worldserver ||:
Run sudo make start from the repo root, just as the upstream doc suggested running sudo make boot.
Where to Go Next
- Post-Installation: Creating accounts, editing
realmlist.wtf, and enabling SOAP are documented in README → Post-Installation Steps. - Module Catalog: Review every module shipped with RealmMaster under README → Complete Module Catalog before toggling flags in
.env. - Script Reference: For backups, migrations, and module management, see README → Script Reference; it replaces the “useful scripts” appendix from the upstream doc.
- Management & Ops: Automated backups, status checks, and database tooling live in README → Management & Operations, covering everything that used to be manual in the older guide.
By following this document side-by-side with the original AzerothCore instructions, seasoned developers can reuse their muscle memory while benefiting from RealmMaster’s automation and profile-driven compose stack.