nchebrov/AzerothCore-RealmMaster

Fork 0

mirror of https://github.com/uprightbass360/AzerothCore-RealmMaster.git synced 2026-02-03 19:03:50 +00:00

Files

uprightbass360 d3484a3aea module setup process

2025-11-17 02:23:53 -05:00

22 KiB

Raw Blame History

AzerothCore Database Management Guide

Version: 1.0 Last Updated: 2025-01-14

This guide covers all aspects of database management in your AzerothCore deployment, including backups, restores, migrations, and troubleshooting.

Overview
Database Structure
Backup System
Restore Procedures
Health Monitoring
Module SQL Management
Migration & Upgrades
Troubleshooting
Best Practices

Overview

Databases in AzerothCore

Your server uses four primary databases:

Database	Purpose	Size (typical)
acore_auth	Account authentication, realm list	Small (< 50MB)
acore_world	Game world data (creatures, quests, items)	Large (1-3GB)
acore_characters	Player character data	Medium (100MB-1GB)
acore_playerbots	Playerbot AI data (if enabled)	Small (< 100MB)

Update System

AzerothCore uses a built-in update system that:

Automatically detects and applies SQL updates on server startup
Tracks applied updates in the updates table (in each database)
Uses SHA1 hashes to prevent duplicate execution
Supports module-specific updates

Database Structure

Core Tables by Database

Auth Database (acore_auth)

account - User accounts
account_access - GM permissions
realmlist - Server realm configuration
updates - Applied SQL updates

World Database (acore_world)

creature - NPC spawns
gameobject - Object spawns
quest_template - Quest definitions
item_template - Item definitions
updates - Applied SQL updates

Characters Database (acore_characters)

characters - Player characters
item_instance - Player items
character_spell - Character spells
character_inventory - Equipped/bagged items
updates - Applied SQL updates

Updates Table Structure

Every database has an updates table:

CREATE TABLE `updates` (
  `name` varchar(200) NOT NULL,           -- Filename (e.g., 2025_01_14_00.sql)
  `hash` char(40) DEFAULT '',             -- SHA1 hash of file
  `state` enum('RELEASED','CUSTOM','MODULE','ARCHIVED','PENDING'),
  `timestamp` timestamp DEFAULT CURRENT_TIMESTAMP,
  `speed` int unsigned DEFAULT '0',       -- Execution time (ms)
  PRIMARY KEY (`name`)
);

Update States:

RELEASED - Official AzerothCore updates
MODULE - Module-specific updates
CUSTOM - Your custom SQL changes
ARCHIVED - Historical updates (consolidated)
PENDING - Queued for application

Backup System

Automated Backups

The system automatically creates backups on two schedules:

Hourly Backups

Frequency: Every N minutes (default: 60)
Retention: Last N hours (default: 6)
Location: storage/backups/hourly/YYYYMMDD_HHMMSS/

Daily Backups

Frequency: Once per day at configured hour (default: 09:00)
Retention: Last N days (default: 3)
Location: storage/backups/daily/YYYYMMDD_HHMMSS/

Configuration

Edit .env to configure backup settings:

# Backup intervals
BACKUP_INTERVAL_MINUTES=60          # Hourly backup frequency
BACKUP_RETENTION_HOURS=6            # How many hourly backups to keep
BACKUP_RETENTION_DAYS=3             # How many daily backups to keep
BACKUP_DAILY_TIME=09                # Daily backup hour (00-23)

# Additional databases
BACKUP_EXTRA_DATABASES=""           # Comma-separated list

Manual Backups

Create an on-demand backup:

./scripts/bash/manual-backup.sh --label my-backup-name

Options:

--label NAME - Custom backup name
--container NAME - Backup container name (default: ac-backup)

Output location: manual-backups/LABEL_YYYYMMDD_HHMMSS/

Export Backups

Create a portable backup for migration:

./scripts/bash/backup-export.sh \
  --password YOUR_MYSQL_PASSWORD \
  --auth-db acore_auth \
  --characters-db acore_characters \
  --world-db acore_world \
  --db auth,characters,world \
  -o ./export-location

This creates: ExportBackup_YYYYMMDD_HHMMSS/ with:

Compressed SQL files (.sql.gz)
manifest.json (metadata)

Restore Procedures

Automatic Restore on Startup

The system automatically detects and restores backups on first startup:

Searches for backups in priority order:
- /backups/daily/ (latest)
- /backups/hourly/ (latest)
- storage/backups/ExportBackup_*/
- manual-backups/
If backup found:
- Restores all databases
- Marks restoration complete
- Skips schema import
If no backup:
- Creates fresh databases
- Runs dbimport to populate schemas
- Applies all pending updates

Restore Safety Checks & Sentinels

Because MySQL stores its hot data in a tmpfs (/var/lib/mysql-runtime) while persisting only backups and status markers under local-storage/mysql-data, it is possible for the runtime data to be wiped (for example, after a host reboot) while the sentinel .restore-completed file still claims the databases are ready. To prevent the worldserver and authserver from entering restart loops, the ac-db-import workflow now performs an explicit sanity check before trusting those markers:

The import script queries MySQL for the combined table count across acore_auth, acore_world, and acore_characters.
If any tables exist, the script logs Backup restoration completed successfully and skips the expensive restore just as before.
If no tables are found or the query fails, the script logs Restoration marker found, but databases are empty - forcing re-import, automatically clears the stale marker, and reruns the backup restore + dbimport pipeline so services always start with real data.

Manual intervention is only required if you intentionally want to force a fresh import despite having data. In that scenario:

Stop the stack: docker compose down
Delete the sentinel: rm -f local-storage/mysql-data/.restore-completed
Run docker compose run --rm ac-db-import

See docs/ADVANCED.md#database-hardening for more background on the tmpfs/persistent split and why the sentinel exists, and review docs/TROUBLESHOOTING.md for quick steps when the automation logs the warning above.

Manual Restore

Restore from backup directory:

./scripts/bash/backup-import.sh \
  --backup-dir ./storage/backups/ExportBackup_20250114_120000 \
  --password YOUR_MYSQL_PASSWORD \
  --auth-db acore_auth \
  --characters-db acore_characters \
  --world-db acore_world \
  --all

Selective restore (only specific databases):

./scripts/bash/backup-import.sh \
  --backup-dir ./path/to/backup \
  --password YOUR_PASSWORD \
  --db characters \
  --characters-db acore_characters

Skip specific databases:

./scripts/bash/backup-import.sh \
  --backup-dir ./path/to/backup \
  --password YOUR_PASSWORD \
  --all \
  --skip world

Merge Backups (Advanced)

Merge accounts/characters from another server:

./scripts/bash/backup-merge.sh \
  --backup-dir ../old-server/backup \
  --password YOUR_PASSWORD \
  --all-accounts \
  --all-characters \
  --exclude-bots

This intelligently:

Remaps GUIDs to avoid conflicts
Preserves existing data
Imports character progression (spells, talents, etc.)
Handles item instances

Options:

--all-accounts - Import all accounts
--all-characters - Import all characters
--exclude-bots - Skip playerbot characters
--account "name1,name2" - Import specific accounts
--dry-run - Show what would be imported

Health Monitoring

Database Health Check

Check overall database health:

./scripts/bash/db-health-check.sh

Output includes:

✅ Database status (exists, responsive)
📊 Update counts (released, module, custom)
🕐 Last update timestamp
💾 Database sizes
📦 Module update summary
👥 Account/character counts

Options:

-v, --verbose - Show detailed information
-p, --pending - Show pending updates
-m, --no-modules - Hide module updates
-c, --container NAME - Specify MySQL container

Example output:

🗄️  AZEROTHCORE DATABASE HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🗄️  Database Status
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✅ Auth DB (acore_auth)
     🔄 Updates: 45 applied
     🕐 Last update: 2025-01-14 14:30:22
     💾 Size: 12.3 MB (23 tables)

  ✅ World DB (acore_world)
     🔄 Updates: 1,234 applied (15 module)
     🕐 Last update: 2025-01-14 14:32:15
     💾 Size: 2.1 GB (345 tables)

  ✅ Characters DB (acore_characters)
     🔄 Updates: 89 applied
     🕐 Last update: 2025-01-14 14:31:05
     💾 Size: 180.5 MB (67 tables)

📊 Server Statistics
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ℹ️  Accounts: 25
  ℹ️  Characters: 145
  ℹ️  Active (24h): 8

💾 Total Database Storage: 2.29 GB

Backup Status

Check backup system status:

./scripts/bash/backup-status.sh

Shows:

Backup tier summary (hourly, daily, manual)
Latest backup timestamps
Storage usage
Next scheduled backups

Options:

-d, --details - Show all available backups
-t, --trends - Show size trends over time

Query Applied Updates

Check which updates have been applied:

-- Show all updates for world database
USE acore_world;
SELECT name, state, timestamp FROM updates ORDER BY timestamp DESC LIMIT 20;

-- Show only module updates
SELECT name, state, timestamp FROM updates WHERE state='MODULE' ORDER BY timestamp DESC;

-- Count updates by state
SELECT state, COUNT(*) as count FROM updates GROUP BY state;

Module SQL Management

How Module SQL Works

When you enable a module that includes SQL changes:

Module Installation: Module is cloned to modules/<module-name>/
SQL Detection: SQL files are found in data/sql/{base,updates,custom}/
SQL Staging: SQL is copied to AzerothCore's update directories
Auto-Application: On next server startup, SQL is auto-applied
Tracking: Updates are tracked in updates table with state='MODULE'

Module SQL Structure

Modules follow this structure:

modules/mod-example/
└── data/
    └── sql/
        ├── base/              # Initial schema (runs once)
        │   ├── db_auth/
        │   ├── db_world/
        │   └── db_characters/
        ├── updates/           # Incremental updates
        │   ├── db_auth/
        │   ├── db_world/
        │   └── db_characters/
        └── custom/            # Optional custom SQL
            └── db_world/

Verifying Module SQL

Check if module SQL was applied:

# Run health check with module details
./scripts/bash/db-health-check.sh --verbose

# Or query directly
mysql -e "SELECT * FROM acore_world.updates WHERE name LIKE '%mod-example%'"

Manual SQL Execution

If you need to run SQL manually:

# Connect to database
docker exec -it ac-mysql mysql -uroot -p

# Select database
USE acore_world;

# Run your SQL
SOURCE /path/to/your/file.sql;

# Or pipe from host
docker exec -i ac-mysql mysql -uroot -pPASSWORD acore_world < yourfile.sql

Module SQL Ledger & Deduplication

./scripts/bash/stage-modules.sh now keeps a lightweight ledger at storage/modules/.modules-meta/module-sql-ledger.txt (also mounted inside containers at /azerothcore/modules/.modules-meta/module-sql-ledger.txt). Each staged SQL file is recorded as:

<database-scope>|<module>|<base_filename>|<hash>

When the script runs again it hashes every module SQL file and skips any entry whose (db, module, filename) already matches with the same hash. This prevents re-copying identical SQL after a backup restore and stops worldserver from reapplying inserts that already exist in the database. If a database restore is detected (local-storage/mysql-data/.restore-completed changed), the ledger is automatically reset so every module SQL file is recopied exactly once. The ledger is automatically updated anytime a file changes so only the modified SQL is restaged.

The stage script also cross-checks MySQL’s updates table before copying files and prunes any staged file whose identifier already exists there. That means even if a file gets stuck in /azerothcore/data/sql/updates/<db> (e.g., after an interrupted run), it is removed before worldserver starts if the database already recorded it.

Restore-Time SQL Reconciliation

During a backup restore the ac-db-import service now runs scripts/bash/restore-and-stage.sh, which consolidates the old restore workflow with module SQL staging. Every backup created by the scheduler now includes a snapshot of the module ledger at module-sql-ledger.txt (for example storage/backups/hourly/20250101_120000/module-sql-ledger.txt). The restore script:

Refreshes storage/modules/.modules-meta/module-sql-ledger.txt using the snapshot bundled with the backup (or rebuilds it from the modules directory if the snapshot is missing).
Writes storage/modules/.modules-meta/.restore-prestaged to signal that the next ./scripts/bash/stage-modules.sh run must repopulate /azerothcore/data/sql/updates/* before worldserver comes online.

The staging script now recopies every module SQL file—regardless of whether it has already been applied—using deterministic names like MODULE_mod-npc-buffer_npc_buffer.sql. AzerothCore’s built-in updater consults the updates tables to decide what should actually run, so already-applied files remain on disk purely to keep history intact and avoid “file missing” warnings. If a legacy backup doesn’t contain the ledger snapshot the helper simply rebuilds it and still sets the flag, so the runtime staging pass behaves the same. Run rm -f storage/modules/.modules-meta/module-sql-ledger.txt and rerun ./scripts/bash/stage-modules.sh --yes if you intentionally need to reseed the ledger from scratch.

This snapshot-driven workflow means restoring a new backup automatically replays any newly added module SQL while avoiding duplicate inserts for modules that were already present. See docs/ADVANCED.md for a deeper look at the marker workflow and container responsibilities.

Forcing a Module SQL Re-stage

If you intentionally need to reapply all module SQL (for example after manually cleaning tables):

Stop services: docker compose down

Remove the SQL ledger so the next run rehashes everything:

rm -f storage/modules/.modules-meta/module-sql-ledger.txt

(Optional) Drop the relevant records from the updates table if you want AzerothCore to rerun them, e.g.:

docker exec -it ac-mysql mysql -uroot -p \
  -e "DELETE FROM acore_characters.updates WHERE name LIKE '%MODULE_mod-ollama-chat%';"

Run ./scripts/bash/stage-modules.sh --yes

Only perform step 3 if you understand the impact—deleting entries causes worldserver to execute those SQL scripts again on next startup.

Migration & Upgrades

Upgrading from Older Backups

When restoring an older backup to a newer AzerothCore version:

Restore the backup as normal
Verification happens automatically - The system runs dbimport after restore
Missing updates are applied - Any new schema changes are detected and applied
Check for errors in worldserver logs

Manual Migration Steps

If automatic migration fails:

# 1. Backup current state
./scripts/bash/manual-backup.sh --label pre-migration

# 2. Run dbimport manually
docker exec -it ac-worldserver /bin/bash
cd /azerothcore/env/dist/bin
./dbimport

# 3. Check for errors
tail -f /azerothcore/env/dist/logs/DBErrors.log

# 4. Verify with health check
./scripts/bash/db-health-check.sh --verbose --pending

Schema Version Checking

Check your database version:

-- World database version
SELECT * FROM acore_world.version;

-- Check latest update
SELECT name, timestamp FROM acore_world.updates ORDER BY timestamp DESC LIMIT 1;

Troubleshooting

Database Won't Start

Symptom: MySQL container keeps restarting

Solutions:

Check logs:

docker logs ac-mysql

Check disk space:

df -h

Reset MySQL data (WARNING: deletes all data):

docker-compose down
rm -rf storage/mysql/*
docker-compose up -d

Updates Not Applying

Symptom: SQL updates in pending_db_* not getting applied

Solutions:

Check Updates.EnableDatabases setting:

grep "Updates.EnableDatabases" storage/config/worldserver.conf
# Should be 7 (auth+char+world) or 15 (all including playerbots)

Check for SQL errors:

docker logs ac-worldserver | grep -i "sql error"

Manually run dbimport:

docker exec -it ac-worldserver /bin/bash
cd /azerothcore/env/dist/bin
./dbimport

Backup Restore Fails

Symptom: Backup import reports errors

Solutions:

Verify backup integrity:

./scripts/bash/verify-backup-complete.sh /path/to/backup

Check SQL file format:

zcat backup.sql.gz | head -20
# Should see SQL statements like CREATE DATABASE, INSERT INTO

Check database names in manifest:

cat backup/manifest.json
# Verify database names match your .env

Try importing individual databases:

# Extract and import manually
zcat backup/acore_world.sql.gz | docker exec -i ac-mysql mysql -uroot -pPASSWORD acore_world

Missing Characters After Restore

Symptom: Characters don't appear in-game

Common Causes:

Wrong database restored - Check you restored characters DB
GUID mismatch - Items reference wrong GUIDs
Incomplete restore - Check for SQL errors during restore

Fix with backup-merge:

# Use merge instead of import to remap GUIDs
./scripts/bash/backup-merge.sh \
  --backup-dir ./path/to/backup \
  --password PASSWORD \
  --all-characters

Duplicate SQL Execution

Symptom: "Duplicate key" errors in logs

Cause: SQL update ran twice

Prevention: The updates table prevents this, but if table is missing:

-- Recreate updates table
CREATE TABLE IF NOT EXISTS `updates` (
  `name` varchar(200) NOT NULL,
  `hash` char(40) DEFAULT '',
  `state` enum('RELEASED','CUSTOM','MODULE','ARCHIVED','PENDING') NOT NULL DEFAULT 'RELEASED',
  `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `speed` int unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Performance Issues

Symptom: Database queries are slow

Solutions:

Check database size:

./scripts/bash/db-health-check.sh

Optimize tables:

USE acore_world;
OPTIMIZE TABLE creature;
OPTIMIZE TABLE gameobject;

USE acore_characters;
OPTIMIZE TABLE characters;
OPTIMIZE TABLE item_instance;

Check MySQL configuration:

docker exec ac-mysql mysql -uroot -pPASSWORD -e "SHOW VARIABLES LIKE 'innodb_buffer_pool_size'"

Increase buffer pool (edit docker-compose.yml):

environment:
  MYSQL_INNODB_BUFFER_POOL_SIZE: 512M  # Increase from 256M

Best Practices

Backup Strategy

✅ DO:

Keep at least 3 days of daily backups
Test restore procedures regularly
Store backups in multiple locations
Monitor backup size trends
Verify backup completion

❌ DON'T:

Rely solely on automated backups
Store backups only on same disk as database
Skip verification of backup integrity
Ignore backup size growth warnings

Update Management

✅ DO:

Let AzerothCore's auto-updater handle SQL
Review DBErrors.log after updates
Keep Updates.EnableDatabases enabled
Test module updates in development first

❌ DON'T:

Manually modify core database tables
Skip module SQL when installing modules
Disable auto-updates in production
Run untested SQL in production

Module Installation

✅ DO:

Enable modules via .env file
Verify module SQL applied via health check
Check module compatibility before enabling
Test modules individually first

❌ DON'T:

Copy SQL files manually
Edit module source SQL
Enable incompatible module combinations
Skip SQL verification after module install

Performance

✅ DO:

Run OPTIMIZE TABLE on large tables monthly
Monitor database size growth
Set appropriate MySQL buffer pool size
Use SSD storage for MySQL data

❌ DON'T:

Store MySQL data on slow HDDs
Run database on same disk as backup
Ignore slow query logs
Leave unused data unarchived

Quick Reference

Essential Commands

# Check database health
./scripts/bash/db-health-check.sh

# Check backup status
./scripts/bash/backup-status.sh

# Create manual backup
./scripts/bash/manual-backup.sh --label my-backup

# Restore from backup
./scripts/bash/backup-import.sh --backup-dir ./path/to/backup --password PASS --all

# Export portable backup
./scripts/bash/backup-export.sh --password PASS --all -o ./export

# Connect to MySQL
docker exec -it ac-mysql mysql -uroot -p

# View worldserver logs
docker logs ac-worldserver -f

# Restart services
docker-compose restart ac-worldserver ac-authserver

Important File Locations

storage/
├── mysql/                    # MySQL data directory
├── backups/
│   ├── hourly/              # Automated hourly backups
│   └── daily/               # Automated daily backups
├── config/                  # Server configuration files
└── logs/                    # Server log files

manual-backups/              # Manual backup storage
local-storage/
└── modules/                 # Installed module files

Support Resources

Health Check: ./scripts/bash/db-health-check.sh --help
Backup Status: ./scripts/bash/backup-status.sh --help
AzerothCore Wiki: https://www.azerothcore.org/wiki
AzerothCore Discord: https://discord.gg/gkt4y2x
Issue Tracker: https://github.com/uprightbass360/AzerothCore-RealmMaster/issues

End of Database Management Guide

22 KiB Raw Blame History Unescape Escape

AzerothCore Database Management Guide

Table of Contents

Overview

Databases in AzerothCore

Update System

Database Structure

Core Tables by Database

Updates Table Structure

Backup System

Automated Backups

Configuration

Manual Backups

Export Backups

Restore Procedures

Automatic Restore on Startup

Restore Safety Checks & Sentinels

Manual Restore

Merge Backups (Advanced)

Health Monitoring

Database Health Check

Backup Status

Query Applied Updates

Module SQL Management

How Module SQL Works

Module SQL Structure

Verifying Module SQL

Manual SQL Execution

Module SQL Ledger & Deduplication

Restore-Time SQL Reconciliation

Forcing a Module SQL Re-stage

Migration & Upgrades

Upgrading from Older Backups

Manual Migration Steps

Schema Version Checking

Troubleshooting

Database Won't Start

Updates Not Applying

Backup Restore Fails

Missing Characters After Restore

Duplicate SQL Execution

Performance Issues

Best Practices

Backup Strategy

Update Management

Module Installation

Performance

Quick Reference

Essential Commands

Important File Locations

Support Resources

22 KiB

Raw Blame History