# AzerothCore Database Management Guide

**Version:** 1.0
**Last Updated:** 2025-01-14

This guide covers all aspects of database management in your AzerothCore deployment, including backups, restores, migrations, and troubleshooting.

---

## Table of Contents

- [Overview](#overview)
- [Database Structure](#database-structure)
- [Backup System](#backup-system)
- [Restore Procedures](#restore-procedures)
- [Health Monitoring](#health-monitoring)
- [Module SQL Management](#module-sql-management)
- [Migration & Upgrades](#migration--upgrades)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)

---

## Overview

### Databases in AzerothCore

Your server uses four primary databases:

| Database | Purpose | Size (typical) |
|----------|---------|----------------|
| **acore_auth** | Account authentication, realm list | Small (< 50MB) |
| **acore_world** | Game world data (creatures, quests, items) | Large (1-3GB) |
| **acore_characters** | Player character data | Medium (100MB-1GB) |
| **acore_playerbots** | Playerbot AI data (if enabled) | Small (< 100MB) |

### Update System

AzerothCore uses a built-in update system that:
- Automatically detects and applies SQL updates on server startup
- Tracks applied updates in the `updates` table (in each database)
- Uses SHA1 hashes to prevent duplicate execution
- Supports module-specific updates

---

## Database Structure

### Core Tables by Database

**Auth Database (acore_auth)**
- `account` - User accounts
- `account_access` - GM permissions
- `realmlist` - Server realm configuration
- `updates` - Applied SQL updates

**World Database (acore_world)**
- `creature` - NPC spawns
- `gameobject` - Object spawns
- `quest_template` - Quest definitions
- `item_template` - Item definitions
- `updates` - Applied SQL updates

**Characters Database (acore_characters)**
- `characters` - Player characters
- `item_instance` - Player items
- `character_spell` - Character spells
- `character_inventory` - Equipped/bagged items
- `updates` - Applied SQL updates

### Updates Table Structure

Every database has an `updates` table:

```sql
CREATE TABLE `updates` (
  `name` varchar(200) NOT NULL,           -- Filename (e.g., 2025_01_14_00.sql)
  `hash` char(40) DEFAULT '',             -- SHA1 hash of file
  `state` enum('RELEASED','CUSTOM','MODULE','ARCHIVED','PENDING'),
  `timestamp` timestamp DEFAULT CURRENT_TIMESTAMP,
  `speed` int unsigned DEFAULT '0',       -- Execution time (ms)
  PRIMARY KEY (`name`)
);
```

**Update States:**
- `RELEASED` - Official AzerothCore updates
- `MODULE` - Module-specific updates
- `CUSTOM` - Your custom SQL changes
- `ARCHIVED` - Historical updates (consolidated)
- `PENDING` - Queued for application

---

## Backup System

### Automated Backups

The system automatically creates backups on two schedules:

**Hourly Backups**
- Frequency: Every N minutes (default: 60)
- Retention: Last N hours (default: 6)
- Location: `storage/backups/hourly/YYYYMMDD_HHMMSS/`

**Daily Backups**
- Frequency: Once per day at configured hour (default: 09:00)
- Retention: Last N days (default: 3)
- Location: `storage/backups/daily/YYYYMMDD_HHMMSS/`

### Configuration

Edit `.env` to configure backup settings:

```bash
# Backup intervals
BACKUP_INTERVAL_MINUTES=60          # Hourly backup frequency
BACKUP_RETENTION_HOURS=6            # How many hourly backups to keep
BACKUP_RETENTION_DAYS=3             # How many daily backups to keep
BACKUP_DAILY_TIME=09                # Daily backup hour (00-23)

# Additional databases
BACKUP_EXTRA_DATABASES=""           # Comma-separated list
```

### Manual Backups

Create an on-demand backup:

```bash
./scripts/bash/manual-backup.sh --label my-backup-name
```

Options:
- `--label NAME` - Custom backup name
- `--container NAME` - Backup container name (default: ac-backup)

Output location: `manual-backups/LABEL_YYYYMMDD_HHMMSS/`

### Export Backups

Create a portable backup for migration:

```bash
./scripts/bash/backup-export.sh \
  --password YOUR_MYSQL_PASSWORD \
  --auth-db acore_auth \
  --characters-db acore_characters \
  --world-db acore_world \
  --db auth,characters,world \
  -o ./export-location
```

This creates: `ExportBackup_YYYYMMDD_HHMMSS/` with:
- Compressed SQL files (.sql.gz)
- manifest.json (metadata)

---

## Restore Procedures

### Automatic Restore on Startup

The system automatically detects and restores backups on first startup:

1. Searches for backups in priority order:
   - `/backups/daily/` (latest)
   - `/backups/hourly/` (latest)
   - `storage/backups/ExportBackup_*/`
   - `manual-backups/`

2. If backup found:
   - Restores all databases
   - Marks restoration complete
   - Skips schema import

3. If no backup:
   - Creates fresh databases
   - Runs `dbimport` to populate schemas
   - Applies all pending updates

### Restore Safety Checks & Sentinels

Because MySQL stores its hot data in a tmpfs (`/var/lib/mysql-runtime`) while persisting only backups and status markers under `local-storage/mysql-data`, it is possible for the runtime data to be wiped (for example, after a host reboot) while the sentinel `.restore-completed` file still claims the databases are ready. To prevent the worldserver and authserver from entering restart loops, the `ac-db-import` workflow now performs an explicit sanity check before trusting those markers:

- The import script queries MySQL for the combined table count across `acore_auth`, `acore_world`, and `acore_characters`.
- If **any tables exist**, the script logs `Backup restoration completed successfully` and skips the expensive restore just as before.
- If **no tables are found or the query fails**, the script logs `Restoration marker found, but databases are empty - forcing re-import`, automatically clears the stale marker, and reruns the backup restore + `dbimport` pipeline so services always start with real data.

Manual intervention is only required if you intentionally want to force a fresh import despite having data. In that scenario:

1. Stop the stack: `docker compose down`
2. Delete the sentinel: `rm -f local-storage/mysql-data/.restore-completed`
3. Run `docker compose run --rm ac-db-import`

See [docs/ADVANCED.md#database-hardening](ADVANCED.md#database-hardening) for more background on the tmpfs/persistent split and why the sentinel exists, and review [docs/TROUBLESHOOTING.md](TROUBLESHOOTING.md#database-connection-issues) for quick steps when the automation logs the warning above.

### Manual Restore

**Restore from backup directory:**

```bash
./scripts/bash/backup-import.sh \
  --backup-dir ./storage/backups/ExportBackup_20250114_120000 \
  --password YOUR_MYSQL_PASSWORD \
  --auth-db acore_auth \
  --characters-db acore_characters \
  --world-db acore_world \
  --all
```

**Selective restore (only specific databases):**

```bash
./scripts/bash/backup-import.sh \
  --backup-dir ./path/to/backup \
  --password YOUR_PASSWORD \
  --db characters \
  --characters-db acore_characters
```

**Skip specific databases:**

```bash
./scripts/bash/backup-import.sh \
  --backup-dir ./path/to/backup \
  --password YOUR_PASSWORD \
  --all \
  --skip world
```

### Merge Backups (Advanced)

Merge accounts/characters from another server:

```bash
./scripts/bash/backup-merge.sh \
  --backup-dir ../old-server/backup \
  --password YOUR_PASSWORD \
  --all-accounts \
  --all-characters \
  --exclude-bots
```

This intelligently:
- Remaps GUIDs to avoid conflicts
- Preserves existing data
- Imports character progression (spells, talents, etc.)
- Handles item instances

Options:
- `--all-accounts` - Import all accounts
- `--all-characters` - Import all characters
- `--exclude-bots` - Skip playerbot characters
- `--account "name1,name2"` - Import specific accounts
- `--dry-run` - Show what would be imported

---

## Health Monitoring

### Database Health Check

Check overall database health:

```bash
./scripts/bash/db-health-check.sh
```

Output includes:
- ✅ Database status (exists, responsive)
- 📊 Update counts (released, module, custom)
- 🕐 Last update timestamp
- 💾 Database sizes
- 📦 Module update summary
- 👥 Account/character counts

**Options:**
- `-v, --verbose` - Show detailed information
- `-p, --pending` - Show pending updates
- `-m, --no-modules` - Hide module updates
- `-c, --container NAME` - Specify MySQL container

**Example output:**

```
🗄️  AZEROTHCORE DATABASE HEALTH CHECK
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🗄️  Database Status
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✅ Auth DB (acore_auth)
     🔄 Updates: 45 applied
     🕐 Last update: 2025-01-14 14:30:22
     💾 Size: 12.3 MB (23 tables)

  ✅ World DB (acore_world)
     🔄 Updates: 1,234 applied (15 module)
     🕐 Last update: 2025-01-14 14:32:15
     💾 Size: 2.1 GB (345 tables)

  ✅ Characters DB (acore_characters)
     🔄 Updates: 89 applied
     🕐 Last update: 2025-01-14 14:31:05
     💾 Size: 180.5 MB (67 tables)

📊 Server Statistics
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ℹ️  Accounts: 25
  ℹ️  Characters: 145
  ℹ️  Active (24h): 8

💾 Total Database Storage: 2.29 GB
```

### Backup Status

Check backup system status:

```bash
./scripts/bash/backup-status.sh
```

Shows:
- Backup tier summary (hourly, daily, manual)
- Latest backup timestamps
- Storage usage
- Next scheduled backups

**Options:**
- `-d, --details` - Show all available backups
- `-t, --trends` - Show size trends over time

### Query Applied Updates

Check which updates have been applied:

```sql
-- Show all updates for world database
USE acore_world;
SELECT name, state, timestamp FROM updates ORDER BY timestamp DESC LIMIT 20;

-- Show only module updates
SELECT name, state, timestamp FROM updates WHERE state='MODULE' ORDER BY timestamp DESC;

-- Count updates by state
SELECT state, COUNT(*) as count FROM updates GROUP BY state;
```

---

## Module SQL Management

### How Module SQL Works

When you enable a module that includes SQL changes:

1. **Module Installation:** Module is cloned to `modules/<module-name>/`
2. **SQL Detection:** SQL files are found in `data/sql/{base,updates,custom}/`
3. **SQL Staging:** SQL is copied to AzerothCore's update directories
4. **Auto-Application:** On next server startup, SQL is auto-applied
5. **Tracking:** Updates are tracked in `updates` table with `state='MODULE'`

### Module SQL Structure

Modules follow this structure:

```
modules/mod-example/
└── data/
    └── sql/
        ├── base/              # Initial schema (runs once)
        │   ├── db_auth/
        │   ├── db_world/
        │   └── db_characters/
        ├── updates/           # Incremental updates
        │   ├── db_auth/
        │   ├── db_world/
        │   └── db_characters/
        └── custom/            # Optional custom SQL
            └── db_world/
```

### Verifying Module SQL

Check if module SQL was applied:

```bash
# Run health check with module details
./scripts/bash/db-health-check.sh --verbose

# Or query directly
mysql -e "SELECT * FROM acore_world.updates WHERE name LIKE '%mod-example%'"
```

### Manual SQL Execution

If you need to run SQL manually:

```bash
# Connect to database
docker exec -it ac-mysql mysql -uroot -p

# Select database
USE acore_world;

# Run your SQL
SOURCE /path/to/your/file.sql;

# Or pipe from host
docker exec -i ac-mysql mysql -uroot -pPASSWORD acore_world < yourfile.sql
```

### Module SQL Ledger & Deduplication

`./scripts/bash/stage-modules.sh` now keeps a lightweight ledger at `storage/modules/.modules-meta/module-sql-ledger.txt` (also mounted inside containers at `/azerothcore/modules/.modules-meta/module-sql-ledger.txt`). Each staged SQL file is recorded as:

```
<database-scope>|<module>|<base_filename>|<hash>
```

When the script runs again it hashes every module SQL file and skips any entry whose `(db, module, filename)` already matches with the same hash. This prevents re-copying identical SQL after a backup restore and stops worldserver from reapplying inserts that already exist in the database. If a database restore is detected (`local-storage/mysql-data/.restore-completed` changed), the ledger is automatically reset so every module SQL file is recopied exactly once. The ledger is automatically updated anytime a file changes so only the modified SQL is restaged.

The stage script also cross-checks MySQL’s `updates` table before copying files and prunes any staged file whose identifier already exists there. That means even if a file gets stuck in `/azerothcore/data/sql/updates/<db>` (e.g., after an interrupted run), it is removed before worldserver starts if the database already recorded it.

### Restore-Time SQL Reconciliation

During a backup restore the `ac-db-import` service now runs `scripts/bash/restore-and-stage.sh`, which consolidates the old restore workflow with module SQL staging. Every backup created by the scheduler now includes a snapshot of the module ledger at `module-sql-ledger.txt` (for example `storage/backups/hourly/20250101_120000/module-sql-ledger.txt`). The restore script:

- Refreshes `storage/modules/.modules-meta/module-sql-ledger.txt` using the snapshot bundled with the backup (or rebuilds it from the modules directory if the snapshot is missing).
- Writes `storage/modules/.modules-meta/.restore-prestaged` to signal that the next `./scripts/bash/stage-modules.sh` run must repopulate `/azerothcore/data/sql/updates/*` before worldserver comes online.

The staging script now recopies every module SQL file—regardless of whether it has already been applied—using deterministic names like `MODULE_mod-npc-buffer_npc_buffer.sql`. AzerothCore’s built-in updater consults the `updates` tables to decide what should actually run, so already-applied files remain on disk purely to keep history intact and avoid “file missing” warnings. If a legacy backup doesn’t contain the ledger snapshot the helper simply rebuilds it and still sets the flag, so the runtime staging pass behaves the same. Run `rm -f storage/modules/.modules-meta/module-sql-ledger.txt` and rerun `./scripts/bash/stage-modules.sh --yes` if you intentionally need to reseed the ledger from scratch.

This snapshot-driven workflow means restoring a new backup automatically replays any newly added module SQL while avoiding duplicate inserts for modules that were already present. See **[docs/ADVANCED.md](ADVANCED.md)** for a deeper look at the marker workflow and container responsibilities.

### Forcing a Module SQL Re-stage

If you intentionally need to reapply all module SQL (for example after manually cleaning tables):

1. Stop services: `docker compose down`
2. Remove the SQL ledger so the next run rehashes everything:
   ```bash
   rm -f storage/modules/.modules-meta/module-sql-ledger.txt
   ```
3. (Optional) Drop the relevant records from the `updates` table if you want AzerothCore to rerun them, e.g.:
   ```bash
   docker exec -it ac-mysql mysql -uroot -p \
     -e "DELETE FROM acore_characters.updates WHERE name LIKE '%MODULE_mod-ollama-chat%';"
   ```
4. Run `./scripts/bash/stage-modules.sh --yes`

Only perform step 3 if you understand the impact—deleting entries causes worldserver to execute those SQL scripts again on next startup.

---

## Migration & Upgrades

### Upgrading from Older Backups

When restoring an older backup to a newer AzerothCore version:

1. **Restore the backup** as normal
2. **Verification happens automatically** - The system runs `dbimport` after restore
3. **Missing updates are applied** - Any new schema changes are detected and applied
4. **Check for errors** in worldserver logs

### Manual Migration Steps

If automatic migration fails:

```bash
# 1. Backup current state
./scripts/bash/manual-backup.sh --label pre-migration

# 2. Run dbimport manually
docker exec -it ac-worldserver /bin/bash
cd /azerothcore/env/dist/bin
./dbimport

# 3. Check for errors
tail -f /azerothcore/env/dist/logs/DBErrors.log

# 4. Verify with health check
./scripts/bash/db-health-check.sh --verbose --pending
```

### Schema Version Checking

Check your database version:

```sql
-- World database version
SELECT * FROM acore_world.version;

-- Check latest update
SELECT name, timestamp FROM acore_world.updates ORDER BY timestamp DESC LIMIT 1;
```

---

## Troubleshooting

### Database Won't Start

**Symptom:** MySQL container keeps restarting

**Solutions:**

1. Check logs:
```bash
docker logs ac-mysql
```

2. Check disk space:
```bash
df -h
```

3. Reset MySQL data (WARNING: deletes all data):
```bash
docker-compose down
rm -rf storage/mysql/*
docker-compose up -d
```

### Updates Not Applying

**Symptom:** SQL updates in `pending_db_*` not getting applied

**Solutions:**

1. Check `Updates.EnableDatabases` setting:
```bash
grep "Updates.EnableDatabases" storage/config/worldserver.conf
# Should be 7 (auth+char+world) or 15 (all including playerbots)
```

2. Check for SQL errors:
```bash
docker logs ac-worldserver | grep -i "sql error"
```

3. Manually run dbimport:
```bash
docker exec -it ac-worldserver /bin/bash
cd /azerothcore/env/dist/bin
./dbimport
```

### Backup Restore Fails

**Symptom:** Backup import reports errors

**Solutions:**

1. Verify backup integrity:
```bash
./scripts/bash/verify-backup-complete.sh /path/to/backup
```

2. Check SQL file format:
```bash
zcat backup.sql.gz | head -20
# Should see SQL statements like CREATE DATABASE, INSERT INTO
```

3. Check database names in manifest:
```bash
cat backup/manifest.json
# Verify database names match your .env
```

4. Try importing individual databases:
```bash
# Extract and import manually
zcat backup/acore_world.sql.gz | docker exec -i ac-mysql mysql -uroot -pPASSWORD acore_world
```

### Missing Characters After Restore

**Symptom:** Characters don't appear in-game

**Common Causes:**

1. **Wrong database restored** - Check you restored characters DB
2. **GUID mismatch** - Items reference wrong GUIDs
3. **Incomplete restore** - Check for SQL errors during restore

**Fix with backup-merge:**

```bash
# Use merge instead of import to remap GUIDs
./scripts/bash/backup-merge.sh \
  --backup-dir ./path/to/backup \
  --password PASSWORD \
  --all-characters
```

### Duplicate SQL Execution

**Symptom:** "Duplicate key" errors in logs

**Cause:** SQL update ran twice

**Prevention:** The `updates` table prevents this, but if table is missing:

```sql
-- Recreate updates table
CREATE TABLE IF NOT EXISTS `updates` (
  `name` varchar(200) NOT NULL,
  `hash` char(40) DEFAULT '',
  `state` enum('RELEASED','CUSTOM','MODULE','ARCHIVED','PENDING') NOT NULL DEFAULT 'RELEASED',
  `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `speed` int unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```

### Performance Issues

**Symptom:** Database queries are slow

**Solutions:**

1. Check database size:
```bash
./scripts/bash/db-health-check.sh
```

2. Optimize tables:
```sql
USE acore_world;
OPTIMIZE TABLE creature;
OPTIMIZE TABLE gameobject;

USE acore_characters;
OPTIMIZE TABLE characters;
OPTIMIZE TABLE item_instance;
```

3. Check MySQL configuration:
```bash
docker exec ac-mysql mysql -uroot -pPASSWORD -e "SHOW VARIABLES LIKE 'innodb_buffer_pool_size'"
```

4. Increase buffer pool (edit docker-compose.yml):
```yaml
environment:
  MYSQL_INNODB_BUFFER_POOL_SIZE: 512M  # Increase from 256M
```

---

## Best Practices

### Backup Strategy

✅ **DO:**
- Keep at least 3 days of daily backups
- Test restore procedures regularly
- Store backups in multiple locations
- Monitor backup size trends
- Verify backup completion

❌ **DON'T:**
- Rely solely on automated backups
- Store backups only on same disk as database
- Skip verification of backup integrity
- Ignore backup size growth warnings

### Update Management

✅ **DO:**
- Let AzerothCore's auto-updater handle SQL
- Review `DBErrors.log` after updates
- Keep `Updates.EnableDatabases` enabled
- Test module updates in development first

❌ **DON'T:**
- Manually modify core database tables
- Skip module SQL when installing modules
- Disable auto-updates in production
- Run untested SQL in production

### Module Installation

✅ **DO:**
- Enable modules via `.env` file
- Verify module SQL applied via health check
- Check module compatibility before enabling
- Test modules individually first

❌ **DON'T:**
- Copy SQL files manually
- Edit module source SQL
- Enable incompatible module combinations
- Skip SQL verification after module install

### Performance

✅ **DO:**
- Run `OPTIMIZE TABLE` on large tables monthly
- Monitor database size growth
- Set appropriate MySQL buffer pool size
- Use SSD storage for MySQL data

❌ **DON'T:**
- Store MySQL data on slow HDDs
- Run database on same disk as backup
- Ignore slow query logs
- Leave unused data unarchived

---

## Quick Reference

### Essential Commands

```bash
# Check database health
./scripts/bash/db-health-check.sh

# Check backup status
./scripts/bash/backup-status.sh

# Create manual backup
./scripts/bash/manual-backup.sh --label my-backup

# Restore from backup
./scripts/bash/backup-import.sh --backup-dir ./path/to/backup --password PASS --all

# Export portable backup
./scripts/bash/backup-export.sh --password PASS --all -o ./export

# Connect to MySQL
docker exec -it ac-mysql mysql -uroot -p

# View worldserver logs
docker logs ac-worldserver -f

# Restart services
docker-compose restart ac-worldserver ac-authserver
```

### Important File Locations

```
storage/
├── mysql/                    # MySQL data directory
├── backups/
│   ├── hourly/              # Automated hourly backups
│   └── daily/               # Automated daily backups
├── config/                  # Server configuration files
└── logs/                    # Server log files

manual-backups/              # Manual backup storage
local-storage/
└── modules/                 # Installed module files
```

### Support Resources

- **Health Check:** `./scripts/bash/db-health-check.sh --help`
- **Backup Status:** `./scripts/bash/backup-status.sh --help`
- **AzerothCore Wiki:** https://www.azerothcore.org/wiki
- **AzerothCore Discord:** https://discord.gg/gkt4y2x
- **Issue Tracker:** https://github.com/uprightbass360/AzerothCore-RealmMaster/issues

---

**End of Database Management Guide**