mirror of
https://github.com/uprightbass360/AzerothCore-RealmMaster.git
synced 2026-01-13 00:58:34 +00:00
This commit introduces major enhancements to the module installation system, database management, and configuration handling for AzerothCore deployments. ## Module System Improvements ### Module SQL Staging & Installation - Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure - Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*) - Implement conditional module database importing based on enabled modules - Add support for both cpp-modules and lua-scripts module types - Handle rsync exit code 23 (permission warnings) gracefully during deployment ### Module Manifest & Automation - Add automated module manifest generation via GitHub Actions workflow - Implement Python-based module manifest updater with comprehensive validation - Add module dependency tracking and SQL file discovery - Support for blocked modules and module metadata management ## Database Management Enhancements ### Database Import System - Add db-guard container for continuous database health monitoring and verification - Implement conditional database import that skips when databases are current - Add backup restoration and SQL staging coordination - Support for Playerbots database (4th database) in all import operations - Add comprehensive database health checking and status reporting ### Database Configuration - Implement 10 new dbimport.conf settings from environment variables: - Database.Reconnect.Seconds/Attempts for connection reliability - Updates.AllowedModules for module auto-update control - Updates.Redundancy for data integrity checks - Worker/Synch thread settings for all three core databases - Auto-apply dbimport.conf settings via auto-post-install.sh - Add environment variable injection for db-import and db-guard containers ### Backup & Recovery - Fix backup scheduler to prevent immediate execution on container startup - Add backup status monitoring script with detailed reporting - Implement backup import/export utilities - Add database verification scripts for SQL update tracking ## User Import Directory - Add new import/ directory for user-provided database files and configurations - Support for custom SQL files, configuration overrides, and example templates - Automatic import of user-provided databases and configs during initialization - Documentation and examples for custom database imports ## Configuration & Environment - Eliminate CLIENT_DATA_VERSION warning by adding default value syntax - Improve CLIENT_DATA_VERSION documentation in .env.template - Add comprehensive database import settings to .env and .env.template - Update setup.sh to handle new configuration variables with proper defaults ## Monitoring & Debugging - Add status dashboard with Go-based terminal UI (statusdash.go) - Implement JSON status output (statusjson.sh) for programmatic access - Add comprehensive database health check script - Add repair-storage-permissions.sh utility for permission issues ## Testing & Documentation - Add Phase 1 integration test suite for module installation verification - Add comprehensive documentation for: - Database management (DATABASE_MANAGEMENT.md) - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md) - Implementation mapping (IMPLEMENTATION_MAP.md) - SQL staging comparison and path coverage - Module assets and DBC file requirements - Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation - Update references from database-import/ to import/ directory ## Breaking Changes - Renamed database-import/ directory to import/ for clarity - Module SQL files now staged to AzerothCore-compatible paths - db-guard container now required for proper database lifecycle management ## Bug Fixes - Fix module SQL staging directory structure for AzerothCore compatibility - Handle rsync exit code 23 gracefully during deployments - Prevent backup from running immediately on container startup - Correct SQL staging paths for proper module installation
499 lines
14 KiB
Markdown
499 lines
14 KiB
Markdown
# Module Assets Analysis - DBC Files and Source Code
|
|
|
|
**Date:** 2025-11-16
|
|
**Purpose:** Verify handling of module DBC files, source code, and client patches
|
|
|
|
---
|
|
|
|
## Module Asset Types Found
|
|
|
|
### 1. Source Code (C++ Modules)
|
|
|
|
**Location:** `/azerothcore/modules/*/src/`
|
|
**Count:** 1,489 C++ files (.cpp and .h) across all enabled modules
|
|
**Purpose:** Server-side gameplay logic
|
|
|
|
**Examples Found:**
|
|
- `/azerothcore/modules/mod-npc-beastmaster/src/`
|
|
- `/azerothcore/modules/mod-global-chat/src/`
|
|
- `/azerothcore/modules/mod-guildhouse/src/`
|
|
|
|
**Status:** ✅ **FULLY HANDLED**
|
|
|
|
**How It Works:**
|
|
1. Modules compiled into Docker image during build
|
|
2. Source code included in image but NOT actively compiled at runtime
|
|
3. C++ code already executed as part of worldserver binary
|
|
4. Runtime module repositories provide:
|
|
- SQL files (staged by us)
|
|
- Configuration files (managed by manage-modules.sh)
|
|
- Documentation/README
|
|
|
|
**Conclusion:** Source code is **build-time only**. Pre-built images already contain compiled module code. No runtime action needed.
|
|
|
|
---
|
|
|
|
### 2. DBC Files (Database Client Files)
|
|
|
|
**Location:** `/azerothcore/modules/*/data/patch/DBFilesClient/`
|
|
**Found in:** mod-worgoblin (custom race module)
|
|
**Count:** 20+ custom DBC files for new race
|
|
|
|
**Example Files Found:**
|
|
```
|
|
/azerothcore/modules/mod-worgoblin/data/patch/DBFilesClient/
|
|
├── ChrRaces.dbc # Race definitions
|
|
├── CharBaseInfo.dbc # Character stats
|
|
├── CharHairGeosets.dbc # Hair models
|
|
├── CharacterFacialHairStyles.dbc
|
|
├── CharStartOutfit.dbc # Starting gear
|
|
├── NameGen.dbc # Name generation
|
|
├── TalentTab.dbc # Talent trees
|
|
├── Faction.dbc # Faction relations
|
|
└── ...
|
|
```
|
|
|
|
**Purpose:** Client-side data that defines:
|
|
- New races/classes
|
|
- Custom spells/items
|
|
- UI elements
|
|
- Character customization
|
|
|
|
**Status:** ⚠️ **NOT AUTOMATICALLY DEPLOYED**
|
|
|
|
---
|
|
|
|
### 3. Client Patch Files (MPQ Archives)
|
|
|
|
**Found in Multiple Modules:**
|
|
```
|
|
storage/modules/aio-blackjack/patch-W.MPQ
|
|
storage/modules/mod-arac/Patch-A.MPQ
|
|
storage/modules/prestige-and-draft-mode/Client Side Files/Mpq Patch/patch-P.mpq
|
|
storage/modules/horadric-cube-for-world-of-warcraft/Client/Data/zhCN/patch-zhCN-5.MPQ
|
|
```
|
|
|
|
**Purpose:** Pre-packaged client patches containing:
|
|
- DBC files
|
|
- Custom textures/models
|
|
- UI modifications
|
|
- Sound files
|
|
|
|
**Status:** ⚠️ **USER MUST MANUALLY DISTRIBUTE**
|
|
|
|
---
|
|
|
|
### 4. Other Client Assets
|
|
|
|
**mod-worgoblin patch directory structure:**
|
|
```
|
|
storage/modules/mod-worgoblin/data/patch/
|
|
├── Character/ # Character models
|
|
├── Creature/ # NPC models
|
|
├── DBFilesClient/ # DBC files
|
|
├── ITEM/ # Item models
|
|
├── Interface/ # UI elements
|
|
├── Sound/ # Audio files
|
|
└── Spells/ # Spell effects
|
|
```
|
|
|
|
**Status:** ⚠️ **NOT PACKAGED OR DEPLOYED**
|
|
|
|
---
|
|
|
|
## How DBC Files Work in AzerothCore
|
|
|
|
### Server-Side DBC
|
|
|
|
**Location:** `/azerothcore/data/dbc/`
|
|
**Purpose:** Server reads these to understand game rules
|
|
**Source:** Extracted from vanilla WoW 3.3.5a client
|
|
|
|
**Current Status:**
|
|
```bash
|
|
$ docker exec ac-worldserver ls /azerothcore/data/dbc | wc -l
|
|
1189 DBC files present
|
|
```
|
|
|
|
✅ Server has standard DBC files (from client-data download)
|
|
|
|
### Client-Side DBC
|
|
|
|
**Location:** Player's `WoW/Data/` folder (or patch MPQ)
|
|
**Purpose:** Client reads these to:
|
|
- Display UI correctly
|
|
- Render spells/models
|
|
- Generate character names
|
|
- Show tooltips
|
|
|
|
**Critical:** Client and server DBCs must match!
|
|
|
|
---
|
|
|
|
## Official AzerothCore DBC Deployment Process
|
|
|
|
### For Module Authors:
|
|
|
|
1. **Create Modified DBCs:**
|
|
- Use DBC editor tools
|
|
- Modify necessary tables
|
|
- Export modified .dbc files
|
|
|
|
2. **Package for Distribution:**
|
|
- Create MPQ patch file (e.g., `Patch-Z.MPQ`)
|
|
- Include all modified DBCs
|
|
- Add any custom assets (models, textures)
|
|
|
|
3. **Server Deployment:**
|
|
- Copy DBCs to `/azerothcore/data/dbc/` (overwrites vanilla)
|
|
- Restart server
|
|
|
|
4. **Client Distribution:**
|
|
- Distribute patch MPQ to all players
|
|
- Players place in `WoW/Data/` directory
|
|
- Players restart game
|
|
|
|
### For Server Admins:
|
|
|
|
**Manual Steps Required:**
|
|
1. Download module patch from README/releases
|
|
2. Apply server-side DBCs
|
|
3. Host patch file for players to download
|
|
4. Instruct players to install patch
|
|
|
|
---
|
|
|
|
## Current Implementation Status
|
|
|
|
### What We Handle Automatically ✅
|
|
|
|
1. **Module SQL** - Staged to core updates directory
|
|
2. **Module Config** - Deployed to worldserver config directory
|
|
3. **Module Compilation** - Pre-built into Docker images
|
|
4. **Standard DBC** - Downloaded via client-data scripts
|
|
|
|
### What We DON'T Handle ⚠️
|
|
|
|
1. **Custom Module DBCs** - Not deployed to server DBC directory
|
|
2. **Client Patch Files** - Not distributed to players
|
|
3. **Client Assets** - Not packaged or made available
|
|
4. **DBC Synchronization** - No validation that client/server match
|
|
|
|
---
|
|
|
|
## Gap Analysis
|
|
|
|
### Modules Requiring Client Patches
|
|
|
|
From our analysis, these modules have client-side requirements:
|
|
|
|
| Module | Client Assets | Server DBCs | Impact if Missing |
|
|
|--------|--------------|-------------|-------------------|
|
|
| **mod-worgoblin** | ✅ Yes (extensive) | ✅ Yes | NEW RACE WON'T WORK |
|
|
| **mod-arac** | ✅ Yes (Patch-A.MPQ) | ✅ Yes | Class/race combos broken |
|
|
| **aio-blackjack** | ✅ Yes (patch-W.MPQ) | ❓ Unknown | UI elements missing |
|
|
| **prestige-and-draft-mode** | ✅ Yes (patch-P.mpq) | ❓ Unknown | Features unavailable |
|
|
| **horadric-cube** | ✅ Yes (patch-zhCN-5.MPQ) | ❓ Unknown | Locale-specific broken |
|
|
|
|
### Severity Assessment
|
|
|
|
**mod-worgoblin (CRITICAL):**
|
|
- Adds entirely new playable race (Worgen/Goblin)
|
|
- Requires 20+ modified DBC files
|
|
- Without patch: Players can't create/see race correctly
|
|
- **Currently broken** - DBCs not deployed
|
|
|
|
**mod-arac (HIGH):**
|
|
- "All Races All Classes" - removes restrictions
|
|
- Requires modified class/race DBC tables
|
|
- Without patch: Restrictions may still apply client-side
|
|
- **Potentially broken** - needs verification
|
|
|
|
**Others (MEDIUM/LOW):**
|
|
- Gameplay features may work server-side
|
|
- UI/visual elements missing client-side
|
|
- Degraded experience but not completely broken
|
|
|
|
---
|
|
|
|
## Why We Don't Auto-Deploy Client Patches
|
|
|
|
### Technical Reasons
|
|
|
|
1. **Client patches are player-specific**
|
|
- Each player must install manually
|
|
- No server-side push mechanism
|
|
- Requires download link/instructions
|
|
|
|
2. **Version control complexity**
|
|
- Different locales (enUS, zhCN, etc.)
|
|
- Different client versions
|
|
- Naming conflicts between modules
|
|
|
|
3. **File hosting requirements**
|
|
- MPQ files can be 10MB+ each
|
|
- Need web server or file host
|
|
- Update distribution mechanism
|
|
|
|
4. **Testing/validation needed**
|
|
- Must verify client compatibility
|
|
- Risk of corrupting client
|
|
- Hard to automate testing
|
|
|
|
### Architectural Reasons
|
|
|
|
1. **Docker images are server-only**
|
|
- Don't interact with player clients
|
|
- Can't modify player installations
|
|
- Out of scope for server deployment
|
|
|
|
2. **Module isolation**
|
|
- Each module maintains own patches
|
|
- No central patch repository
|
|
- Version conflicts possible
|
|
|
|
3. **Admin responsibility**
|
|
- Server admin chooses which modules
|
|
- Must communicate requirements to players
|
|
- Custom instructions per module
|
|
|
|
---
|
|
|
|
## Recommended Approach
|
|
|
|
### Current Best Practice ✅
|
|
|
|
**Our Implementation:**
|
|
1. ✅ Deploy module source (pre-compiled in image)
|
|
2. ✅ Deploy module SQL (runtime staging)
|
|
3. ✅ Deploy module config files (manage-modules.sh)
|
|
4. ⚠️ **Document client patch requirements** (user responsibility)
|
|
|
|
**This matches official AzerothCore guidance:**
|
|
- Server-side automation where possible
|
|
- Client-side patches distributed manually
|
|
- Admin reads module README for requirements
|
|
|
|
### Enhanced Documentation 📝
|
|
|
|
**What We Should Add:**
|
|
|
|
1. **Module README Scanner**
|
|
- Detect client patch requirements
|
|
- Warn admin during deployment
|
|
- Link to download instructions
|
|
|
|
2. **Client Patch Detection**
|
|
- Scan for `*.MPQ`, `*.mpq` files
|
|
- Check for `data/patch/` directories
|
|
- Report found patches in deployment log
|
|
|
|
3. **Deployment Checklist**
|
|
- List modules with client requirements
|
|
- Provide download links (from module repos)
|
|
- Instructions for player distribution
|
|
|
|
**Example Output:**
|
|
```
|
|
⚠️ Client Patches Required:
|
|
|
|
mod-worgoblin:
|
|
📦 Patch: storage/modules/mod-worgoblin/Patch-Z.MPQ
|
|
📋 Instructions: See storage/modules/mod-worgoblin/README.md
|
|
🔗 Download: https://github.com/azerothcore/mod-worgoblin/releases
|
|
|
|
mod-arac:
|
|
📦 Patch: storage/modules/mod-arac/Patch-A.MPQ
|
|
📋 Instructions: Players must install to WoW/Data/
|
|
|
|
⚠️ Server admins must distribute these patches to players!
|
|
```
|
|
|
|
---
|
|
|
|
## Server-Side DBC Deployment (Possible Enhancement)
|
|
|
|
### What Could Be Automated
|
|
|
|
**If modules include server DBCs:**
|
|
```
|
|
modules/mod-worgoblin/
|
|
└── data/
|
|
├── sql/ # ✅ We handle this
|
|
├── dbc/ # ❌ We don't handle this
|
|
│ ├── ChrRaces.dbc
|
|
│ └── ...
|
|
└── patch/ # ❌ Client-side (manual)
|
|
└── ...
|
|
```
|
|
|
|
**Potential Enhancement:**
|
|
```bash
|
|
# In stage-modules.sh, add DBC staging:
|
|
if [ -d "$module_dir/data/dbc" ]; then
|
|
echo "📦 Staging server DBCs for $module_name..."
|
|
cp -r "$module_dir/data/dbc/"* /azerothcore/data/dbc/
|
|
echo "⚠️ Server restart required to load new DBCs"
|
|
fi
|
|
```
|
|
|
|
**Risks:**
|
|
- ⚠️ Overwrites vanilla DBCs (could break other modules)
|
|
- ⚠️ No conflict detection between modules
|
|
- ⚠️ No rollback mechanism
|
|
- ⚠️ Requires worldserver restart (not just reload)
|
|
|
|
**Recommendation:** **DON'T AUTO-DEPLOY** server DBCs
|
|
- Too risky without validation
|
|
- Better to document in README
|
|
- Admin can manually copy if needed
|
|
|
|
---
|
|
|
|
## Source Code Compilation
|
|
|
|
### How It Works in Standard Setup
|
|
|
|
**Official Process:**
|
|
1. Clone module to `/modules/` directory
|
|
2. Run CMake (detects new module)
|
|
3. Recompile entire core
|
|
4. Module C++ code compiled into worldserver binary
|
|
|
|
**CMake Module Detection:**
|
|
```cmake
|
|
# CMake scans for modules during configuration
|
|
foreach(module_dir ${CMAKE_SOURCE_DIR}/modules/*)
|
|
if(EXISTS ${module_dir}/CMakeLists.txt)
|
|
add_subdirectory(${module_dir})
|
|
endif()
|
|
endforeach()
|
|
```
|
|
|
|
### How It Works With Pre-Built Images
|
|
|
|
**Docker Image Build Process:**
|
|
1. Modules cloned during image build
|
|
2. CMake runs with all enabled modules
|
|
3. Worldserver compiled with modules included
|
|
4. Binary contains all module code
|
|
|
|
**Runtime (Our Deployment):**
|
|
1. Image already has compiled modules
|
|
2. Mount module repositories for:
|
|
- SQL files (we stage these)
|
|
- Config files (we deploy these)
|
|
- README/docs (reference only)
|
|
3. Source code in repository is **NOT compiled**
|
|
|
|
**Verification:**
|
|
```bash
|
|
# Module code is inside the binary
|
|
$ docker exec ac-worldserver worldserver --version
|
|
# Shows compiled modules
|
|
|
|
# Source code exists but isn't used
|
|
$ docker exec ac-worldserver ls /azerothcore/modules/mod-*/src/
|
|
# Files present but not actively compiled
|
|
```
|
|
|
|
### Status: ✅ **FULLY HANDLED**
|
|
|
|
No action needed for source code:
|
|
- Pre-built images contain all enabled modules
|
|
- Source repositories provide SQL/config only
|
|
- Recompilation would require custom build (out of scope)
|
|
|
|
---
|
|
|
|
## Comparison: Official vs. Our Implementation
|
|
|
|
| Asset Type | Official Process | Our Implementation | Status |
|
|
|------------|------------------|-------------------|--------|
|
|
| **C++ Source** | Compile at build | ✅ Pre-compiled in image | ✅ COMPLETE |
|
|
| **SQL Files** | Applied by DBUpdater | ✅ Runtime staging | ✅ COMPLETE |
|
|
| **Config Files** | Manual deployment | ✅ Automated by manage-modules | ✅ COMPLETE |
|
|
| **Server DBCs** | Manual copy to /data/dbc | ❌ Not deployed | ⚠️ DOCUMENTED |
|
|
| **Client Patches** | Distribute to players | ❌ Not distributed | ⚠️ USER RESPONSIBILITY |
|
|
| **Client Assets** | Package in MPQ | ❌ Not packaged | ⚠️ MANUAL |
|
|
|
|
---
|
|
|
|
## Recommendations
|
|
|
|
### Keep Current Approach ✅
|
|
|
|
**What we do well:**
|
|
1. SQL staging - automated and secure
|
|
2. Config management - fully automated
|
|
3. Source handling - correctly uses pre-built binaries
|
|
4. Clear separation of server vs. client concerns
|
|
|
|
### Add Documentation 📝
|
|
|
|
**Enhance deployment output:**
|
|
1. Detect modules with client patches
|
|
2. Warn admin about distribution requirements
|
|
3. Provide links to patch files and instructions
|
|
4. Create post-deployment checklist
|
|
|
|
### Don't Implement (Too Risky) ⛔
|
|
|
|
**What NOT to automate:**
|
|
1. Server DBC deployment - risk of conflicts
|
|
2. Client patch distribution - technically impossible from server
|
|
3. Module recompilation - requires custom build process
|
|
4. Client asset packaging - out of scope
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
### Current Status: ✅ **SOUND ARCHITECTURE**
|
|
|
|
**What We Handle:**
|
|
- ✅ Module source code (via pre-built images)
|
|
- ✅ Module SQL (runtime staging)
|
|
- ✅ Module configuration (automated deployment)
|
|
|
|
**What Requires Manual Steps:**
|
|
- ⚠️ Server DBC deployment (module README instructions)
|
|
- ⚠️ Client patch distribution (admin responsibility)
|
|
- ⚠️ Player communication (outside automation scope)
|
|
|
|
### No Critical Gaps
|
|
|
|
All gaps identified are **by design**:
|
|
- Client-side patches can't be auto-deployed (technical limitation)
|
|
- Server DBCs shouldn't be auto-deployed (safety concern)
|
|
- Module READMEs must be read (standard practice)
|
|
|
|
**Our implementation correctly handles what can be automated while documenting what requires manual steps.**
|
|
|
|
---
|
|
|
|
## Modules Requiring Special Attention
|
|
|
|
### High Priority (Client Patches Required)
|
|
|
|
**mod-worgoblin:**
|
|
- Status: Likely broken without client patch
|
|
- Action: Check README, distribute Patch-Z.MPQ to players
|
|
- Impact: New race completely unavailable
|
|
|
|
**mod-arac:**
|
|
- Status: Needs verification
|
|
- Action: Distribute Patch-A.MPQ to players
|
|
- Impact: Race/class restrictions may apply incorrectly
|
|
|
|
### Medium Priority (Enhanced Features)
|
|
|
|
**aio-blackjack, prestige-and-draft-mode, horadric-cube:**
|
|
- Status: Core functionality may work, UI missing
|
|
- Action: Optional patch distribution for full experience
|
|
- Impact: Degraded but functional
|
|
|
|
---
|
|
|
|
**Conclusion:** Our implementation is complete for automated deployment. Client patches and server DBCs correctly remain manual tasks with proper documentation.
|