nchebrov/AzerothCore-RealmMaster
1
0
Fork 0
mirror of https://github.com/uprightbass360/AzerothCore-RealmMaster.git synced 2026-01-13 00:58:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: comprehensive module system and database management improvements

Browse Source 
This commit introduces major enhancements to the module installation system,
database management, and configuration handling for AzerothCore deployments.

## Module System Improvements

### Module SQL Staging & Installation
- Refactor module SQL staging to properly handle AzerothCore's sql/ directory structure
- Fix SQL staging path to use correct AzerothCore format (sql/custom/db_*/*)
- Implement conditional module database importing based on enabled modules
- Add support for both cpp-modules and lua-scripts module types
- Handle rsync exit code 23 (permission warnings) gracefully during deployment

### Module Manifest & Automation
- Add automated module manifest generation via GitHub Actions workflow
- Implement Python-based module manifest updater with comprehensive validation
- Add module dependency tracking and SQL file discovery
- Support for blocked modules and module metadata management

## Database Management Enhancements

### Database Import System
- Add db-guard container for continuous database health monitoring and verification
- Implement conditional database import that skips when databases are current
- Add backup restoration and SQL staging coordination
- Support for Playerbots database (4th database) in all import operations
- Add comprehensive database health checking and status reporting

### Database Configuration
- Implement 10 new dbimport.conf settings from environment variables:
  - Database.Reconnect.Seconds/Attempts for connection reliability
  - Updates.AllowedModules for module auto-update control
  - Updates.Redundancy for data integrity checks
  - Worker/Synch thread settings for all three core databases
- Auto-apply dbimport.conf settings via auto-post-install.sh
- Add environment variable injection for db-import and db-guard containers

### Backup & Recovery
- Fix backup scheduler to prevent immediate execution on container startup
- Add backup status monitoring script with detailed reporting
- Implement backup import/export utilities
- Add database verification scripts for SQL update tracking

## User Import Directory

- Add new import/ directory for user-provided database files and configurations
- Support for custom SQL files, configuration overrides, and example templates
- Automatic import of user-provided databases and configs during initialization
- Documentation and examples for custom database imports

## Configuration & Environment

- Eliminate CLIENT_DATA_VERSION warning by adding default value syntax
- Improve CLIENT_DATA_VERSION documentation in .env.template
- Add comprehensive database import settings to .env and .env.template
- Update setup.sh to handle new configuration variables with proper defaults

## Monitoring & Debugging

- Add status dashboard with Go-based terminal UI (statusdash.go)
- Implement JSON status output (statusjson.sh) for programmatic access
- Add comprehensive database health check script
- Add repair-storage-permissions.sh utility for permission issues

## Testing & Documentation

- Add Phase 1 integration test suite for module installation verification
- Add comprehensive documentation for:
  - Database management (DATABASE_MANAGEMENT.md)
  - Module SQL analysis (AZEROTHCORE_MODULE_SQL_ANALYSIS.md)
  - Implementation mapping (IMPLEMENTATION_MAP.md)
  - SQL staging comparison and path coverage
  - Module assets and DBC file requirements
- Update SCRIPTS.md, ADVANCED.md, and troubleshooting documentation
- Update references from database-import/ to import/ directory

## Breaking Changes

- Renamed database-import/ directory to import/ for clarity
- Module SQL files now staged to AzerothCore-compatible paths
- db-guard container now required for proper database lifecycle management

## Bug Fixes

- Fix module SQL staging directory structure for AzerothCore compatibility
- Handle rsync exit code 23 gracefully during deployments
- Prevent backup from running immediately on container startup
- Correct SQL staging paths for proper module installation
This commit is contained in:
uprightbass360
2025-11-20 18:26:00 -05:00
committed by Deckard
parent 0d83f01995
commit e6231bb4a4
56 changed files with 11298 additions and 487 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
docs/TROUBLESHOOTING.md
View File

@@ -41,10 +41,64 @@ ls storage/config/mod_*.conf*
# Verify MySQL is running and responsive
docker exec ac-mysql mysql -u root -p -e "SELECT 1;"
# Starting with the 2025-11-17 release the import job checks if
# the runtime tables exist before trusting restoration markers. If you see
# "Restoration marker found, but databases are empty - forcing re-import" in
# `docker logs ac-db-import`, just let the container finish; it will automatically
# clear stale markers and replay the latest backup so the services never boot
# against an empty tmpfs volume. See docs/DATABASE_MANAGEMENT.md#restore-safety-checks--sentinels
# for full details.
# Forcing a fresh import (if schema missing/invalid)
# 1. Stop the stack
docker compose down
# 2. Remove the sentinel created after a successful restore (inside the docker volume)
docker run --rm -v mysql-data:/var/lib/mysql-persistent alpine sh -c 'rm -f /var/lib/mysql-persistent/.restore-completed'
# 3. Re-run the import pipeline (either stand-alone or via stage-modules)
docker compose run --rm ac-db-import
# or
./scripts/bash/stage-modules.sh --yes
#
# See docs/ADVANCED.md#database-hardening for details on the sentinel workflow and why it's required.
**Permission denied writing to local-storage or storage**
```bash
# Reset ownership/permissions on the shared directories
./scripts/bash/repair-storage-permissions.sh
```
> This script reuses the same helper container as the staging workflow to `chown`
> `storage/`, `local-storage/`, and module metadata paths back to the current
> host UID/GID so tools like `scripts/python/modules.py` can regenerate
> `modules.env` without manual intervention.
# Check database initialization
docker logs ac-db-init
docker logs ac-db-import
```
> Need more context on why the sentinel exists or how the restore-aware SQL stage cooperates with backups? See [docs/ADVANCED.md#database-hardening](ADVANCED.md#database-hardening) for the full architecture notes.
**Worldserver restart loop (duplicate module SQL)**
> After a backup restore the ledger snapshot is synced and `.restore-prestaged` is set so the next `./scripts/bash/stage-modules.sh` run recopies EVERY module SQL file into `/azerothcore/data/sql/updates/*` with deterministic names. Check `docker logs ac-worldserver` to confirm it sees those files; the `updates` table still prevents reapplication, but the files remain on disk so the server never complains about missing history.
```bash
# 1. Inspect the worldserver log for errors like
# "Duplicate entry ... MODULE_<module_name>_<file>"
docker logs ac-worldserver
# 2. Remove the staged SQL file that keeps replaying:
docker exec ac-worldserver rm /azerothcore/data/sql/updates/<db>/<filename>.sql
# 3. Re-run the staging workflow
./scripts/bash/stage-modules.sh --yes
# 4. Restart the worldserver container
docker compose restart ac-worldserver-playerbots # or the profile you use
# See docs/DATABASE_MANAGEMENT.md#module-sql-management for details on the workflow.
```
**Legacy backup missing module SQL snapshot**
Legacy backups behave the same as new ones now—just rerun `./scripts/bash/stage-modules.sh --yes` after a restore and the updater will apply whatever the database still needs.
**Source rebuild issues**
```bash