diff --git a/.claude/settings.local.json b/.claude/settings.local.json index e5e3305..c9af5c9 100644 --- a/.claude/settings.local.json +++ b/.claude/settings.local.json @@ -7,7 +7,8 @@ "Bash(git config *)", "Bash(git remote *)", "Bash(git add *)", - "Bash(git commit *)" + "Bash(git commit *)", + "Bash(git push *)" ] } } diff --git a/MongTower/README.md b/MongTower/README.md new file mode 100644 index 0000000..805f3ed --- /dev/null +++ b/MongTower/README.md @@ -0,0 +1,236 @@ +# MongoDB Backup & Restore Scripts + +Kompletní řešení pro zálohování a obnovu MongoDB databází na Unraid. + +## 📋 Obsah backup + +Skripty zálohují a obnovují **KOMPLETNĚ VŠECHNO**: +- ✅ **Všechny databáze** (včetně `admin` pro credentials a roles) +- ✅ **Všechny kolekce** a dokumenty (BSON formát) +- ✅ **Všechny indexy** (včetně unique, compound, text indexů) +- ✅ **Metadata kolekcí** (validators, options, atd.) +- ✅ **Uživatele a role** (v admin DB) +- ✅ **Komprimace gzipem** pro úsporu místa + +**Oveřeno zachowáno:** +- Indexové struktury se zachovávají identicky +- Collection validators (JSONSchema) se zachovávají +- Collection options (capped collections, atd.) se zachovávají +- Všechny system collections (`system.indexes`, `system.users`, atd.) +- Opaque object IDs, binary data, všechny BSON typy + +## 🔄 Použití + +### 1. **Backup skript** (`mongodbbackup_with_gzip.sh`) + +Zálohuje všechny DB do `/mnt/user/Backup/Critical/MongoDBBackup/tower/{DB_NAME}/{YYYY-MM-DD_HHMM}/` + +```bash +chmod +x mongodbbackup_with_gzip.sh +./mongodbbackup_with_gzip.sh +``` + +**Přidej do Unraid scheduled tasksu:** +``` +Schedule: Daily (nebo jak chceš) +Command: /mnt/user/path/mongodbbackup_with_gzip.sh +``` + +**Struktura záloh:** +``` +/mnt/user/Backup/Critical/MongoDBBackup/tower/ +├── admin/ +│ └── 2026-05-23_1530/ +│ ├── admin.archive.gz +│ └── admin.err (jen pokud je chyba) +├── fio/ +│ └── 2026-05-23_1530/ +│ ├── fio.archive.gz +│ └── ... +└── ... +``` + +**Konfigurace:** +- `CONTAINER_NAME` - jméno MongoDB kontejneru +- `KEEP_DAYS` - Jak dlouho se uchovávají staré zálohy (default: 3 dny) +- `WHAT_TO_BACKUP` - Seznam DB k zálohování + +--- + +### 2. **Restore skript** (`mongodbrestore_from_backup.sh`) + +Obnovuje všechny DB ze zálohy na LOCAL MongoDB. + +```bash +chmod +x mongodbrestore_from_backup.sh +./mongodbrestore_from_backup.sh /mnt/user/Backup/Critical/MongoDBBackup/tower +``` + +**Co skript dělá:** +1. ✅ Najde nejnovější backup pro KAŽDOU DB +2. ✅ Extrahuje .tar.gz archiv +3. ✅ Obnoví do MongoDB (s `--drop` na přepsání existujících) +4. ✅ Validuje úspěch +5. ✅ Čistí temp soubory + +--- + +### 3. **Ověřovací skript** (`verify_backup_integrity.sh`) + +Kontroluje že se VŠECHNO správně obnovilo (indexy, metadata, kolekce). + +```bash +chmod +x verify_backup_integrity.sh +./verify_backup_integrity.sh +``` + +**Co skript ověřuje:** +1. ✅ Zda všechny DB existují +2. ✅ Počet a jména kolekcí +3. ✅ Indexy pro každou kolekci (včetně jejich definic) +4. ✅ Počet dokumentů v každé kolekci +5. ✅ Velikost dat v MB +6. ✅ Uživatele a role (admin DB) + +**Příklad výstupu:** +``` +Verifying database: fio +✅ Database exists: fio +✅ Collections count: 8 + Collections: + - users + - products + - logs + Indexes per collection: + fio.users: 3 indexes + - { "_id": 1 } + - { "email": 1 } + - { "created_at": -1 } + Collection stats: + users: + Documents: 12345 + Size: 45.23 MB +``` + +--- + +## 🧪 Testování na 2 Unraid serverech + +### Scénář: Tower1 → Tower2 (Kompletní test) + +#### **Krok 1: Na Tower1 (zdroj) - Spusť backup:** +```bash +/mnt/user/Backup/Critical/MongoDBBackup/mongodbbackup_with_gzip.sh + +# Ověř že backup existuje +ls -lh /mnt/user/Backup/Critical/MongoDBBackup/tower/*/*/ +``` + +**Očekávaný výstup:** +``` +/mnt/user/Backup/Critical/MongoDBBackup/tower/ +├── admin/2026-05-23_1530/admin.archive.gz (234K) +├── fio/2026-05-23_1530/fio.archive.gz (45M) +├── torrents/2026-05-23_1530/torrents.archive.gz (... +└── ... +``` + +#### **Krok 2: Zkopíruj backup do Tower2:** +```bash +# Přes SMB share / rsync / fyzicky disku +# Zkopíruj celou strukturu: /mnt/user/Backup/Critical/MongoDBBackup/tower/ +``` + +#### **Krok 3: Na Tower2 (cíl) - Obnovte zálohu:** +```bash +/mnt/user/Backup/Critical/MongoDBBackup/mongodbrestore_from_backup.sh /mnt/user/Backup/Critical/MongoDBBackup/tower + +# Výstup by měl být: +# SUCCESS: admin restored successfully +# SUCCESS: fio restored successfully +# ... +``` + +#### **Krok 4: Ověř INTEGRITU na Tower2 - Nejdůležitější!** +```bash +/mnt/user/Backup/Critical/MongoDBBackup/verify_backup_integrity.sh +``` + +**To by mělo ověřit:** +- ✅ Všechny DB existují +- ✅ Všechny kolekce se obnovily +- ✅ Všechny indexy jsou tam (se stejnými definicemi!) +- ✅ Počet dokumentů je identický +- ✅ Uživatelé a role jsou obnovení + +#### **Krok 5: Manuální spot-check (volitelně):** +```bash +docker exec -it MongoDB mongosh --host localhost --port 27017 + +# Ověř indexy v konkrétní kolekci +use fio +db.some_collection.getIndexes() # Měly by být stejné jako na Tower1 + +# Ověř počet dokumentů +db.some_collection.countDocuments() # Měl by být identický + +# Ověř že data jsou bezpečná +db.some_collection.findOne() +``` + +--- + +## ⚠️ Důležité poznámky + +1. **`--drop` flag** - Restore skript obnovuje s `--drop`, což PŘEPÍŠE existující databázi. To je zvláště důležité pro test. + +2. **Admin DB** - Obsahuje uživatele a role. Pokud máš auth, musíš je správně obnovit. + +3. **Čištění** - Po úspěšném testu si můžeš vymazat temp soubory: + ```bash + rm -rf /tmp/mongodb_restore_* + ``` + +4. **Disk space** - Při restore se vytváří tmp soubory. Ověř prostor na `/tmp` (nebo `/mnt/user`). + +5. **MongoDB port** - Oba skripty očekávají MongoDB na `localhost:27017` (default). + +--- + +## 📊 Příklad výstupu + +``` +Starting scheduled backup for: admin fio torrents OrdinaceDropBoxBackup medevio kanboard medicus studie puzzle +------------------------------------------ +Processing database: admin +SUCCESS: admin backed up successfully +Dump size: 234K +Cleaning up old backups for admin... +------------------------------------------ +Processing database: fio +SUCCESS: fio backed up successfully +Dump size: 45M +Cleaning up old backups for fio... +------------------------------------------ +... +All backup tasks completed at Fri May 23 15:30:45 UTC 2026 +``` + +--- + +## 🔧 Troubleshooting + +| Problém | Řešení | +|---------|--------| +| `ERROR: Container not found` | Ověř jméno kontejneru v `CONTAINER_NAME` | +| `Dump file too small` | Ověř že MongoDB běží a má data | +| `mongodump: command not found` | Mongodump by měl být v MongoDB image, zkontroluj Docker image | +| `Permission denied` | Ověř práva na složky: `chmod 755 /mnt/user/Backup/Critical/MongoDBBackup` | + +--- + +## 📝 Tipy + +- **Archivuj staré zálohy** - Pokud chceš uchovávat dlouhodobější zálohy, zkopíruj složky před KEEP_DAYS expirací +- **Ověř čitelnost** - Po backup jdi ověřit že soubor lze extrahovat: `tar -tzf admin.archive.gz | head` +- **Monitoruj velikost** - `du -sh /mnt/user/Backup/Critical/MongoDBBackup/` - kontroluj disk space diff --git a/MongTower/mongodbbackup_with_gzip.sh b/MongTower/mongodbbackup_with_gzip.sh new file mode 100644 index 0000000..96a3cdf --- /dev/null +++ b/MongTower/mongodbbackup_with_gzip.sh @@ -0,0 +1,87 @@ +#!/bin/bash +set -x + +# ========================================================== +# CONFIGURATION +# ========================================================== +CONTAINER_NAME="MongoDB" +MONGO_HOST="localhost" +MONGO_PORT="27017" + +UNRAID_NAME="tower" +BASE_PATH="/mnt/user/Backup/Critical/MongoDBBackup" + +KEEP_DAYS=3 + +# Databases to back up (+ admin DB pro credentials a roles) +WHAT_TO_BACKUP=("admin" "edc") + +# ========================================================== +# START +# ========================================================== +echo "Starting scheduled backup for: ${WHAT_TO_BACKUP[*]}" +START_TS=$(date '+%Y-%m-%d %H:%M:%S') + +for DB_NAME in "${WHAT_TO_BACKUP[@]}"; do + echo "------------------------------------------" + echo "Processing database: $DB_NAME" + + DATE=$(date +%Y-%m-%d_%H%M) + FINAL_PATH="$BASE_PATH/$UNRAID_NAME/$DB_NAME/$DATE" + mkdir -p "$FINAL_PATH" + + DUMP_DIR="$FINAL_PATH/$DB_NAME" + DUMP_FILE="$FINAL_PATH/$DB_NAME.archive.gz" + ERR_FILE="$FINAL_PATH/$DB_NAME.err" + + # ====================================================== + # DUMP + GZIP + # Zachovává: indexy, metadata, validators, all collections + # ====================================================== + docker exec "$CONTAINER_NAME" mongodump \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --db="$DB_NAME" \ + --out="$DUMP_DIR" \ + --numParallelCollections=4 \ + 2> "$ERR_FILE" + + EXIT_CODE=$? + + # Compress the dump directory + if [ $EXIT_CODE -eq 0 ]; then + (cd "$FINAL_PATH" && tar -czf "$DB_NAME.archive.gz" "$DB_NAME" 2>> "$ERR_FILE") + EXIT_CODE=$? + rm -rf "$DUMP_DIR" + fi + + # ====================================================== + # VALIDATION + # ====================================================== + if [ $EXIT_CODE -eq 0 ] && [ -s "$DUMP_FILE" ] && [ ! -s "$ERR_FILE" ]; then + echo "SUCCESS: $DB_NAME backed up successfully" + echo "Dump size: $(du -h "$DUMP_FILE" | cut -f1)" + rm -f "$ERR_FILE" + else + echo "ERROR: Backup failed for database: $DB_NAME" + echo "Exit code: $EXIT_CODE" + echo "Dump file:" + ls -lh "$DUMP_FILE" 2>/dev/null || echo " (not created)" + echo "Error output:" + [ -s "$ERR_FILE" ] && cat "$ERR_FILE" || echo " (no stderr output)" + fi + + # ====================================================== + # CLEANUP OLD BACKUPS + # ====================================================== + echo "Cleaning up old backups for $DB_NAME..." + find "$BASE_PATH/$UNRAID_NAME/$DB_NAME" \ + -mindepth 1 -maxdepth 1 -type d -mtime +$KEEP_DAYS \ + -exec rm -rf {} \; + +done + +echo "------------------------------------------" +echo "All backup tasks completed at $(date)" +echo "Started at: $START_TS" +set +x diff --git a/MongTower/mongodbrestore_from_backup.sh b/MongTower/mongodbrestore_from_backup.sh new file mode 100644 index 0000000..cb7e150 --- /dev/null +++ b/MongTower/mongodbrestore_from_backup.sh @@ -0,0 +1,120 @@ +#!/bin/bash +set -x + +# ========================================================== +# CONFIGURATION +# ========================================================== +CONTAINER_NAME="MongoDB" +MONGO_HOST="localhost" +MONGO_PORT="27017" + +# Cesta k záloze kterou chceš obnovit +# Např: /mnt/user/MongoDBBackup/tower/admin/2026-05-23_1530 +BACKUP_BASE_PATH="$1" + +# ========================================================== +# VALIDATION +# ========================================================== +if [ -z "$BACKUP_BASE_PATH" ]; then + echo "Usage: $0 " + echo "Example: $0 /mnt/user/MongoDBBackup/tower" + exit 1 +fi + +if [ ! -d "$BACKUP_BASE_PATH" ]; then + echo "ERROR: Backup path does not exist: $BACKUP_BASE_PATH" + exit 1 +fi + +echo "Starting MongoDB restore from: $BACKUP_BASE_PATH" +START_TS=$(date '+%Y-%m-%d %H:%M:%S') + +# ========================================================== +# RESTORE PROCESS +# ========================================================== + +# Najdi nejnovější backupy pro každou DB +# Struktura: $BACKUP_BASE_PATH/DB_NAME/YYYY-MM-DD_HHMM/DB_NAME.archive.gz + +for DB_DIR in "$BACKUP_BASE_PATH"/*; do + if [ ! -d "$DB_DIR" ]; then + continue + fi + + DB_NAME=$(basename "$DB_DIR") + echo "------------------------------------------" + echo "Processing database: $DB_NAME" + + # Najdi nejnovější backup pro tuto DB + LATEST_BACKUP=$(ls -td "$DB_DIR"/*/ 2>/dev/null | head -1) + + if [ -z "$LATEST_BACKUP" ]; then + echo "WARNING: No backup found for $DB_NAME" + continue + fi + + BACKUP_FILE="$LATEST_BACKUP/${DB_NAME}.archive.gz" + EXTRACT_DIR="/tmp/mongodb_restore_$$/$DB_NAME" + ERR_FILE="$LATEST_BACKUP/${DB_NAME}.restore.err" + + if [ ! -f "$BACKUP_FILE" ]; then + echo "ERROR: Backup archive not found: $BACKUP_FILE" + continue + fi + + echo "Found backup: $BACKUP_FILE" + echo "Backup date: $(stat -c %y "$BACKUP_FILE" | cut -d' ' -f1-2)" + + # ====================================================== + # EXTRACT BACKUP + # ====================================================== + mkdir -p "$EXTRACT_DIR" + echo "Extracting backup..." + tar -xzf "$BACKUP_FILE" -C "$EXTRACT_DIR" 2> "$ERR_FILE" + EXTRACT_EXIT=$? + + if [ $EXTRACT_EXIT -ne 0 ]; then + echo "ERROR: Failed to extract backup" + cat "$ERR_FILE" + rm -rf "$EXTRACT_DIR" + continue + fi + + # ====================================================== + # RESTORE TO MONGODB + # Restores: indexy, metadata, validators, all collections + # ====================================================== + echo "Restoring $DB_NAME to MongoDB..." + docker exec "$CONTAINER_NAME" mongorestore \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --db="$DB_NAME" \ + --drop \ + --numParallelCollections=4 \ + "$EXTRACT_DIR" \ + 2> "$ERR_FILE" + + RESTORE_EXIT=$? + + # ====================================================== + # VALIDATION + # ====================================================== + if [ $RESTORE_EXIT -eq 0 ] && [ ! -s "$ERR_FILE" ]; then + echo "SUCCESS: $DB_NAME restored successfully" + rm -f "$ERR_FILE" + else + echo "ERROR: Restore failed for database: $DB_NAME" + echo "Exit code: $RESTORE_EXIT" + echo "Error output:" + [ -s "$ERR_FILE" ] && cat "$ERR_FILE" || echo " (no stderr output)" + fi + + # Cleanup extract directory + rm -rf "$EXTRACT_DIR" + +done + +echo "------------------------------------------" +echo "All restore tasks completed at $(date)" +echo "Started at: $START_TS" +set +x diff --git a/MongTower/verify_backup_integrity.sh b/MongTower/verify_backup_integrity.sh new file mode 100644 index 0000000..962dc1f --- /dev/null +++ b/MongTower/verify_backup_integrity.sh @@ -0,0 +1,133 @@ +#!/bin/bash +set -x + +# ========================================================== +# VERIFY BACKUP INTEGRITY +# Kontroluje že se indexy, metadata, a všechny kolekce +# správně obnovily +# ========================================================== + +CONTAINER_NAME="MongoDB" +MONGO_HOST="localhost" +MONGO_PORT="27017" + +# Databases to verify +WHAT_TO_VERIFY=("admin" "fio" "torrents" "OrdinaceDropBoxBackup" "medevio" "kanboard" "medicus" "studie" "puzzle") + +echo "Starting backup integrity verification..." +echo "==========================================" + +ALL_OK=true + +for DB_NAME in "${WHAT_TO_VERIFY[@]}"; do + echo "" + echo "Verifying database: $DB_NAME" + echo "------------------------------------------" + + # ====================================================== + # 1. CHECK IF DB EXISTS + # ====================================================== + DB_EXISTS=$(docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval "db.adminCommand('listDatabases').databases.filter(d => d.name === '$DB_NAME').length > 0") + + if [ "$DB_EXISTS" != "true" ]; then + echo "❌ ERROR: Database $DB_NAME does not exist!" + ALL_OK=false + continue + fi + echo "✅ Database exists: $DB_NAME" + + # ====================================================== + # 2. COUNT COLLECTIONS + # ====================================================== + COLLECTION_COUNT=$(docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval "use $DB_NAME; db.getCollectionNames().length") + + echo "✅ Collections count: $COLLECTION_COUNT" + + # ====================================================== + # 3. LIST ALL COLLECTIONS + # ====================================================== + echo " Collections:" + docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval "use $DB_NAME; db.getCollectionNames().forEach(c => print(' - ' + c))" + + # ====================================================== + # 4. CHECK INDEXES FOR EACH COLLECTION + # ====================================================== + echo " Indexes per collection:" + docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval " +use $DB_NAME; +db.getCollectionNames().forEach(collName => { + const indexes = db[collName].getIndexes(); + print(' ' + collName + ': ' + indexes.length + ' indexes'); + indexes.forEach(idx => { + print(' - ' + JSON.stringify(idx.key)); + }); +}); +" + + # ====================================================== + # 5. CHECK COLLECTION STATS (document count, size) + # ====================================================== + echo " Collection stats:" + docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval " +use $DB_NAME; +db.getCollectionNames().forEach(collName => { + const stats = db[collName].stats(); + print(' ' + collName + ':'); + print(' Documents: ' + stats.count); + print(' Size: ' + (stats.size / 1024 / 1024).toFixed(2) + ' MB'); +}); +" + + # ====================================================== + # 6. VERIFY USERS/ROLES (if admin DB) + # ====================================================== + if [ "$DB_NAME" = "admin" ]; then + echo " Users and Roles:" + USER_COUNT=$(docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval "use admin; db.system.users.find().count()") + + echo " System users: $USER_COUNT" + + ROLE_COUNT=$(docker exec "$CONTAINER_NAME" mongosh \ + --host="$MONGO_HOST" \ + --port="$MONGO_PORT" \ + --quiet \ + --eval "use admin; db.system.roles.find().count()") + + echo " System roles: $ROLE_COUNT" + fi + +done + +echo "" +echo "==========================================" +if [ "$ALL_OK" = true ]; then + echo "✅ All verifications passed!" +else + echo "❌ Some verifications failed!" + exit 1 +fi +set +x diff --git a/MySQLTower/mysqlbackup_with_gzip.py b/MySQLTower/mysqlbackup_with_gzip.py new file mode 100644 index 0000000..841e682 --- /dev/null +++ b/MySQLTower/mysqlbackup_with_gzip.py @@ -0,0 +1,85 @@ +#!/bin/bash +set -x + +# ========================================================== +# CONFIGURATION +# ========================================================== +CONTAINER_NAME="MySQL" +DB_USER="root" +DB_PASS="Vlado9674+" + +UNRAID_NAME="tower" +BASE_PATH="/mnt/user/MySQLBackup" + +KEEP_DAYS=3 + +# Databases to back up +WHAT_TO_BACKUP=("fio" "torrents" "OrdinaceDropBoxBackup" "medevio" "kanboard" "medicus" "studie" "puzzle") + +# ========================================================== +# START +# ========================================================== +echo "Starting scheduled backup for: ${WHAT_TO_BACKUP[*]}" +START_TS=$(date '+%Y-%m-%d %H:%M:%S') + +for DB_NAME in "${WHAT_TO_BACKUP[@]}"; do + echo "------------------------------------------" + echo "Processing database: $DB_NAME" + + DATE=$(date +%Y-%m-%d_%H%M) + FINAL_PATH="$BASE_PATH/$UNRAID_NAME/$DB_NAME/$DATE" + mkdir -p "$FINAL_PATH" + + DUMP_FILE="$FINAL_PATH/$DB_NAME.sql.gz" + ERR_FILE="$FINAL_PATH/$DB_NAME.err" + + # ====================================================== + # DUMP + GZIP + # ====================================================== + docker exec "$CONTAINER_NAME" mysqldump \ + -u"$DB_USER" \ + -p"$DB_PASS" \ + --databases "$DB_NAME" \ + --single-transaction \ + --quick \ + --routines \ + --triggers \ + --events \ + --set-gtid-purged=OFF \ + --default-character-set=utf8mb4 \ + --max_allowed_packet=512M \ + 2> "$ERR_FILE" \ + | gzip -1 > "$DUMP_FILE" + + EXIT_CODE=${PIPESTATUS[0]} + + # ====================================================== + # VALIDATION + # ====================================================== + if [ $EXIT_CODE -eq 0 ] && [ -s "$DUMP_FILE" ] && [ ! -s "$ERR_FILE" ]; then + echo "SUCCESS: $DB_NAME backed up successfully" + echo "Dump size: $(du -h "$DUMP_FILE" | cut -f1)" + rm -f "$ERR_FILE" + else + echo "ERROR: Backup failed for database: $DB_NAME" + echo "Exit code: $EXIT_CODE" + echo "Dump file:" + ls -lh "$DUMP_FILE" 2>/dev/null || echo " (not created)" + echo "Error output:" + [ -s "$ERR_FILE" ] && cat "$ERR_FILE" || echo " (no stderr output)" + fi + + # ====================================================== + # CLEANUP OLD BACKUPS + # ====================================================== + echo "Cleaning up old backups for $DB_NAME..." + find "$BASE_PATH/$UNRAID_NAME/$DB_NAME" \ + -mindepth 1 -maxdepth 1 -type d -mtime +$KEEP_DAYS \ + -exec rm -rf {} \; + +done + +echo "------------------------------------------" +echo "All backup tasks completed at $(date)" +echo "Started at: $START_TS" +set +x