administrator
6bcb721eb4
Sjednocení ukládání příloh do jednoho blob storu na Tower1 (SeaweedFS Filer), content-addressed cesta /mail-attachments/ab/cd/<sha256> přes sdílený seaweed_store.py. Tři zdroje, jeden dedup: - mailstore: mailstore_attachments_poc.py (pole seaweed_attachments[]) - Graph: 3_download_attachments v1.4→v1.5 (upload při stažení nové přílohy; attachments_index dostává seaweed_path/url/synced_at) + backfill graph - JNJ: jnj_tower_ingest v1.2→v1.3 (upload při parse .msg; attachments[] dostává sha256/seaweed_path/url + doc-level seaweed_synced_at) + backfill jnj Backfill skripty jsou idempotentní (batch+resume, --retry-errors). Výpadek SeaweedFS žádnou pipeline neshodí (jen warning, doplní backfill). Ověřeno: 114 726 objektů / 53.3 GB, 0 nesynchronizovaných dokumentů, globální dedup mezi větvemi funguje. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
41 lines
1.2 KiB
Python
41 lines
1.2 KiB
Python
#!/usr/bin/env python3
|
|
# -*- coding: utf-8 -*-
|
|
"""Deploy seaweed zmen do /mnt/user/Scripts na Unraidu (.76) pres SFTP."""
|
|
import sys, paramiko
|
|
from pathlib import Path
|
|
|
|
LOCAL = Path(r"U:\janssen\Python-runner")
|
|
REMOTE = "/mnt/user/Scripts"
|
|
FILES = [
|
|
"seaweed_store.py",
|
|
"seaweed_attachments_backfill_graph.py",
|
|
"3_download_attachments_v1.5.py",
|
|
"0_run_pipeline_v1.0.py",
|
|
]
|
|
|
|
c = paramiko.SSHClient()
|
|
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
|
c.connect("192.168.1.76", username="root", password="7309208104", timeout=10)
|
|
sftp = c.open_sftp()
|
|
|
|
for f in FILES:
|
|
lp = LOCAL / f
|
|
rp = f"{REMOTE}/{f}"
|
|
sftp.put(str(lp), rp)
|
|
print(f"PUT {f} ({lp.stat().st_size} B)")
|
|
|
|
# smaz stary v1.4
|
|
old = f"{REMOTE}/3_download_attachments_v1.4.py"
|
|
try:
|
|
sftp.remove(old)
|
|
print("DEL 3_download_attachments_v1.4.py")
|
|
except IOError as e:
|
|
print(f"(v1.4 uz neni / {e})")
|
|
|
|
# overeni
|
|
_, out, _ = c.exec_command(f"ls -la {REMOTE}/seaweed_store.py "
|
|
f"{REMOTE}/seaweed_attachments_backfill_graph.py "
|
|
f"{REMOTE}/3_download_attachments_v1.5.py")
|
|
print(out.read().decode())
|
|
sftp.close(); c.close()
|