z230
This commit is contained in:
@@ -0,0 +1,29 @@
|
|||||||
|
============================================================
|
||||||
|
SPUSTENI V JNJ — jnj_unsent_probe (diagnostika neodeslani)
|
||||||
|
Zkopiruj cely radek do cmd / PowerShell na JNJ stroji.
|
||||||
|
Skript JEN CTE, nic nezapisuje ani nenahrava.
|
||||||
|
============================================================
|
||||||
|
|
||||||
|
# 1) Sonda na HUSTAKA (vse, vcetne polozek s Message-ID) — klicovy test:
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_unsent_probe_v1.0.py" --to hustak --all
|
||||||
|
|
||||||
|
|
||||||
|
# 2) Sonda na celou kampan ICOTROKINRA (jen podezrele bez Message-ID, 60 dni):
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_unsent_probe_v1.0.py" --subject icotrokinra --days 60
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# TIP: vystup rovnou do souboru (pak mi ho posli):
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
# 1b) hustak -> soubor na plochu:
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_unsent_probe_v1.0.py" --to hustak --all > "%USERPROFILE%\Desktop\probe_hustak.txt" 2>&1
|
||||||
|
|
||||||
|
# 2b) icotrokinra -> soubor na plochu:
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_unsent_probe_v1.0.py" --subject icotrokinra --days 60 > "%USERPROFILE%\Desktop\probe_icotrokinra.txt" 2>&1
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# (VOLITELNE) jnj_mailbox_sync v1.4 — refresh vc. slozky Archive
|
||||||
|
# ============================================================
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_mailbox_sync_v1.5.py" --mode full-update --days 0
|
||||||
@@ -68,8 +68,9 @@ za poslední 60 dní. To je doporučený běh pro „aktualizovat i neodeslané"
|
|||||||
|
|
||||||
## Revert
|
## Revert
|
||||||
|
|
||||||
Stará verze: `Trash/jnj_mailbox_sync_v1.2.py` (bez detekce změny). Server v2.4 zůstává
|
Stará verze: `Trash/jnj_mailbox_sync_v1.3.py` (bez skenování Archive), `Trash/…_v1.2.py`
|
||||||
zpětně kompatibilní (overwrite je opt-in), takže revert na JNJ straně nevyžaduje zásah na serveru.
|
(bez detekce změny). Server v2.4 zůstává zpětně kompatibilní (overwrite je opt-in),
|
||||||
|
takže revert na JNJ straně nevyžaduje zásah na serveru.
|
||||||
|
|
||||||
## Historie
|
## Historie
|
||||||
|
|
||||||
@@ -80,3 +81,5 @@ zpětně kompatibilní (overwrite je opt-in), takže revert na JNJ straně nevy
|
|||||||
položky, které se po zachycení změnily (např. dopsaná chyba `SendAsDenied`), se znovu
|
položky, které se po zachycení změnily (např. dopsaná chyba `SendAsDenied`), se znovu
|
||||||
nahrávají s `overwrite=1`. Nové sloupce `last_mod_time`, `content_uploads`,
|
nahrávají s `overwrite=1`. Nové sloupce `last_mod_time`, `content_uploads`,
|
||||||
`runs.content_updated`. Vyžaduje app.py ≥ v2.4.
|
`runs.content_updated`. Vyžaduje app.py ≥ v2.4.
|
||||||
|
- **1.4.0** — + skenování složky **Archive** v primární schránce (hledá se podle jména
|
||||||
|
pod kořenem schránky, ne přes default folder; Online Archive se neskenuje).
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# jnj_mailbox_sync v1.5.0
|
||||||
|
|
||||||
|
**Soubor:** `jnj_mailbox_sync_v1.5.py`
|
||||||
|
**Datum:** 2026-06-16
|
||||||
|
**Autor:** vladimir.buzalka
|
||||||
|
**Běží:** JNJ stroj (Outlook MAPI), Python z Thonny.
|
||||||
|
|
||||||
|
## Co to je
|
||||||
|
|
||||||
|
Synchronizace JNJ Outlooku (MAPI) → osobní schránka (přes msgreceiver) + bookkeeping
|
||||||
|
v SQLite (`C:\Users\vbuzalka\SQLITE\jnjemails.db`). Sleduje přesuny e-mailů mezi
|
||||||
|
složkami a příznak „už není ve schránce" — bez opětovného přenosu těla.
|
||||||
|
Skenované složky: **Inbox + Sent Items + Deleted Items + Archive** (vč. podsložek).
|
||||||
|
|
||||||
|
## Novinka v1.5 — provenance verze skriptu na úrovni entry
|
||||||
|
|
||||||
|
Do tabulky `messages` přidány dva sloupce (jen pro náhled, **Tower je nezpracovává** —
|
||||||
|
nejsou v mirroru do `jnj_messages`):
|
||||||
|
|
||||||
|
| Sloupec | Význam |
|
||||||
|
|---|---|
|
||||||
|
| `captured_by_version` | verze skriptu, která entry **poprvé zachytila/odeslala** (set při INSERT) |
|
||||||
|
| `last_upload_version` | verze, která naposledy **re-uploadla tělo** (set při INSERT i při re-uploadu) |
|
||||||
|
|
||||||
|
Smysl: kdykoliv se podívat (`jnjemails` SQL), kterou verzí byl daný e-mail přenesen.
|
||||||
|
**Pravidlo:** při jakékoliv změně skriptu vždy bumpni verzi (`SCRIPT_VERSION`) — jinak
|
||||||
|
tahle stopa ztrácí smysl. Migrace přes `ALTER TABLE` (staré řádky = NULL).
|
||||||
|
|
||||||
|
## Novinka v1.4 — skenování složky Archive (primární schránka)
|
||||||
|
|
||||||
|
Přidána složka **Archive** (jednoklikové archivování v Outlooku) v **primární** schránce.
|
||||||
|
Archive **není** default folder, takže se hledá podle jména `"Archive"` pod kořenem
|
||||||
|
primární schránky (`Inbox.Parent`) a přidává se do `scanned_roots` (aby se její položky
|
||||||
|
nehodnotily jako „opustilo schránku"). **Online Archive** (samostatný store) se i nadále
|
||||||
|
**neskenuje**. Řeší případy, kdy odeslaná kopie skončila v Archive (jinak chyběla domácímu
|
||||||
|
přehledu i párování dvojčat).
|
||||||
|
|
||||||
|
## Novinka v1.3 — detekce změny obsahu (re-upload změněného e-mailu)
|
||||||
|
|
||||||
|
**Problém:** e-mail **bez Message-ID** (typicky **NEODESLANÝ** Sent kvůli `SendAsDenied`,
|
||||||
|
nebo čerstvě odeslaný, kde Exchange ještě nedoplnil Message-ID) má **stabilní EntryID**.
|
||||||
|
Když do něj Outlook **po zachycení** dopíše chybu odeslání, obsah se změní, ale identita
|
||||||
|
(`entryid:<EID>`) zůstane → starý sync to vyhodnotil jako „známé, beze změny" a
|
||||||
|
aktualizovaný (chybový) e-mail už domů **nepřenesl**. Naproti tomu úspěšně odeslaný
|
||||||
|
e-mail dostane **nové EntryID + Message-ID**, takže se zachytil jako nový. Vznikla
|
||||||
|
asymetrie: failed-update se ztrácel.
|
||||||
|
|
||||||
|
**Řešení:** identita zůstává (Message-ID / `entryid:`), ale navíc se sleduje **verzní otisk**
|
||||||
|
= `PR_LAST_MODIFICATION_TIME` (`0x30080040`). U **známé položky bez Message-ID**
|
||||||
|
(`mid` začíná `entryid:`) se otisk porovná; když se posunul, e-mail se znovu uloží
|
||||||
|
(`SaveAs`) a nahraje s `overwrite=1` → server přepíše původní `.msg` na místě → Tower ho
|
||||||
|
přeparsuje → dokument v Mongu se aktualizuje (vč. těla s chybou).
|
||||||
|
|
||||||
|
- Hlídání je **levné** — druhé čtení property jen u známých no-ID položek (desítky kusů);
|
||||||
|
položky s Message-ID jsou finalizované a nesledují se.
|
||||||
|
- Re-upload běží jen v režimech, které smějí nahrávat (**capture, full-update**), a posílá se
|
||||||
|
s `folder=""` → server **nedělá** Graph re-import (žádný duplikát v Graph zrcadle).
|
||||||
|
- **Vyžaduje msgreceiver app.py ≥ v2.4** (overwrite na `/upload`). Bez něj se re-upload chová
|
||||||
|
jako starý skip (nepřepíše, ale nic nerozbije) — pořadí nasazení server → JNJ bez výpadku.
|
||||||
|
|
||||||
|
## Nové sloupce SQLite
|
||||||
|
|
||||||
|
- `messages.last_mod_time` — PR_LAST_MODIFICATION_TIME při posledním zachycení (otisk).
|
||||||
|
- `messages.content_uploads` — kolikrát se tělo nahrálo (1 = jen první zachycení).
|
||||||
|
- `runs.content_updated` — kolik e-mailů se v běhu re-uploadlo kvůli změně obsahu.
|
||||||
|
|
||||||
|
(Migrace přes stávající `ALTER TABLE` smyčku — staré `jnjemails.db` se doplní automaticky.)
|
||||||
|
|
||||||
|
## Argumenty
|
||||||
|
|
||||||
|
`--mode {capture,update-paths,full-update}` (default capture), `--days N`
|
||||||
|
(0 = celé), `--dry-run`, `--limit N`, `--no-db-upload`.
|
||||||
|
|
||||||
|
## Spouštění (JNJ stroj, plné cesty)
|
||||||
|
|
||||||
|
```
|
||||||
|
"C:\Users\vbuzalka\AppData\Local\Programs\Thonny\python.exe" "c:\Users\vbuzalka\OneDrive - JNJ\##JNJPrenos\Python\jnj_mailbox_sync_v1.5.py" --mode full-update --days 60
|
||||||
|
```
|
||||||
|
|
||||||
|
`full-update --days 60` = dorovná chybějící + **re-uploadne změněné** (chybové) Sent položky
|
||||||
|
za poslední 60 dní. To je doporučený běh pro „aktualizovat i neodeslané".
|
||||||
|
|
||||||
|
## Revert
|
||||||
|
|
||||||
|
Stará verze: `Trash/jnj_mailbox_sync_v1.4.py` (bez provenance sloupců),
|
||||||
|
`…_v1.3.py` (bez skenování Archive), `…_v1.2.py` (bez detekce změny). Server v2.4
|
||||||
|
zůstává zpětně kompatibilní (overwrite je opt-in), takže revert na JNJ straně
|
||||||
|
nevyžaduje zásah na serveru.
|
||||||
|
|
||||||
|
## Historie
|
||||||
|
|
||||||
|
- **1.0.0** — režimy capture/update-paths/full-update, sledování přesunů, updated_at.
|
||||||
|
- **1.1.0** — + Deleted Items do skenovaných složek.
|
||||||
|
- **1.2.0** — upload SQLite komprimován (lzma/xz max) + šifrován (Fernet) → `.db.xz.enc`.
|
||||||
|
- **1.3.0** — + detekce změny obsahu přes `PR_LAST_MODIFICATION_TIME`: známé no-ID
|
||||||
|
položky, které se po zachycení změnily (např. dopsaná chyba `SendAsDenied`), se znovu
|
||||||
|
nahrávají s `overwrite=1`. Nové sloupce `last_mod_time`, `content_uploads`,
|
||||||
|
`runs.content_updated`. Vyžaduje app.py ≥ v2.4.
|
||||||
|
- **1.4.0** — + skenování složky **Archive** v primární schránce (hledá se podle jména
|
||||||
|
pod kořenem schránky, ne přes default folder; Online Archive se neskenuje).
|
||||||
|
- **1.5.0** — + provenance verze na úrovni entry: sloupce `captured_by_version`
|
||||||
|
a `last_upload_version` (jen náhled, Tower nezpracovává).
|
||||||
@@ -0,0 +1,707 @@
|
|||||||
|
"""
|
||||||
|
jnj_mailbox_sync v1.5
|
||||||
|
Nazev: jnj_mailbox_sync_v1.5.py
|
||||||
|
Verze: 1.5.0
|
||||||
|
Datum: 2026-06-16
|
||||||
|
Autor: vladimir.buzalka
|
||||||
|
|
||||||
|
Popis:
|
||||||
|
Synchronizace JNJ Outlooku (MAPI) -> osobni schranka + bookkeeping v SQLite.
|
||||||
|
Nasledník inbox_full_sync_v1.1 / jnj_mailbox_sync_v1.2. Sleduje PRESUN emailu
|
||||||
|
mezi slozkami a priznak "uz neni ve schrance" — BEZ opetovneho prenosu tela.
|
||||||
|
|
||||||
|
Scope: primarni schranka, Inbox + Sent Items + Deleted Items + Archive
|
||||||
|
vcetne vsech podsložek. Slozka Archive (jednoklikove archivovani v Outlooku)
|
||||||
|
NENI default folder — hleda se podle jmena pod korenem primarni schranky.
|
||||||
|
Online Archive (samostatny store) se i nadale NEskenuje.
|
||||||
|
|
||||||
|
Identita emailu = Internet Message-ID (stabilni pres presuny). Kdyz Message-ID
|
||||||
|
chybi (typicky cerstve odeslane / NEODESLANE Sent polozky — Exchange ho doplni
|
||||||
|
az po skutecnem transportu), pouzije se fallback "entryid:<EntryID>".
|
||||||
|
|
||||||
|
Sloupce cest v SQLite:
|
||||||
|
folder = cesta pri PRVNIM zachyceni (historie, neprepisuje se)
|
||||||
|
jnj_folder = AKTUALNI ziva cesta (prepisuje se pri presunu)
|
||||||
|
updated_at se bumpne pri insertu i kazde zmene — watermark pro domaci sync.
|
||||||
|
|
||||||
|
NOVINKA v1.3 — DETEKCE ZMENY OBSAHU (re-upload zmeneneho emailu)
|
||||||
|
Problem: e-mail bez Message-ID (napr. NEODESLANY Sent kvuli SendAsDenied) ma
|
||||||
|
STABILNI EntryID. Kdyz do nej Outlook PO zachyceni dopise chybu odeslani,
|
||||||
|
obsah se zmeni, ale identita (entryid:<EID>) zustane — stary sync to vyhodnotil
|
||||||
|
jako "zname, beze zmeny" a aktualizovany (chybovy) e-mail uz domu NEPRENESL.
|
||||||
|
Naproti tomu uspesne odeslany e-mail dostane NOVE EntryID + Message-ID, takze
|
||||||
|
se zachytil jako novy. Vznikla asymetrie: failed-update se ztracel.
|
||||||
|
|
||||||
|
Reseni: identita zustava (Message-ID / entryid:), ale navic se sleduje VERZNI
|
||||||
|
OTISK = PR_LAST_MODIFICATION_TIME (0x30080040). U ZNAMEHO emailu BEZ Message-ID
|
||||||
|
(mid zacina "entryid:") se otisk porovna; kdyz se posunul, e-mail se znovu
|
||||||
|
ulozi (SaveAs) a nahraje s priznakem overwrite=true (server prepise puvodni
|
||||||
|
.msg na miste -> Tower ho preparsuje -> dokument v Mongu se aktualizuje, vc.
|
||||||
|
tela s chybou). Tim doteche i "zmeneny hustak". Hlidani je levne — druhe cteni
|
||||||
|
property jen u znamych no-ID polozek (desitky kusu); polozky s Message-ID jsou
|
||||||
|
finalizovane a nesleduji se.
|
||||||
|
|
||||||
|
Re-upload bezi jen v rezimech, ktere smeji nahravat (capture, full-update),
|
||||||
|
a posila se BEZ folderu (folder="") => server NEdela Graph re-import (zadny
|
||||||
|
duplikat v Graph zrcadle); jen prepise /msgs soubor pro Tower parse.
|
||||||
|
|
||||||
|
Vyzaduje msgreceiver app.py >= v2.4 (overwrite na /upload). Bez nej se
|
||||||
|
re-upload chova jako "exists" (stary skip) — neprepise, ale nic nerozbije.
|
||||||
|
|
||||||
|
Upload SQLite (zustava z v1.2): DB se pred odeslanim KOMPRIMUJE (lzma/xz, max) a
|
||||||
|
SIFRUJE (Fernet, klic z TOKENu) a nahrava jako .db.xz.enc.
|
||||||
|
|
||||||
|
Rezimy (--mode):
|
||||||
|
capture (default) Projde cely Inbox+Sent+Deleted, nove emaily ulozi a
|
||||||
|
nahraje + NOVE re-uploadne zmenene znamé no-ID polozky.
|
||||||
|
Okno --days se IGNORUJE (bere VSE).
|
||||||
|
update-paths Jen METADATA cesty/precteno + "opustilo schranku". NIC nenahrava
|
||||||
|
(ani re-upload).
|
||||||
|
full-update update-paths + dorovna chybejici (SaveAs+upload) + re-upload
|
||||||
|
zmenenych znamých no-ID polozek.
|
||||||
|
|
||||||
|
Argumenty:
|
||||||
|
--mode {capture,update-paths,full-update} default capture
|
||||||
|
--days N velikost okna ve dnech (default 30). 0 = cely Inbox+Sent.
|
||||||
|
--dry-run NIC nezapise/nenahraje, jen vypise co by udelal.
|
||||||
|
--limit N zpracovat max N polozek (rychly test).
|
||||||
|
--no-db-upload na konci nenahravat SQLite na server.
|
||||||
|
|
||||||
|
Spousteni:
|
||||||
|
# Refresh poslednich 60 dni + zachytit zmenene (chybove) Sent polozky:
|
||||||
|
python jnj_mailbox_sync_v1.3.py --mode full-update --days 60
|
||||||
|
|
||||||
|
Zavislosti:
|
||||||
|
pywin32, requests, cryptography, sqlite3 + lzma (stdlib).
|
||||||
|
Python 3.10+, Windows, Outlook musi byt spusteny a prihlaseny.
|
||||||
|
|
||||||
|
Historie verzi:
|
||||||
|
1.0.0 2026-06-09 Rezimy capture/update-paths/full-update, sledovani presunu,
|
||||||
|
not_in_mailbox_anymore, updated_at watermark.
|
||||||
|
1.1.0 2026-06-10 + Deleted Items do SYNC_FOLDERS.
|
||||||
|
1.2.0 2026-06-10 Upload SQLite komprimovan (lzma) + sifrovan (Fernet) ->
|
||||||
|
.db.xz.enc. Vyzaduje app.py >= v2.1.
|
||||||
|
1.3.0 2026-06-16 + DETEKCE ZMENY OBSAHU pres PR_LAST_MODIFICATION_TIME:
|
||||||
|
zname no-ID polozky (entryid:), ktere se po zachyceni
|
||||||
|
zmenily (napr. dopsana chyba SendAsDenied), se znovu
|
||||||
|
nahravaji s overwrite=true. Nove SQLite sloupce
|
||||||
|
last_mod_time, content_uploads; runs.content_updated.
|
||||||
|
Vyzaduje app.py >= v2.4 (overwrite na /upload).
|
||||||
|
1.4.0 2026-06-16 + skenovani slozky Archive v PRIMARNI schrance (ne Online
|
||||||
|
Archive). Archive neni default folder -> hleda se podle
|
||||||
|
jmena ("Archive") pod korenem primarni schranky a pridava
|
||||||
|
se do scanned_roots (aby se jeji polozky nehodnotily jako
|
||||||
|
GONE). Resi pripady, kdy odeslana kopie skoncila v Archive.
|
||||||
|
1.5.0 2026-06-16 + provenance verze skriptu na urovni entry: nove SQLite
|
||||||
|
sloupce captured_by_version (verze, ktera entry POPRVE
|
||||||
|
zachytila) a last_upload_version (verze, ktera naposledy
|
||||||
|
re-uploadla telo). JEN pro nahled — Tower je NEzpracovava
|
||||||
|
(nejsou v mirroru do jnj_messages). Pravidlo: pri kazde
|
||||||
|
zmene skriptu verzovat, aby tahle stopa byla uzitecna.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import lzma
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import win32com.client
|
||||||
|
import requests
|
||||||
|
import urllib3
|
||||||
|
from cryptography.fernet import Fernet
|
||||||
|
|
||||||
|
if hasattr(sys.stdout, "reconfigure"):
|
||||||
|
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||||
|
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
|
||||||
|
|
||||||
|
# ─── KONFIGURACE ──────────────────────────────────────────────────────────────
|
||||||
|
TOKEN = "13e1bb01-9fd5-44a8-8ce9-4ee27133d340"
|
||||||
|
UPLOAD_URL = "https://msgs.buzalka.cz/upload"
|
||||||
|
DB_UPLOAD_URL = "https://msgs.buzalka.cz/upload-db"
|
||||||
|
DB_PATH = r"C:\Users\vbuzalka\SQLITE\jnjemails.db"
|
||||||
|
LOG_PATH = r"C:\Users\vbuzalka\SQLITE\jnj_mailbox_sync_errors.log"
|
||||||
|
PR_INTERNET_MESSAGE_ID = "http://schemas.microsoft.com/mapi/proptag/0x1035001E"
|
||||||
|
PR_LAST_MOD_TIME = "http://schemas.microsoft.com/mapi/proptag/0x30080040" # PR_LAST_MODIFICATION_TIME
|
||||||
|
SCRIPT_NAME = "jnj_mailbox_sync"
|
||||||
|
SCRIPT_VERSION = "1.5.0"
|
||||||
|
|
||||||
|
# olFolderInbox=6, olFolderSentMail=5, olFolderDeletedItems=3
|
||||||
|
SYNC_FOLDERS = [(6, "Inbox"), (5, "Sent Items"), (3, "Deleted Items")]
|
||||||
|
OLSAVE_MSG = 3 # OlSaveAsType.olMSG
|
||||||
|
|
||||||
|
# Sifrovaci klic odvozeny z TOKENu (stejny algoritmus jako server)
|
||||||
|
_FERNET = Fernet(base64.urlsafe_b64encode(hashlib.sha256(TOKEN.encode()).digest()))
|
||||||
|
|
||||||
|
logging.basicConfig(
|
||||||
|
filename=LOG_PATH,
|
||||||
|
level=logging.ERROR,
|
||||||
|
format="%(asctime)s | %(message)s",
|
||||||
|
datefmt="%Y-%m-%d %H:%M:%S",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
# ──────────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
# ─── SQLite ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def init_db(conn):
|
||||||
|
conn.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS messages (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
message_id TEXT NOT NULL,
|
||||||
|
subject TEXT,
|
||||||
|
sender TEXT,
|
||||||
|
received_at TEXT,
|
||||||
|
folder TEXT,
|
||||||
|
source TEXT,
|
||||||
|
uploaded_at TEXT DEFAULT (datetime('now')),
|
||||||
|
entry_id TEXT,
|
||||||
|
graph_id TEXT,
|
||||||
|
is_read INTEGER DEFAULT 0,
|
||||||
|
jnj_folder TEXT,
|
||||||
|
not_in_mailbox_anymore INTEGER DEFAULT 0,
|
||||||
|
left_mailbox_at TEXT,
|
||||||
|
updated_at TEXT,
|
||||||
|
last_mod_time TEXT,
|
||||||
|
content_uploads INTEGER DEFAULT 1,
|
||||||
|
captured_by_version TEXT,
|
||||||
|
last_upload_version TEXT
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
conn.execute("CREATE UNIQUE INDEX IF NOT EXISTS idx_message_id ON messages(message_id)")
|
||||||
|
|
||||||
|
conn.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS runs (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
script TEXT NOT NULL,
|
||||||
|
version TEXT,
|
||||||
|
started_at TEXT NOT NULL,
|
||||||
|
finished_at TEXT,
|
||||||
|
mode TEXT,
|
||||||
|
window_days INTEGER,
|
||||||
|
dry_run INTEGER DEFAULT 0,
|
||||||
|
found INTEGER DEFAULT 0,
|
||||||
|
new_captured INTEGER DEFAULT 0,
|
||||||
|
path_updated INTEGER DEFAULT 0,
|
||||||
|
read_updated INTEGER DEFAULT 0,
|
||||||
|
returned INTEGER DEFAULT 0,
|
||||||
|
left_mailbox INTEGER DEFAULT 0,
|
||||||
|
content_updated INTEGER DEFAULT 0,
|
||||||
|
skipped INTEGER DEFAULT 0,
|
||||||
|
errors INTEGER DEFAULT 0
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
|
||||||
|
conn.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS log (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
run_id INTEGER REFERENCES runs(id),
|
||||||
|
level TEXT NOT NULL,
|
||||||
|
event TEXT NOT NULL,
|
||||||
|
subject TEXT,
|
||||||
|
folder TEXT,
|
||||||
|
graph_id TEXT,
|
||||||
|
detail TEXT,
|
||||||
|
created_at TEXT DEFAULT (datetime('now'))
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
conn.execute("CREATE INDEX IF NOT EXISTS idx_log_run_id ON log(run_id)")
|
||||||
|
|
||||||
|
# Migrace existujici jnjemails.db — pridej chybejici sloupce
|
||||||
|
for col, ddl in [
|
||||||
|
("entry_id", "TEXT"), ("graph_id", "TEXT"), ("is_read", "INTEGER DEFAULT 0"),
|
||||||
|
("jnj_folder", "TEXT"), ("not_in_mailbox_anymore", "INTEGER DEFAULT 0"),
|
||||||
|
("left_mailbox_at", "TEXT"), ("updated_at", "TEXT"),
|
||||||
|
("last_mod_time", "TEXT"), ("content_uploads", "INTEGER DEFAULT 1"),
|
||||||
|
("captured_by_version", "TEXT"), ("last_upload_version", "TEXT"),
|
||||||
|
]:
|
||||||
|
try:
|
||||||
|
conn.execute(f"ALTER TABLE messages ADD COLUMN {col} {ddl}")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
for col, ddl in [
|
||||||
|
("mode", "TEXT"), ("window_days", "INTEGER"), ("dry_run", "INTEGER DEFAULT 0"),
|
||||||
|
("found", "INTEGER DEFAULT 0"), ("new_captured", "INTEGER DEFAULT 0"),
|
||||||
|
("path_updated", "INTEGER DEFAULT 0"), ("read_updated", "INTEGER DEFAULT 0"),
|
||||||
|
("returned", "INTEGER DEFAULT 0"), ("left_mailbox", "INTEGER DEFAULT 0"),
|
||||||
|
("content_updated", "INTEGER DEFAULT 0"),
|
||||||
|
]:
|
||||||
|
try:
|
||||||
|
conn.execute(f"ALTER TABLE runs ADD COLUMN {col} {ddl}")
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
conn.execute("CREATE INDEX IF NOT EXISTS idx_updated_at ON messages(updated_at)")
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def start_run(conn, mode, days, dry):
|
||||||
|
cur = conn.execute(
|
||||||
|
"""INSERT INTO runs (script, version, started_at, mode, window_days, dry_run)
|
||||||
|
VALUES (?, ?, datetime('now'), ?, ?, ?)""",
|
||||||
|
(SCRIPT_NAME, SCRIPT_VERSION, mode, days, 1 if dry else 0),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
return cur.lastrowid
|
||||||
|
|
||||||
|
|
||||||
|
def finish_run(conn, run_id, stats):
|
||||||
|
conn.execute(
|
||||||
|
"""UPDATE runs SET finished_at=datetime('now'),
|
||||||
|
found=?, new_captured=?, path_updated=?, read_updated=?,
|
||||||
|
returned=?, left_mailbox=?, content_updated=?, skipped=?, errors=?
|
||||||
|
WHERE id=?""",
|
||||||
|
(stats["found"], stats["new_captured"], stats["path_updated"],
|
||||||
|
stats["read_updated"], stats["returned"], stats["left_mailbox"],
|
||||||
|
stats["content_updated"], stats["skipped"], stats["errors"], run_id),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def db_log(conn, run_id, level, event, subject=None, folder=None, graph_id=None, detail=None):
|
||||||
|
conn.execute(
|
||||||
|
"""INSERT INTO log (run_id, level, event, subject, folder, graph_id, detail)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?)""",
|
||||||
|
(run_id, level, event, subject, folder, graph_id, detail),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def info(conn, run_id, event, **kw):
|
||||||
|
db_log(conn, run_id, "INFO", event, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def error(conn, run_id, event, **kw):
|
||||||
|
db_log(conn, run_id, "ERROR", event, **kw)
|
||||||
|
|
||||||
|
|
||||||
|
def db_get(conn, mid):
|
||||||
|
cur = conn.execute(
|
||||||
|
"""SELECT message_id, folder, jnj_folder, is_read, not_in_mailbox_anymore,
|
||||||
|
last_mod_time, content_uploads
|
||||||
|
FROM messages WHERE message_id=?""", (mid,))
|
||||||
|
r = cur.fetchone()
|
||||||
|
if not r:
|
||||||
|
return None
|
||||||
|
return {"message_id": r[0], "folder": r[1], "jnj_folder": r[2],
|
||||||
|
"is_read": r[3], "not_in_mailbox_anymore": r[4],
|
||||||
|
"last_mod_time": r[5], "content_uploads": r[6]}
|
||||||
|
|
||||||
|
|
||||||
|
def apply_update(conn, mid, changes):
|
||||||
|
sets, vals = [], []
|
||||||
|
for k, v in changes.items():
|
||||||
|
sets.append(f"{k}=?")
|
||||||
|
vals.append(v)
|
||||||
|
sets.append("updated_at=datetime('now')")
|
||||||
|
vals.append(mid)
|
||||||
|
conn.execute(f"UPDATE messages SET {', '.join(sets)} WHERE message_id=?", vals)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
|
# ─── Outlook / prenos ────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def get_mid(item) -> str:
|
||||||
|
try:
|
||||||
|
mid = item.PropertyAccessor.GetProperty(PR_INTERNET_MESSAGE_ID)
|
||||||
|
except Exception:
|
||||||
|
mid = None
|
||||||
|
return mid or f"entryid:{item.EntryID}"
|
||||||
|
|
||||||
|
|
||||||
|
def get_lastmod(item):
|
||||||
|
"""PR_LAST_MODIFICATION_TIME jako ISO string (verzni otisk). None pri chybe."""
|
||||||
|
try:
|
||||||
|
v = item.PropertyAccessor.GetProperty(PR_LAST_MOD_TIME)
|
||||||
|
if v is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return v.isoformat()
|
||||||
|
except Exception:
|
||||||
|
return str(v)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def upload_msg(msg_path, filename, folder="", overwrite=False):
|
||||||
|
with open(msg_path, "rb") as f:
|
||||||
|
encrypted = _FERNET.encrypt(f.read())
|
||||||
|
enc_filename = Path(filename).stem + ".emsg"
|
||||||
|
data = {"folder": folder}
|
||||||
|
if overwrite:
|
||||||
|
data["overwrite"] = "1"
|
||||||
|
resp = requests.post(
|
||||||
|
UPLOAD_URL,
|
||||||
|
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||||
|
files={"file": (enc_filename, encrypted, "application/octet-stream")},
|
||||||
|
data=data,
|
||||||
|
timeout=60,
|
||||||
|
)
|
||||||
|
if not resp.ok:
|
||||||
|
raise requests.HTTPError(f"{resp.status_code} {resp.reason} | {resp.text[:200]}")
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
def save_and_upload(item, folder="", overwrite=False):
|
||||||
|
"""SaveAs do temp -> upload (sifrovane). Vraci (filename, server_json)."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
safe = f"{item.EntryID[-20:]}.msg"
|
||||||
|
p = Path(tmp) / safe
|
||||||
|
item.SaveAs(str(p), OLSAVE_MSG)
|
||||||
|
result = upload_msg(p, safe, folder, overwrite=overwrite)
|
||||||
|
return safe, result
|
||||||
|
|
||||||
|
|
||||||
|
def capture_new(conn, run_id, item, mid, current, is_read, subject, stats):
|
||||||
|
"""Novy email: SaveAs -> upload -> insert. Vraci True pri uspechu."""
|
||||||
|
_, result = save_and_upload(item, current, overwrite=False)
|
||||||
|
graph_id = result.get("graph_id")
|
||||||
|
lm = get_lastmod(item)
|
||||||
|
try:
|
||||||
|
received = item.ReceivedTime.isoformat() if item.ReceivedTime else None
|
||||||
|
except Exception:
|
||||||
|
received = None
|
||||||
|
try:
|
||||||
|
sender = item.SenderEmailAddress or ""
|
||||||
|
except Exception:
|
||||||
|
sender = ""
|
||||||
|
conn.execute(
|
||||||
|
"""INSERT OR IGNORE INTO messages
|
||||||
|
(message_id, subject, sender, received_at, folder, source,
|
||||||
|
entry_id, graph_id, is_read, jnj_folder,
|
||||||
|
not_in_mailbox_anymore, updated_at, last_mod_time, content_uploads,
|
||||||
|
captured_by_version, last_upload_version)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, datetime('now'), ?, 1, ?, ?)""",
|
||||||
|
(mid, subject, sender, received, current, SCRIPT_NAME,
|
||||||
|
item.EntryID, graph_id, is_read, current, lm,
|
||||||
|
SCRIPT_VERSION, SCRIPT_VERSION),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
info(conn, run_id, "captured", subject=subject, folder=current, graph_id=graph_id)
|
||||||
|
print(f" NEW | {subject[:70]}")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def reupload_changed(item, current):
|
||||||
|
"""Znovu nahraj zmeneny (znamy) email — overwrite na serveru.
|
||||||
|
Folder="" => server NEdela Graph re-import (jen prepise /msgs soubor)."""
|
||||||
|
save_and_upload(item, folder="", overwrite=True)
|
||||||
|
|
||||||
|
|
||||||
|
def process_item(conn, run_id, item, current, stats, seen, mode, dry):
|
||||||
|
try:
|
||||||
|
mid = get_mid(item)
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
seen.add(mid)
|
||||||
|
stats["found"] += 1
|
||||||
|
|
||||||
|
try:
|
||||||
|
is_read = 0 if item.UnRead else 1
|
||||||
|
except Exception:
|
||||||
|
is_read = 0
|
||||||
|
subject = str(getattr(item, "Subject", "") or "")
|
||||||
|
|
||||||
|
row = db_get(conn, mid)
|
||||||
|
|
||||||
|
# ── Novy email (neni v DB) ────────────────────────────────────────────
|
||||||
|
if row is None:
|
||||||
|
if mode in ("capture", "full-update"):
|
||||||
|
if dry:
|
||||||
|
stats["new_captured"] += 1
|
||||||
|
print(f" NEW* | {subject[:70]}")
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
if capture_new(conn, run_id, item, mid, current, is_read, subject, stats):
|
||||||
|
stats["new_captured"] += 1
|
||||||
|
except Exception as e:
|
||||||
|
stats["errors"] += 1
|
||||||
|
error(conn, run_id, "capture_error", subject=subject, folder=current, detail=str(e))
|
||||||
|
print(f" CHYBA NEW | {subject[:50]} | {e}")
|
||||||
|
else: # update-paths — telo nemame, nelze dorovnat
|
||||||
|
stats["new_uncaptured"] += 1
|
||||||
|
return
|
||||||
|
|
||||||
|
# ── Znamy email — porovnej zmeny ──────────────────────────────────────
|
||||||
|
changes = {}
|
||||||
|
current_known = row.get("jnj_folder") or row.get("folder")
|
||||||
|
if current_known != current:
|
||||||
|
changes["jnj_folder"] = current
|
||||||
|
stats["path_updated"] += 1
|
||||||
|
if row.get("is_read") != is_read:
|
||||||
|
changes["is_read"] = is_read
|
||||||
|
stats["read_updated"] += 1
|
||||||
|
if row.get("not_in_mailbox_anymore"):
|
||||||
|
changes["not_in_mailbox_anymore"] = 0
|
||||||
|
changes["left_mailbox_at"] = None
|
||||||
|
stats["returned"] += 1
|
||||||
|
|
||||||
|
# ── DETEKCE ZMENY OBSAHU (v1.3) ───────────────────────────────────────
|
||||||
|
# Jen u znamých polozek BEZ Message-ID (mid zacina "entryid:") — tam ma
|
||||||
|
# EntryID stabilni a obsah se muze zmenit pod stejnou identitou (napr.
|
||||||
|
# dopsana chyba SendAsDenied). Polozky s Message-ID jsou finalizovane.
|
||||||
|
# Re-upload jen v rezimech, ktere smeji nahravat, a ne v dry-run.
|
||||||
|
if (mode in ("capture", "full-update") and mid.startswith("entryid:")):
|
||||||
|
cur_lm = get_lastmod(item)
|
||||||
|
if cur_lm and cur_lm != row.get("last_mod_time"):
|
||||||
|
stats["content_updated"] += 1
|
||||||
|
if dry:
|
||||||
|
# DRY-RUN: jen napocitej + ukaz, NIC nenahrava (nahled pred ostrym behem)
|
||||||
|
print(f" REUP* | {subject[:55]} | obsah zmenen -> by se re-uploadl")
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
reupload_changed(item, current)
|
||||||
|
changes["last_mod_time"] = cur_lm
|
||||||
|
changes["content_uploads"] = (row.get("content_uploads") or 1) + 1
|
||||||
|
changes["last_upload_version"] = SCRIPT_VERSION
|
||||||
|
print(f" REUP | {subject[:55]} | obsah zmenen -> re-upload")
|
||||||
|
info(conn, run_id, "content_reupload", subject=subject, folder=current,
|
||||||
|
detail=f"last_mod {row.get('last_mod_time')} -> {cur_lm}")
|
||||||
|
except Exception as e:
|
||||||
|
stats["content_updated"] -= 1
|
||||||
|
stats["errors"] += 1
|
||||||
|
error(conn, run_id, "reupload_error", subject=subject, folder=current, detail=str(e))
|
||||||
|
print(f" CHYBA REUP | {subject[:50]} | {e}")
|
||||||
|
|
||||||
|
if changes:
|
||||||
|
if not dry:
|
||||||
|
apply_update(conn, mid, changes)
|
||||||
|
what = []
|
||||||
|
if "jnj_folder" in changes:
|
||||||
|
what.append(f"-> {current}")
|
||||||
|
if "is_read" in changes:
|
||||||
|
what.append("precteno" if is_read else "neprecteno")
|
||||||
|
if "not_in_mailbox_anymore" in changes:
|
||||||
|
what.append("vraceno do schranky")
|
||||||
|
if "last_mod_time" in changes:
|
||||||
|
what.append("obsah aktualizovan")
|
||||||
|
marker = "*" if dry else " "
|
||||||
|
print(f" UPD{marker} | {subject[:55]} | {', '.join(what)}")
|
||||||
|
info(conn, run_id, "path_update", subject=subject, folder=current, detail="; ".join(what))
|
||||||
|
else:
|
||||||
|
stats["skipped"] += 1
|
||||||
|
|
||||||
|
|
||||||
|
def walk(conn, run_id, folder, folder_path, cutoff_local, stats, seen, mode, dry, limit):
|
||||||
|
current = f"{folder_path}/{folder.Name}"
|
||||||
|
try:
|
||||||
|
items = folder.Items
|
||||||
|
if cutoff_local is not None:
|
||||||
|
restrict = ("@SQL=\"urn:schemas:httpmail:datereceived\" >= '%s'"
|
||||||
|
% cutoff_local.strftime("%Y/%m/%d %H:%M:%S"))
|
||||||
|
items = items.Restrict(restrict)
|
||||||
|
items.Sort("[ReceivedTime]", True) # newest first
|
||||||
|
except Exception as e:
|
||||||
|
print(f" CHYBA slozka {current}: {e}")
|
||||||
|
error(conn, run_id, "folder_error", folder=current, detail=str(e))
|
||||||
|
return
|
||||||
|
|
||||||
|
n = 0
|
||||||
|
for item in items:
|
||||||
|
if limit and stats["found"] >= limit:
|
||||||
|
break
|
||||||
|
try:
|
||||||
|
if not str(getattr(item, "MessageClass", "")).upper().startswith("IPM.NOTE"):
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
process_item(conn, run_id, item, current, stats, seen, mode, dry)
|
||||||
|
n += 1
|
||||||
|
|
||||||
|
print(f" {current}: {n} polozek")
|
||||||
|
info(conn, run_id, "folder_done", folder=current, detail=str(n))
|
||||||
|
|
||||||
|
try:
|
||||||
|
subs = list(folder.Folders)
|
||||||
|
except Exception:
|
||||||
|
subs = []
|
||||||
|
for sub in subs:
|
||||||
|
if limit and stats["found"] >= limit:
|
||||||
|
break
|
||||||
|
walk(conn, run_id, sub, current, cutoff_local, stats, seen, mode, dry, limit)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_dt(s):
|
||||||
|
if not s:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
dt = datetime.fromisoformat(s)
|
||||||
|
if dt.tzinfo:
|
||||||
|
dt = dt.astimezone().replace(tzinfo=None)
|
||||||
|
return dt
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def flag_left_mailbox(conn, run_id, cutoff_local, seen, scanned_roots, stats, dry):
|
||||||
|
"""Emaily v DB v okne, ktere jsme ve SKENOVANE casti schranky NEvideli ->
|
||||||
|
opustily pracovni schranku. Ponecha posledni znamou cestu, nastavi priznak."""
|
||||||
|
cur = conn.execute(
|
||||||
|
"""SELECT message_id, received_at, jnj_folder, folder, not_in_mailbox_anymore
|
||||||
|
FROM messages""")
|
||||||
|
to_flag = []
|
||||||
|
for mid, received_at, jnjf, fld, flag in cur.fetchall():
|
||||||
|
if mid in seen or flag:
|
||||||
|
continue
|
||||||
|
path = jnjf or fld or ""
|
||||||
|
if not any(path.startswith(root) for root in scanned_roots):
|
||||||
|
continue
|
||||||
|
rec = _parse_dt(received_at)
|
||||||
|
if rec is None or rec < cutoff_local:
|
||||||
|
continue
|
||||||
|
to_flag.append((mid, path))
|
||||||
|
|
||||||
|
for mid, path in to_flag:
|
||||||
|
if not dry:
|
||||||
|
conn.execute(
|
||||||
|
"""UPDATE messages SET not_in_mailbox_anymore=1,
|
||||||
|
left_mailbox_at=datetime('now'), updated_at=datetime('now')
|
||||||
|
WHERE message_id=?""", (mid,))
|
||||||
|
stats["left_mailbox"] += 1
|
||||||
|
print(f" GONE{'*' if dry else ' '} | {path}")
|
||||||
|
if not dry and to_flag:
|
||||||
|
conn.commit()
|
||||||
|
info(conn, run_id, "left_mailbox", detail=str(len(to_flag)))
|
||||||
|
|
||||||
|
|
||||||
|
# ─── MAIN ─────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description=f"jnj_mailbox_sync v{SCRIPT_VERSION}")
|
||||||
|
ap.add_argument("--mode", choices=["capture", "update-paths", "full-update"],
|
||||||
|
default="capture")
|
||||||
|
ap.add_argument("--days", type=int, default=30,
|
||||||
|
help="Okno ve dnech pro update-paths/full-update (0 = vse)")
|
||||||
|
ap.add_argument("--dry-run", action="store_true",
|
||||||
|
help="Nic nezapise/nenahraje, jen vypise co by udelal")
|
||||||
|
ap.add_argument("--limit", type=int, default=0, help="Max N polozek (test)")
|
||||||
|
ap.add_argument("--no-db-upload", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
mode, dry = args.mode, args.dry_run
|
||||||
|
|
||||||
|
if mode == "capture":
|
||||||
|
cutoff_local = None
|
||||||
|
else:
|
||||||
|
cutoff_local = None if args.days == 0 else (datetime.now() - timedelta(days=args.days))
|
||||||
|
|
||||||
|
win = "vse" if cutoff_local is None else f"{args.days} dni (od {cutoff_local:%Y-%m-%d %H:%M})"
|
||||||
|
print(f"=== jnj_mailbox_sync v{SCRIPT_VERSION} ===")
|
||||||
|
print(f"Start: {datetime.now():%Y-%m-%d %H:%M:%S}")
|
||||||
|
print(f"Rezim: {mode} Okno: {win} {'[DRY-RUN — nic se nemeni]' if dry else ''}")
|
||||||
|
print(f"DB: {DB_PATH}")
|
||||||
|
|
||||||
|
conn = sqlite3.connect(DB_PATH)
|
||||||
|
init_db(conn)
|
||||||
|
run_id = start_run(conn, mode, args.days, dry)
|
||||||
|
|
||||||
|
outlook = win32com.client.Dispatch("Outlook.Application")
|
||||||
|
ns = outlook.GetNamespace("MAPI")
|
||||||
|
|
||||||
|
stats = {"found": 0, "new_captured": 0, "new_uncaptured": 0, "path_updated": 0,
|
||||||
|
"read_updated": 0, "returned": 0, "left_mailbox": 0, "content_updated": 0,
|
||||||
|
"skipped": 0, "errors": 0}
|
||||||
|
seen = set()
|
||||||
|
|
||||||
|
scanned_roots = set()
|
||||||
|
for fid, label in SYNC_FOLDERS:
|
||||||
|
root = ns.GetDefaultFolder(fid)
|
||||||
|
mailbox = root.Parent.Name
|
||||||
|
scanned_roots.add(f"/{mailbox}/{root.Name}")
|
||||||
|
print(f"\n=== {label} ({mailbox}) ===")
|
||||||
|
walk(conn, run_id, root, f"/{mailbox}", cutoff_local, stats, seen, mode, dry, args.limit)
|
||||||
|
|
||||||
|
# ── Archive v PRIMARNI schrance (v1.4) ─────────────────────────────────
|
||||||
|
# Archive (jednoklikove archivovani) NENI default folder -> hleda se podle
|
||||||
|
# jmena pod korenem primarni schranky (inbox.Parent = koren te same schranky,
|
||||||
|
# takze Online Archive = jiny store se SEM nepriplete).
|
||||||
|
try:
|
||||||
|
mbox_root = ns.GetDefaultFolder(6).Parent
|
||||||
|
mailbox = mbox_root.Name
|
||||||
|
archive = None
|
||||||
|
for f in mbox_root.Folders:
|
||||||
|
try:
|
||||||
|
if str(f.Name).strip().lower() == "archive":
|
||||||
|
archive = f
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if archive is not None:
|
||||||
|
scanned_roots.add(f"/{mailbox}/{archive.Name}")
|
||||||
|
print(f"\n=== Archive ({mailbox}) ===")
|
||||||
|
walk(conn, run_id, archive, f"/{mailbox}", cutoff_local, stats, seen, mode, dry, args.limit)
|
||||||
|
else:
|
||||||
|
print("\n(Archive slozka v primarni schrance nenalezena -> preskakuji)")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n(Archive scan preskocen: {e})")
|
||||||
|
|
||||||
|
if mode in ("update-paths", "full-update") and cutoff_local is not None and not (args.limit):
|
||||||
|
print("\n--- Kontrola 'opustilo schranku' (v okne, Inbox/Sent/Deleted) ---")
|
||||||
|
flag_left_mailbox(conn, run_id, cutoff_local, seen, scanned_roots, stats, dry)
|
||||||
|
elif args.limit:
|
||||||
|
print("\n(--limit aktivni -> detekce 'opustilo schranku' preskocena)")
|
||||||
|
|
||||||
|
finish_run(conn, run_id, stats)
|
||||||
|
|
||||||
|
# ── Souhrn ─────────────────────────────────────────────────────────────
|
||||||
|
print(f"\n{'='*60}")
|
||||||
|
print(f"SOUHRN [{mode}{' / DRY-RUN' if dry else ''}]")
|
||||||
|
print(f" Nalezeno ve schrance: {stats['found']}")
|
||||||
|
if mode in ("capture", "full-update"):
|
||||||
|
lbl = "by se nahralo" if dry else "nahrano"
|
||||||
|
print(f" Nove zachyceno ({lbl}): {stats['new_captured']}")
|
||||||
|
else:
|
||||||
|
print(f" Nove (bez tela, nedorovnano):{stats['new_uncaptured']}")
|
||||||
|
print(f" Aktualizovana cesta: {stats['path_updated']}")
|
||||||
|
print(f" Zmena precteno/neprecteno: {stats['read_updated']}")
|
||||||
|
print(f" Vraceno do schranky: {stats['returned']}")
|
||||||
|
print(f" Obsah zmenen (re-upload): {stats['content_updated']}")
|
||||||
|
print(f" Opustilo schranku (GONE): {stats['left_mailbox']}")
|
||||||
|
print(f" Beze zmeny (skip): {stats['skipped']}")
|
||||||
|
print(f" Chyby: {stats['errors']}")
|
||||||
|
print(f"{'='*60}")
|
||||||
|
|
||||||
|
if dry:
|
||||||
|
print("DRY-RUN: SQLite ani server se NEMENILY.")
|
||||||
|
elif not args.no_db_upload:
|
||||||
|
print("\nUpload SQLite na server...")
|
||||||
|
upload_db(DB_PATH)
|
||||||
|
|
||||||
|
print(f"\nKonec: {datetime.now():%Y-%m-%d %H:%M:%S}")
|
||||||
|
if stats["errors"]:
|
||||||
|
print(f"Chyby logovany do: {LOG_PATH}")
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def upload_db(db_path):
|
||||||
|
"""Komprese (lzma/xz, max) -> Fernet sifra -> upload jako .db.xz.enc."""
|
||||||
|
ts = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||||
|
filename = f"jnjemails_{ts}.db"
|
||||||
|
try:
|
||||||
|
with open(db_path, "rb") as f:
|
||||||
|
raw = f.read()
|
||||||
|
compressed = lzma.compress(raw, preset=9 | lzma.PRESET_EXTREME)
|
||||||
|
encrypted = _FERNET.encrypt(compressed)
|
||||||
|
enc_filename = filename + ".xz.enc"
|
||||||
|
resp = requests.post(
|
||||||
|
DB_UPLOAD_URL,
|
||||||
|
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||||
|
files={"file": (enc_filename, encrypted, "application/octet-stream")},
|
||||||
|
timeout=300,
|
||||||
|
)
|
||||||
|
mb_raw, mb_xz, mb_enc = (len(raw) / 1048576,
|
||||||
|
len(compressed) / 1048576,
|
||||||
|
len(encrypted) / 1048576)
|
||||||
|
print(f" DB upload: {resp.json()} "
|
||||||
|
f"({mb_raw:.1f} MB -> xz {mb_xz:.1f} MB -> enc {mb_enc:.1f} MB)")
|
||||||
|
except Exception as e:
|
||||||
|
print(f" DB upload CHYBA: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,272 @@
|
|||||||
|
"""
|
||||||
|
jnj_unsent_probe v1.1
|
||||||
|
Nazev: jnj_unsent_probe_v1.0.py (verze 1.1.0 — bohatsi vypis)
|
||||||
|
Verze: 1.1.0
|
||||||
|
Datum: 2026-06-16
|
||||||
|
Autor: vladimir.buzalka
|
||||||
|
Bezi: JNJ stroj (Outlook MAPI), Python z Thonny. JEN CTE, nic nezapisuje/nenahrava.
|
||||||
|
|
||||||
|
UCEL (diagnostika):
|
||||||
|
Cte e-maily PRIMO z ziveho Outlooku (MAPI) a vypisuje "identifikatory
|
||||||
|
neodeslani", ktere se pri exportu do .msg ztraci nebo nejsou spolehlive.
|
||||||
|
Slouzi k OVERENI, ktery zivy priznak spolehlive oznaci NEODESLANY e-mail
|
||||||
|
(napr. hustakova nabidka, kterou Exchange odmitl SendAsDenied).
|
||||||
|
|
||||||
|
Pro kazdou nalezenou polozku vypise vedle sebe:
|
||||||
|
- folder, subject, prijemce
|
||||||
|
- item.Sent (object model bool — odeslano?)
|
||||||
|
- PR_MESSAGE_FLAGS + dekodovane bity UNSENT / SUBMIT / READ
|
||||||
|
- ma Internet Message-ID? (PR_0x1035)
|
||||||
|
- ma PR_CLIENT_SUBMIT_TIME? (0x0039)
|
||||||
|
- PR_LAST_VERB_EXECUTED (0x1081)
|
||||||
|
- body_has_error (zive item.Body obsahuje SendAsDenied / could not be sent?)
|
||||||
|
- pokud ano -> vypise i snippet chyby
|
||||||
|
|
||||||
|
DULEZITE: tohle je SONDA. Z jejiho vystupu se rozhodne, ktery priznak je
|
||||||
|
spolehlivy detektor, a teprve pak se z toho udela produkcni flagovani.
|
||||||
|
|
||||||
|
Filtry (argumenty):
|
||||||
|
--to SUBSTR jen polozky, jejichz prijemce obsahuje SUBSTR (napr. hustak)
|
||||||
|
--subject SUBSTR jen polozky s SUBSTR v predmetu (napr. icotrokinra)
|
||||||
|
--days N okno poslednich N dni dle ReceivedTime (default 90; 0 = vse)
|
||||||
|
--all vypsat VSE (jinak jen "podezrele" = bez Internet Message-ID)
|
||||||
|
--limit N max N vypsanych polozek (default 60)
|
||||||
|
--folders LIST carkou oddelene: inbox,sent,drafts,deleted,outbox,archive
|
||||||
|
(default vse uvedene)
|
||||||
|
|
||||||
|
Priklady:
|
||||||
|
python jnj_unsent_probe_v1.0.py --to hustak --all
|
||||||
|
python jnj_unsent_probe_v1.0.py --subject icotrokinra --days 60
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
|
import win32com.client
|
||||||
|
|
||||||
|
if hasattr(sys.stdout, "reconfigure"):
|
||||||
|
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||||
|
|
||||||
|
# MAPI proptagy
|
||||||
|
PR_MESSAGE_FLAGS = "http://schemas.microsoft.com/mapi/proptag/0x0E070003"
|
||||||
|
PR_INTERNET_MSG_ID = "http://schemas.microsoft.com/mapi/proptag/0x1035001E"
|
||||||
|
PR_CLIENT_SUBMIT_TIME = "http://schemas.microsoft.com/mapi/proptag/0x00390040"
|
||||||
|
PR_LAST_VERB = "http://schemas.microsoft.com/mapi/proptag/0x10810003"
|
||||||
|
|
||||||
|
# MSGFLAG bity
|
||||||
|
MSGFLAG_READ = 0x1
|
||||||
|
MSGFLAG_UNSENT = 0x8
|
||||||
|
MSGFLAG_SUBMIT = 0x4
|
||||||
|
|
||||||
|
# Default folder ID (OlDefaultFolders)
|
||||||
|
DEFAULT_FOLDERS = {
|
||||||
|
"inbox": 6, "sent": 5, "drafts": 16, "deleted": 3, "outbox": 4,
|
||||||
|
}
|
||||||
|
|
||||||
|
ERR_MARKERS = ("SendAsDenied", "could not be sent", "TransportSend",
|
||||||
|
"MapiExceptionSendAs", "nemáte oprávnění", "on behalf of")
|
||||||
|
|
||||||
|
|
||||||
|
def prop(item, tag, default=None):
|
||||||
|
try:
|
||||||
|
v = item.PropertyAccessor.GetProperty(tag)
|
||||||
|
return v if v is not None else default
|
||||||
|
except Exception:
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def get_to(item):
|
||||||
|
try:
|
||||||
|
return item.To or ""
|
||||||
|
except Exception:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def body_error_snippet(item):
|
||||||
|
"""Zive telo (item.Body) — obsahuje stopu chyby odeslani?"""
|
||||||
|
try:
|
||||||
|
b = item.Body or ""
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
for m in ERR_MARKERS:
|
||||||
|
i = b.find(m)
|
||||||
|
if i >= 0:
|
||||||
|
return b[max(0, i - 10):i + 90].replace("\r", " ").replace("\n", " ")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def describe(item):
|
||||||
|
subj = str(getattr(item, "Subject", "") or "")[:42]
|
||||||
|
to = get_to(item)[:32]
|
||||||
|
try:
|
||||||
|
sent = bool(item.Sent)
|
||||||
|
except Exception:
|
||||||
|
sent = None
|
||||||
|
flags = prop(item, PR_MESSAGE_FLAGS, 0) or 0
|
||||||
|
unsent = bool(flags & MSGFLAG_UNSENT)
|
||||||
|
submit = bool(flags & MSGFLAG_SUBMIT)
|
||||||
|
read = bool(flags & MSGFLAG_READ)
|
||||||
|
mid = prop(item, PR_INTERNET_MSG_ID)
|
||||||
|
if not mid:
|
||||||
|
mid = prop(item, "http://schemas.microsoft.com/mapi/proptag/0x1035001F") # unicode varianta
|
||||||
|
has_mid = bool(mid)
|
||||||
|
submit_time = prop(item, PR_CLIENT_SUBMIT_TIME)
|
||||||
|
last_verb = prop(item, PR_LAST_VERB)
|
||||||
|
err = body_error_snippet(item)
|
||||||
|
try:
|
||||||
|
rdate = item.ReceivedTime.strftime("%Y-%m-%d %H:%M") if item.ReceivedTime else "?"
|
||||||
|
except Exception:
|
||||||
|
rdate = "?"
|
||||||
|
try:
|
||||||
|
eid = str(item.EntryID)[-20:]
|
||||||
|
except Exception:
|
||||||
|
eid = "?"
|
||||||
|
return {
|
||||||
|
"subject": subj, "to": to, "sent": sent, "flags": flags,
|
||||||
|
"unsent": unsent, "submit": submit, "read": read,
|
||||||
|
"has_mid": has_mid, "mid_val": (str(mid)[:60] if mid else "-"),
|
||||||
|
"submit_time": bool(submit_time),
|
||||||
|
"last_verb": last_verb, "err": err, "rdate": rdate, "eid": eid,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def matches(item, args):
|
||||||
|
if args.to:
|
||||||
|
if args.to.lower() not in get_to(item).lower():
|
||||||
|
try:
|
||||||
|
# zkus i recipients
|
||||||
|
rec = "; ".join(str(r.Address or r.Name or "") for r in item.Recipients)
|
||||||
|
except Exception:
|
||||||
|
rec = ""
|
||||||
|
if args.to.lower() not in rec.lower():
|
||||||
|
return False
|
||||||
|
if args.subject:
|
||||||
|
if args.subject.lower() not in str(getattr(item, "Subject", "") or "").lower():
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def walk(folder, path, args, cutoff, out, counters):
|
||||||
|
cur = f"{path}/{folder.Name}"
|
||||||
|
try:
|
||||||
|
items = folder.Items
|
||||||
|
try:
|
||||||
|
items.Sort("[ReceivedTime]", True)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
for item in items:
|
||||||
|
if len(out) >= args.limit:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
if not str(getattr(item, "MessageClass", "")).upper().startswith("IPM.NOTE"):
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if cutoff is not None:
|
||||||
|
try:
|
||||||
|
rt = item.ReceivedTime
|
||||||
|
if rt is not None and rt.replace(tzinfo=None) < cutoff:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if not matches(item, args):
|
||||||
|
continue
|
||||||
|
counters["seen"] += 1
|
||||||
|
d = describe(item)
|
||||||
|
if (not args.all) and d["has_mid"]:
|
||||||
|
continue # ma Message-ID -> neni podezrely (pokud neni --all)
|
||||||
|
d["folder"] = cur
|
||||||
|
out.append(d)
|
||||||
|
try:
|
||||||
|
subs = list(folder.Folders)
|
||||||
|
except Exception:
|
||||||
|
subs = []
|
||||||
|
for sub in subs:
|
||||||
|
if len(out) >= args.limit:
|
||||||
|
return
|
||||||
|
walk(sub, cur, args, cutoff, out, counters)
|
||||||
|
|
||||||
|
|
||||||
|
def find_archive(ns):
|
||||||
|
try:
|
||||||
|
root = ns.GetDefaultFolder(6).Parent
|
||||||
|
for f in root.Folders:
|
||||||
|
try:
|
||||||
|
if str(f.Name).strip().lower() == "archive":
|
||||||
|
return f, root.Name
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description="jnj_unsent_probe v1.0 (diagnostika)")
|
||||||
|
ap.add_argument("--to", default="")
|
||||||
|
ap.add_argument("--subject", default="")
|
||||||
|
ap.add_argument("--days", type=int, default=90)
|
||||||
|
ap.add_argument("--all", action="store_true")
|
||||||
|
ap.add_argument("--limit", type=int, default=60)
|
||||||
|
ap.add_argument("--folders", default="inbox,sent,drafts,deleted,outbox,archive")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
cutoff = None if args.days == 0 else (datetime.now() - timedelta(days=args.days))
|
||||||
|
want = [x.strip().lower() for x in args.folders.split(",") if x.strip()]
|
||||||
|
|
||||||
|
print(f"=== jnj_unsent_probe v1.0 ===")
|
||||||
|
print(f"Filtr: to~'{args.to}' subject~'{args.subject}' okno={'vse' if cutoff is None else str(args.days)+'d'} "
|
||||||
|
f"| {'VSE' if args.all else 'jen bez Message-ID'} | slozky={want}")
|
||||||
|
|
||||||
|
outlook = win32com.client.Dispatch("Outlook.Application")
|
||||||
|
ns = outlook.GetNamespace("MAPI")
|
||||||
|
|
||||||
|
out = []
|
||||||
|
counters = {"seen": 0}
|
||||||
|
for name in want:
|
||||||
|
if len(out) >= args.limit:
|
||||||
|
break
|
||||||
|
if name == "archive":
|
||||||
|
arch, mbox = find_archive(ns)
|
||||||
|
if arch is not None:
|
||||||
|
walk(arch, f"/{mbox}", args, cutoff, out, counters)
|
||||||
|
else:
|
||||||
|
print(" (Archive nenalezena)")
|
||||||
|
continue
|
||||||
|
fid = DEFAULT_FOLDERS.get(name)
|
||||||
|
if not fid:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
root = ns.GetDefaultFolder(fid)
|
||||||
|
except Exception as e:
|
||||||
|
print(f" ({name} nedostupna: {e})")
|
||||||
|
continue
|
||||||
|
walk(root, f"/{root.Parent.Name}", args, cutoff, out, counters)
|
||||||
|
|
||||||
|
print(f"\nProsmatrovano polozek: {counters['seen']} vypsano: {len(out)}\n")
|
||||||
|
n_unsent = n_noid = n_err = 0
|
||||||
|
for i, d in enumerate(out, 1):
|
||||||
|
if d["unsent"]:
|
||||||
|
n_unsent += 1
|
||||||
|
if not d["has_mid"]:
|
||||||
|
n_noid += 1
|
||||||
|
if d["err"]:
|
||||||
|
n_err += 1
|
||||||
|
print(f"[{i}] {d['folder']} ({d['rdate']})")
|
||||||
|
print(f" subject : {d['subject']}")
|
||||||
|
print(f" to : {d['to']}")
|
||||||
|
print(f" Sent={d['sent']} UNSENT={d['unsent']} SUBMIT={d['submit']} "
|
||||||
|
f"has_MsgID={d['has_mid']} submit_time={d['submit_time']} ERR={'YES' if d['err'] else '-'}")
|
||||||
|
print(f" MsgID : {d['mid_val']}")
|
||||||
|
print(f" EntryID[-20:] (=jmeno .msg): {d['eid']}")
|
||||||
|
if d["err"]:
|
||||||
|
print(f" ERR : ...{d['err']}...")
|
||||||
|
print()
|
||||||
|
|
||||||
|
print(f"SOUHRN: vypsano={len(out)} UNSENT-flag={n_unsent} bez-MsgID={n_noid} s-chybou-v-tele={n_err}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user