feat: harden dashcam transfer tegen flakke SD-lezer + stale-mount cleanup

- dashcam-transfer.py: log echte OSError (shutil.copy2 i.p.v. cp), pre-flight
  leesbaarheidscheck voor ruimte-wissen, retry (3x) + early-abort (5x), guards
  op bestandssysteem-calls, WhatsApp GESTOPT bij defecte kaart
- dashcam-transfer-mount.sh: skip als service al draait, read-only fsck.exfat
  health check, %k device-naam doorgeven
- dashcam-transfer-cleanup.sh: nieuw - ruimt stale mount op bij udev REMOVE
- udev/99-dashcam-trigger.rules: ADD + REMOVE regels met %k
- tests: 20 python + 7 mount + 12 cleanup checks
- README: architectuur, robuustheid, deploy-instructies
This commit is contained in:
Eddy de Vink 2026-08-11 20:23:24 +02:00
commit 8cc04ea409
9 changed files with 779 additions and 76 deletions

4
.gitignore vendored
View file

@ -19,6 +19,10 @@ build/
.env.test.local .env.test.local
.env.production.local .env.production.local
# Python
__pycache__/
*.pyc
# Logs # Logs
logs/ logs/
*.log *.log

View file

@ -9,17 +9,51 @@ organisatie per datum en camera-positie (front/back). Draait op de **nextcloud**
SD-kaart insteken SD-kaart insteken
→ udev detecteert vendor:product (14cd:1212 of 05e3:0751) → udev detecteert vendor:product (14cd:1212 of 05e3:0751)
→ systemd-run start /usr/local/bin/dashcam-transfer-mount %k → systemd-run start /usr/local/bin/dashcam-transfer-mount %k
→ skip als de transfer-service al draait (flaky lezer = geen 2e mount)
→ wacht tot /dev/sdX1 verschijnt (max 60s) → wacht tot /dev/sdX1 verschijnt (max 60s)
→ mount apparaat dynamisch op /mnt/sdcard (uid=1000, exfat) → mount apparaat dynamisch op /mnt/sdcard (uid=1000, exfat)
→ read-only fsck.exfat health check (logt, blokkeert niet)
→ systemctl start dashcam-transfer.service → systemctl start dashcam-transfer.service
→ /usr/local/bin/dashcam-transfer.py → /usr/local/bin/dashcam-transfer
1. Check mounts (/mnt/sdcard, /mnt/ultratouch) 1. Check mounts (/mnt/sdcard, /mnt/ultratouch)
2. Scan /mnt/sdcard/Novatek/Movie/ op *.MP4 2. Scan /mnt/sdcard/Novatek/Movie/ op *.MP4
3. Check vrije ruimte — verwijder oudste datum-mappen indien nodig 3. Leesbaarheidscheck bron (voorkomt ruimte wissen bij defecte kaart)
4. Copy + verify + delete per bestand 4. Check vrije ruimte — verwijder oudste datum-mappen indien nodig
5. Stuur WhatsApp notificatie via wwebjs-api 5. Copy + verify + delete per bestand, met retry
6. Early-abort bij aanhoudende I/O-fouten (SD-kaart defect)
7. Stuur WhatsApp notificatie via wwebjs-api
SD-kaart uittrekken
→ udev REMOVE trigger (zelfde vendor:product)
→ systemd-run start /usr/local/bin/dashcam-transfer-cleanup (%k)
→ stale mount? (device-node weg of source = verwijderd device)
→ stop dashcam-transfer.service
→ umount /mnt/sdcard (fallback: umount -l)
``` ```
## Robuustheid
Het script is ontworpen tegen de hierboven geziene productie-fout (defecte/flakke
SD-kaartlezer die midden in de transfer offline klapt, `I/O error`, `device offline`):
- **Echte fout zichtbaar** — kopieerfouten worden als `OSError` met reden gelogd
(i.p.v. alleen "exit status 1"); de eerdere `cp`-stderr ging verloren.
- **Pre-flight leesbaarheidscheck** — voordat oude datum-mappen op de SSD worden
verwijderd, wordt eerst een bestand van de kaart gelezen. Een onleesbare kaart
resulteert in een duidelijke WhatsApp-"GESTOPT" en **niets** wordt gewist.
- **Retry per bestand** — 3 pogingen met 2s pauze; vangt transiënte USB-glitches.
- **Early-abort** — bij 5 opeenvolgende fouten stopt het script en stuurt het een
WhatsApp met de laatste fout. Bronbestanden worden nooit verwijderd zonder
geverifieerde kopie.
- **Geen crash mid-loop** — alle bestandssysteem-calls zijn afgeschermd.
- **fsck.exfat** — na mount draait het mount-script een read-only health check;
een dirty/defecte kaart wordt daarmee zichtbaar in syslog.
- **Geen dubbele runs** — het mount-script skip als de transfer al actief is.
- **Stale-mount opruiming** — als de kaartlezer wordt uitgetrokken (of de kaart
offline klapt) ruimt een udev REMOVE-trigger de mount op: de service wordt
gestopt en `/mnt/sdcard` netjes ontkoppeld (lazy als dat mislukt). Zo blijft er
nooit een dood device in de `mount`-output hangen.
## Hardware ## Hardware
| Component | Mount | Device | Doel | | Component | Mount | Device | Doel |
@ -82,15 +116,25 @@ De container: `docker-compose.yml` in `/home/aiaiaiaiagent/wwebjs-api/`.
```udev ```udev
ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="14cd", ENV{ID_MODEL_ID}=="1212", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k" ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="14cd", ENV{ID_MODEL_ID}=="1212", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k"
ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="05e3", ENV{ID_MODEL_ID}=="0751", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k" ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="05e3", ENV{ID_MODEL_ID}=="0751", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k"
ACTION=="remove", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="14cd", ENV{ID_MODEL_ID}=="1212", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-cleanup %k"
ACTION=="remove", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="05e3", ENV{ID_MODEL_ID}=="0751", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-cleanup %k"
``` ```
`%k` geeft het kernel device (b.v. `sde1`) door aan het mount script. `%k` geeft het kernel device (b.v. `sde1`) door aan het mount- en cleanup-script,
zodat de kaart dynamisch gemount wordt ongeacht het filesystem UUID. Na wijziging:
`sudo udevadm control --reload-rules`.
### Mount script (`/usr/local/bin/dashcam-transfer-mount.sh`) ### Mount script (`/usr/local/bin/dashcam-transfer-mount.sh`)
Wacht tot het doorgegeven device beschikbaar is, unmount eventuele stale mounts, Wacht tot het doorgegeven device beschikbaar is, unmount eventuele stale mounts,
mount de SD-kaart en start de service. mount de SD-kaart en start de service.
### Cleanup script (`/usr/local/bin/dashcam-transfer-cleanup.sh`)
Draait op udev REMOVE-trigger. Checkt of `/mnt/sdcard` een stale mount is (device-node
weg of gemount op een verwijderd device) en ruimt hem dan op: service stoppen,
`umount` (fallback `umount -l`). Bij een gezonde mount (ander device) doet het niets.
### Systemd service (`/etc/systemd/system/dashcam-transfer.service`) ### Systemd service (`/etc/systemd/system/dashcam-transfer.service`)
```ini ```ini
@ -121,9 +165,20 @@ Alle activiteit wordt gelogd naar `/tmp/dashcam-transfer.log` op nextcloud.
## Ontwikkeling ## Ontwikkeling
Lokale repo: `~/src/dashcam-transfer/` op Mac Mini. Lokale repo: `~/src/dashcam-transfer/` op Mac Mini.
Tests: `python3 -m unittest discover -s tests`, `./tests/test_mount_script.sh`
en `./tests/test_cleanup_script.sh`.
Deployen naar nextcloud: Deployen naar nextcloud:
```bash ```bash
scp dashcam-transfer.py aiaiaiaiagent@nextcloud:/tmp/ scp dashcam-transfer.py aiaiaiaiagent@nextcloud:/tmp/
ssh aiaiaiaiagent@nextcloud 'sudo -A cp /tmp/dashcam-transfer.py /usr/local/bin/dashcam-transfer' ssh aiaiaiaiagent@nextcloud 'sudo -A cp /tmp/dashcam-transfer.py /usr/local/bin/dashcam-transfer'
scp dashcam-transfer-mount.sh aiaiaiaiagent@nextcloud:/tmp/
ssh aiaiaiaiagent@nextcloud 'sudo -A cp /tmp/dashcam-transfer-mount.sh /usr/local/bin/dashcam-transfer-mount'
scp dashcam-transfer-cleanup.sh aiaiaiaiagent@nextcloud:/tmp/
ssh aiaiaiaiagent@nextcloud 'sudo -A cp /tmp/dashcam-transfer-cleanup.sh /usr/local/bin/dashcam-transfer-cleanup'
scp udev/99-dashcam-trigger.rules aiaiaiaiagent@nextcloud:/tmp/
ssh aiaiaiaiagent@nextcloud 'sudo -A cp /tmp/99-dashcam-trigger.rules /etc/udev/rules.d/ && sudo -A udevadm control --reload-rules'
``` ```
Controleer na deploy altijd met `sudo sha256sum` dat de hash op de server gelijk
is aan de lokale `shasum -a 256`.

39
dashcam-transfer-cleanup.sh Executable file
View file

@ -0,0 +1,39 @@
#!/bin/bash
# Remove-trigger cleanup for the dashcam SD card.
# Called by udev rule 99-dashcam-trigger.rules via systemd-run on
# ACTION=="remove" with %k as $1. Unmounts a stale /mnt/sdcard mount and
# stops a transfer that can no longer complete.
# Idempotent: does nothing when /mnt/sdcard is not mounted or still healthy.
logger -t dashcam-transfer "Remove trigger: block device weggevallen"
src=$(findmnt -no SOURCE /mnt/sdcard 2>/dev/null)
if [ -z "$src" ]; then
logger -t dashcam-transfer "Remove: /mnt/sdcard niet gemount, niets te doen"
exit 0
fi
case "$src" in
/*) ;;
*) logger -t dashcam-transfer "Remove: onbekende source ($src), niets te doen"; exit 0 ;;
esac
stale=0
if [ ! -e "$src" ]; then
stale=1
elif [ -n "$1" ] && [[ "$src" == *"/$1"* ]]; then
stale=1
fi
if [ "$stale" -eq 0 ]; then
logger -t dashcam-transfer "Remove: device $src nog aanwezig, mount blijft"
exit 0
fi
logger -t dashcam-transfer "Remove: device $src verdwenen — transfer stoppen, mount opruimen"
systemctl stop dashcam-transfer.service 2>/dev/null || true
sleep 1
if ! umount /mnt/sdcard 2>/dev/null; then
logger -t dashcam-transfer "Remove: umount mislukt, lazy unmount"
umount -l /mnt/sdcard 2>/dev/null || true
fi
logger -t dashcam-transfer "Remove: stale mount opgeruimd"

View file

@ -1,15 +1,22 @@
#!/bin/bash #!/bin/bash
# Mount SD card and start dashcam transfer. # Mount SD card by kernel device name (udev %k) and start dashcam transfer.
# Called by udev rule 99-dashcam-trigger.rules via systemd-run. # Called by udev rule 99-dashcam-trigger.rules via systemd-run.
# systemd-run is used (not direct udev exec) because udev context # systemd-run is used (not direct udev exec) because udev context
# lacks mount/umount permissions on Fedora with SELinux enforcing. # lacks mount/umount permissions on Fedora with SELinux enforcing.
# MAX_WAIT="${DASHCAM_MAX_WAIT:-60}"
# The udev rule passes the kernel name (e.g. "sde1") as $1 so the
# device is mounted dynamically, independent of its filesystem UUID.
DEV=${1:+/dev/$1} DEV=${1:+/dev/$1}
logger -t dashcam-transfer "Trigger: SD card detected ($DEV)" logger -t dashcam-transfer "Trigger: SD card detected ($DEV)"
# Never interrupt a running transfer: a flaky reader can cause a second
# udev event (USB re-enumeration) which must not unmount a busy card.
if systemctl is-active --quiet dashcam-transfer.service; then
logger -t dashcam-transfer "Trigger: transfer service al bezig — skip"
exit 0
fi
if [ -n "$1" ]; then if [ -n "$1" ]; then
for ((i=0; i<60; i++)); do for ((i=0; i<MAX_WAIT; i++)); do
if [ -e "$DEV" ]; then if [ -e "$DEV" ]; then
logger -t dashcam-transfer "Trigger: device ready after ${i}s" logger -t dashcam-transfer "Trigger: device ready after ${i}s"
break break
@ -19,7 +26,7 @@ if [ -n "$1" ]; then
else else
UUID=$(lsblk -rno UUID /dev/sde1 2>/dev/null) UUID=$(lsblk -rno UUID /dev/sde1 2>/dev/null)
DEV=/dev/disk/by-uuid/$UUID DEV=/dev/disk/by-uuid/$UUID
for ((i=0; i<60; i++)); do for ((i=0; i<MAX_WAIT; i++)); do
if [ -e "$DEV" ]; then if [ -e "$DEV" ]; then
logger -t dashcam-transfer "Trigger: device ready after ${i}s" logger -t dashcam-transfer "Trigger: device ready after ${i}s"
break break
@ -27,12 +34,19 @@ else
sleep 1 sleep 1
done done
fi fi
while mountpoint -q /mnt/sdcard 2>/dev/null; do while mountpoint -q /mnt/sdcard 2>/dev/null; do
umount /mnt/sdcard 2>/dev/null || break umount /mnt/sdcard 2>/dev/null || break
done done
mount "$DEV" /mnt/sdcard -o uid=1000,gid=1000,fmask=0000,dmask=0000 mount "$DEV" /mnt/sdcard -o uid=1000,gid=1000,fmask=0000,dmask=0000
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
logger -t dashcam-transfer "Trigger: mounted, starting transfer" logger -t dashcam-transfer "Trigger: mounted, starting transfer"
# Read-only exFAT health check; logs warnings but never blocks the transfer.
if command -v fsck.exfat >/dev/null 2>&1; then
fsck.exfat -n "$DEV" 2>&1 | logger -t dashcam-transfer
logger -t dashcam-transfer "Trigger: fsck.exfat exit ${PIPESTATUS[0]}"
fi
systemctl start dashcam-transfer systemctl start dashcam-transfer
else else
logger -t dashcam-transfer "TRIGGER ERROR: mount failed" logger -t dashcam-transfer "TRIGGER ERROR: mount failed"

View file

@ -8,8 +8,8 @@ import logging
import os import os
import re import re
import shutil import shutil
import subprocess
import sys import sys
import time
import urllib.request import urllib.request
SD_CARD = "/mnt/sdcard" SD_CARD = "/mnt/sdcard"
@ -22,6 +22,27 @@ LOCK_FILE = "/tmp/dashcam-transfer.lock"
CAMERA_MAP = {"F": "front", "B": "back"} CAMERA_MAP = {"F": "front", "B": "back"}
FILE_PATTERN = re.compile(r"^(\d{8})\d{6}_\d+([FB])\.MP4$") FILE_PATTERN = re.compile(r"^(\d{8})\d{6}_\d+([FB])\.MP4$")
MAX_RETRIES = 3
RETRY_DELAY = 2
MAX_CONSECUTIVE_ERRORS = 5
WHATSAPP_API = "http://localhost:3001/client/sendMessage/EdGpt" # localhost = nextcloud server, niet Mac Mini!
WHATSAPP_CHAT = "31620248011@c.us"
UNREADABLE_REASON = (
"SD-kaart onleesbaar (I/O-fout): lezer/kaart defect of niet netjes afgemeld. "
"fsck.exfat en lezer/poort controleren. Geen bronbestanden verwijderd."
)
class TransferAborted(Exception):
def __init__(self, ok, errors, consecutive, reason):
self.ok = ok
self.errors = errors
self.consecutive = consecutive
self.reason = reason
super().__init__(reason)
def setup_logging(): def setup_logging():
logging.basicConfig( logging.basicConfig(
@ -49,16 +70,37 @@ def check_mount(path, name):
def get_mp4_files(src_dir): def get_mp4_files(src_dir):
"""Return matching (fname, date, camera) tuples, [] when absent, None on I/O error."""
if not os.path.isdir(src_dir): if not os.path.isdir(src_dir):
return [] return []
try:
entries = os.listdir(src_dir)
except OSError as e:
logging.error("SD-kaart directory niet leesbaar: %s", e)
return None
result = [] result = []
for fname in os.listdir(src_dir): for fname in entries:
match = FILE_PATTERN.match(fname) match = FILE_PATTERN.match(fname)
if match: if match:
result.append((fname, match.group(1), match.group(2).upper())) result.append((fname, match.group(1), match.group(2).upper()))
return result return result
def check_source_readable(src_dir, files):
"""Smoke-test that the SD card actually returns data, before freeing space."""
try:
with os.scandir(src_dir):
pass
if files:
sample = os.path.join(src_dir, files[0][0])
with open(sample, "rb") as f:
f.read(1)
except OSError as e:
logging.error("SD-kaart niet leesbaar (I/O-fout): %s", e)
return False
return True
def ensure_free_space(needed_bytes, target_base): def ensure_free_space(needed_bytes, target_base):
stat = os.statvfs(target_base) stat = os.statvfs(target_base)
free = stat.f_frsize * stat.f_bavail free = stat.f_frsize * stat.f_bavail
@ -105,19 +147,113 @@ def ensure_free_space(needed_bytes, target_base):
logging.info("Ruimte vrijgemaakt: %d bytes door %d mappen", freed, len(removed)) logging.info("Ruimte vrijgemaakt: %d bytes door %d mappen", freed, len(removed))
WHATSAPP_API = "http://localhost:3001/client/sendMessage/EdGpt" # localhost = nextcloud server, niet Mac Mini! def copy_file(src, dst, attempts=MAX_RETRIES, delay=RETRY_DELAY):
WHATSAPP_CHAT = "31620248011@c.us" """Copy src → dst with retry. Raises the last OSError when all attempts fail."""
last_error = None
for attempt in range(1, attempts + 1):
try:
shutil.copy2(src, dst)
return
except OSError as e:
last_error = e
logging.warning(
"Kopieerpoging %d/%d mislukt voor %s: %s",
attempt, attempts, os.path.basename(src), e,
)
if attempt < attempts:
time.sleep(delay)
raise last_error
def send_whatsapp_notification(ok, errors, total_bytes): def _safe_remove(path, name):
if errors == 0: try:
prefix = "✅" os.remove(path)
status = "geen fouten" except OSError as e:
else: logging.warning("Bron %s kon niet verwijderd worden (blijft op kaart): %s", name, e)
prefix = "❌"
status = f"{errors} fout(en)"
def transfer_files(files, source_dir, target_base):
"""Copy+verify+delete per file. Returns (ok, errors) or raises TransferAborted."""
ok = 0
errors = 0
consecutive = 0
last_error = None
for fname, date_str, cam in files:
cam_name = CAMERA_MAP[cam]
target_dir = os.path.join(target_base, date_str, cam_name)
src = os.path.join(source_dir, fname)
dst = os.path.join(target_dir, fname)
try:
os.makedirs(target_dir, exist_ok=True)
if os.path.exists(dst):
src_size = os.path.getsize(src)
dst_size = os.path.getsize(dst)
if src_size == dst_size:
logging.info(
"Bestaat al en klopt: %s → %s/%s/", fname, date_str, cam_name
)
_safe_remove(src, fname)
ok += 1
consecutive = 0
continue
logging.warning(
"Doel %s bestaat maar is anders (%d vs %d bytes) — overschrijf",
dst, dst_size, src_size,
)
os.remove(dst)
copy_file(src, dst)
src_size = os.path.getsize(src)
dst_size = os.path.getsize(dst)
if dst_size != src_size:
raise OSError("Verificatie mislukt: %d vs %d bytes" % (src_size, dst_size))
except OSError as e:
errors += 1
consecutive += 1
last_error = e
logging.error("Fout bij overzetten %s: %s", fname, e)
if consecutive >= MAX_CONSECUTIVE_ERRORS:
logging.error(
"Te veel opeenvolgende fouten (%d) — transfer gestopt", consecutive
)
raise TransferAborted(
ok, errors, consecutive,
f"SD-kaart onleesbaar (I/O-fout) — gestopt na {consecutive} "
f"opeenvolgende fouten. Laatste fout: {last_error}. "
f"Geen bronbestanden verwijderd.",
)
continue
_safe_remove(src, fname)
logging.info(
"Overgezet: %s → %s/%s/ (%d bytes)", fname, date_str, cam_name, src_size
)
ok += 1
consecutive = 0
return ok, errors
def send_whatsapp_notification(ok, errors, total_bytes, aborted=False, reason=None):
gb = total_bytes / (1024 ** 3) gb = total_bytes / (1024 ** 3)
body = f"{prefix} Dashcam transfer voltooid: {ok} bestanden ({gb:.1f} GB) overgezet, {status}" if aborted:
body = (
f"❌ Dashcam transfer GESTOPT: {reason} "
f"({ok} ok, {errors} fout(en) tot dusver)"
)
elif errors == 0:
body = (
f"✅ Dashcam transfer voltooid: {ok} bestanden ({gb:.1f} GB) "
f"overgezet, geen fouten"
)
else:
body = (
f"❌ Dashcam transfer voltooid: {ok} bestanden ({gb:.1f} GB) "
f"overgezet, {errors} fout(en)"
)
payload = {"chatId": WHATSAPP_CHAT, "content": body, "contentType": "string"} payload = {"chatId": WHATSAPP_CHAT, "content": body, "contentType": "string"}
try: try:
data = json.dumps(payload).encode() data = json.dumps(payload).encode()
@ -148,6 +284,9 @@ def main():
check_mount(SSD, "Ultratouch SSD") check_mount(SSD, "Ultratouch SSD")
files = get_mp4_files(SOURCE_DIR) files = get_mp4_files(SOURCE_DIR)
if files is None:
send_whatsapp_notification(0, 0, 0, aborted=True, reason=UNREADABLE_REASON)
return
if not files: if not files:
logging.info("Geen MP4 bestanden gevonden in %s", SOURCE_DIR) logging.info("Geen MP4 bestanden gevonden in %s", SOURCE_DIR)
return return
@ -155,63 +294,36 @@ def main():
if args.limit > 0: if args.limit > 0:
files = files[:args.limit] files = files[:args.limit]
total_size = sum( if not check_source_readable(SOURCE_DIR, files):
os.path.getsize(os.path.join(SOURCE_DIR, fname)) for fname, _, _ in files send_whatsapp_notification(0, 0, 0, aborted=True, reason=UNREADABLE_REASON)
) return
logging.info(
"Start transfer: %d bestanden, %d bytes", len(files), total_size
)
ensure_free_space(total_size, TARGET_BASE) total_size = 0
for fname, _, _ in files:
try:
total_size += os.path.getsize(os.path.join(SOURCE_DIR, fname))
except OSError as e:
logging.error("Grootte van %s niet leesbaar: %s", fname, e)
send_whatsapp_notification(0, 0, 0, aborted=True, reason=UNREADABLE_REASON)
return
ok = 0 logging.info("Start transfer: %d bestanden, %d bytes", len(files), total_size)
errors = 0
for fname, date_str, cam in files:
target_dir = os.path.join(TARGET_BASE, date_str, CAMERA_MAP[cam])
os.makedirs(target_dir, exist_ok=True)
src = os.path.join(SOURCE_DIR, fname)
dst = os.path.join(target_dir, fname)
if os.path.exists(dst):
src_size = os.path.getsize(src)
dst_size = os.path.getsize(dst)
if src_size == dst_size:
logging.info(
"Bestaat al en klopt: %s → %s/%s/", fname, date_str, CAMERA_MAP[cam]
)
os.remove(src)
ok += 1
continue
else:
logging.warning(
"Doel %s bestaat maar is anders (%d vs %d bytes) — overschrijf",
dst, dst_size, src_size,
)
os.remove(dst)
try: try:
subprocess.run( ensure_free_space(total_size, TARGET_BASE)
["cp", src, dst], check=True, capture_output=True, timeout=600 except SystemExit:
send_whatsapp_notification(
0, 0, total_size, aborted=True,
reason="Onvoldoende vrije ruimte op de SSD, ook na opschonen.",
) )
except (subprocess.CalledProcessError, OSError) as e: raise
logging.error("Fout bij kopiëren %s: %s", fname, e)
errors += 1
continue
dst_size = os.path.getsize(dst) try:
src_size = os.path.getsize(src) ok, errors = transfer_files(files, SOURCE_DIR, TARGET_BASE)
if dst_size != src_size: except TransferAborted as e:
logging.error( logging.info("Transfer gestopt: %d ok, %d fouten", e.ok, e.errors)
"Verificatie mislukt %s: %d vs %d bytes", fname, src_size, dst_size send_whatsapp_notification(e.ok, e.errors, total_size, aborted=True, reason=e.reason)
) return
errors += 1
continue
os.remove(src)
logging.info(
"Overgezet: %s → %s/%s/ (%d bytes)", fname, date_str, CAMERA_MAP[cam], src_size
)
ok += 1
logging.info("Transfer voltooid: %d ok, %d fouten", ok, errors) logging.info("Transfer voltooid: %d ok, %d fouten", ok, errors)
send_whatsapp_notification(ok, errors, total_size) send_whatsapp_notification(ok, errors, total_size)

118
tests/test_cleanup_script.sh Executable file
View file

@ -0,0 +1,118 @@
#!/bin/bash
# Functional test for dashcam-transfer-cleanup.sh using fake tools on PATH.
set -u
SCRIPT="$(cd "$(dirname "$0")/.." && pwd)/dashcam-transfer-cleanup.sh"
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
FAKEBIN="$WORK/bin"
mkdir -p "$FAKEBIN"
LOGGER_FILE="$WORK/logger.log"
UMOUNT_MARKER="$WORK/umounted"
LAZY_MARKER="$WORK/lazy"
STOP_MARKER="$WORK/stopped"
export LOGGER_FILE UMOUNT_MARKER LAZY_MARKER STOP_MARKER FAKE_SRC FAKE_UMOUNT_FAIL
cat > "$FAKEBIN/logger" <<'EOF'
#!/bin/bash
echo "logger: $*" >> "$LOGGER_FILE"
EOF
cat > "$FAKEBIN/findmnt" <<'EOF'
#!/bin/bash
if [ -z "$FAKE_SRC" ]; then
exit 1
fi
echo "$FAKE_SRC"
EOF
cat > "$FAKEBIN/systemctl" <<'EOF'
#!/bin/bash
if [ "$1" = "stop" ]; then
touch "$STOP_MARKER"
fi
EOF
cat > "$FAKEBIN/umount" <<'EOF'
#!/bin/bash
if [ "${1:-}" = "-l" ]; then
touch "$LAZY_MARKER"
exit 0
fi
if [ "$FAKE_UMOUNT_FAIL" = "1" ]; then
exit 1
fi
touch "$UMOUNT_MARKER"
EOF
chmod +x "$FAKEBIN"/*
pass=0
fail=0
check() { # $1=desc, $2=condition
if eval "$2"; then
echo "PASS: $1"
pass=$((pass + 1))
else
echo "FAIL: $1"
fail=$((fail + 1))
fi
}
reset() {
rm -f "$LOGGER_FILE" "$UMOUNT_MARKER" "$LAZY_MARKER" "$STOP_MARKER"
unset FAKE_UMOUNT_FAIL
}
# Test 1: niet gemount → niets te doen
reset
FAKE_SRC=""
PATH="$FAKEBIN:$PATH" "$SCRIPT"
check "niet gemount → geen umount" 'test ! -e "$UMOUNT_MARKER"'
check "niet gemount → geen service stop" 'test ! -e "$STOP_MARKER"'
# Test 2: device nog aanwezig → mount blijft
reset
FAKE_SRC="$WORK/fake-device"
touch "$FAKE_SRC"
PATH="$FAKEBIN:$PATH" "$SCRIPT"
check "device aanwezig → geen umount" 'test ! -e "$UMOUNT_MARKER"'
check "device aanwezig → geen service stop" 'test ! -e "$STOP_MARKER"'
# Test 3: device verdwenen → stop + umount
reset
FAKE_SRC="$WORK/gone-device"
PATH="$FAKEBIN:$PATH" "$SCRIPT"
check "device weg → service gestopt" 'test -e "$STOP_MARKER"'
check "device weg → umount uitgevoerd" 'test -e "$UMOUNT_MARKER"'
check "device weg → geen lazy nodig" 'test ! -e "$LAZY_MARKER"'
# Test 4: umount mislukt → lazy umount
reset
FAKE_SRC="$WORK/gone-device"
export FAKE_UMOUNT_FAIL=1
PATH="$FAKEBIN:$PATH" "$SCRIPT"
check "umount faalt → lazy umount" 'test -e "$LAZY_MARKER"'
# Test 5: node bestaat nog maar source = verwijderd device → stale
reset
mkdir -p "$WORK/dev"
touch "$WORK/dev/sde1"
FAKE_SRC="$WORK/dev/sde1"
PATH="$FAKEBIN:$PATH" "$SCRIPT" sde1
check "source matcht verwijderd device → umount" 'test -e "$UMOUNT_MARKER"'
check "source matcht verwijderd device → service gestopt" 'test -e "$STOP_MARKER"'
# Test 6: ander device verwijdert, sdcard-healthy → niets doen
reset
touch "$WORK/dev/sdb2"
FAKE_SRC="$WORK/dev/sde1"
PATH="$FAKEBIN:$PATH" "$SCRIPT" sdb2
check "ander device → geen umount" 'test ! -e "$UMOUNT_MARKER"'
check "ander device → geen service stop" 'test ! -e "$STOP_MARKER"'
echo "=== $pass pass, $fail fail ==="
[ "$fail" -eq 0 ]

View file

@ -0,0 +1,251 @@
import errno
import importlib.util
import json
import logging
import os
import sys
import tempfile
import unittest
from unittest import mock
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MODULE_PATH = os.path.join(REPO_ROOT, "dashcam-transfer.py")
spec = importlib.util.spec_from_file_location("dashcam_transfer", MODULE_PATH)
dashcam = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dashcam)
sys.modules["dashcam_transfer"] = dashcam
logging.getLogger().addHandler(logging.NullHandler())
def make_mp4(path, size):
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "wb") as f:
f.write(b"0" * size)
class GetMp4FilesTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
def tearDown(self):
self.tmp.cleanup()
def test_matches_front_and_back_files(self):
src = self.tmp.name
make_mp4(os.path.join(src, "20260622050346_0000001F.MP4"), 10)
make_mp4(os.path.join(src, "20260622050346_0000002B.MP4"), 10)
make_mp4(os.path.join(src, "negeer.txt"), 10)
result = dashcam.get_mp4_files(src)
self.assertEqual(len(result), 2)
self.assertIn(("20260622050346_0000001F.MP4", "20260622", "F"), result)
self.assertIn(("20260622050346_0000002B.MP4", "20260622", "B"), result)
def test_missing_dir_returns_empty_list(self):
self.assertEqual(dashcam.get_mp4_files(os.path.join(self.tmp.name, "nope")), [])
def test_listdir_error_returns_none(self):
with mock.patch.object(dashcam.os, "listdir", side_effect=OSError(errno.EIO, "I/O error")):
self.assertIsNone(dashcam.get_mp4_files(self.tmp.name))
class CheckSourceReadableTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.src = self.tmp.name
def tearDown(self):
self.tmp.cleanup()
def test_readable_dir_returns_true(self):
fname = "20260622050346_0000001F.MP4"
make_mp4(os.path.join(self.src, fname), 10)
self.assertTrue(dashcam.check_source_readable(self.src, [(fname, "20260622", "F")]))
def test_io_error_on_read_returns_false(self):
fname = "20260622050346_0000001F.MP4"
make_mp4(os.path.join(self.src, fname), 10)
with mock.patch("builtins.open", side_effect=OSError(errno.EIO, "I/O error")):
self.assertFalse(dashcam.check_source_readable(self.src, [(fname, "20260622", "F")]))
class CopyFileTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.src = os.path.join(self.tmp.name, "src.mp4")
self.dst = os.path.join(self.tmp.name, "dst.mp4")
make_mp4(self.src, 100)
def tearDown(self):
self.tmp.cleanup()
def test_copies_file(self):
dashcam.copy_file(self.src, self.dst, attempts=2, delay=0)
self.assertEqual(os.path.getsize(self.dst), 100)
def test_retries_then_succeeds(self):
real_copy2 = dashcam.shutil.copy2
calls = {"n": 0}
def flaky(s, d):
calls["n"] += 1
if calls["n"] < 3:
raise OSError(errno.EIO, "I/O error")
real_copy2(s, d)
with mock.patch.object(dashcam.shutil, "copy2", side_effect=flaky):
dashcam.copy_file(self.src, self.dst, attempts=3, delay=0)
self.assertEqual(calls["n"], 3)
self.assertEqual(os.path.getsize(self.dst), 100)
def test_raises_after_exhausting_attempts(self):
with mock.patch.object(dashcam.shutil, "copy2", side_effect=OSError(errno.EIO, "I/O error")):
with self.assertRaises(OSError):
dashcam.copy_file(self.src, self.dst, attempts=2, delay=0)
class TransferFilesTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.src = os.path.join(self.tmp.name, "src")
self.base = os.path.join(self.tmp.name, "base")
os.makedirs(self.src)
os.makedirs(self.base)
def tearDown(self):
self.tmp.cleanup()
def _make(self, fname, size):
make_mp4(os.path.join(self.src, fname), size)
return (fname, fname[:8], fname[-5])
def test_transfers_files_to_camera_subdirs(self):
files = [
self._make("20260622050346_0000001F.MP4", 100),
self._make("20260622050346_0000002B.MP4", 200),
]
ok, errors = dashcam.transfer_files(files, self.src, self.base)
self.assertEqual((ok, errors), (2, 0))
self.assertFalse(os.path.exists(os.path.join(self.src, files[0][0])))
self.assertEqual(
os.path.getsize(os.path.join(self.base, "20260622", "front", files[0][0])), 100
)
self.assertEqual(
os.path.getsize(os.path.join(self.base, "20260622", "back", files[1][0])), 200
)
def test_skips_existing_matching_destination(self):
files = [self._make("20260622050346_0000001F.MP4", 100)]
dst = os.path.join(self.base, "20260622", "front", files[0][0])
make_mp4(dst, 100)
ok, errors = dashcam.transfer_files(files, self.src, self.base)
self.assertEqual((ok, errors), (1, 0))
self.assertFalse(os.path.exists(os.path.join(self.src, files[0][0])))
self.assertEqual(os.path.getsize(dst), 100)
def test_overwrites_existing_different_destination(self):
files = [self._make("20260622050346_0000001F.MP4", 100)]
dst = os.path.join(self.base, "20260622", "front", files[0][0])
make_mp4(dst, 10)
ok, errors = dashcam.transfer_files(files, self.src, self.base)
self.assertEqual((ok, errors), (1, 0))
self.assertEqual(os.path.getsize(dst), 100)
def test_verification_mismatch_counts_as_error_and_keeps_source(self):
files = [self._make("20260622050346_0000001F.MP4", 100)]
def bad_copy2(s, d):
with open(d, "wb") as f:
f.write(b"0" * 50)
with mock.patch.object(dashcam.shutil, "copy2", side_effect=bad_copy2), mock.patch.object(
dashcam.time, "sleep"
):
ok, errors = dashcam.transfer_files(files, self.src, self.base)
self.assertEqual((ok, errors), (0, 1))
self.assertTrue(os.path.exists(os.path.join(self.src, files[0][0])))
def test_aborts_after_max_consecutive_errors(self):
files = [
self._make("2026062205034%d_000000%dF.MP4" % (i, i + 1), 10)
for i in range(dashcam.MAX_CONSECUTIVE_ERRORS + 3)
]
with mock.patch.object(
dashcam.shutil, "copy2", side_effect=OSError(errno.EIO, "I/O error")
), mock.patch.object(dashcam.time, "sleep"):
with self.assertRaises(dashcam.TransferAborted) as ctx:
dashcam.transfer_files(files, self.src, self.base)
self.assertEqual(ctx.exception.consecutive, dashcam.MAX_CONSECUTIVE_ERRORS)
self.assertEqual(ctx.exception.ok, 0)
for fname, _, _ in files:
self.assertTrue(os.path.exists(os.path.join(self.src, fname)))
class EnsureFreeSpaceTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.base = self.tmp.name
def tearDown(self):
self.tmp.cleanup()
@staticmethod
def _statvfs(free_bytes):
frsize = 4096
return mock.Mock(f_frsize=frsize, f_bavail=free_bytes // frsize)
def test_no_deletion_when_enough_space(self):
with mock.patch.object(dashcam.os, "statvfs", return_value=self._statvfs(10_000_000)):
dashcam.ensure_free_space(1000, self.base)
def test_deletes_oldest_date_dirs_until_enough(self):
for name, fname in (("20260301", "a.mp4"), ("20260310", "b.mp4"), ("20260811", "c.mp4")):
make_mp4(os.path.join(self.base, name, fname), 5000)
with mock.patch.object(dashcam.os, "statvfs", return_value=self._statvfs(4096)):
dashcam.ensure_free_space(5000, self.base)
self.assertFalse(os.path.exists(os.path.join(self.base, "20260301")))
self.assertTrue(os.path.exists(os.path.join(self.base, "20260310")))
self.assertTrue(os.path.exists(os.path.join(self.base, "20260811")))
def test_exits_when_still_insufficient(self):
make_mp4(os.path.join(self.base, "20260301", "a.mp4"), 100)
with mock.patch.object(dashcam.os, "statvfs", return_value=self._statvfs(0)):
with self.assertRaises(SystemExit):
dashcam.ensure_free_space(10_000, self.base)
class SendWhatsappTest(unittest.TestCase):
def setUp(self):
patcher = mock.patch.object(dashcam.urllib.request, "urlopen")
self.mock_urlopen = patcher.start()
self.mock_urlopen.return_value.__enter__.return_value = mock.Mock()
self.addCleanup(patcher.stop)
def _body(self):
req = self.mock_urlopen.call_args[0][0]
return json.loads(req.data)["content"]
def test_success_message(self):
dashcam.send_whatsapp_notification(5, 0, 381_000_000)
self.assertIn("5 bestanden", self._body())
self.assertIn("geen fouten", self._body())
def test_errors_message(self):
dashcam.send_whatsapp_notification(3, 2, 100_000_000)
self.assertIn("2 fout(en)", self._body())
def test_aborted_message(self):
dashcam.send_whatsapp_notification(
1, 4, 100_000_000, aborted=True, reason="SD-kaart onleesbaar"
)
self.assertIn("GESTOPT", self._body())
self.assertIn("SD-kaart onleesbaar", self._body())
def test_api_failure_is_swallowed(self):
self.mock_urlopen.side_effect = Exception("boom")
dashcam.send_whatsapp_notification(0, 0, 0)
if __name__ == "__main__":
unittest.main()

98
tests/test_mount_script.sh Executable file
View file

@ -0,0 +1,98 @@
#!/bin/bash
# Functional test for dashcam-transfer-mount.sh using fake tools on PATH.
set -u
SCRIPT="$(cd "$(dirname "$0")/.." && pwd)/dashcam-transfer-mount.sh"
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
FAKEBIN="$WORK/bin"
mkdir -p "$FAKEBIN"
LOGGER_FILE="$WORK/logger.log"
MOUNT_LOG="$WORK/mount.log"
FSCK_LOG="$WORK/fsck.log"
MOUNT_MARKER="$WORK/mounted"
START_MARKER="$WORK/started"
export LOGGER_FILE MOUNT_LOG FSCK_LOG MOUNT_MARKER START_MARKER
cat > "$FAKEBIN/logger" <<'EOF'
#!/bin/bash
echo "logger: $*" >> "$LOGGER_FILE"
EOF
cat > "$FAKEBIN/systemctl" <<'EOF'
#!/bin/bash
if [ "$1" = "is-active" ]; then
case "$2" in
--quiet) ;;
*) echo "$SYSTEMCTL_STATE" ;;
esac
[ "$SYSTEMCTL_STATE" = "active" ]
elif [ "$1" = "start" ]; then
touch "$START_MARKER"
fi
EOF
cat > "$FAKEBIN/mount" <<'EOF'
#!/bin/bash
echo "mount $*" >> "$MOUNT_LOG"
touch "$MOUNT_MARKER"
exit 0
EOF
cat > "$FAKEBIN/umount" <<'EOF'
#!/bin/bash
exit 0
EOF
cat > "$FAKEBIN/mountpoint" <<'EOF'
#!/bin/bash
exit 1
EOF
cat > "$FAKEBIN/fsck.exfat" <<'EOF'
#!/bin/bash
echo "fsck $*" >> "$FSCK_LOG"
exit 0
EOF
cat > "$FAKEBIN/lsblk" <<'EOF'
#!/bin/bash
exit 0
EOF
chmod +x "$FAKEBIN"/*
pass=0
fail=0
check() { # $1=desc, $2=condition
if eval "$2"; then
echo "PASS: $1"
pass=$((pass + 1))
else
echo "FAIL: $1"
fail=$((fail + 1))
fi
}
# Test 1: service already active → skip, never unmount/mount/start
export SYSTEMCTL_STATE=active
rm -f "$MOUNT_MARKER" "$START_MARKER" "$LOGGER_FILE" "$MOUNT_LOG" "$FSCK_LOG"
PATH="$FAKEBIN:$PATH" DASHCAM_MAX_WAIT=0 "$SCRIPT" sde1
check "service actief → exit zonder mount" 'test ! -e "$MOUNT_MARKER"'
check "service actief → geen service start" 'test ! -e "$START_MARKER"'
check "service actief → skip gelogd" 'grep -q "al bezig" "$LOGGER_FILE"'
# Test 2: service inactive → mount, fsck, start
export SYSTEMCTL_STATE=inactive
rm -f "$MOUNT_MARKER" "$START_MARKER" "$LOGGER_FILE" "$MOUNT_LOG" "$FSCK_LOG"
PATH="$FAKEBIN:$PATH" DASHCAM_MAX_WAIT=0 "$SCRIPT" sde1
check "service inactief → gemount" 'test -e "$MOUNT_MARKER"'
check "service inactief → service gestart" 'test -e "$START_MARKER"'
check "fsck.exfat uitgevoerd" 'test -s "$FSCK_LOG"'
check "mount gelogd" 'grep -q "mounted, starting transfer" "$LOGGER_FILE"'
echo "=== $pass pass, $fail fail ==="
[ "$fail" -eq 0 ]

View file

@ -0,0 +1,12 @@
# Dashcam SD card trigger — deploy naar /etc/udev/rules.d/99-dashcam-trigger.rules
# udev draait deze via systemd-run (niet direct) omdat de udev-context geen
# mount/umount-rechten heeft op Fedora met SELinux enforcing.
# %k = kernel device name (bv. sde1), doorgegeven als $1.
# ADD: kaart ingeplugd → mount + start transfer
ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="14cd", ENV{ID_MODEL_ID}=="1212", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k"
ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="05e3", ENV{ID_MODEL_ID}=="0751", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-mount %k"
# REMOVE: kaartlezer eruit → stale mount opruimen + mislukte transfer stoppen
ACTION=="remove", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="14cd", ENV{ID_MODEL_ID}=="1212", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-cleanup %k"
ACTION=="remove", SUBSYSTEM=="block", KERNEL=="sd*[0-9]", ENV{ID_VENDOR_ID}=="05e3", ENV{ID_MODEL_ID}=="0751", RUN+="/usr/bin/systemd-run --no-block --collect /usr/local/bin/dashcam-transfer-cleanup %k"