feat(deploy): air-gapped offline bundle + systemd unit for edge daemon

Offline-installable bundle (wheelhouse + model + source) built on a machine with internet and installed on the air-gapped Pi with --no-index (zero fetches), plus a systemd unit (auto-restart, device access, hardening) and env template. Runs the package from source via python -m edge (no offline build backend). Drift-guard test keeps the env template in sync with what the daemon reads and rejects inline comments that systemd EnvironmentFile= would fold into values.
2026-07-12 17:51:45 +09:00 · 2026-07-12 17:51:45 +09:00 · 2ffa74b639
commit 2ffa74b639
parent 6c8fd9318f
6 changed files with 398 additions and 0 deletions
--- a/deploy/README.md
+++ b/deploy/README.md
@ -0,0 +1,86 @@
+# Edge deployment (Raspberry Pi 5, air-gapped / 폐쇄망)
+
+Offline-installable bundle + systemd unit for the `move-detect` edge daemon.
+The bundle is fully self-contained (all wheels + pose model + source): the Pi
+performs **zero network fetches** at install or run time. The only runtime
+network use is the cloud POST / config-poll / heartbeat, and even that tolerates
+being offline (events buffer in the local sqlite queue and flush on reconnect).
+
+> Scope: this covers the **CPU MoveNet + USB (UVC) webcam** path — the one the
+> daemon wires today (`edge/daemon.py:main`) and the one verified end-to-end.
+> Hailo-8 acceleration (`edge/adapters/hailo_pose.py`) and the CSI camera
+> (`edge/adapters/picamera_source.py`) are separate device-validated paths — see
+> `docs/hardware-bringup.md`.
+
+## Files
+
+| File | Runs on | Purpose |
+|------|---------|---------|
+| `bundle/build_offline_bundle.sh` | build machine (has internet) | downloads wheels + model, stages source, emits one tarball |
+| `bundle/install_offline.sh` | the Pi (no internet) | venv + `--no-index` install, systemd unit, service account |
+| `systemd/move-detect-edge.service` | the Pi | the daemon unit (auto-restart, device access, hardening) |
+| `systemd/move-detect-edge.env.example` | the Pi | env template → `/etc/move-detect/edge.env` |
+
+## 1. Build the bundle (on a machine WITH internet)
+
+**Most reliable: build on an aarch64 host** (a Pi 5, or an arm64 CI runner) so the
+downloaded wheels match the target exactly:
+
+```bash
+deploy/bundle/build_offline_bundle.sh            # native aarch64 wheels
+```
+
+From an x86 box you can attempt a cross build (best-effort — onnxruntime/opencv
+wheel tags may not resolve; if it fails, build natively):
+
+```bash
+deploy/bundle/build_offline_bundle.sh --cross    # target aarch64 / cp311
+```
+
+Output: `dist/move-detect-edge-bundle-aarch64-cp311.tar.gz` (+ `MANIFEST.txt`
+listing the bundled wheels). This single file crosses the air gap.
+
+## 2. Transfer + install (on the air-gapped Pi)
+
+Copy the tarball across the gap (USB, one-way transfer, etc.), then:
+
+```bash
+tar xzf move-detect-edge-bundle-aarch64-cp311.tar.gz
+cd move-detect-edge-bundle
+sudo ./install_offline.sh
+```
+
+The installer: creates the `movedetect` service account (in `video`/`render`/
+`plugdev`/`hailo` groups), lays down source + model under `/opt/move-detect`,
+builds a venv and installs the bundled wheels with `--no-index`, verifies the
+package imports, seeds `/etc/move-detect/edge.env`, and installs the systemd unit.
+
+## 3. Configure + start
+
+```bash
+sudo nano /etc/move-detect/edge.env      # DEVICE_ID, DEVICE_SECRET, CAFE24_URL, ...
+sudo systemctl enable --now move-detect-edge
+journalctl -u move-detect-edge -f
+```
+
+`DEVICE_ID`/`DEVICE_SECRET` must match a `devices` row in the cloud (see
+`cloud/docs/cloud-deploy.md` §6). `MOVENET_ONNX` should point at the bundled
+model (`/opt/move-detect/models/movenet_lightning.onnx`).
+
+## 4. Verify the air gap (do this before trusting a build)
+
+- [ ] On a host with **networking disabled**, `install_offline.sh` completes —
+      the `--no-index` installs prove nothing is fetched. If pip tries to reach
+      PyPI, a wheel is missing from the bundle: rebuild (step 1) natively.
+- [ ] Step [4/6] of the installer prints `edge import OK` (onnxruntime + cv2 +
+      numpy + requests all import from the venv).
+- [ ] `systemctl status move-detect-edge` is `active (running)`; the log shows
+      frames processed and (if the cloud is reachable) heartbeats.
+- [ ] Pull the network cable → events keep queueing (`QUEUE_DB` grows) → restore
+      → the queue flushes. (End-to-end behavior verified in the edge e2e harness.)
+
+## Not bundled (by design)
+
+- **MySQL / the cloud app** — a separate host; see `cloud/docs/cloud-deploy.md`.
+- **HailoRT + `yolov8s_pose.hef`** — the Hailo runtime and HEF are installed via
+  Hailo's own Pi packages; the CPU MoveNet path here needs neither.
--- a/deploy/bundle/build_offline_bundle.sh
+++ b/deploy/bundle/build_offline_bundle.sh
@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+# Build a fully self-contained, offline-installable bundle for the move-detect
+# edge daemon (air-gapped / 폐쇄망 target — see rules/common/build-deployment.md).
+#
+# Run this on a BUILD machine that HAS internet. The resulting tarball carries
+# every runtime wheel + the pose model + the edge source + the systemd unit, so
+# the air-gapped Pi installs with ZERO network fetches (see install_offline.sh).
+#
+# RELIABILITY NOTE: onnxruntime and opencv publish per-architecture wheels. The
+# most reliable build is a NATIVE one — run this ON an aarch64 host (a Pi 5, or
+# an arm64 CI runner) so pip downloads wheels for the exact target. Cross-downloading
+# aarch64 wheels from an x86 box (--cross) is best-effort and can miss a wheel whose
+# tags don't match; if it fails, build natively.
+#
+# Usage:
+#   deploy/bundle/build_offline_bundle.sh                 # native (host arch)
+#   deploy/bundle/build_offline_bundle.sh --cross         # cross to aarch64/cp311
+#   TARGET_ARCH=aarch64 PYVER=3.11 deploy/bundle/build_offline_bundle.sh --cross
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+OUT_DIR="${OUT_DIR:-$REPO_ROOT/dist}"
+STAGE="$OUT_DIR/move-detect-edge-bundle"
+WHEELHOUSE="$STAGE/wheelhouse"
+PYVER="${PYVER:-3.11}"                       # Pi OS Bookworm ships CPython 3.11
+PYABI="cp${PYVER//./}"                        # cp311
+TARGET_ARCH="${TARGET_ARCH:-aarch64}"
+RUNTIME_PKGS=(numpy requests onnxruntime opencv-python-headless)
+
+MODE="native"
+[ "${1:-}" = "--cross" ] && MODE="cross"
+
+echo ">> build mode      : $MODE"
+echo ">> python target   : $PYVER ($PYABI)"
+echo ">> target arch     : $TARGET_ARCH"
+echo ">> output          : $OUT_DIR"
+
+rm -rf "$STAGE"
+mkdir -p "$WHEELHOUSE" "$STAGE/src" "$STAGE/models" "$STAGE/deploy/systemd" "$STAGE/docs"
+
+echo ">> [1/5] downloading runtime wheels -> wheelhouse/"
+if [ "$MODE" = "cross" ]; then
+  # Best-effort cross download. Try the common manylinux aarch64 tags.
+  python3 -m pip download \
+    --only-binary=:all: \
+    --python-version "$PYVER" --implementation cp --abi "$PYABI" \
+    --platform "manylinux2014_${TARGET_ARCH}" \
+    --platform "manylinux_2_28_${TARGET_ARCH}" \
+    -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
+else
+  # Native: wheels for THIS host's interpreter/arch (run on the Pi / arm64 CI).
+  # Guard the ABI: numpy/onnxruntime ship version-specific cpXY wheels, so a build
+  # host whose python3 != target PYVER produces a wheelhouse the Pi's cp311 venv
+  # cannot install offline. Fail loudly instead of shipping a broken bundle.
+  HOST_PYVER="$(python3 -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')"
+  if [ "$HOST_PYVER" != "$PYVER" ]; then
+    echo "ERROR: native build host python3 is $HOST_PYVER but target PYVER is $PYVER." >&2
+    echo "       Run with python$PYVER (e.g. 'PYVER=$HOST_PYVER ...' only if the Pi runs $HOST_PYVER)," >&2
+    echo "       or use --cross to pin aarch64/$PYABI wheels explicitly." >&2
+    exit 1
+  fi
+  python3 -m pip download -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
+fi
+
+echo ">> [2/5] fetching MoveNet ONNX model (build-machine only)"
+( cd "$REPO_ROOT" && python3 -m edge.tools.fetch_movenet )
+cp "$REPO_ROOT/models/movenet_lightning.onnx" "$STAGE/models/"
+
+echo ">> [3/5] staging edge source (package + pyproject)"
+cp -r "$REPO_ROOT/edge" "$STAGE/src/edge"
+cp "$REPO_ROOT/pyproject.toml" "$STAGE/src/"
+# do not ship compiled caches
+find "$STAGE/src" -name '__pycache__' -type d -prune -exec rm -rf {} +
+
+echo ">> [4/5] staging deploy assets + install script"
+cp "$REPO_ROOT/deploy/systemd/move-detect-edge.service"     "$STAGE/deploy/systemd/"
+cp "$REPO_ROOT/deploy/systemd/move-detect-edge.env.example" "$STAGE/deploy/systemd/"
+cp "$REPO_ROOT/deploy/bundle/install_offline.sh"            "$STAGE/"
+cp "$REPO_ROOT/docs/pilot-runbook.md"                       "$STAGE/docs/" 2>/dev/null || true
+chmod +x "$STAGE/install_offline.sh"
+
+{
+  echo "bundle: move-detect-edge"
+  echo "built_mode: $MODE"
+  echo "python: $PYVER ($PYABI)"
+  echo "target_arch: $TARGET_ARCH"
+  echo "wheels:"
+  ls -1 "$WHEELHOUSE"
+} > "$STAGE/MANIFEST.txt"
+
+echo ">> [5/5] packaging tarball"
+TARBALL="$OUT_DIR/move-detect-edge-bundle-${TARGET_ARCH}-${PYABI}.tar.gz"
+tar -C "$OUT_DIR" -czf "$TARBALL" "move-detect-edge-bundle"
+echo ">> DONE: $TARBALL"
+echo ">> transfer this single file across the air gap, then run install_offline.sh on the Pi."
--- a/deploy/bundle/install_offline.sh
+++ b/deploy/bundle/install_offline.sh
@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# Offline installer for the move-detect edge daemon. Runs on the air-gapped Pi 5
+# (NO internet). Unpack the bundle produced by build_offline_bundle.sh, cd into it,
+# then: sudo ./install_offline.sh
+#
+# Installs everything from the bundled wheelhouse with --no-index (zero fetches),
+# runs the package straight from source (no build backend needed offline).
+set -euo pipefail
+
+if [ "$(id -u)" -ne 0 ]; then
+  echo "must run as root (sudo ./install_offline.sh)" >&2
+  exit 1
+fi
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PREFIX=/opt/move-detect
+STATE=/var/lib/move-detect
+ETC=/etc/move-detect
+SVC_USER=movedetect
+WHEELHOUSE="$HERE/wheelhouse"
+RUNTIME_PKGS=(numpy requests onnxruntime opencv-python-headless)
+
+echo ">> [1/6] service account + groups"
+if ! id -u "$SVC_USER" >/dev/null 2>&1; then
+  useradd --system --no-create-home --shell /usr/sbin/nologin "$SVC_USER"
+fi
+# camera (video), Hailo accelerator (render/plugdev vary by image) — ignore missing groups
+for g in video render plugdev hailo; do
+  getent group "$g" >/dev/null 2>&1 && usermod -aG "$g" "$SVC_USER" || true
+done
+
+echo ">> [2/6] lay down source, model, docs under $PREFIX"
+install -d -m 0755 "$PREFIX" "$PREFIX/src" "$PREFIX/models" "$PREFIX/docs"
+# Fully replace the package tree on re-install/upgrade. Without the rm, `cp -r SRC DEST`
+# nests SRC *inside* an existing DEST (edge/edge/...) and the OLD code keeps running.
+rm -rf "$PREFIX/src/edge"
+cp -r "$HERE/src/edge" "$PREFIX/src/edge"
+cp "$HERE/src/pyproject.toml" "$PREFIX/src/" 2>/dev/null || true
+cp "$HERE/models/"*.onnx "$PREFIX/models/"
+cp "$HERE/docs/"*.md "$PREFIX/docs/" 2>/dev/null || true
+
+echo ">> [3/6] python venv + offline wheel install (--no-index)"
+python3 -m venv "$PREFIX/venv"
+"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" --upgrade pip >/dev/null 2>&1 || true
+"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
+
+echo ">> [4/6] verify the edge package imports offline"
+PYTHONPATH="$PREFIX/src" "$PREFIX/venv/bin/python" -c "import edge.daemon, onnxruntime, cv2, numpy, requests; print('edge import OK')"
+
+echo ">> [5/6] state dir + env file"
+install -d -m 0750 "$STATE"
+chown "$SVC_USER":"$SVC_USER" "$STATE"
+install -d -m 0750 "$ETC"
+if [ ! -f "$ETC/edge.env" ]; then
+  install -m 0640 "$HERE/deploy/systemd/move-detect-edge.env.example" "$ETC/edge.env"
+  chown root:"$SVC_USER" "$ETC/edge.env"
+  echo "   -> wrote $ETC/edge.env from template (EDIT IT: DEVICE_ID, DEVICE_SECRET, CAFE24_URL)"
+else
+  echo "   -> $ETC/edge.env already exists; left unchanged"
+fi
+
+echo ">> [6/6] systemd unit"
+install -m 0644 "$HERE/deploy/systemd/move-detect-edge.service" /etc/systemd/system/
+systemctl daemon-reload
+
+echo ""
+echo "Installed. Next:"
+echo "  1. edit $ETC/edge.env (device id/secret, cloud URL; MOVENET_ONNX=$PREFIX/models/movenet_lightning.onnx)"
+echo "  2. sudo systemctl enable --now move-detect-edge"
+echo "  3. journalctl -u move-detect-edge -f"
--- a/deploy/systemd/move-detect-edge.env.example
+++ b/deploy/systemd/move-detect-edge.env.example
@ -0,0 +1,40 @@
+# move-detect edge daemon environment — copy to /etc/move-detect/edge.env and fill in.
+# Consumed by edge.config.load_config() and edge.daemon.main().
+# Keep this file 0640 root:movedetect — it holds the per-device HMAC secret.
+#
+# IMPORTANT: systemd EnvironmentFile= does NOT strip inline comments, so every active
+# line below is a bare KEY=value. All comments live on their own lines.
+
+# --- identity / cloud (required) ---
+# DEVICE_ID must match the devices.device_id row in the cloud.
+DEVICE_ID=pi-01
+# DEVICE_SECRET: per-device HMAC secret; must match devices.secret.
+DEVICE_SECRET=CHANGEME
+# CAFE24_URL: cloud base URL (POST events / poll config / heartbeat).
+CAFE24_URL=https://cloud.example.com
+# CAMERA_ID: stamped on events and matched to dashboard zones.
+CAMERA_ID=cam-01
+
+# --- pose backend / capture ---
+# POSE_BACKEND: "movenet" (CPU, needs MOVENET_ONNX) or "hailo" (needs HAILO_HEF).
+POSE_BACKEND=movenet
+# MOVENET_ONNX: required when POSE_BACKEND=movenet.
+MOVENET_ONNX=/opt/move-detect/models/movenet_lightning.onnx
+# HAILO_HEF: required when POSE_BACKEND=hailo (uncomment and set).
+#HAILO_HEF=/opt/move-detect/models/yolov8s_pose.hef
+# UVC_DEVICE: OpenCV VideoCapture index for the USB (UVC) webcam.
+UVC_DEVICE=0
+# TARGET_FPS: frame pacing = 1/TARGET_FPS.
+TARGET_FPS=12
+
+# --- offline queue (required) ---
+# QUEUE_DB: sqlite buffer for events while the cloud is unreachable.
+QUEUE_DB=/var/lib/move-detect/queue.db
+
+# --- detector tuning (optional; defaults shown) ---
+#FALL_PRONE_S=3.0
+#FALL_NO_MOTION_S=10.0
+#IMMOBILE_S=120.0
+#GHOST_REAPPEAR_S=10.0
+#ZONE_DEBOUNCE_S=0.75
+#COOLDOWN_S=30.0
--- a/deploy/systemd/move-detect-edge.service
+++ b/deploy/systemd/move-detect-edge.service
@ -0,0 +1,50 @@
+# move-detect edge daemon — systemd unit (Raspberry Pi 5, air-gapped/폐쇄망).
+# Install: copy to /etc/systemd/system/, fill /etc/move-detect/edge.env, then
+#   sudo systemctl daemon-reload && sudo systemctl enable --now move-detect-edge
+# The daemon tolerates the cloud being offline (events buffer in the local queue
+# and flush on reconnect), so this unit only *wants* the network, not requires it.
+
+[Unit]
+Description=move-detect edge daemon (workplace-safety AI CCTV)
+Documentation=file:/opt/move-detect/docs/pilot-runbook.md
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+Type=simple
+# Dedicated unprivileged account; must be in the groups that own the camera and
+# the Hailo accelerator device nodes.
+User=movedetect
+Group=movedetect
+SupplementaryGroups=video render plugdev
+
+WorkingDirectory=/opt/move-detect
+EnvironmentFile=/etc/move-detect/edge.env
+# Run the package straight from source (no pip build step needed offline).
+Environment=PYTHONPATH=/opt/move-detect/src
+ExecStart=/opt/move-detect/venv/bin/python -m edge
+
+# Unattended resilience: always restart; the loop also isolates per-frame and
+# periodic-op exceptions internally so a single glitch never exits the process.
+Restart=always
+RestartSec=5
+StartLimitIntervalSec=0
+
+# Writable state (offline queue sqlite lives here; keep QUEUE_DB pointed at it).
+# systemd creates/owns /var/lib/move-detect for the service user.
+StateDirectory=move-detect
+StateDirectoryMode=0750
+
+# Camera + accelerator device access (USB UVC = /dev/video*, Hailo = /dev/hailo0).
+DeviceAllow=/dev/hailo0 rw
+DeviceAllow=char-video4linux rw
+
+# Conservative hardening (loosen if it blocks device access on your image).
+NoNewPrivileges=true
+ProtectSystem=strict
+ReadWritePaths=/var/lib/move-detect
+ProtectHome=true
+PrivateTmp=true
+
+[Install]
+WantedBy=multi-user.target
--- a/tests/test_deploy_artifacts.py
+++ b/tests/test_deploy_artifacts.py
@ -0,0 +1,57 @@
+"""Guard: the systemd unit + env template must stay in sync with what the daemon
+actually reads. If someone renames an env var in edge/config.py or edge/daemon.py
+(or edits the unit), this fails instead of shipping a broken bundle.
+"""
+import os
+
+DEPLOY = os.path.join(os.path.dirname(__file__), "..", "deploy", "systemd")
+ENV_EXAMPLE = os.path.join(DEPLOY, "move-detect-edge.env.example")
+UNIT = os.path.join(DEPLOY, "move-detect-edge.service")
+
+# Keys the daemon consumes (edge/config.py:load_config + edge/daemon.py:main).
+REQUIRED_ENV = {
+    "DEVICE_ID", "DEVICE_SECRET", "CAFE24_URL", "CAMERA_ID",
+    "MOVENET_ONNX", "UVC_DEVICE", "TARGET_FPS", "QUEUE_DB",
+}
+
+
+def _env_keys(text):
+    keys = set()
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        keys.add(line.split("=", 1)[0].strip())
+    return keys
+
+
+def test_env_example_lists_every_required_key():
+    with open(ENV_EXAMPLE, encoding="utf-8") as f:
+        keys = _env_keys(f.read())
+    missing = REQUIRED_ENV - keys
+    assert not missing, f"env template missing required keys: {sorted(missing)}"
+
+
+def test_env_example_values_have_no_inline_comments():
+    # systemd EnvironmentFile= does NOT strip inline comments — a trailing "# ..." on
+    # a KEY=value line becomes part of the value and breaks int()/float()/path/secret.
+    with open(ENV_EXAMPLE, encoding="utf-8") as f:
+        offenders = []
+        for line in f.read().splitlines():
+            s = line.strip()
+            if not s or s.startswith("#") or "=" not in s:
+                continue
+            value = s.split("=", 1)[1]
+            if "#" in value:
+                offenders.append(s)
+    assert not offenders, f"inline comments in env values break systemd: {offenders}"
+
+
+def test_systemd_unit_wiring():
+    with open(UNIT, encoding="utf-8") as f:
+        unit = f.read()
+    assert "EnvironmentFile=/etc/move-detect/edge.env" in unit
+    assert "PYTHONPATH=/opt/move-detect/src" in unit
+    assert "python -m edge" in unit          # runs the package entrypoint
+    assert "Restart=always" in unit          # unattended resilience
+    assert "WantedBy=multi-user.target" in unit