fix(bundle): carry the Hailo HEF in the offline bundle, ONNX optional (P5-3)

The air-gapped bundle never fetched the Hailo HEF, so an offline Hailo
install had no pose model. Two latent breaks also made the offline path
unusable regardless of P5-3:

- build [2/5] ran `fetch_movenet` unconditionally; with the MoveNet
  source unreachable that aborted the whole build under `set -e`.
- install_offline copied only `*.onnx`, so even a HEF placed in the
  bundle never reached $PREFIX/models for the backend gate to find.

Fixes:
- build_offline_bundle.sh fetches the $HAILO_ARCH (default hailo8) HEF
  via edge.tools.fetch_hailo_hef (pinned sha256, aborts on mismatch) and
  stages yolov8s_pose.hef. MoveNet ONNX is now optional (guarded fetch →
  Hailo-only bundle if unavailable). onnxruntime moves to a best-effort
  optional wheel. MANIFEST lists models + available pose_backends.
- install_offline.sh copies *.hef (and tolerates a missing *.onnx).

Tests (test_deploy_artifacts.py): bundle fetches+stages the HEF, MoveNet
is optional (guarded), installer copies the HEF. Both scripts pass
`bash -n`. Full python suite: 152 passed, 2 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Xmn8KZWqiQUguEMRtn6rW

deploy/bundle/build_offline_bundle.sh

@@ -25,7 +25,12 @@ WHEELHOUSE="$STAGE/wheelhouse"
 PYVER="${PYVER:-3.11}"                       # Pi OS Bookworm ships CPython 3.11
 PYABI="cp${PYVER//./}"                        # cp311
 TARGET_ARCH="${TARGET_ARCH:-aarch64}"
-RUNTIME_PKGS=(numpy requests onnxruntime opencv-python-headless)
+HAILO_ARCH="${HAILO_ARCH:-hailo8}"            # 26 TOPS AI HAT+ (Hailo-8); set hailo8l for the 13 TOPS AI Kit
+# Mandatory for EVERY backend (cv2/numpy/requests are on the unconditional startup chain).
+# onnxruntime is the CPU-MoveNet fallback only and publishes no 32-bit-ARM wheel / no sdist, so
+# it is downloaded best-effort — a Hailo bundle need not carry it (install_offline treats it optional).
+RUNTIME_PKGS=(numpy requests opencv-python-headless)
+OPTIONAL_PKGS=(onnxruntime)
 
 MODE="native"
 [ "${1:-}" = "--cross" ] && MODE="cross"
@@ -33,12 +38,13 @@ MODE="native"
 echo ">> build mode      : $MODE"
 echo ">> python target   : $PYVER ($PYABI)"
 echo ">> target arch     : $TARGET_ARCH"
+echo ">> hailo arch      : $HAILO_ARCH"
 echo ">> output          : $OUT_DIR"
 
 rm -rf "$STAGE"
 mkdir -p "$WHEELHOUSE" "$STAGE/src" "$STAGE/models" "$STAGE/deploy/systemd" "$STAGE/docs"
 
-echo ">> [1/5] downloading runtime wheels -> wheelhouse/"
+echo ">> [1/6] downloading runtime wheels -> wheelhouse/"
 if [ "$MODE" = "cross" ]; then
   # Best-effort cross download. Try the common manylinux aarch64 tags.
   python3 -m pip download \
@@ -47,6 +53,14 @@ if [ "$MODE" = "cross" ]; then
     --platform "manylinux2014_${TARGET_ARCH}" \
     --platform "manylinux_2_28_${TARGET_ARCH}" \
     -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
+  # CPU fallback wheel, best-effort — onnxruntime may have no matching aarch64/$PYABI tag.
+  python3 -m pip download \
+    --only-binary=:all: \
+    --python-version "$PYVER" --implementation cp --abi "$PYABI" \
+    --platform "manylinux2014_${TARGET_ARCH}" \
+    --platform "manylinux_2_28_${TARGET_ARCH}" \
+    -d "$WHEELHOUSE" "${OPTIONAL_PKGS[@]}" \
+    || echo "   -> onnxruntime aarch64/$PYABI wheel unavailable; Hailo-only bundle (no CPU MoveNet)."
 else
   # Native: wheels for THIS host's interpreter/arch (run on the Pi / arm64 CI).
   # Guard the ABI: numpy/onnxruntime ship version-specific cpXY wheels, so a build
@@ -69,35 +83,56 @@ else
     exit 1
   fi
   python3 -m pip download -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
+  # CPU fallback wheel, best-effort — a Hailo bundle is complete without it.
+  python3 -m pip download -d "$WHEELHOUSE" "${OPTIONAL_PKGS[@]}" \
+    || echo "   -> onnxruntime unavailable for this host; Hailo-only bundle (no CPU MoveNet)."
 fi
 
-echo ">> [2/5] fetching MoveNet ONNX model (build-machine only)"
+echo ">> [2/6] fetching MoveNet ONNX model (optional CPU fallback; build-machine only)"
-( cd "$REPO_ROOT" && python3 -m edge.tools.fetch_movenet )
+# Optional: the MoveNet source may be unreachable, and a Hailo-8 delivery runs off the HEF. Do
-cp "$REPO_ROOT/models/movenet_lightning.onnx" "$STAGE/models/"
+# not abort the whole bundle over the CPU fallback — stage it if we can, otherwise carry on.
+if ( cd "$REPO_ROOT" && python3 -m edge.tools.fetch_movenet ); then
+  cp "$REPO_ROOT/models/movenet_lightning.onnx" "$STAGE/models/"
+else
+  echo "   -> MoveNet ONNX unavailable; Hailo-only bundle (CPU fallback not included)."
+fi
 
-echo ">> [3/5] staging edge source (package + pyproject)"
+echo ">> [3/6] fetching Hailo $HAILO_ARCH yolov8s_pose HEF (build-machine only)"
+# Mandatory for a Hailo delivery: this is the primary pose backend. fetch_hailo_hef verifies the
+# download against a pinned sha256 and refuses a mismatch, so a bad/tampered HEF aborts the build.
+( cd "$REPO_ROOT" && HAILO_ARCH="$HAILO_ARCH" python3 -m edge.tools.fetch_hailo_hef --arch "$HAILO_ARCH" )
+cp "$REPO_ROOT/models/yolov8s_pose.hef" "$STAGE/models/"
+
+echo ">> [4/6] staging edge source (package + pyproject)"
 cp -r "$REPO_ROOT/edge" "$STAGE/src/edge"
 cp "$REPO_ROOT/pyproject.toml" "$STAGE/src/"
 # do not ship compiled caches
 find "$STAGE/src" -name '__pycache__' -type d -prune -exec rm -rf {} +
 
-echo ">> [4/5] staging deploy assets + install script"
+echo ">> [5/6] staging deploy assets + install script"
 cp "$REPO_ROOT/deploy/systemd/move-detect-edge.service"     "$STAGE/deploy/systemd/"
 cp "$REPO_ROOT/deploy/systemd/move-detect-edge.env.example" "$STAGE/deploy/systemd/"
 cp "$REPO_ROOT/deploy/bundle/install_offline.sh"            "$STAGE/"
 cp "$REPO_ROOT/docs/pilot-runbook.md"                       "$STAGE/docs/" 2>/dev/null || true
 chmod +x "$STAGE/install_offline.sh"
 
+# Which pose backends this bundle can actually run, so the operator knows before it crosses the
+# air gap. Hailo needs the HEF (present) + system hailo_platform on the Pi; MoveNet needs the ONNX.
+HAS_ONNX=no; [ -f "$STAGE/models/movenet_lightning.onnx" ] && HAS_ONNX=yes
 {
   echo "bundle: move-detect-edge"
   echo "built_mode: $MODE"
   echo "python: $PYVER ($PYABI)"
   echo "target_arch: $TARGET_ARCH"
+  echo "hailo_arch: $HAILO_ARCH"
+  echo "pose_backends: hailo (yolov8s_pose.hef)$([ "$HAS_ONNX" = yes ] && echo ', movenet (movenet_lightning.onnx)')"
+  echo "models:"
+  ls -1 "$STAGE/models"
   echo "wheels:"
   ls -1 "$WHEELHOUSE"
 } > "$STAGE/MANIFEST.txt"
 
-echo ">> [5/5] packaging tarball"
+echo ">> [6/6] packaging tarball"
 TARBALL="$OUT_DIR/move-detect-edge-bundle-${TARGET_ARCH}-${PYABI}.tar.gz"
 tar -C "$OUT_DIR" -czf "$TARBALL" "move-detect-edge-bundle"
 echo ">> DONE: $TARBALL"

deploy/bundle/install_offline.sh

@@ -39,7 +39,11 @@ install -d -m 0755 "$PREFIX" "$PREFIX/src" "$PREFIX/models" "$PREFIX/docs"
 rm -rf "$PREFIX/src/edge"
 cp -r "$HERE/src/edge" "$PREFIX/src/edge"
 cp "$HERE/src/pyproject.toml" "$PREFIX/src/" 2>/dev/null || true
-cp "$HERE/models/"*.onnx "$PREFIX/models/"
+# Copy whichever pose models the bundle carries. Each is optional on its own — a Hailo bundle
+# ships only the .hef, a CPU bundle only the .onnx — so a missing glob must not abort the install
+# (the [5/6] backend gate below is what refuses a bundle with NO usable backend).
+cp "$HERE/models/"*.hef  "$PREFIX/models/" 2>/dev/null || true
+cp "$HERE/models/"*.onnx "$PREFIX/models/" 2>/dev/null || true
 cp "$HERE/docs/"*.md "$PREFIX/docs/" 2>/dev/null || true
 
 echo ">> [3/6] python venv + offline wheel install (--no-index)"

docs/delivery-audit.md

@@ -206,9 +206,16 @@ misconfig/degradation fail LOUD.
 - [x] **P5-2 [H]** `deploy/bundle/build_offline_bundle.sh:55` — a native build on x86 produces
   an aarch64-named tarball full of x86_64 wheels. **Fix:** assert `uname -m == TARGET_ARCH`.
   DONE — native branch now hard-fails when uname -m != TARGET_ARCH.
-- [ ] **P5-3 [M]** `deploy/bundle/build_offline_bundle.sh:65` — the air-gapped bundle never
+- [x] **P5-3 [M]** `deploy/bundle/build_offline_bundle.sh:65` — the air-gapped bundle never
   carries the Hailo HEF, so offline Hailo install is impossible. **Fix:** stage the HEF (via
   `fetch_hailo_hef --arch`) into the bundle.
+  DONE — build now fetches the `$HAILO_ARCH` (default hailo8) HEF via `fetch_hailo_hef` (pinned
+  sha256, aborts on mismatch) and stages `yolov8s_pose.hef`. Also fixed two latent breaks that
+  made the offline path unusable regardless: the MoveNet ONNX fetch was unconditional so its dead
+  source aborted the whole build under `set -e` (now guarded/optional → Hailo-only bundle), and
+  `install_offline.sh` copied only `*.onnx` so a bundled HEF never reached `$PREFIX/models` for the
+  backend gate to find (now copies `*.hef` too). MANIFEST lists models + available pose_backends.
+  Tests: fetches/stages HEF, MoveNet optional, installer copies the HEF.
 - [ ] **P5-4 [M]** `deploy/bundle/build_offline_bundle.sh:28` — pip deps unpinned, no digests,
   MANIFEST has no hashes, bundle unsigned; spec requires SHA256/GPG + SBOM. **Fix:** pin +
   hash + manifest digests.

tests/test_deploy_artifacts.py

@@ -10,6 +10,8 @@ UNIT = os.path.join(DEPLOY, "move-detect-edge.service")
 SETUP_ONLINE = os.path.join(os.path.dirname(__file__), "..", "deploy", "setup_online.sh")
 INSTALL_OFFLINE = os.path.join(os.path.dirname(__file__), "..", "deploy", "bundle",
                                "install_offline.sh")
+BUILD_OFFLINE = os.path.join(os.path.dirname(__file__), "..", "deploy", "bundle",
+                             "build_offline_bundle.sh")
 
 # Keys the daemon consumes (edge/config.py:load_config + edge/daemon.py:main).
 REQUIRED_ENV = {
@@ -214,3 +216,30 @@ def test_offline_installer_gates_the_pose_backend():
     assert "no usable pose backend" in sh
     # It must WRITE the reconciled backend into the installed env, not leave the template's.
     assert "POSE_BACKEND=$POSE_BACKEND" in sh or "POSE_BACKEND=${POSE_BACKEND}" in sh
+
+
+def test_offline_bundle_fetches_and_stages_the_hailo_hef():
+    with open(BUILD_OFFLINE, encoding="utf-8") as f:
+        sh = f.read()
+    # A Hailo-8 delivery runs off the HEF; the offline bundle must carry it, fetched on the build
+    # machine (the air-gapped Pi cannot). Without this the installer's backend gate finds no HEF.
+    assert "edge.tools.fetch_hailo_hef" in sh
+    assert "yolov8s_pose.hef" in sh  # staged into the bundle's models/
+
+
+def test_offline_bundle_treats_movenet_onnx_as_optional():
+    with open(BUILD_OFFLINE, encoding="utf-8") as f:
+        sh = f.read()
+    # The MoveNet source can be unreachable and a Hailo bundle is complete without the CPU
+    # fallback, so the fetch must be GUARDED (inside an `if`) — never an unconditional line that
+    # aborts the whole build under `set -e`.
+    assert "fetch_movenet ); then" in sh
+    assert "Hailo-only bundle" in sh
+
+
+def test_offline_installer_copies_the_hef_into_place():
+    with open(INSTALL_OFFLINE, encoding="utf-8") as f:
+        sh = f.read()
+    # install_offline used to copy only *.onnx, so a HEF in the bundle never reached
+    # $PREFIX/models and the Hailo backend gate ([5/6]) could never see it.
+    assert 'cp "$HERE/models/"*.hef' in sh