2ea0420de7
Two independent adversarial reviews (codex CLI + a 6-dimension verification workflow) of the delivery-hardening branch surfaced ~40 findings; this closes the confirmed correctness/security/deploy blockers plus a codex re-review pass. edge: - geometry: track_displacement per-second threshold recalibrated (0.02->0.25, env-tunable via MOTION_THR, validated finite/positive); confidence-filter keypoints with a bbox-center fallback so hallucinated points don't fake motion - fall: low-confidence frames no longer cancel an active FALLING/FALLEN incident (freeze + reset recovery timer so recovery needs a fresh confident streak); baseline height only seeded from trusted frames - queue: PostClient.send() returns a tri-state SendResult; only definite 4xx count toward the poison drop cap, so a network outage (incl. 408/425/429/5xx) never deletes queued safety events; legacy rows persist their minted uuid - config: reject placeholder / http / short DEVICE_SECRET at startup - zones: warn+skip legacy pixel-space polygons that would silently match nothing cloud: - zone endpoints wired into P3-2b site-scope (cross-site read/write/delete closed) + role gate on list; catch narrowed so DB faults surface as 500 - snapshot purge keeps the DB row when unlink() fails (no untrackable PIPA orphan) - ingest dedup path re-stores missing snapshots (+ short-write guard); UNIQUE (event_id, content_hash) makes recovery idempotent under retry/race - event list cursor carries full-precision occurred_at (no float rounding at the page boundary); string keyset end-to-end - SiteScope '' fails closed (only NULL is global); dashboard fetches full page cap - runnable migration (site/event_uuid + backfill ordering + indexes); rate-limit reap fails open deploy: - offline bundle hashes EVERY file (fatal on missing/failed/injected; models in SBOM; arch in tarball name; REQUIRE_SIGNATURE option) - installer verifies before any mutation, Hailo arch preflight before install, atomic src swap, backend reconcile preserves a working operator choice - systemd DeviceAllow=char-dma_heap (the /dev/dma_heap/ dir form was inert) Verified: edge pytest 165 passed / 2 skipped; cloud phpunit 109 passed / 4 skipped; deploy scripts bash -n clean. Reviewed by codex twice (findings + fixes) with the second-pass findings folded in.
195 lines
10 KiB
Bash
Executable file
195 lines
10 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Build a fully self-contained, offline-installable bundle for the move-detect
|
|
# edge daemon (air-gapped / 폐쇄망 target — see rules/common/build-deployment.md).
|
|
#
|
|
# Run this on a BUILD machine that HAS internet. The resulting tarball carries
|
|
# every runtime wheel + the pose model + the edge source + the systemd unit, so
|
|
# the air-gapped Pi installs with ZERO network fetches (see install_offline.sh).
|
|
#
|
|
# RELIABILITY NOTE: onnxruntime and opencv publish per-architecture wheels. The
|
|
# most reliable build is a NATIVE one — run this ON an aarch64 host (a Pi 5, or
|
|
# an arm64 CI runner) so pip downloads wheels for the exact target. Cross-downloading
|
|
# aarch64 wheels from an x86 box (--cross) is best-effort and can miss a wheel whose
|
|
# tags don't match; if it fails, build natively.
|
|
#
|
|
# Usage:
|
|
# deploy/bundle/build_offline_bundle.sh # native (host arch)
|
|
# deploy/bundle/build_offline_bundle.sh --cross # cross to aarch64/cp311
|
|
# TARGET_ARCH=aarch64 PYVER=3.11 deploy/bundle/build_offline_bundle.sh --cross
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
OUT_DIR="${OUT_DIR:-$REPO_ROOT/dist}"
|
|
STAGE="$OUT_DIR/move-detect-edge-bundle"
|
|
WHEELHOUSE="$STAGE/wheelhouse"
|
|
PYVER="${PYVER:-3.11}" # Pi OS Bookworm ships CPython 3.11
|
|
PYABI="cp${PYVER//./}" # cp311
|
|
TARGET_ARCH="${TARGET_ARCH:-aarch64}"
|
|
HAILO_ARCH="${HAILO_ARCH:-hailo8}" # 26 TOPS AI HAT+ (Hailo-8); set hailo8l for the 13 TOPS AI Kit
|
|
# Mandatory for EVERY backend (cv2/numpy/requests are on the unconditional startup chain).
|
|
# onnxruntime is the CPU-MoveNet fallback only and publishes no 32-bit-ARM wheel / no sdist, so
|
|
# it is downloaded best-effort — a Hailo bundle need not carry it (install_offline treats it optional).
|
|
RUNTIME_PKGS=(numpy requests opencv-python-headless)
|
|
OPTIONAL_PKGS=(onnxruntime)
|
|
|
|
MODE="native"
|
|
[ "${1:-}" = "--cross" ] && MODE="cross"
|
|
|
|
echo ">> build mode : $MODE"
|
|
echo ">> python target : $PYVER ($PYABI)"
|
|
echo ">> target arch : $TARGET_ARCH"
|
|
echo ">> hailo arch : $HAILO_ARCH"
|
|
echo ">> output : $OUT_DIR"
|
|
|
|
rm -rf "$STAGE"
|
|
mkdir -p "$WHEELHOUSE" "$STAGE/src" "$STAGE/models" "$STAGE/deploy/systemd" "$STAGE/docs"
|
|
|
|
echo ">> [1/6] downloading runtime wheels -> wheelhouse/"
|
|
if [ "$MODE" = "cross" ]; then
|
|
# Best-effort cross download. Try the common manylinux aarch64 tags.
|
|
python3 -m pip download \
|
|
--only-binary=:all: \
|
|
--python-version "$PYVER" --implementation cp --abi "$PYABI" \
|
|
--platform "manylinux2014_${TARGET_ARCH}" \
|
|
--platform "manylinux_2_28_${TARGET_ARCH}" \
|
|
-d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
|
# CPU fallback wheel, best-effort — onnxruntime may have no matching aarch64/$PYABI tag.
|
|
python3 -m pip download \
|
|
--only-binary=:all: \
|
|
--python-version "$PYVER" --implementation cp --abi "$PYABI" \
|
|
--platform "manylinux2014_${TARGET_ARCH}" \
|
|
--platform "manylinux_2_28_${TARGET_ARCH}" \
|
|
-d "$WHEELHOUSE" "${OPTIONAL_PKGS[@]}" \
|
|
|| echo " -> onnxruntime aarch64/$PYABI wheel unavailable; Hailo-only bundle (no CPU MoveNet)."
|
|
else
|
|
# Native: wheels for THIS host's interpreter/arch (run on the Pi / arm64 CI).
|
|
# Guard the ABI: numpy/onnxruntime ship version-specific cpXY wheels, so a build
|
|
# host whose python3 != target PYVER produces a wheelhouse the Pi's cp311 venv
|
|
# cannot install offline. Fail loudly instead of shipping a broken bundle.
|
|
HOST_PYVER="$(python3 -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')"
|
|
if [ "$HOST_PYVER" != "$PYVER" ]; then
|
|
echo "ERROR: native build host python3 is $HOST_PYVER but target PYVER is $PYVER." >&2
|
|
echo " Run with python$PYVER (e.g. 'PYVER=$HOST_PYVER ...' only if the Pi runs $HOST_PYVER)," >&2
|
|
echo " or use --cross to pin aarch64/$PYABI wheels explicitly." >&2
|
|
exit 1
|
|
fi
|
|
# Guard the ARCH too: a native build on an x86_64 workstation would download x86_64 wheels
|
|
# while the tarball is still named/declared aarch64 (line 27/86/92) — a bundle that installs
|
|
# then fails to import on the Pi. numpy/opencv/onnxruntime are per-arch, so this is fatal.
|
|
HOST_ARCH="$(uname -m)"
|
|
if [ "$HOST_ARCH" != "$TARGET_ARCH" ]; then
|
|
echo "ERROR: native build host arch is $HOST_ARCH but TARGET_ARCH is $TARGET_ARCH." >&2
|
|
echo " Build natively ON an $TARGET_ARCH host (a Pi 5 / arm64 CI), or use --cross." >&2
|
|
exit 1
|
|
fi
|
|
python3 -m pip download -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
|
# CPU fallback wheel, best-effort — a Hailo bundle is complete without it.
|
|
python3 -m pip download -d "$WHEELHOUSE" "${OPTIONAL_PKGS[@]}" \
|
|
|| echo " -> onnxruntime unavailable for this host; Hailo-only bundle (no CPU MoveNet)."
|
|
fi
|
|
|
|
echo ">> [2/6] fetching MoveNet ONNX model (optional CPU fallback; build-machine only)"
|
|
# Optional: the MoveNet source may be unreachable, and a Hailo-8 delivery runs off the HEF. Do
|
|
# not abort the whole bundle over the CPU fallback — stage it if we can, otherwise carry on.
|
|
if ( cd "$REPO_ROOT" && python3 -m edge.tools.fetch_movenet ); then
|
|
cp "$REPO_ROOT/models/movenet_lightning.onnx" "$STAGE/models/"
|
|
else
|
|
echo " -> MoveNet ONNX unavailable; Hailo-only bundle (CPU fallback not included)."
|
|
fi
|
|
|
|
echo ">> [3/6] fetching Hailo $HAILO_ARCH yolov8s_pose HEF (build-machine only)"
|
|
# Mandatory for a Hailo delivery: this is the primary pose backend. fetch_hailo_hef verifies the
|
|
# download against a pinned sha256 and refuses a mismatch, so a bad/tampered HEF aborts the build.
|
|
( cd "$REPO_ROOT" && HAILO_ARCH="$HAILO_ARCH" python3 -m edge.tools.fetch_hailo_hef --arch "$HAILO_ARCH" )
|
|
cp "$REPO_ROOT/models/yolov8s_pose.hef" "$STAGE/models/"
|
|
|
|
echo ">> [4/6] staging edge source (package + pyproject)"
|
|
cp -r "$REPO_ROOT/edge" "$STAGE/src/edge"
|
|
cp "$REPO_ROOT/pyproject.toml" "$STAGE/src/"
|
|
# do not ship compiled caches
|
|
find "$STAGE/src" -name '__pycache__' -type d -prune -exec rm -rf {} +
|
|
|
|
echo ">> [5/6] staging deploy assets + install script"
|
|
cp "$REPO_ROOT/deploy/systemd/move-detect-edge.service" "$STAGE/deploy/systemd/"
|
|
cp "$REPO_ROOT/deploy/systemd/move-detect-edge.env.example" "$STAGE/deploy/systemd/"
|
|
cp "$REPO_ROOT/deploy/bundle/install_offline.sh" "$STAGE/"
|
|
cp "$REPO_ROOT/docs/pilot-runbook.md" "$STAGE/docs/" 2>/dev/null || true
|
|
chmod +x "$STAGE/install_offline.sh"
|
|
|
|
# Supply-chain integrity (air-gap: the target can't reach PyPI to re-resolve, so pin + hash here).
|
|
# requirements.lock — exact name==version of every bundled wheel, for a reproducible rebuild.
|
|
# sbom.json — CycloneDX component list: wheels AND models (name, version, sha256).
|
|
# SHA256SUMS — sha256 of EVERY staged file (code, unit, installer, env template, models,
|
|
# wheels, lock, sbom, MANIFEST). Generated LAST so it covers everything
|
|
# install_offline.sh lays down and runs as root — not just the wheels/models.
|
|
echo ">> generating requirements.lock + sbom.json"
|
|
: > "$STAGE/requirements.lock"
|
|
{
|
|
printf '{\n "bomFormat": "CycloneDX",\n "specVersion": "1.5",\n "version": 1,\n "components": [\n'
|
|
sep=""
|
|
for whl in "$WHEELHOUSE"/*.whl; do
|
|
[ -e "$whl" ] || continue
|
|
base="$(basename "$whl")"
|
|
name="$(echo "$base" | cut -d- -f1)" # wheel: {name}-{version}-{pyabi}-...-{plat}.whl
|
|
ver="$(echo "$base" | cut -d- -f2)"
|
|
sum="$(sha256sum "$whl" | cut -d' ' -f1)"
|
|
echo "${name}==${ver}" >> "$STAGE/requirements.lock"
|
|
printf '%s {"type":"library","name":"%s","version":"%s","purl":"pkg:pypi/%s@%s","hashes":[{"alg":"SHA-256","content":"%s"}]}' \
|
|
"$sep" "$name" "$ver" "$name" "$ver" "$sum"
|
|
sep=$',\n'
|
|
done
|
|
# ML weights are shipped artifacts too — record them so a customer-side SBOM audit is complete.
|
|
for mdl in "$STAGE"/models/*; do
|
|
[ -e "$mdl" ] || continue
|
|
mbase="$(basename "$mdl")"
|
|
msum="$(sha256sum "$mdl" | cut -d' ' -f1)"
|
|
printf '%s {"type":"file","name":"%s","hashes":[{"alg":"SHA-256","content":"%s"}]}' \
|
|
"$sep" "$mbase" "$msum"
|
|
sep=$',\n'
|
|
done
|
|
printf '\n ]\n}\n'
|
|
} > "$STAGE/sbom.json"
|
|
sort -u -o "$STAGE/requirements.lock" "$STAGE/requirements.lock"
|
|
|
|
# Which pose backends this bundle can actually run, so the operator knows before it crosses the
|
|
# air gap. Hailo needs the HEF (present) + system hailo_platform on the Pi; MoveNet needs the ONNX.
|
|
HAS_ONNX=no; [ -f "$STAGE/models/movenet_lightning.onnx" ] && HAS_ONNX=yes
|
|
SIGNED=no; [ -n "${SIGN_KEY:-}" ] && SIGNED=yes
|
|
{
|
|
echo "bundle: move-detect-edge"
|
|
echo "built_mode: $MODE"
|
|
echo "python: $PYVER ($PYABI)"
|
|
echo "target_arch: $TARGET_ARCH"
|
|
echo "hailo_arch: $HAILO_ARCH"
|
|
echo "pose_backends: hailo (yolov8s_pose.hef)$([ "$HAS_ONNX" = yes ] && echo ', movenet (movenet_lightning.onnx)')"
|
|
echo "integrity: SHA256SUMS covers ALL bundle files (verified on install)$([ "$SIGNED" = yes ] && echo ', SHA256SUMS.asc (signed)')"
|
|
echo "models:"
|
|
ls -1 "$STAGE/models"
|
|
echo "wheels:"
|
|
ls -1 "$WHEELHOUSE"
|
|
} > "$STAGE/MANIFEST.txt"
|
|
|
|
# SHA256SUMS LAST: hash every staged file EXCEPT the manifest itself and its detached signature,
|
|
# so the daemon source, systemd unit, installer, env template, models and wheels are ALL covered.
|
|
# A tampered edge/daemon.py can no longer pass `sha256sum -c` and then be installed and run.
|
|
( cd "$STAGE" && find . -type f ! -name SHA256SUMS ! -name 'SHA256SUMS.asc' -print0 \
|
|
| sort -z | xargs -0 sha256sum > SHA256SUMS )
|
|
|
|
# Optional detached GPG signature over the integrity file. When present, install_offline.sh
|
|
# REQUIRES it to verify (a failed signature is fatal there); when absent the bundle offers
|
|
# corruption-detection only. Provenance-critical deliveries MUST set SIGN_KEY.
|
|
if [ -n "${SIGN_KEY:-}" ]; then
|
|
gpg --local-user "$SIGN_KEY" --armor --detach-sign --output "$STAGE/SHA256SUMS.asc" "$STAGE/SHA256SUMS"
|
|
echo ">> signed SHA256SUMS with key $SIGN_KEY -> SHA256SUMS.asc"
|
|
else
|
|
echo ">> NOTE: SHA256SUMS is unsigned (corruption-detection only). Set SIGN_KEY=<gpg-id> for a"
|
|
echo ">> detached signature; install_offline.sh will then require it to verify."
|
|
fi
|
|
|
|
echo ">> [6/6] packaging tarball"
|
|
# Embed the Hailo arch in the filename: hailo8 and hailo8l bundles are byte-different but were
|
|
# otherwise identically named, so the wrong one could be shipped to a board and crash-loop.
|
|
TARBALL="$OUT_DIR/move-detect-edge-bundle-${TARGET_ARCH}-${PYABI}-${HAILO_ARCH}.tar.gz"
|
|
tar -C "$OUT_DIR" -czf "$TARBALL" "move-detect-edge-bundle"
|
|
echo ">> DONE: $TARBALL"
|
|
echo ">> transfer this single file across the air gap, then run install_offline.sh on the Pi."
|