Merge feat/p1c-deploy-hailo: air-gapped edge deploy + real Hailo pose adapter
(a) Offline-installable bundle (wheelhouse + model + source) + systemd unit for the air-gapped Pi 5 edge daemon (--no-index install, zero network fetches). (b) Real HailoRT yolov8s_pose adapter (host DFL box + keypoint decode + NMS, COCO-17), replacing the stub, selectable via POSE_BACKEND; device-validated on bringup. Adversarial review clean on the decode/API; three confirmed deploy-config defects (systemd inline-comment crash, non-idempotent install, unpinned wheel ABI) fixed.
This commit is contained in:
commit
2335faa568
10 changed files with 666 additions and 10 deletions
6
.gitattributes
vendored
Normal file
6
.gitattributes
vendored
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# Shell scripts and the systemd env file run on Linux (the Pi). Force LF so a
|
||||
# CRLF checkout on Windows never ships back a script that fails with
|
||||
# "bad interpreter: /usr/bin/env bash^M" or a systemd env file with trailing \r.
|
||||
*.sh text eol=lf
|
||||
*.service text eol=lf
|
||||
move-detect-edge.env.example text eol=lf
|
||||
86
deploy/README.md
Normal file
86
deploy/README.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Edge deployment (Raspberry Pi 5, air-gapped / 폐쇄망)
|
||||
|
||||
Offline-installable bundle + systemd unit for the `move-detect` edge daemon.
|
||||
The bundle is fully self-contained (all wheels + pose model + source): the Pi
|
||||
performs **zero network fetches** at install or run time. The only runtime
|
||||
network use is the cloud POST / config-poll / heartbeat, and even that tolerates
|
||||
being offline (events buffer in the local sqlite queue and flush on reconnect).
|
||||
|
||||
> Scope: this covers the **CPU MoveNet + USB (UVC) webcam** path — the one the
|
||||
> daemon wires today (`edge/daemon.py:main`) and the one verified end-to-end.
|
||||
> Hailo-8 acceleration (`edge/adapters/hailo_pose.py`) and the CSI camera
|
||||
> (`edge/adapters/picamera_source.py`) are separate device-validated paths — see
|
||||
> `docs/hardware-bringup.md`.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Runs on | Purpose |
|
||||
|------|---------|---------|
|
||||
| `bundle/build_offline_bundle.sh` | build machine (has internet) | downloads wheels + model, stages source, emits one tarball |
|
||||
| `bundle/install_offline.sh` | the Pi (no internet) | venv + `--no-index` install, systemd unit, service account |
|
||||
| `systemd/move-detect-edge.service` | the Pi | the daemon unit (auto-restart, device access, hardening) |
|
||||
| `systemd/move-detect-edge.env.example` | the Pi | env template → `/etc/move-detect/edge.env` |
|
||||
|
||||
## 1. Build the bundle (on a machine WITH internet)
|
||||
|
||||
**Most reliable: build on an aarch64 host** (a Pi 5, or an arm64 CI runner) so the
|
||||
downloaded wheels match the target exactly:
|
||||
|
||||
```bash
|
||||
deploy/bundle/build_offline_bundle.sh # native aarch64 wheels
|
||||
```
|
||||
|
||||
From an x86 box you can attempt a cross build (best-effort — onnxruntime/opencv
|
||||
wheel tags may not resolve; if it fails, build natively):
|
||||
|
||||
```bash
|
||||
deploy/bundle/build_offline_bundle.sh --cross # target aarch64 / cp311
|
||||
```
|
||||
|
||||
Output: `dist/move-detect-edge-bundle-aarch64-cp311.tar.gz` (+ `MANIFEST.txt`
|
||||
listing the bundled wheels). This single file crosses the air gap.
|
||||
|
||||
## 2. Transfer + install (on the air-gapped Pi)
|
||||
|
||||
Copy the tarball across the gap (USB, one-way transfer, etc.), then:
|
||||
|
||||
```bash
|
||||
tar xzf move-detect-edge-bundle-aarch64-cp311.tar.gz
|
||||
cd move-detect-edge-bundle
|
||||
sudo ./install_offline.sh
|
||||
```
|
||||
|
||||
The installer: creates the `movedetect` service account (in `video`/`render`/
|
||||
`plugdev`/`hailo` groups), lays down source + model under `/opt/move-detect`,
|
||||
builds a venv and installs the bundled wheels with `--no-index`, verifies the
|
||||
package imports, seeds `/etc/move-detect/edge.env`, and installs the systemd unit.
|
||||
|
||||
## 3. Configure + start
|
||||
|
||||
```bash
|
||||
sudo nano /etc/move-detect/edge.env # DEVICE_ID, DEVICE_SECRET, CAFE24_URL, ...
|
||||
sudo systemctl enable --now move-detect-edge
|
||||
journalctl -u move-detect-edge -f
|
||||
```
|
||||
|
||||
`DEVICE_ID`/`DEVICE_SECRET` must match a `devices` row in the cloud (see
|
||||
`cloud/docs/cloud-deploy.md` §6). `MOVENET_ONNX` should point at the bundled
|
||||
model (`/opt/move-detect/models/movenet_lightning.onnx`).
|
||||
|
||||
## 4. Verify the air gap (do this before trusting a build)
|
||||
|
||||
- [ ] On a host with **networking disabled**, `install_offline.sh` completes —
|
||||
the `--no-index` installs prove nothing is fetched. If pip tries to reach
|
||||
PyPI, a wheel is missing from the bundle: rebuild (step 1) natively.
|
||||
- [ ] Step [4/6] of the installer prints `edge import OK` (onnxruntime + cv2 +
|
||||
numpy + requests all import from the venv).
|
||||
- [ ] `systemctl status move-detect-edge` is `active (running)`; the log shows
|
||||
frames processed and (if the cloud is reachable) heartbeats.
|
||||
- [ ] Pull the network cable → events keep queueing (`QUEUE_DB` grows) → restore
|
||||
→ the queue flushes. (End-to-end behavior verified in the edge e2e harness.)
|
||||
|
||||
## Not bundled (by design)
|
||||
|
||||
- **MySQL / the cloud app** — a separate host; see `cloud/docs/cloud-deploy.md`.
|
||||
- **HailoRT + `yolov8s_pose.hef`** — the Hailo runtime and HEF are installed via
|
||||
Hailo's own Pi packages; the CPU MoveNet path here needs neither.
|
||||
95
deploy/bundle/build_offline_bundle.sh
Normal file
95
deploy/bundle/build_offline_bundle.sh
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
#!/usr/bin/env bash
|
||||
# Build a fully self-contained, offline-installable bundle for the move-detect
|
||||
# edge daemon (air-gapped / 폐쇄망 target — see rules/common/build-deployment.md).
|
||||
#
|
||||
# Run this on a BUILD machine that HAS internet. The resulting tarball carries
|
||||
# every runtime wheel + the pose model + the edge source + the systemd unit, so
|
||||
# the air-gapped Pi installs with ZERO network fetches (see install_offline.sh).
|
||||
#
|
||||
# RELIABILITY NOTE: onnxruntime and opencv publish per-architecture wheels. The
|
||||
# most reliable build is a NATIVE one — run this ON an aarch64 host (a Pi 5, or
|
||||
# an arm64 CI runner) so pip downloads wheels for the exact target. Cross-downloading
|
||||
# aarch64 wheels from an x86 box (--cross) is best-effort and can miss a wheel whose
|
||||
# tags don't match; if it fails, build natively.
|
||||
#
|
||||
# Usage:
|
||||
# deploy/bundle/build_offline_bundle.sh # native (host arch)
|
||||
# deploy/bundle/build_offline_bundle.sh --cross # cross to aarch64/cp311
|
||||
# TARGET_ARCH=aarch64 PYVER=3.11 deploy/bundle/build_offline_bundle.sh --cross
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
OUT_DIR="${OUT_DIR:-$REPO_ROOT/dist}"
|
||||
STAGE="$OUT_DIR/move-detect-edge-bundle"
|
||||
WHEELHOUSE="$STAGE/wheelhouse"
|
||||
PYVER="${PYVER:-3.11}" # Pi OS Bookworm ships CPython 3.11
|
||||
PYABI="cp${PYVER//./}" # cp311
|
||||
TARGET_ARCH="${TARGET_ARCH:-aarch64}"
|
||||
RUNTIME_PKGS=(numpy requests onnxruntime opencv-python-headless)
|
||||
|
||||
MODE="native"
|
||||
[ "${1:-}" = "--cross" ] && MODE="cross"
|
||||
|
||||
echo ">> build mode : $MODE"
|
||||
echo ">> python target : $PYVER ($PYABI)"
|
||||
echo ">> target arch : $TARGET_ARCH"
|
||||
echo ">> output : $OUT_DIR"
|
||||
|
||||
rm -rf "$STAGE"
|
||||
mkdir -p "$WHEELHOUSE" "$STAGE/src" "$STAGE/models" "$STAGE/deploy/systemd" "$STAGE/docs"
|
||||
|
||||
echo ">> [1/5] downloading runtime wheels -> wheelhouse/"
|
||||
if [ "$MODE" = "cross" ]; then
|
||||
# Best-effort cross download. Try the common manylinux aarch64 tags.
|
||||
python3 -m pip download \
|
||||
--only-binary=:all: \
|
||||
--python-version "$PYVER" --implementation cp --abi "$PYABI" \
|
||||
--platform "manylinux2014_${TARGET_ARCH}" \
|
||||
--platform "manylinux_2_28_${TARGET_ARCH}" \
|
||||
-d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
||||
else
|
||||
# Native: wheels for THIS host's interpreter/arch (run on the Pi / arm64 CI).
|
||||
# Guard the ABI: numpy/onnxruntime ship version-specific cpXY wheels, so a build
|
||||
# host whose python3 != target PYVER produces a wheelhouse the Pi's cp311 venv
|
||||
# cannot install offline. Fail loudly instead of shipping a broken bundle.
|
||||
HOST_PYVER="$(python3 -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')"
|
||||
if [ "$HOST_PYVER" != "$PYVER" ]; then
|
||||
echo "ERROR: native build host python3 is $HOST_PYVER but target PYVER is $PYVER." >&2
|
||||
echo " Run with python$PYVER (e.g. 'PYVER=$HOST_PYVER ...' only if the Pi runs $HOST_PYVER)," >&2
|
||||
echo " or use --cross to pin aarch64/$PYABI wheels explicitly." >&2
|
||||
exit 1
|
||||
fi
|
||||
python3 -m pip download -d "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
||||
fi
|
||||
|
||||
echo ">> [2/5] fetching MoveNet ONNX model (build-machine only)"
|
||||
( cd "$REPO_ROOT" && python3 -m edge.tools.fetch_movenet )
|
||||
cp "$REPO_ROOT/models/movenet_lightning.onnx" "$STAGE/models/"
|
||||
|
||||
echo ">> [3/5] staging edge source (package + pyproject)"
|
||||
cp -r "$REPO_ROOT/edge" "$STAGE/src/edge"
|
||||
cp "$REPO_ROOT/pyproject.toml" "$STAGE/src/"
|
||||
# do not ship compiled caches
|
||||
find "$STAGE/src" -name '__pycache__' -type d -prune -exec rm -rf {} +
|
||||
|
||||
echo ">> [4/5] staging deploy assets + install script"
|
||||
cp "$REPO_ROOT/deploy/systemd/move-detect-edge.service" "$STAGE/deploy/systemd/"
|
||||
cp "$REPO_ROOT/deploy/systemd/move-detect-edge.env.example" "$STAGE/deploy/systemd/"
|
||||
cp "$REPO_ROOT/deploy/bundle/install_offline.sh" "$STAGE/"
|
||||
cp "$REPO_ROOT/docs/pilot-runbook.md" "$STAGE/docs/" 2>/dev/null || true
|
||||
chmod +x "$STAGE/install_offline.sh"
|
||||
|
||||
{
|
||||
echo "bundle: move-detect-edge"
|
||||
echo "built_mode: $MODE"
|
||||
echo "python: $PYVER ($PYABI)"
|
||||
echo "target_arch: $TARGET_ARCH"
|
||||
echo "wheels:"
|
||||
ls -1 "$WHEELHOUSE"
|
||||
} > "$STAGE/MANIFEST.txt"
|
||||
|
||||
echo ">> [5/5] packaging tarball"
|
||||
TARBALL="$OUT_DIR/move-detect-edge-bundle-${TARGET_ARCH}-${PYABI}.tar.gz"
|
||||
tar -C "$OUT_DIR" -czf "$TARBALL" "move-detect-edge-bundle"
|
||||
echo ">> DONE: $TARBALL"
|
||||
echo ">> transfer this single file across the air gap, then run install_offline.sh on the Pi."
|
||||
70
deploy/bundle/install_offline.sh
Normal file
70
deploy/bundle/install_offline.sh
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
#!/usr/bin/env bash
|
||||
# Offline installer for the move-detect edge daemon. Runs on the air-gapped Pi 5
|
||||
# (NO internet). Unpack the bundle produced by build_offline_bundle.sh, cd into it,
|
||||
# then: sudo ./install_offline.sh
|
||||
#
|
||||
# Installs everything from the bundled wheelhouse with --no-index (zero fetches),
|
||||
# runs the package straight from source (no build backend needed offline).
|
||||
set -euo pipefail
|
||||
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
echo "must run as root (sudo ./install_offline.sh)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PREFIX=/opt/move-detect
|
||||
STATE=/var/lib/move-detect
|
||||
ETC=/etc/move-detect
|
||||
SVC_USER=movedetect
|
||||
WHEELHOUSE="$HERE/wheelhouse"
|
||||
RUNTIME_PKGS=(numpy requests onnxruntime opencv-python-headless)
|
||||
|
||||
echo ">> [1/6] service account + groups"
|
||||
if ! id -u "$SVC_USER" >/dev/null 2>&1; then
|
||||
useradd --system --no-create-home --shell /usr/sbin/nologin "$SVC_USER"
|
||||
fi
|
||||
# camera (video), Hailo accelerator (render/plugdev vary by image) — ignore missing groups
|
||||
for g in video render plugdev hailo; do
|
||||
getent group "$g" >/dev/null 2>&1 && usermod -aG "$g" "$SVC_USER" || true
|
||||
done
|
||||
|
||||
echo ">> [2/6] lay down source, model, docs under $PREFIX"
|
||||
install -d -m 0755 "$PREFIX" "$PREFIX/src" "$PREFIX/models" "$PREFIX/docs"
|
||||
# Fully replace the package tree on re-install/upgrade. Without the rm, `cp -r SRC DEST`
|
||||
# nests SRC *inside* an existing DEST (edge/edge/...) and the OLD code keeps running.
|
||||
rm -rf "$PREFIX/src/edge"
|
||||
cp -r "$HERE/src/edge" "$PREFIX/src/edge"
|
||||
cp "$HERE/src/pyproject.toml" "$PREFIX/src/" 2>/dev/null || true
|
||||
cp "$HERE/models/"*.onnx "$PREFIX/models/"
|
||||
cp "$HERE/docs/"*.md "$PREFIX/docs/" 2>/dev/null || true
|
||||
|
||||
echo ">> [3/6] python venv + offline wheel install (--no-index)"
|
||||
python3 -m venv "$PREFIX/venv"
|
||||
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" --upgrade pip >/dev/null 2>&1 || true
|
||||
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
||||
|
||||
echo ">> [4/6] verify the edge package imports offline"
|
||||
PYTHONPATH="$PREFIX/src" "$PREFIX/venv/bin/python" -c "import edge.daemon, onnxruntime, cv2, numpy, requests; print('edge import OK')"
|
||||
|
||||
echo ">> [5/6] state dir + env file"
|
||||
install -d -m 0750 "$STATE"
|
||||
chown "$SVC_USER":"$SVC_USER" "$STATE"
|
||||
install -d -m 0750 "$ETC"
|
||||
if [ ! -f "$ETC/edge.env" ]; then
|
||||
install -m 0640 "$HERE/deploy/systemd/move-detect-edge.env.example" "$ETC/edge.env"
|
||||
chown root:"$SVC_USER" "$ETC/edge.env"
|
||||
echo " -> wrote $ETC/edge.env from template (EDIT IT: DEVICE_ID, DEVICE_SECRET, CAFE24_URL)"
|
||||
else
|
||||
echo " -> $ETC/edge.env already exists; left unchanged"
|
||||
fi
|
||||
|
||||
echo ">> [6/6] systemd unit"
|
||||
install -m 0644 "$HERE/deploy/systemd/move-detect-edge.service" /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
|
||||
echo ""
|
||||
echo "Installed. Next:"
|
||||
echo " 1. edit $ETC/edge.env (device id/secret, cloud URL; MOVENET_ONNX=$PREFIX/models/movenet_lightning.onnx)"
|
||||
echo " 2. sudo systemctl enable --now move-detect-edge"
|
||||
echo " 3. journalctl -u move-detect-edge -f"
|
||||
40
deploy/systemd/move-detect-edge.env.example
Normal file
40
deploy/systemd/move-detect-edge.env.example
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
# move-detect edge daemon environment — copy to /etc/move-detect/edge.env and fill in.
|
||||
# Consumed by edge.config.load_config() and edge.daemon.main().
|
||||
# Keep this file 0640 root:movedetect — it holds the per-device HMAC secret.
|
||||
#
|
||||
# IMPORTANT: systemd EnvironmentFile= does NOT strip inline comments, so every active
|
||||
# line below is a bare KEY=value. All comments live on their own lines.
|
||||
|
||||
# --- identity / cloud (required) ---
|
||||
# DEVICE_ID must match the devices.device_id row in the cloud.
|
||||
DEVICE_ID=pi-01
|
||||
# DEVICE_SECRET: per-device HMAC secret; must match devices.secret.
|
||||
DEVICE_SECRET=CHANGEME
|
||||
# CAFE24_URL: cloud base URL (POST events / poll config / heartbeat).
|
||||
CAFE24_URL=https://cloud.example.com
|
||||
# CAMERA_ID: stamped on events and matched to dashboard zones.
|
||||
CAMERA_ID=cam-01
|
||||
|
||||
# --- pose backend / capture ---
|
||||
# POSE_BACKEND: "movenet" (CPU, needs MOVENET_ONNX) or "hailo" (needs HAILO_HEF).
|
||||
POSE_BACKEND=movenet
|
||||
# MOVENET_ONNX: required when POSE_BACKEND=movenet.
|
||||
MOVENET_ONNX=/opt/move-detect/models/movenet_lightning.onnx
|
||||
# HAILO_HEF: required when POSE_BACKEND=hailo (uncomment and set).
|
||||
#HAILO_HEF=/opt/move-detect/models/yolov8s_pose.hef
|
||||
# UVC_DEVICE: OpenCV VideoCapture index for the USB (UVC) webcam.
|
||||
UVC_DEVICE=0
|
||||
# TARGET_FPS: frame pacing = 1/TARGET_FPS.
|
||||
TARGET_FPS=12
|
||||
|
||||
# --- offline queue (required) ---
|
||||
# QUEUE_DB: sqlite buffer for events while the cloud is unreachable.
|
||||
QUEUE_DB=/var/lib/move-detect/queue.db
|
||||
|
||||
# --- detector tuning (optional; defaults shown) ---
|
||||
#FALL_PRONE_S=3.0
|
||||
#FALL_NO_MOTION_S=10.0
|
||||
#IMMOBILE_S=120.0
|
||||
#GHOST_REAPPEAR_S=10.0
|
||||
#ZONE_DEBOUNCE_S=0.75
|
||||
#COOLDOWN_S=30.0
|
||||
50
deploy/systemd/move-detect-edge.service
Normal file
50
deploy/systemd/move-detect-edge.service
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
# move-detect edge daemon — systemd unit (Raspberry Pi 5, air-gapped/폐쇄망).
|
||||
# Install: copy to /etc/systemd/system/, fill /etc/move-detect/edge.env, then
|
||||
# sudo systemctl daemon-reload && sudo systemctl enable --now move-detect-edge
|
||||
# The daemon tolerates the cloud being offline (events buffer in the local queue
|
||||
# and flush on reconnect), so this unit only *wants* the network, not requires it.
|
||||
|
||||
[Unit]
|
||||
Description=move-detect edge daemon (workplace-safety AI CCTV)
|
||||
Documentation=file:/opt/move-detect/docs/pilot-runbook.md
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
# Dedicated unprivileged account; must be in the groups that own the camera and
|
||||
# the Hailo accelerator device nodes.
|
||||
User=movedetect
|
||||
Group=movedetect
|
||||
SupplementaryGroups=video render plugdev
|
||||
|
||||
WorkingDirectory=/opt/move-detect
|
||||
EnvironmentFile=/etc/move-detect/edge.env
|
||||
# Run the package straight from source (no pip build step needed offline).
|
||||
Environment=PYTHONPATH=/opt/move-detect/src
|
||||
ExecStart=/opt/move-detect/venv/bin/python -m edge
|
||||
|
||||
# Unattended resilience: always restart; the loop also isolates per-frame and
|
||||
# periodic-op exceptions internally so a single glitch never exits the process.
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
StartLimitIntervalSec=0
|
||||
|
||||
# Writable state (offline queue sqlite lives here; keep QUEUE_DB pointed at it).
|
||||
# systemd creates/owns /var/lib/move-detect for the service user.
|
||||
StateDirectory=move-detect
|
||||
StateDirectoryMode=0750
|
||||
|
||||
# Camera + accelerator device access (USB UVC = /dev/video*, Hailo = /dev/hailo0).
|
||||
DeviceAllow=/dev/hailo0 rw
|
||||
DeviceAllow=char-video4linux rw
|
||||
|
||||
# Conservative hardening (loosen if it blocks device access on your image).
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/var/lib/move-detect
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -1,16 +1,198 @@
|
|||
"""Hailo yolov8s_pose HEF 어댑터 (실장비). PosePort 구현.
|
||||
개발 PC에선 인스턴스화 불가(hailort 부재). Pi에서 HailoRT로 실제 추론 배선."""
|
||||
"""Hailo yolov8s_pose HEF adapter (실장비 Pi5 + Hailo-8/8L). Implements PosePort.
|
||||
|
||||
yolov8s_pose has NO on-chip NMS: the HEF emits 9 raw feature-map tensors (3 strides
|
||||
x {box DFL (H,W,64), class score (H,W,1, already sigmoid), keypoints (H,W,51)}). This
|
||||
adapter does the host-side DFL box-decode + keypoint-decode + NMS and remaps to
|
||||
COCO-17 `list[Person]`.
|
||||
|
||||
The pure-numpy decode below is ported from Hailo's MIT-licensed reference
|
||||
(Hailo-Application-Code-Examples/runtime/python/pose_estimation/pose_estimation_utils.py,
|
||||
class PoseEstPostProcessing) and locked by tests/test_hailo_pose.py via an injected
|
||||
fake infer_fn (no hardware needed).
|
||||
|
||||
DEVICE VALIDATION (bringup): the HailoRT wiring in `_build_hailo_infer` cannot run off
|
||||
Pi/hailort. Validate it on device per docs/hardware-bringup.md, and confirm the model's
|
||||
input (640x640 uint8 RGB, on-chip normalization) and the 9-tensor output shapes against
|
||||
the actual HEF before trusting inference. The remap/decode math is what the unit tests fix.
|
||||
"""
|
||||
import cv2
|
||||
import numpy as np
|
||||
|
||||
from edge.models import Person, BBox, Keypoint
|
||||
|
||||
# COCO-17 keypoint order (yolov8_pose): nose, l/r eye, l/r ear, l/r shoulder,
|
||||
# l/r elbow, l/r wrist, l/r hip, l/r knee, l/r ankle. Emitted in this order, so the
|
||||
# 17 keypoints map positionally to edge.models.Keypoint without reindexing.
|
||||
_NUM_KPTS = 17
|
||||
|
||||
|
||||
def _sigmoid(x):
|
||||
return 1.0 / (1.0 + np.exp(-x))
|
||||
|
||||
|
||||
def _softmax(x):
|
||||
e = np.exp(x - np.max(x, axis=-1, keepdims=True))
|
||||
return e / np.sum(e, axis=-1, keepdims=True)
|
||||
|
||||
|
||||
def _letterbox(frame, size):
|
||||
"""Aspect-preserving resize to size x size with centered padding; BGR->RGB uint8.
|
||||
Returns (canvas, scale, (pad_x, pad_y)) so results can be mapped back to `frame`."""
|
||||
h, w = frame.shape[:2]
|
||||
scale = min(size / w, size / h)
|
||||
nw, nh = max(1, round(w * scale)), max(1, round(h * scale))
|
||||
resized = cv2.resize(frame, (nw, nh))
|
||||
rgb = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)
|
||||
canvas = np.full((size, size, 3), 114, dtype=np.uint8)
|
||||
pad_x, pad_y = (size - nw) // 2, (size - nh) // 2
|
||||
canvas[pad_y:pad_y + nh, pad_x:pad_x + nw] = rgb
|
||||
return canvas, scale, (pad_x, pad_y)
|
||||
|
||||
|
||||
def _pick(outs, size, channels):
|
||||
for t in outs:
|
||||
if t.shape[1] == size and t.shape[2] == size and t.shape[3] == channels:
|
||||
return t
|
||||
raise ValueError(f"missing yolov8s_pose tensor for size={size} channels={channels}")
|
||||
|
||||
|
||||
def _iou(box, boxes):
|
||||
x1 = np.maximum(box[0], boxes[:, 0]); y1 = np.maximum(box[1], boxes[:, 1])
|
||||
x2 = np.minimum(box[2], boxes[:, 2]); y2 = np.minimum(box[3], boxes[:, 3])
|
||||
inter = np.clip(x2 - x1, 0, None) * np.clip(y2 - y1, 0, None)
|
||||
area = (box[2] - box[0]) * (box[3] - box[1])
|
||||
areas = (boxes[:, 2] - boxes[:, 0]) * (boxes[:, 3] - boxes[:, 1])
|
||||
union = area + areas - inter
|
||||
return np.where(union > 0, inter / union, 0.0)
|
||||
|
||||
|
||||
def _nms(pred, conf_thres, iou_thres, max_det):
|
||||
"""pred: (N, 4 + 1 + 51) = [xywh, score, 17*(x,y,score)]. Greedy IoU NMS (1 class)."""
|
||||
keep_mask = pred[:, 4] > conf_thres
|
||||
pred = pred[keep_mask]
|
||||
if pred.shape[0] == 0:
|
||||
return []
|
||||
xywh = pred[:, :4]
|
||||
xyxy = np.empty_like(xywh)
|
||||
xyxy[:, 0] = xywh[:, 0] - xywh[:, 2] / 2
|
||||
xyxy[:, 1] = xywh[:, 1] - xywh[:, 3] / 2
|
||||
xyxy[:, 2] = xywh[:, 0] + xywh[:, 2] / 2
|
||||
xyxy[:, 3] = xywh[:, 1] + xywh[:, 3] / 2
|
||||
scores = pred[:, 4]
|
||||
kpts = pred[:, 5:].reshape(-1, _NUM_KPTS, 3)
|
||||
order = np.argsort(-scores)
|
||||
dets = []
|
||||
while order.size > 0 and len(dets) < max_det:
|
||||
i = order[0]
|
||||
dets.append({"bbox": xyxy[i], "score": float(scores[i]), "kpts": kpts[i]})
|
||||
if order.size == 1:
|
||||
break
|
||||
ious = _iou(xyxy[i], xyxy[order[1:]])
|
||||
order = order[1:][ious <= iou_thres]
|
||||
return dets
|
||||
|
||||
|
||||
def decode_yolov8_pose(outs, input_size, strides, reg_max,
|
||||
conf_thres, iou_thres, max_det):
|
||||
"""Host decode of the 9 raw yolov8s_pose tensors -> NMS'd detections (letterbox px).
|
||||
Ported from Hailo's MIT PoseEstPostProcessing (decoder + extract + NMS)."""
|
||||
reg_range = np.arange(reg_max + 1)
|
||||
boxes_all = scores_all = kpts_all = None
|
||||
for stride in strides:
|
||||
size = input_size // stride
|
||||
box = _pick(outs, size, 4 * (reg_max + 1)).astype(np.float32)
|
||||
score = _pick(outs, size, 1).astype(np.float32)
|
||||
kpt = _pick(outs, size, _NUM_KPTS * 3).astype(np.float32).copy()
|
||||
|
||||
gx, gy = np.meshgrid(np.arange(size) + 0.5, np.arange(size) + 0.5)
|
||||
ct_col, ct_row = gx.flatten() * stride, gy.flatten() * stride
|
||||
center = np.stack((ct_col, ct_row, ct_col, ct_row), axis=1) # (HW, 4)
|
||||
|
||||
bd = box.reshape(1, size * size, 4, reg_max + 1)
|
||||
bd = _softmax(bd) * reg_range.reshape(1, 1, 1, -1)
|
||||
bd = np.sum(bd, axis=-1) * stride # (1, HW, 4) distances (px)
|
||||
bd = np.concatenate([bd[:, :, :2] * (-1), bd[:, :, 2:]], axis=-1)
|
||||
decode_box = center[None] + bd # xyxy
|
||||
xmin, ymin = decode_box[:, :, 0], decode_box[:, :, 1]
|
||||
xmax, ymax = decode_box[:, :, 2], decode_box[:, :, 3]
|
||||
xywh = np.transpose(np.array([(xmin + xmax) / 2, (ymin + ymax) / 2,
|
||||
xmax - xmin, ymax - ymin]), [1, 2, 0])
|
||||
|
||||
sc = score.reshape(1, size * size, 1)
|
||||
kp = kpt.reshape(1, size * size, _NUM_KPTS, 3)
|
||||
kp[..., :2] *= 2
|
||||
kp[..., :2] = stride * (kp[..., :2] - 0.5) + center[:, :2].reshape(size * size, 1, 2)
|
||||
kp = kp.reshape(1, size * size, _NUM_KPTS * 3)
|
||||
|
||||
boxes_all = xywh if boxes_all is None else np.concatenate([boxes_all, xywh], axis=1)
|
||||
scores_all = sc if scores_all is None else np.concatenate([scores_all, sc], axis=1)
|
||||
kpts_all = kp if kpts_all is None else np.concatenate([kpts_all, kp], axis=1)
|
||||
|
||||
pred = np.concatenate([boxes_all, scores_all, kpts_all], axis=2)[0]
|
||||
return _nms(pred, conf_thres, iou_thres, max_det)
|
||||
|
||||
|
||||
class HailoPoseAdapter:
|
||||
def __init__(self, hef_path: str):
|
||||
def __init__(self, hef_path, input_size=640, infer_fn=None,
|
||||
score_threshold=0.5, nms_iou=0.7, reg_max=15, strides=(8, 16, 32),
|
||||
min_person_score=0.2, max_detections=100):
|
||||
self.hef_path = hef_path
|
||||
# 실장비 배선 지점: HailoRT VDevice/InferModel 로드, yolov8s_pose 후처리.
|
||||
# 참고: hailo-rpi5-examples pose 파이프라인 (MIT).
|
||||
raise NotImplementedError("실장비 전용: HailoRT 배선 필요")
|
||||
self.input_size = input_size
|
||||
self.score_threshold = score_threshold
|
||||
self.nms_iou = nms_iou
|
||||
self.reg_max = reg_max
|
||||
self.strides = tuple(strides)
|
||||
self.min_person_score = min_person_score
|
||||
self.max_detections = max_detections
|
||||
self._infer_fn = infer_fn if infer_fn is not None else self._build_hailo_infer(hef_path)
|
||||
|
||||
def _build_hailo_infer(self, hef_path): # pragma: no cover (device-only: needs hailort + HEF)
|
||||
"""Lazily build the HailoRT inference closure. Validate on device (bringup).
|
||||
Returns infer_fn(frame_640_uint8_rgb) -> list of the 9 output tensors."""
|
||||
from hailo_platform import HEF, VDevice, FormatType, HailoSchedulingAlgorithm
|
||||
|
||||
params = VDevice.create_params()
|
||||
params.scheduling_algorithm = HailoSchedulingAlgorithm.ROUND_ROBIN
|
||||
vdevice = VDevice(params)
|
||||
hef = HEF(hef_path)
|
||||
infer_model = vdevice.create_infer_model(hef_path)
|
||||
infer_model.set_batch_size(1)
|
||||
infer_model.input().set_format_type(FormatType.UINT8) # on-chip normalization
|
||||
out_infos = hef.get_output_vstream_infos()
|
||||
for info in out_infos:
|
||||
infer_model.output(info.name).set_format_type(FormatType.FLOAT32) # dequantized
|
||||
configured = infer_model.configure()
|
||||
self._hailo_ctx = (vdevice, configured) # keep alive for the session
|
||||
|
||||
def infer_fn(frame):
|
||||
buffers = {info.name: np.empty(infer_model.output(info.name).shape, dtype=np.float32)
|
||||
for info in out_infos}
|
||||
bindings = configured.create_bindings(output_buffers=buffers)
|
||||
bindings.input().set_buffer(np.ascontiguousarray(frame))
|
||||
configured.run([bindings], timeout_ms=10000)
|
||||
return [buffers[info.name][None, ...] if buffers[info.name].ndim == 3
|
||||
else buffers[info.name] for info in out_infos]
|
||||
|
||||
return infer_fn
|
||||
|
||||
def infer(self, frame) -> list:
|
||||
# frame(640x640 RGB) -> yolov8s_pose -> list[Person(bbox, 17 keypoints)]
|
||||
raise NotImplementedError
|
||||
h, w = frame.shape[:2]
|
||||
canvas, scale, (pad_x, pad_y) = _letterbox(frame, self.input_size)
|
||||
outs = self._infer_fn(canvas)
|
||||
dets = decode_yolov8_pose(outs, self.input_size, self.strides, self.reg_max,
|
||||
self.score_threshold, self.nms_iou, self.max_detections)
|
||||
persons = []
|
||||
for d in dets:
|
||||
if d["score"] < self.min_person_score:
|
||||
continue
|
||||
x1, y1, x2, y2 = d["bbox"]
|
||||
bbox = BBox((x1 - pad_x) / scale, (y1 - pad_y) / scale,
|
||||
(x2 - pad_x) / scale, (y2 - pad_y) / scale)
|
||||
keypoints = []
|
||||
for j in range(_NUM_KPTS):
|
||||
kx, ky, ksc = d["kpts"][j]
|
||||
keypoints.append(Keypoint(float((kx - pad_x) / scale),
|
||||
float((ky - pad_y) / scale),
|
||||
float(_sigmoid(ksc))))
|
||||
persons.append(Person(bbox, keypoints))
|
||||
return persons
|
||||
|
|
|
|||
|
|
@ -41,7 +41,6 @@ def main() -> None: # pragma: no cover (integration entrypoint, exercised on de
|
|||
from edge.queue import OfflineQueue
|
||||
from edge.runtime import PipelineRuntime
|
||||
from edge.adapters.uvc_source import UvcSource
|
||||
from edge.adapters.movenet_pose import MoveNetPoseAdapter
|
||||
|
||||
cfg = load_config(dict(os.environ))
|
||||
nonce = lambda: os.urandom(8).hex()
|
||||
|
|
@ -51,6 +50,13 @@ def main() -> None: # pragma: no cover (integration entrypoint, exercised on de
|
|||
post = PostClient(base + "/api/events", cfg.device_id, cfg.secret, nonce, now)
|
||||
queue = OfflineQueue(os.environ.get("QUEUE_DB", "queue.db"), post)
|
||||
source = UvcSource(int(os.environ.get("UVC_DEVICE", "0")))
|
||||
# POSE_BACKEND selects the accelerator path: "hailo" (Pi5 + Hailo-8, needs HAILO_HEF)
|
||||
# or the default CPU "movenet" (needs MOVENET_ONNX).
|
||||
if os.environ.get("POSE_BACKEND", "movenet").lower() == "hailo":
|
||||
from edge.adapters.hailo_pose import HailoPoseAdapter
|
||||
pose = HailoPoseAdapter(os.environ["HAILO_HEF"])
|
||||
else:
|
||||
from edge.adapters.movenet_pose import MoveNetPoseAdapter
|
||||
pose = MoveNetPoseAdapter(os.environ["MOVENET_ONNX"])
|
||||
runtime = PipelineRuntime(cfg, source, pose, queue)
|
||||
run_loop(runtime, client, queue, clock=now, sleep=time.sleep, iterations=None,
|
||||
|
|
|
|||
57
tests/test_deploy_artifacts.py
Normal file
57
tests/test_deploy_artifacts.py
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
"""Guard: the systemd unit + env template must stay in sync with what the daemon
|
||||
actually reads. If someone renames an env var in edge/config.py or edge/daemon.py
|
||||
(or edits the unit), this fails instead of shipping a broken bundle.
|
||||
"""
|
||||
import os
|
||||
|
||||
DEPLOY = os.path.join(os.path.dirname(__file__), "..", "deploy", "systemd")
|
||||
ENV_EXAMPLE = os.path.join(DEPLOY, "move-detect-edge.env.example")
|
||||
UNIT = os.path.join(DEPLOY, "move-detect-edge.service")
|
||||
|
||||
# Keys the daemon consumes (edge/config.py:load_config + edge/daemon.py:main).
|
||||
REQUIRED_ENV = {
|
||||
"DEVICE_ID", "DEVICE_SECRET", "CAFE24_URL", "CAMERA_ID",
|
||||
"MOVENET_ONNX", "UVC_DEVICE", "TARGET_FPS", "QUEUE_DB",
|
||||
}
|
||||
|
||||
|
||||
def _env_keys(text):
|
||||
keys = set()
|
||||
for line in text.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
keys.add(line.split("=", 1)[0].strip())
|
||||
return keys
|
||||
|
||||
|
||||
def test_env_example_lists_every_required_key():
|
||||
with open(ENV_EXAMPLE, encoding="utf-8") as f:
|
||||
keys = _env_keys(f.read())
|
||||
missing = REQUIRED_ENV - keys
|
||||
assert not missing, f"env template missing required keys: {sorted(missing)}"
|
||||
|
||||
|
||||
def test_env_example_values_have_no_inline_comments():
|
||||
# systemd EnvironmentFile= does NOT strip inline comments — a trailing "# ..." on
|
||||
# a KEY=value line becomes part of the value and breaks int()/float()/path/secret.
|
||||
with open(ENV_EXAMPLE, encoding="utf-8") as f:
|
||||
offenders = []
|
||||
for line in f.read().splitlines():
|
||||
s = line.strip()
|
||||
if not s or s.startswith("#") or "=" not in s:
|
||||
continue
|
||||
value = s.split("=", 1)[1]
|
||||
if "#" in value:
|
||||
offenders.append(s)
|
||||
assert not offenders, f"inline comments in env values break systemd: {offenders}"
|
||||
|
||||
|
||||
def test_systemd_unit_wiring():
|
||||
with open(UNIT, encoding="utf-8") as f:
|
||||
unit = f.read()
|
||||
assert "EnvironmentFile=/etc/move-detect/edge.env" in unit
|
||||
assert "PYTHONPATH=/opt/move-detect/src" in unit
|
||||
assert "python -m edge" in unit # runs the package entrypoint
|
||||
assert "Restart=always" in unit # unattended resilience
|
||||
assert "WantedBy=multi-user.target" in unit
|
||||
64
tests/test_hailo_pose.py
Normal file
64
tests/test_hailo_pose.py
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
"""TDD for HailoPoseAdapter — locks the yolov8s_pose host decode (DFL box + keypoint
|
||||
decode + NMS + COCO-17 assembly + letterbox un-mapping) WITHOUT hardware/hailort,
|
||||
by injecting a fake infer_fn that returns synthetic raw output tensors.
|
||||
|
||||
yolov8s_pose emits 9 raw tensors (no on-chip NMS): per stride (8/16/32) a box(H,W,64
|
||||
DFL), score(H,W,1 already-sigmoid), and kpts(H,W,51 = 17*(x,y,score)).
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
from edge.adapters.hailo_pose import HailoPoseAdapter
|
||||
|
||||
INPUT = 640
|
||||
STRIDES = (8, 16, 32)
|
||||
|
||||
|
||||
def _blank_outputs():
|
||||
outs = []
|
||||
for s in STRIDES:
|
||||
size = INPUT // s
|
||||
outs.append(np.zeros((1, size, size, 64), np.float32)) # box DFL
|
||||
outs.append(np.zeros((1, size, size, 1), np.float32)) # class score (sigmoid'd)
|
||||
outs.append(np.zeros((1, size, size, 51), np.float32)) # keypoints
|
||||
return outs
|
||||
|
||||
|
||||
def _one_person_outputs():
|
||||
outs = _blank_outputs()
|
||||
# activate a single cell on the stride-8 map (size 80), grid cell (40, 40)
|
||||
box, score, kpts = outs[0], outs[1], outs[2]
|
||||
r = c = 40
|
||||
score[0, r, c, 0] = 0.9 # person prob (already sigmoid on-chip)
|
||||
# DFL: one-hot each of the 4 sides at bin index 2 -> distance = 2 cells * stride8 = 16 px
|
||||
b = box[0, r, c].reshape(4, 16)
|
||||
b[:, 2] = 50.0
|
||||
box[0, r, c] = b.reshape(64)
|
||||
# keypoints: raw x=y=0.5, raw score=3.0 for all 17 joints
|
||||
k = kpts[0, r, c].reshape(17, 3)
|
||||
k[:, 0] = 0.5
|
||||
k[:, 1] = 0.5
|
||||
k[:, 2] = 3.0
|
||||
kpts[0, r, c] = k.reshape(51)
|
||||
return outs
|
||||
|
||||
|
||||
def test_infer_maps_yolov8_pose_to_coco17_person():
|
||||
frame = np.zeros((640, 640, 3), np.uint8) # already 640x640 -> scale 1, no padding
|
||||
outs = _one_person_outputs()
|
||||
a = HailoPoseAdapter("unused.hef", infer_fn=lambda f: outs)
|
||||
persons = a.infer(frame)
|
||||
assert len(persons) == 1
|
||||
p = persons[0]
|
||||
assert len(p.keypoints) == 17
|
||||
# grid center = (40 + 0.5) * 8 = 324 ; box +/- 16 -> [308, 340]
|
||||
assert abs(p.bbox.x1 - 308) < 2 and abs(p.bbox.y1 - 308) < 2
|
||||
assert abs(p.bbox.x2 - 340) < 2 and abs(p.bbox.y2 - 340) < 2
|
||||
# keypoint decode: stride*(2*0.5 - 0.5) + 324 = 8*0.5 + 324 = 328
|
||||
assert abs(p.keypoints[0].x - 328) < 2 and abs(p.keypoints[0].y - 328) < 2
|
||||
# joint score sigmoided: sigmoid(3.0) ~= 0.9526
|
||||
assert abs(p.keypoints[0].conf - 0.9526) < 0.01
|
||||
|
||||
|
||||
def test_no_detection_returns_empty():
|
||||
a = HailoPoseAdapter("unused.hef", infer_fn=lambda f: _blank_outputs())
|
||||
assert a.infer(np.zeros((640, 640, 3), np.uint8)) == []
|
||||
Loading…
Reference in a new issue