Merge feat/p1d-hailo-csi: CSI camera + Hailo hardware path
Implement the Pi5 + Hailo-8 HAT + CSI camera premise: real PiCameraSource (picamera2), CAMERA_BACKEND=csi|uvc switch, and an offline install that exposes the Pi system packages (HailoRT, picamera2) to the venv via --system-site-packages, with Hailo/CSI prerequisites documented. Adversarial review clean.
This commit is contained in:
commit
d9d149d4a1
6 changed files with 112 additions and 9 deletions
|
|
@ -79,8 +79,40 @@ model (`/opt/move-detect/models/movenet_lightning.onnx`).
|
||||||
- [ ] Pull the network cable → events keep queueing (`QUEUE_DB` grows) → restore
|
- [ ] Pull the network cable → events keep queueing (`QUEUE_DB` grows) → restore
|
||||||
→ the queue flushes. (End-to-end behavior verified in the edge e2e harness.)
|
→ the queue flushes. (End-to-end behavior verified in the edge e2e harness.)
|
||||||
|
|
||||||
|
## Hailo-8 HAT + CSI camera path (optional, faster)
|
||||||
|
|
||||||
|
The default bundle runs CPU MoveNet + USB webcam. To use the Hailo-8 accelerator
|
||||||
|
and/or a CSI camera, add these SYSTEM packages on the Pi (they are apt packages, not
|
||||||
|
pip wheels — the venv sees them because it is created with `--system-site-packages`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install -y hailo-all python3-picamera2 python3-opencv
|
||||||
|
```
|
||||||
|
|
||||||
|
- `hailo-all` — HailoRT runtime + driver + the Python `hailo_platform` bindings.
|
||||||
|
- `python3-picamera2` — CSI camera stack (libcamera).
|
||||||
|
|
||||||
|
Then place the pose model and switch backends in `/etc/move-detect/edge.env`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
POSE_BACKEND=hailo
|
||||||
|
HAILO_HEF=/opt/move-detect/models/yolov8s_pose.hef # copy the compiled HEF here
|
||||||
|
CAMERA_BACKEND=csi
|
||||||
|
CSI_DEVICE=0
|
||||||
|
```
|
||||||
|
|
||||||
|
Validate on device per `docs/hardware-bringup.md` before trusting output (the Hailo
|
||||||
|
decode is unit-locked, but the HailoRT inference + picamera2 color order are hardware-
|
||||||
|
validated). USB webcam stays available anytime via `CAMERA_BACKEND=uvc`.
|
||||||
|
|
||||||
|
> **Air gap:** `hailo-all` / `python3-picamera2` are the ONE piece this bundle can't
|
||||||
|
> carry (Hailo's own driver + apt tree). Pre-install them on the Pi image, or transfer
|
||||||
|
> their `.deb`s separately, before running `install_offline.sh`. The pose HEF is
|
||||||
|
> likewise supplied out-of-band (compiled from the Hailo Model Zoo).
|
||||||
|
|
||||||
## Not bundled (by design)
|
## Not bundled (by design)
|
||||||
|
|
||||||
- **MySQL / the cloud app** — a separate host; see `cloud/docs/cloud-deploy.md`.
|
- **MySQL / the cloud app** — a separate host; see `cloud/docs/cloud-deploy.md`.
|
||||||
- **HailoRT + `yolov8s_pose.hef`** — the Hailo runtime and HEF are installed via
|
- **HailoRT + `yolov8s_pose.hef` + picamera2** — Hailo's runtime/driver, the compiled
|
||||||
Hailo's own Pi packages; the CPU MoveNet path here needs neither.
|
HEF, and the CSI camera stack are Pi system packages installed out-of-band (above);
|
||||||
|
the default CPU MoveNet + USB path needs none of them.
|
||||||
|
|
|
||||||
|
|
@ -40,7 +40,13 @@ cp "$HERE/models/"*.onnx "$PREFIX/models/"
|
||||||
cp "$HERE/docs/"*.md "$PREFIX/docs/" 2>/dev/null || true
|
cp "$HERE/docs/"*.md "$PREFIX/docs/" 2>/dev/null || true
|
||||||
|
|
||||||
echo ">> [3/6] python venv + offline wheel install (--no-index)"
|
echo ">> [3/6] python venv + offline wheel install (--no-index)"
|
||||||
python3 -m venv "$PREFIX/venv"
|
# --system-site-packages so the venv can import the Pi SYSTEM packages that are NOT
|
||||||
|
# pip-installable: hailo_platform (HailoRT, apt: hailo-all) for POSE_BACKEND=hailo, and
|
||||||
|
# picamera2 (apt: python3-picamera2) for CAMERA_BACKEND=csi. The wheelhouse install
|
||||||
|
# below adds our deps into the venv; where a system copy already satisfies a wheel's
|
||||||
|
# version constraint pip keeps it (ABI-compatible on the target images), otherwise the
|
||||||
|
# bundled wheel installs into the venv and shadows the system copy.
|
||||||
|
python3 -m venv --system-site-packages "$PREFIX/venv"
|
||||||
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" --upgrade pip >/dev/null 2>&1 || true
|
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" --upgrade pip >/dev/null 2>&1 || true
|
||||||
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
"$PREFIX/venv/bin/pip" install --no-index --find-links "$WHEELHOUSE" "${RUNTIME_PKGS[@]}"
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -22,8 +22,12 @@ POSE_BACKEND=movenet
|
||||||
MOVENET_ONNX=/opt/move-detect/models/movenet_lightning.onnx
|
MOVENET_ONNX=/opt/move-detect/models/movenet_lightning.onnx
|
||||||
# HAILO_HEF: required when POSE_BACKEND=hailo (uncomment and set).
|
# HAILO_HEF: required when POSE_BACKEND=hailo (uncomment and set).
|
||||||
#HAILO_HEF=/opt/move-detect/models/yolov8s_pose.hef
|
#HAILO_HEF=/opt/move-detect/models/yolov8s_pose.hef
|
||||||
|
# CAMERA_BACKEND: "uvc" (USB webcam, uses UVC_DEVICE) or "csi" (Pi CSI camera, uses CSI_DEVICE).
|
||||||
|
CAMERA_BACKEND=uvc
|
||||||
# UVC_DEVICE: OpenCV VideoCapture index for the USB (UVC) webcam.
|
# UVC_DEVICE: OpenCV VideoCapture index for the USB (UVC) webcam.
|
||||||
UVC_DEVICE=0
|
UVC_DEVICE=0
|
||||||
|
# CSI_DEVICE: picamera2 camera_num for the CSI camera (used when CAMERA_BACKEND=csi).
|
||||||
|
CSI_DEVICE=0
|
||||||
# TARGET_FPS: frame pacing = 1/TARGET_FPS.
|
# TARGET_FPS: frame pacing = 1/TARGET_FPS.
|
||||||
TARGET_FPS=12
|
TARGET_FPS=12
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,29 @@
|
||||||
"""CSI 카메라 소스 (실장비). 카메라당 별 프로세스로 picamera2 구동."""
|
"""CSI 카메라 소스 (실장비 Pi5, picamera2). Implements SourcePort.
|
||||||
|
|
||||||
|
picamera2 is a SYSTEM package on Raspberry Pi OS (apt: python3-picamera2 + libcamera),
|
||||||
|
NOT a pip wheel — the venv sees it via --system-site-packages (see deploy/README).
|
||||||
|
|
||||||
|
COLOR NOTE: configured for "RGB888", which picamera2 delivers as a 3-channel array in
|
||||||
|
**BGR** byte order (documented picamera2 quirk). The downstream pose adapters
|
||||||
|
(MoveNet/Hailo) expect BGR (they do cv2.cvtColor(BGR2RGB)), same as cv2.VideoCapture,
|
||||||
|
so read() returns the array as-is. Validate the channel order on device during bringup
|
||||||
|
(a BGR<->RGB swap yields plausible-but-wrong keypoints).
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
class PiCameraSource:
|
class PiCameraSource:
|
||||||
def __init__(self, index: int = 0):
|
def __init__(self, index: int = 0, size=(640, 480), picam2=None):
|
||||||
self.index = index
|
self.index = index
|
||||||
raise NotImplementedError("실장비 전용: picamera2 배선 필요")
|
if picam2 is None: # pragma: no cover (device-only: needs picamera2 + CSI camera)
|
||||||
|
from picamera2 import Picamera2 # system apt package (python3-picamera2)
|
||||||
|
picam2 = Picamera2(camera_num=index)
|
||||||
|
picam2.configure(picam2.create_video_configuration(
|
||||||
|
main={"size": tuple(size), "format": "RGB888"}))
|
||||||
|
picam2.start()
|
||||||
|
self.picam2 = picam2
|
||||||
|
|
||||||
def read(self):
|
def read(self):
|
||||||
raise NotImplementedError
|
frame = self.picam2.capture_array()
|
||||||
|
if frame is None:
|
||||||
|
return False, None
|
||||||
|
return True, frame
|
||||||
|
|
|
||||||
|
|
@ -40,7 +40,6 @@ def main() -> None: # pragma: no cover (integration entrypoint, exercised on de
|
||||||
from edge.alert import PostClient
|
from edge.alert import PostClient
|
||||||
from edge.queue import OfflineQueue
|
from edge.queue import OfflineQueue
|
||||||
from edge.runtime import PipelineRuntime
|
from edge.runtime import PipelineRuntime
|
||||||
from edge.adapters.uvc_source import UvcSource
|
|
||||||
|
|
||||||
cfg = load_config(dict(os.environ))
|
cfg = load_config(dict(os.environ))
|
||||||
nonce = lambda: os.urandom(8).hex()
|
nonce = lambda: os.urandom(8).hex()
|
||||||
|
|
@ -49,7 +48,14 @@ def main() -> None: # pragma: no cover (integration entrypoint, exercised on de
|
||||||
client = CloudClient(base, cfg.device_id, cfg.secret, nonce, now)
|
client = CloudClient(base, cfg.device_id, cfg.secret, nonce, now)
|
||||||
post = PostClient(base + "/api/events", cfg.device_id, cfg.secret, nonce, now)
|
post = PostClient(base + "/api/events", cfg.device_id, cfg.secret, nonce, now)
|
||||||
queue = OfflineQueue(os.environ.get("QUEUE_DB", "queue.db"), post)
|
queue = OfflineQueue(os.environ.get("QUEUE_DB", "queue.db"), post)
|
||||||
source = UvcSource(int(os.environ.get("UVC_DEVICE", "0")))
|
# CAMERA_BACKEND selects the capture source: "csi" (Pi CSI cam via picamera2,
|
||||||
|
# uses CSI_DEVICE) or the default "uvc" (USB webcam via OpenCV, uses UVC_DEVICE).
|
||||||
|
if os.environ.get("CAMERA_BACKEND", "uvc").lower() == "csi":
|
||||||
|
from edge.adapters.picamera_source import PiCameraSource
|
||||||
|
source = PiCameraSource(int(os.environ.get("CSI_DEVICE", "0")))
|
||||||
|
else:
|
||||||
|
from edge.adapters.uvc_source import UvcSource
|
||||||
|
source = UvcSource(int(os.environ.get("UVC_DEVICE", "0")))
|
||||||
# POSE_BACKEND selects the accelerator path: "hailo" (Pi5 + Hailo-8, needs HAILO_HEF)
|
# POSE_BACKEND selects the accelerator path: "hailo" (Pi5 + Hailo-8, needs HAILO_HEF)
|
||||||
# or the default CPU "movenet" (needs MOVENET_ONNX).
|
# or the default CPU "movenet" (needs MOVENET_ONNX).
|
||||||
if os.environ.get("POSE_BACKEND", "movenet").lower() == "hailo":
|
if os.environ.get("POSE_BACKEND", "movenet").lower() == "hailo":
|
||||||
|
|
|
||||||
36
tests/test_picamera_source.py
Normal file
36
tests/test_picamera_source.py
Normal file
|
|
@ -0,0 +1,36 @@
|
||||||
|
"""TDD for PiCameraSource (CSI camera via picamera2) — no hardware/picamera2 needed,
|
||||||
|
by injecting a fake Picamera2 object. Locks the SourcePort contract: read() -> (ok, frame)."""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
from edge.adapters.picamera_source import PiCameraSource
|
||||||
|
|
||||||
|
|
||||||
|
class _FakePicam2:
|
||||||
|
def __init__(self, frames):
|
||||||
|
self.frames = list(frames)
|
||||||
|
self.i = 0
|
||||||
|
self.started = False
|
||||||
|
|
||||||
|
def capture_array(self):
|
||||||
|
frame = self.frames[self.i % len(self.frames)]
|
||||||
|
self.i += 1
|
||||||
|
return frame
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_returns_ok_and_frame():
|
||||||
|
frame = np.full((480, 640, 3), 128, np.uint8)
|
||||||
|
src = PiCameraSource(picam2=_FakePicam2([frame]))
|
||||||
|
ok, f = src.read()
|
||||||
|
assert ok is True
|
||||||
|
assert f.shape == (480, 640, 3)
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_none_frame_returns_false():
|
||||||
|
class _Empty:
|
||||||
|
def capture_array(self):
|
||||||
|
return None
|
||||||
|
|
||||||
|
src = PiCameraSource(picam2=_Empty())
|
||||||
|
ok, f = src.read()
|
||||||
|
assert ok is False
|
||||||
|
assert f is None
|
||||||
Loading…
Reference in a new issue