fix(edge): confidence-gate foot/nose keypoints, warn on MoveNet single-person (P2-9/10/11)
Three keypoint-trust hardening fixes that reuse the codebase's existing "visible" confidence convention (no new tuning knobs): - P2-9 zone.py: _foot averaged both ankles unconditionally, so a hallucinated ankle (conf~0, often pinned to (0,0)) dragged the foot point into or out of a zone. Now averages only ankles with conf >= 0.2 and falls back to the bbox bottom-centre when neither is trustworthy. - P2-11 geometry.py: head_below_hips used the NOSE regardless of confidence. Now returns False when the nose is below the visibility threshold, so an unconfident/hallucinated nose can't fire the head-below-hips fall cue. The confident-nose path is unchanged (test_fall.py stays green). - P2-10 movenet_pose.py: the CPU MoveNet fallback is single-person, so at a multi-worker site every worker but one is silently invisible to fall/zone/ghost detection. Construction now logs a loud startup WARNING pointing to the Hailo backend. Tests: test_zone (bbox fallback), test_geometry (low-conf nose), and test_movenet_pose (caplog warning). Full python suite: 157 passed, 2 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Xmn8KZWqiQUguEMRtn6rW
This commit is contained in:
parent
949c6d05c3
commit
f1ee7f0a36
7 changed files with 75 additions and 8 deletions
|
|
@ -99,14 +99,22 @@ misconfig/degradation fail LOUD.
|
||||||
arms a CRITICAL ghost if the person then leaves. **Fix:** require sustained prone.
|
arms a CRITICAL ghost if the person then leaves. **Fix:** require sustained prone.
|
||||||
- [ ] **P2-8 [M]** `edge/app.py:30` — detection dropouts keep a track 'active' with a frozen
|
- [ ] **P2-8 [M]** `edge/app.py:30` — detection dropouts keep a track 'active' with a frozen
|
||||||
frame; time-based timers accrue on stale data. **Fix:** treat a stale track as no-data.
|
frame; time-based timers accrue on stale data. **Fix:** treat a stale track as no-data.
|
||||||
- [ ] **P2-9 [M]** `edge/detectors/zone.py:45` — foot point averages both ankles with no
|
- [x] **P2-9 [M]** `edge/detectors/zone.py:45` — foot point averages both ankles with no
|
||||||
confidence check; a hallucinated ankle at (0,0) drags the foot into/out of a zone. **Fix:**
|
confidence check; a hallucinated ankle at (0,0) drags the foot into/out of a zone. **Fix:**
|
||||||
confidence-gate the ankles, fall back to bbox-bottom-center.
|
confidence-gate the ankles, fall back to bbox-bottom-center.
|
||||||
- [ ] **P2-10 [M]** `edge/adapters/movenet_pose.py:34` — the CPU MoveNet fallback is
|
DONE — `_foot` averages only ankles with conf ≥ MIN_ANKLE_CONF (0.2, the existing 'visible'
|
||||||
|
gate); with neither trustworthy it falls back to bbox bottom-centre. No tuning knob — reuses
|
||||||
|
the codebase's keypoint-visible convention. test_zone.py locks the fallback.
|
||||||
|
- [x] **P2-10 [M]** `edge/adapters/movenet_pose.py:34` — the CPU MoveNet fallback is
|
||||||
single-person; with 2+ workers all but one are invisible to every detector, no warning.
|
single-person; with 2+ workers all but one are invisible to every detector, no warning.
|
||||||
**Fix:** warn loudly that MoveNet fallback is single-person (Hailo is multi-person).
|
**Fix:** warn loudly that MoveNet fallback is single-person (Hailo is multi-person).
|
||||||
- [ ] **P2-11 [L]** `edge/geometry.py:39` — `head_below_hips` uses the NOSE keypoint
|
DONE — MoveNetPoseAdapter.__init__ logs a startup WARNING ("SINGLE-PERSON … use POSE_BACKEND=
|
||||||
|
hailo at multi-person sites"). test_movenet_pose.py asserts the warning via caplog.
|
||||||
|
- [x] **P2-11 [L]** `edge/geometry.py:39` — `head_below_hips` uses the NOSE keypoint
|
||||||
regardless of confidence. **Fix:** confidence-gate.
|
regardless of confidence. **Fix:** confidence-gate.
|
||||||
|
DONE — `head_below_hips` returns False when nose conf < nose_thr (0.2), so an unconfident/
|
||||||
|
hallucinated nose no longer fires the fall cue. test_geometry.py locks it; test_fall.py stays
|
||||||
|
green (the cue's confident-nose path is unchanged).
|
||||||
|
|
||||||
## Phase 3 — Privacy / PIPA (delivery compliance)
|
## Phase 3 — Privacy / PIPA (delivery compliance)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,5 @@
|
||||||
|
import logging
|
||||||
|
|
||||||
import cv2
|
import cv2
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
|
@ -17,6 +19,13 @@ class MoveNetPoseAdapter:
|
||||||
session = onnxruntime.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
|
session = onnxruntime.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
|
||||||
self.session = session
|
self.session = session
|
||||||
self._input_name = session.get_inputs()[0].name
|
self._input_name = session.get_inputs()[0].name
|
||||||
|
# MoveNet SinglePose returns ONE person per frame. At a multi-worker site every worker
|
||||||
|
# but one is invisible to fall/zone/ghost detection with no other symptom — so say it
|
||||||
|
# loudly at startup. The Hailo backend (yolov8s_pose) is multi-person.
|
||||||
|
logging.getLogger("edge.pose").warning(
|
||||||
|
"MoveNet CPU fallback is SINGLE-PERSON: only one worker is tracked per frame; any "
|
||||||
|
"additional workers are invisible to fall/zone/ghost detection. Use the Hailo "
|
||||||
|
"backend (POSE_BACKEND=hailo) at multi-person sites.")
|
||||||
|
|
||||||
def infer(self, frame) -> list:
|
def infer(self, frame) -> list:
|
||||||
h, w = frame.shape[:2]
|
h, w = frame.shape[:2]
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,9 @@ from dataclasses import dataclass
|
||||||
from edge.tracking import Track
|
from edge.tracking import Track
|
||||||
from edge.models import L_ANKLE, R_ANKLE
|
from edge.models import L_ANKLE, R_ANKLE
|
||||||
|
|
||||||
|
# Below this an ankle keypoint is treated as unusable (typically hallucinated at conf~0).
|
||||||
|
MIN_ANKLE_CONF = 0.2
|
||||||
|
|
||||||
|
|
||||||
def point_in_polygon(x: float, y: float, poly: list[tuple[float, float]]) -> bool:
|
def point_in_polygon(x: float, y: float, poly: list[tuple[float, float]]) -> bool:
|
||||||
inside = False
|
inside = False
|
||||||
|
|
@ -41,9 +44,18 @@ class ZoneIntrusionDetector:
|
||||||
self._fired = {k for k in self._fired if k[0] != track_id}
|
self._fired = {k for k in self._fired if k[0] != track_id}
|
||||||
|
|
||||||
def _foot(self, track: Track, frame_wh=None) -> tuple[float, float]:
|
def _foot(self, track: Track, frame_wh=None) -> tuple[float, float]:
|
||||||
kps = track.latest.keypoints
|
p = track.latest
|
||||||
a, b = kps[L_ANKLE], kps[R_ANKLE]
|
a, b = p.keypoints[L_ANKLE], p.keypoints[R_ANKLE]
|
||||||
fx, fy = (a.x + b.x) / 2, (a.y + b.y) / 2
|
# Average only the ankles the model is confident about. A hallucinated ankle (conf~0,
|
||||||
|
# often pinned to (0,0) or a frame corner) would otherwise drag the foot point into or
|
||||||
|
# out of a zone. With neither ankle trustworthy, fall back to the bbox bottom-centre — a
|
||||||
|
# robust standing-foot proxy (same bbox-fallback discipline as fall.py's cues).
|
||||||
|
good = [k for k in (a, b) if k.conf >= MIN_ANKLE_CONF]
|
||||||
|
if good:
|
||||||
|
fx = sum(k.x for k in good) / len(good)
|
||||||
|
fy = sum(k.y for k in good) / len(good)
|
||||||
|
else:
|
||||||
|
fx, fy = p.bbox.cx, p.bbox.y2
|
||||||
# Zones are stored NORMALIZED (0..1 of the image) by the dashboard, which stretches the
|
# Zones are stored NORMALIZED (0..1 of the image) by the dashboard, which stretches the
|
||||||
# camera snapshot to a fixed canvas. The foot is camera-native pixels, so normalize it
|
# camera snapshot to a fixed canvas. The foot is camera-native pixels, so normalize it
|
||||||
# by the frame size to compare in the same space — otherwise every zone is misplaced
|
# by the frame size to compare in the same space — otherwise every zone is misplaced
|
||||||
|
|
|
||||||
|
|
@ -34,9 +34,15 @@ def torso_angle_deg(keypoints: list[Keypoint]) -> float:
|
||||||
return math.degrees(math.atan2(abs(vx), abs(vy))) if (vx or vy) else 0.0
|
return math.degrees(math.atan2(abs(vx), abs(vy))) if (vx or vy) else 0.0
|
||||||
|
|
||||||
|
|
||||||
def head_below_hips(keypoints: list[Keypoint]) -> bool:
|
def head_below_hips(keypoints: list[Keypoint], nose_thr: float = 0.2) -> bool:
|
||||||
|
# Only trust the nose if the model is confident about it. An unconfident nose (commonly
|
||||||
|
# hallucinated at (0,0) / a frame corner) would otherwise assert — or silently suppress —
|
||||||
|
# this fall cue on data we don't trust. Same 'visible' gate as visible_count().
|
||||||
|
nose = keypoints[NOSE]
|
||||||
|
if nose.conf < nose_thr:
|
||||||
|
return False
|
||||||
_, hy = _hip_center(keypoints)
|
_, hy = _hip_center(keypoints)
|
||||||
return keypoints[NOSE].y > hy
|
return nose.y > hy
|
||||||
|
|
||||||
|
|
||||||
def visible_count(keypoints: list[Keypoint], thr: float = 0.2) -> int:
|
def visible_count(keypoints: list[Keypoint], thr: float = 0.2) -> int:
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,15 @@ def test_head_below_hips_true_when_nose_lower_on_screen():
|
||||||
kps = _kps({0: (5, 120, 1), 11: (0, 100, 1), 12: (10, 100, 1)})
|
kps = _kps({0: (5, 120, 1), 11: (0, 100, 1), 12: (10, 100, 1)})
|
||||||
assert head_below_hips(kps) is True
|
assert head_below_hips(kps) is True
|
||||||
|
|
||||||
|
def test_head_below_hips_false_when_nose_low_confidence():
|
||||||
|
# A hallucinated low-confidence nose must not fire the head-below-hips fall cue, even when
|
||||||
|
# its coordinate happens to sit below the hips.
|
||||||
|
low = _kps({0: (5, 120, 0.1), 11: (0, 100, 1), 12: (10, 100, 1)})
|
||||||
|
assert head_below_hips(low) is False
|
||||||
|
# ...but a confident nose in the same position still fires.
|
||||||
|
ok = _kps({0: (5, 120, 0.9), 11: (0, 100, 1), 12: (10, 100, 1)})
|
||||||
|
assert head_below_hips(ok) is True
|
||||||
|
|
||||||
def test_visible_count_and_core_visible():
|
def test_visible_count_and_core_visible():
|
||||||
kps = _kps({5: (0, 0, 0.9), 6: (1, 0, 0.9), 11: (0, 1, 0.9), 12: (1, 1, 0.9)})
|
kps = _kps({5: (0, 0, 0.9), 6: (1, 0, 0.9), 11: (0, 1, 0.9), 12: (1, 1, 0.9)})
|
||||||
assert visible_count(kps, thr=0.2) == 4
|
assert visible_count(kps, thr=0.2) == 4
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,4 @@
|
||||||
|
import logging
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from edge.adapters.movenet_pose import MoveNetPoseAdapter
|
from edge.adapters.movenet_pose import MoveNetPoseAdapter
|
||||||
|
|
||||||
|
|
@ -30,3 +31,10 @@ def test_low_score_person_dropped():
|
||||||
return [np.zeros((1, 1, 17, 3), dtype=np.float32)] # all score 0
|
return [np.zeros((1, 1, 17, 3), dtype=np.float32)] # all score 0
|
||||||
a = MoveNetPoseAdapter("unused.onnx", session=Low())
|
a = MoveNetPoseAdapter("unused.onnx", session=Low())
|
||||||
assert a.infer(np.zeros((10, 10, 3), np.uint8)) == []
|
assert a.infer(np.zeros((10, 10, 3), np.uint8)) == []
|
||||||
|
|
||||||
|
def test_warns_that_movenet_is_single_person(caplog):
|
||||||
|
# MoveNet tracks only ONE person; at a multi-worker site the rest are silently invisible to
|
||||||
|
# every detector. Construction must warn loudly so the operator knows to use the Hailo backend.
|
||||||
|
with caplog.at_level(logging.WARNING, logger="edge.pose"):
|
||||||
|
MoveNetPoseAdapter("unused.onnx", session=_FakeSession())
|
||||||
|
assert any("SINGLE-PERSON" in r.getMessage() for r in caplog.records)
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,21 @@ def test_leaving_zone_resets():
|
||||||
assert out == []
|
assert out == []
|
||||||
|
|
||||||
|
|
||||||
|
def _track_lowconf_ankles(bbox, t):
|
||||||
|
kps = [Keypoint(0, 0, 0.0) for _ in range(17)] # every keypoint hallucinated at the origin
|
||||||
|
return Track(1, deque([TrackFrame(t, bbox, kps)], maxlen=90))
|
||||||
|
|
||||||
|
def test_foot_falls_back_to_bbox_when_ankles_low_confidence():
|
||||||
|
# Both ankles are hallucinated at (0,0) with conf 0. Averaging them would pin the foot to the
|
||||||
|
# image origin — outside any centred zone. The detector must instead use the bbox
|
||||||
|
# bottom-centre, where the worker actually stands, so the intrusion still fires.
|
||||||
|
z = Zone("z1", "restricted", SQUARE)
|
||||||
|
det = ZoneIntrusionDetector([z], ZoneConfig(debounce_s=0.0))
|
||||||
|
bbox = BBox(40, 10, 60, 50) # bottom-centre = (cx=50, y2=50), inside SQUARE
|
||||||
|
out = det.update(_track_lowconf_ankles(bbox, 0.0), now=0.0)
|
||||||
|
assert out == [{"zone_id": "z1", "level": "restricted"}]
|
||||||
|
|
||||||
|
|
||||||
def test_zone_uses_normalized_coordinates_when_frame_size_given():
|
def test_zone_uses_normalized_coordinates_when_frame_size_given():
|
||||||
# The dashboard stores zones as 0..1 of the image (it stretches the snapshot to a fixed
|
# The dashboard stores zones as 0..1 of the image (it stretches the snapshot to a fixed
|
||||||
# 960x540 canvas). The edge foot is camera-native pixels. Normalizing the foot by the
|
# 960x540 canvas). The edge foot is camera-native pixels. Normalizing the foot by the
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue