두 갈래의 작업이 한 커밋에 들어 있다. llm_assistance.py 에서 T0-4 가 앞선
작업의 프롬프트 수정을 대체했기 때문에(프롬프트 리터럴이 llm_prompts.py 로
이동), 분리하려면 실제로 존재한 적 없는 중간 상태를 만들어야 한다.
## (1) 앞서 진행 중이던 review-intelligence 작업
- decision_stats(): 위험밴드 x 실제판정 교차표 + 캘리브레이션 지표
- cleared_reference 지식엔트리 타입: 이전에 클리어된 이미지 매칭은 +80 이
아니라 0점 긍정 신호
- LLM 요약 본문 영속화(_evidence_payload 에 summary/sourceUrls) — 그전까지
요약이 저장되지 않아 운영자에게 한 번도 보인 적이 없었다
- 운영자 GUI 반영
## (2) 실사용 투입 전 T0 — 첫 판정 이후에는 소급 불가한 것들
- T0-1 initialRiskScore/initialRiskBand/scorerVersion 각인.
_rescore_submission 이 12곳에서 riskScore 를 덮어쓰므로 저장된 점수는 늘
왕복 점수였고, 그것을 재채점과 비교하는 테스트는 항상 통과하는 빈 테스트다.
레거시 행은 scorerVersion="legacy" 로 표시만 하고 점수를 날조하지 않는다.
- T0-2 얼굴검출 Haar -> YuNet, 인물 등급(identifiable/background) 도입.
기존의 평평한 +35 는 사람이 찍힌 모든 사진을 medium 문턱 위로 올려 밴드
분포를 붕괴시켰다(실측 high 4 / medium 2 / low 0). 등급은 payload 라운드
트립으로 보존한다 — 잃으면 재채점마다 조용히 +35 로 되돌아간다.
가중치는 models/(gitignore) 에 두고 offline-install.md 에 sha256 반입 절차.
가중치가 없으면 Haar 로 강등하고 1회만 경고한다.
- T0-3 판정 시점 피처 스냅샷. 큐가 riskScore 내림차순이고 화면이 점수를 먼저
보여준 뒤 판정을 받으므로, 스냅샷 없이 모은 라벨은 "이 근거가 반려를
예측한다"가 아니라 "운영자가 내가 보여준 점수에 동의했다"를 기록한다.
- T0-4 모델/프롬프트 버전 + latency_ms 스탬프. latency_ms 는 _evidence_id
해시에서 제외한다 — 포함하면 같은 요약이 재생성될 때마다 id 가 바뀐다.
## (3) 임베딩 저장 금지 정책 개정
금지의 취지는 생체 식별이지 이미지 유사도가 아니므로 범위를 좁힌다.
장면 임베딩(전체 이미지 의미 벡터)을 이미지 유사도 판정에 한해 허용하되
얼굴 마스킹 / 인물 지배 프레임 제외 / 목적 제한 / 별도 저장 클래스 /
연쇄 삭제 5조건을 assert_scene_embedding_allowed 로 강제한다. 마스킹 증명이
없으면 거부한다 — 누락은 마스킹을 건너뛴 것과 구별할 수 없다.
얼굴 임베딩 금지는 landmarks/face_descriptor/face_encoding 추가로 강화.
pytest 502 passed (착수 전 463).
- Keyboard review mode: J/K to move, A/H to decide and jump to the next
unreviewed case, R to focus the memo, N for next unreviewed, ? for help, with
a live position/backlog indicator; disabled while typing.
- Batch decisions: POST /api/submissions/decision-batch applies one decision to
many selected cases through per-case transactions/audit/watchlist, reports
per-case success and failure, caps at 200, requires a memo for reject and
confirms when high-risk cases are included. Lifts the v1 "no bulk
approve/reject" spec restriction under those guardrails (spec revised, static
layout test updated to assert the guardrails instead of the absence).
- Decision-memo presets: GET/POST /api/decision-memo-presets (upsert by label),
one-click chips in the decision pane, "save current memo" action.
- Submission duplicate / prior-case index: seed backfills fingerprints onto the
submission record (idempotent), exact + pHash>=0.97 clustering, queue badge
and group select, prior-decision chips and a conflicting-decision flag.
- Decision export: GET /api/decisions/export with separate internal and
applicant field whitelists (applicant view never carries score/evidence),
CSV with a BOM.
- Online backup/restore: POST /api/admin/backup snapshots the WAL database via
the sqlite backup API plus the media dirs into a timestamped zip with
retention rotation; `--restore` is an offline CLI that keeps the pre-restore
database. GET /api/admin/backups lists snapshots.
Also folds in the co-located HTTP hardening (non-object/non-UTF8 JSON body ->
400, non-ASCII Authorization/cookie compared as bytes -> 401, media OSError ->
404) and the line-buffered startup logging that share these files.
Verified against a live server (auth gate, seeding, duplicate index, batch
decision guardrails, presets, export, backup + restore round-trip). pytest
tests/rights_filter tests/operator_gui: 457 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6GD9gxUtfUiVB3JHejGpU
HIGH:
- face detection: guard OpenCV builds lacking CascadeClassifier and isolate
per-submission analysis failures in BatchAnalyzer, so a broken cv2 or one bad
image no longer takes down every seed/reload/import/upload/rerun with no HTTP
response
- risk scoring: gate Google custom image search (match=search_image) to 0
points so the persistence round-trip (result_url aliased to url) can no longer
flip the same evidence to +45 and inflate bands
- serialization: exclude SEARCH_SKIPPED/ENRICHMENT_FAILURE rows from the auto
search dedup signatures so a quota-skipped query stays retryable instead of
being treated as already done forever
- external_policy: add a daily-window reset + lock (mirrors SearchApiPolicy) so
Vision is not permanently disabled once the daily limit is hit
- image_store: skip a malformed manifest row (and coerce bad width/height)
instead of letting one ValueError paralyse loading/re-analysis for the queue
MEDIUM/LOW:
- enrichment: only count a face crop as transmitted when it actually left the
system, removing false usage debits and a false "personal image sent to
external provider" audit record; pick lastFailure from the failing item;
preserve the collection set when every search was quota-skipped; dedup
candidate counts; attribute Vision failures to the google provider badge
- store: search coverage requires a real result, not a failure row
- scoring: FACE_PERSON contributes once regardless of how many rows exist
- fingerprints: refuse near-matches on degenerate (near-uniform) average
hashes while keeping identical-hash matches
- preprocessing: apply EXIF orientation for external derivatives, face crops,
pHash and the PIL decode fallback
- promoter: scan title/description only, never the URL path/host tokens
- integrations: map Vision responses inside the adapter try; cap HTTPError
bodies; skip the connect probe when a proxy applies; reservation-style quota
accounting (try_reserve/release) to make the check-then-act atomic
- governance/admin: enforce the LLM-summary rule on the real persistence path;
reject unknown manual-entry fields; upsert the rejected_image knowledge entry
on repeat rejections; split policy skips out of the failures group
- env_file: strip a trailing inline comment from unquoted values
Environment drift fixed out-of-tree: removed the user-site OpenCV 5.0.0 that
shadowed the pinned opencv-python-headless==4.13.0.92.
Regression tests added; pytest tests/rights_filter is green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E6GD9gxUtfUiVB3JHejGpU
- Trusted app-shell responses now carry a strict script-src CSP plus
X-Frame-Options: DENY and nosniff (img-src stays permissive for external
evidence thumbnails; the GUI uses no inline scripts).
- Session cookie stores an HMAC-derived tag, not the raw COPYRIGHTER_AUTH_TOKEN,
so a leaked cookie no longer reveals the token/bearer/.env secret.
- Cookie-authenticated state-changing requests require a same-origin
Origin/Referer (CSRF); header-authenticated requests are not forgeable.
- _body rejects negative/non-integer Content-Length (was reaching
rfile.read(-1), an unbounded read-to-EOF) and returns a controlled 400.
- Handler timeout=30 bounds slow-client thread exhaustion; uploads use
exclusive-create ('xb') to close a concurrent same-name overwrite TOCTOU.
Tests: app-shell security headers, negative Content-Length -> 400, cookie tag
(not raw token), cookie-authed cross-origin POST -> 403 / same-origin -> 200.
- cloud_vision: raise on a per-image error inside a 200 response so the adapter
records a FAILURE instead of a silent "no results" that also debited quota;
send the key via X-goog-api-key header (verified for Vision) not the URL.
- search_policy: reset the daily_limit counter on a calendar-day rollover (it
never re-windowed, so a hit limit disabled the provider until restart) and
guard calls_made with a lock.
- google_custom_search: map the API response inside the adapter's try so a
malformed payload becomes ENRICHMENT_FAILURE, not an uncaught crash. (Key
stays a query param — the header is unverified for this endpoint.)
- http_json: cap the response body (10 MiB) and convert decode/read/OS errors
into HttpJsonError so nothing escapes the transport contract.
- store_remote_fetch: SSRF guard rejects any non-global address (`not
is_global`), covering CGNAT 100.64/10 the flag list missed.
- env_clients: a malformed daily-limit env falls back to a conservative cap
(not fail-open unlimited); clamp NAVER display sizes.
- preprocessing: warn when external-derivative re-encoding falls back to the
original bytes (EXIF/GPS may not have been stripped).
Tests: Vision per-image error raises / converts to FAILURE; daily-limit
malformed falls back conservatively.
Under ThreadingHTTPServer several store writers ran unlocked read-modify-write
cycles, so concurrent operator actions could silently lose each other's
updates or revert an in-flight change.
Concurrency (_write_lock; external I/O kept OUTSIDE the lock):
- record_decision now takes _write_lock so a decision isn't reverted by a
concurrent rescore/sweep.
- manual_search runs the network search first, then re-reads the submission
under the lock before the queryHistory RMW (was clobbering concurrent edits).
- _apply_provider_usage_delta merges only usage/status onto the fresh row
instead of rewriting a stale pre-search snapshot (kept an in-flight emergency
disable from being reverted).
- _sync_provider_payloads preserves an operator's disable via an
operatorDisabled flag (connectivity-only disables still reconnect).
- set_provider_enabled / emergency_disable / _sync_submission_provider_state /
collect_keyword_candidates now lock (+ single transaction where applicable).
Robustness:
- seed: a zero-byte/corrupt image now yields a 'failed' submission + audit
instead of an IndexError that aborted the whole import.
- ensure_queue: key the UPDATE/SELECT on the existing row id on a folder_path
match (was matching zero rows and returning None -> seed TypeError); audit
queue activation / legacy migration.
- store_enrichment: lastFailure uses the first failed item; auto-search dedup
reads only the submission's evidence; log a swallowed face-crop load error.
- image_store: documents the pre-existing same-stem id limitation (a prior
sibling-dependent fix was reverted — it resurrected decided submissions).
Tests: operator-disable survives reseed, usage delta keeps an in-flight
disable, seed survives an unanalyzable image, ensure_queue mismatched-id,
dismiss persistence.
Production scores come from _rescore_submission, which re-reads evidence via
_domain_evidence_from_ui and re-scores. Serialization dropped the promoted /
entity fields and the precise EvidenceSource, so on every rescore Naver
promotion (+40) and Google web-entity (+40) silently scored 0 and skip
evidence was mis-restored as FAILURE (+30 phantom).
- store_serialization: persist domainSource / promoted / promotionReason /
entity / category in _evidence_payload and restore them in
_domain_evidence_from_ui; resolve the source from the persisted enum
(keyword heuristic kept only as a legacy fallback).
- search_result_promoter: structural no-result detection (covers image/blog/
web variants), scan only result fields not the query, add Korean rights
markers (excluding over-generic substrings).
- risk_scoring: dedup promoted Naver hits by query_signature; drop zero-point
skip reasons from the operator-facing reasons list.
Tests: round-trip regression (promoted naver / web entity / two distinct
entities score > 0; skip adds no phantom points) + skip-reason exclusion.
Backend (data-integrity / audit-trail):
- mark_evidence_status: make the read-modify-write + audit atomic under
_write_lock + _transaction (was 4 unlocked connections, last-writer-wins,
audit could diverge from DB state).
- promote_collection_candidates: acquire _write_lock, use a deterministic
_stable_id, wrap writes in _transaction, and guard already-promoted batches
so a retry/double-click cannot mint duplicate knowledge entries.
- risk_scoring: bucket web-detection entities by name so two distinct entities
in one image both count (was collapsed to one 'entity' bucket, under-scoring
high->medium). Identical entities still dedup.
Frontend (silent loss of operator actions):
- runManualSearch: in-flight guard; rerunHistoricalQuery voids the call,
preventing duplicate/racing search POSTs.
- deactivateCorrectionEntry: persist via the real knowledge-deactivate
endpoint instead of a local-only mutation + fake audit event.
- provider-retry button: report honestly (no backend endpoint) instead of
writing a fake 'Provider called' audit event.
Tests: +4 regression tests (mark_evidence_status rollback, batch-promote
idempotency, two distinct entities -> high, identical entity dedup).
Remaining Phase 1 item #6 (_auto_naver/_auto_google refactor) deferred.
Backend suite: 361 passed. app.js: node --check OK, 41 static tests pass.
Introduce a reusable browser-test harness under tests/operator_gui/:
- conftest.py: shared fixtures (operator_server, browser with graceful skip,
operator_page with screenshot-on-failure artifacts)
- pages/: OperatorWorkbench Page Object encapsulating selectors and actions
- refactor test_browser_smoke.py onto the fixtures + POM (drops duplicated
server/browser plumbing)
- test_decision_flow.py: approve/hold/reject E2E against the real server,
covering the reject-requires-memo guard and decision persistence
Pin playwright in requirements-dev.txt (Chromium bundled offline for the
air-gapped target) and ignore the artifacts/ screenshot dir.
Move knowledge-entry lifecycle, rerun/auto/manual search drivers, LLM summary
management, manual knowledge registration, and keyword-candidate collection/
promotion into a mixin; CopyrighterStore inherits it. Drop now-unused imports;
point the rollback test at store_serialization._stable_id. sqlite_store.py 1598
-> 874 lines (5333 -> 874, -84%).
Move provider-state sync, local face evidence/crop sync + retention purge,
internal/Google re-analysis, face-crop web detection, and the auto Google/Naver
search drivers into a mixin; CopyrighterStore inherits it. Drop now-unused
imports. Tests now also patch HeuristicFacePersonDetector on store_enrichment
(face detection moved there). sqlite_store.py 2358 -> 1598 lines (-70% overall).
Move the pure text helpers (_text_list, _unique_texts) into store_text and the
~950-line page/CSS/JSON/srcset image-URL extraction (the _PageImageParser and
its helpers) into store_page_scrape. Both behavior-preserving; store_page_scrape
depends only on stdlib + url/text helpers + domain Evidence (no store coupling).
sqlite_store.py 4955 -> 3992 lines.
Move the pure network layer (image/page/stylesheet fetchers, SSRF guard,
redirect-validating opener) into store_remote_fetch, and the DDL/typed-column/
constraint-migration helpers into store_schema. Both are behavior-preserving
relocations imported back into sqlite_store; tests repoint their fetch
monkeypatches to store_remote_fetch. sqlite_store.py 5333 -> 4948 lines.
Governance: purge biometric face crops past a retention window (env
COPYRIGHTER_FACE_CROP_RETENTION_DAYS, default 90d) with an audit trail, run
at startup and reload; audit personal-image transmission to external Vision.
Concurrency: a process write lock + atomic provider-usage delta stop lost
counter updates; candidate promotion is idempotent (deterministic id + status
guard); seeding is serialized. Correctness: skip LLM summarize when a summary
already exists; constraint migration cleans orphan temp tables on failure.
Add provider-readiness startup log. Tests pin all of the above plus risk-band
boundaries (29/30/69/70, 100 cap) and media path-traversal guards.
Address commit security review: replace the ?token= query fallback (which
leaked the token into logs/referrers) with an HttpOnly, SameSite=Strict
session cookie minted on the first header-authenticated request, so <img>
media loads authenticate without a URL token. Use hmac.compare_digest for
constant-time comparison and add Cache-Control: no-store + Referrer-Policy:
no-referrer on untrusted biometric media. Also cover upload/import boundary
validation (400) at the HTTP layer.
Resolve each candidate image/page/stylesheet URL and refuse loopback,
RFC1918, link-local (cloud-metadata), reserved, multicast, and unspecified
targets before fetching; re-validate on every redirect hop via a custom
opener. URLs originate from external search-result content, so this closes
the operator server fetching internal services.
Enable WAL + busy_timeout in _connect (ThreadingHTTPServer concurrent
operators no longer hit 'database is locked'), add a _transaction helper and
thread an optional conn through _put/_get/add_audit_event so record_decision
commits its status change, watchlist entry, and audit event atomically.
Remove wildcard CORS (prevented cross-origin reads of biometric/case data
from localhost), add optional shared-token auth gate on data routes
(COPYRIGHTER_AUTH_TOKEN; GUI shell + /health stay open), cap request body
size (413), and map malformed JSON to 400 and SQLite lock contention to 503.
Correctness:
- Make the local-artifact audit test skip on fresh clones (data/ is
gitignored), so the suite passes outside this workstation
- Drop the transform from the viewRise entrance animation: an animated
transform made .view.active a containing block for 320ms and threw
the fixed decision panel off-screen on every workbench entry
- Collapse the queue toolbar at 1380px instead of 1180px; 1280x800
laptops no longer get a horizontal scrollbar (verified live)
- Serve .woff2 as font/woff2 with an immutable cache header so the
2MB bundled font is fetched once, not per page load (with test)
- Clip overflow on top-bar status chips (long apiError strings spilled
over neighbors at 981-1180px)
- Give queue-row selection a selector that outranks the even-row
zebra stripe (selection background was parity-dependent)
Cleanup:
- Replace the stale old-palette focus ring and ::selection literals
with color-mix over var(--teal)
- Delete dead tokens: unused back-compat aliases (the comment claiming
they were referenced was false), --rail-bot, --ochre-deep, and
--font-stamp (identical to --font-ui since the Pretendard switch)
- Tokenize scattered raw colors: rail ink scale, soft tint levels,
inset-well and bevel shadows, naver/internal source-chip triplets
- Remove the asset-preload div and three orphan SVGs nothing renders;
tests now reject reintroducing them
Verified: 359 tests pass; Playwright audit at 1440/1280/390 shows zero
horizontal overflow on all views, Pretendard active, decision panel
fixed at the viewport corner mid-animation.
Image rights / copyright detection system: SQLite store, HTTP app,
search integrations (Naver, Google Custom Search, Google Cloud Vision
web detection), image analysis (fingerprints, face/person detection,
evidence enrichment, risk scoring), an admin/review layer, governance
and retention policies, batch jobs, and a browser-based operator GUI.
This baseline incorporates a full code-review remediation pass
(46 fixes; 358 tests passing). Highlights:
CRITICAL
- Prevent evidence cascade-delete during the schema-constraint
migration by disabling FK enforcement around the table rebuild.
Security
- Sandbox served media (neutralize stored XSS from uploaded/collected
SVGs) via CSP + nosniff on the untrusted media routes.
- Strip embedded EXIF/GPS from external image derivatives before they
are sent to third-party APIs.
- Return a clean 404 (not an uncaught StopIteration) for PATCH on an
unknown provider.
Correctness
- LLM-summary failures no longer add +30 to the risk score.
- Decode only explicit JS escapes so Korean image URLs are not mangled.
- Consume search quota only after a successful request.
- Naver/Google adapters map responses inside the failure boundary, so a
malformed response degrades to evidence instead of crashing enrichment.
- Domain-aware provider attribution; face-box IoU de-duplication; count
searches (not result items); per-box crop isolation; clamp evidence
confidence and Google CSE num; real submittedEpoch; and more.
Robustness
- Offline LLM connect fast-fails (short connect timeout) so seed/reload
requests are not stalled; full read timeout preserved for generation.
- Malformed numeric env vars fall back to defaults instead of crashing
startup.
Performance
- Per-submission evidence reads (no full-table scan per rescore),
audit-log LIMIT, lazy active-store lookup, hoisted timestamps.
Tests
- ~24 regression tests added pinning the above fixes.
Runtime data (data/, outputs/, *.sqlite3, *.log), secrets (.env), and
node_modules are gitignored.
