feat: LLM 을 요약기에서 증거 판독기로 (구조화 인용 + 입력 경계 + 3B)
사용자 결정: LLM 역할을 "요약"에서 "증거 판독 + 구조화 판단"으로 확대,
타깃에 GPU 있음. 최종 판정은 여전히 사람이 한다.
## 입력 경계
프롬프트가 증거 data 딕셔너리를 통째로 f-string 에 박아 제출건당
13,733~62,291 자를 만들고 있었다. 화이트리스트 9개 필드 + 상위 12건으로
제한한다. promotion_reason 을 제외한 것이 핵심 — 모델이 그대로 베껴서
"요약"이 증거 목록 복붙이 됐다. 얼굴 크롭 좌표 / privacy_note /
thumbnail_url 도 모델 입력에서 뺀다(요약 입력이 아니다).
선별은 위험을 실제로 지는 행 우선: 지문 유사도 0.9 이상 -> full/partial/page
매칭과 promoted -> 얼굴 -> 나머지.
## 구조화 인용과 검증
format=json 으로 {claims:[{text, evidence}], needs_check:[]} 를 받고, 각
claim 이 인용한 라벨이 실제로 프롬프트에 들어간 라벨 집합의 부분집합인지
검사해 아니면 버린다. 화면에 렌더되는 텍스트는 검증을 통과한 claim 으로만
조립하므로 근거 없는 문장이 운영자에게 도달할 경로가 없다.
인용을 JSON 필드로 받는 것은 의도적이다. 산문을 문장 분해해 개체(URL, 숫자)를
대조하는 방식은 'confirming the image's authenticity [E3]' 같은 근거 없는
서술어를 100% 통과시켜, 오히려 환각 문장에 "근거 확인됨" 도장을 찍는다.
집합 포함 검사는 그 실패 모드가 구조적으로 발생하지 않는다.
증거 id 대신 [E1] 라벨을 쓴다 — 24자 해시를 소형 모델에게 받아쓰게 하면
오타와 조작을 구별할 수 없다.
grounded_claims / dropped_claims 를 증거에 기록한다. 모델이 지어내기
시작하면 수치로 드러난다.
## 모델과 컨텍스트
기본 모델 qwen2.5:0.5b-instruct -> qwen2.5:3b-instruct. 0.5B 는 출력 계약을
지키지 못했다(영어로 답하고 증거 목록을 반복했다).
num_ctx 를 명시한다. Ollama 기본 컨텍스트는 수천 토큰이고 넘으면 조용히
자르는데, source_urls / source_evidence_ids 는 잘리지 않은 전체 증거에서
뽑혀 붙었다. 즉 모델이 읽은 적 없는 근거를 근거라고 표시하고 있었다.
num_predict / keep_alive / timeout(30->120s) 도 환경변수로 노출.
## 시드 잠금 해제
_ensure_llm_summary 호출 2곳을 _write_lock 밖으로 옮긴다. record_decision 이
같은 락을 쓰므로, 폴더 재적재 중에는 운영자가 보고 있지도 않은 이미지의
요약이 끝날 때까지 판정 버튼이 대기했다. 시드 1회당 20건 상한을 두되 조용히
자르지 않고 로그를 남긴다(only-if-missing 이라 다음 재적재에서 이어진다).
이 이동은 provider 커버리지 회귀를 만들었다 —
_sync_submission_provider_state 가 잠금 구간 안에서, 즉 나레이션 전에 돌아
요약이 존재하는데도 llm 을 not_run 으로 보고했다. 나레이션이 실제로
발생했을 때만 커버리지를 다시 계산한다.
pytest 515 passed.
This commit is contained in:
changukyu
parent
4f140efd9b
commit
6be263a69d
9 changed files with 514 additions and 36 deletions
13
.env.example
13
.env.example
|
|
@ -45,4 +45,15 @@ COPYRIGHTER_GOOGLE_CUSTOM_SEARCH_DAILY_LIMIT=100
|
|||
COPYRIGHTER_LLM_DAILY_LIMIT=100
|
||||
|
||||
OLLAMA_BASE_URL=http://localhost:11434
|
||||
OLLAMA_MODEL=qwen2.5:0.5b-instruct
|
||||
# 3B is the floor for reading evidence and citing it. 0.5B ignored the output
|
||||
# contract (answered in English, reprinted the evidence list). Pull it on the
|
||||
# build machine: `ollama pull qwen2.5:3b-instruct`.
|
||||
OLLAMA_MODEL=qwen2.5:3b-instruct
|
||||
# Context window. Ollama defaults to a couple of thousand tokens and silently
|
||||
# truncates past it — the summary would then cite evidence the model never read.
|
||||
OLLAMA_NUM_CTX=8192
|
||||
OLLAMA_NUM_PREDICT=512
|
||||
# Seconds. Structured output on a 3B model needs more headroom than the old 30s.
|
||||
OLLAMA_TIMEOUT=120
|
||||
# Keep weights resident between cases so a batch does not reload per case.
|
||||
OLLAMA_KEEP_ALIVE=10m
|
||||
|
|
|
|||
|
|
@ -1,18 +1,49 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from rights_filter.analysis.llm_prompts import (
|
||||
EMPTY_SUMMARY_TEXT,
|
||||
EVIDENCE_LABEL_PREFIX,
|
||||
LEGACY_PROMPT_VERSION,
|
||||
NEEDS_CHECK_HEADING,
|
||||
PROMPT_ID,
|
||||
PROMPT_VERSION,
|
||||
SUMMARY_HEADING,
|
||||
SYSTEM_PROMPT,
|
||||
)
|
||||
from rights_filter.domain.records import Evidence, EvidenceSource
|
||||
from rights_filter.integrations.http_json import UrllibJsonTransport
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The model reads at most this many evidence rows. Real cases carry ~49 rows and
|
||||
# dumping all of them produced 13k-62k character prompts that overran the context
|
||||
# window: the model saw a fraction while the stored summary still claimed every
|
||||
# row as its source. Capping the input is what makes the citation honest.
|
||||
MAX_PROMPT_EVIDENCE = 12
|
||||
|
||||
# Data fields the model is allowed to see. A whitelist rather than a blocklist:
|
||||
# evidence payloads grow, and a field nobody vetted must not reach the prompt by
|
||||
# default. Notably absent — promotion_reason (the model copies it verbatim and
|
||||
# the "summary" becomes a reprint of the evidence list) and the face-crop fields
|
||||
# (crop geometry and privacy notes are not summarization input).
|
||||
PROMPT_DATA_FIELDS = (
|
||||
"match",
|
||||
"similarity",
|
||||
"page_title",
|
||||
"domain",
|
||||
"query",
|
||||
"entity",
|
||||
"label",
|
||||
"knowledge_name",
|
||||
"tier",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LlmSummaryResult:
|
||||
|
|
@ -23,6 +54,11 @@ class LlmSummaryResult:
|
|||
prompt_id: str = ""
|
||||
prompt_version: str = LEGACY_PROMPT_VERSION
|
||||
latency_ms: int = 0
|
||||
# Claims that cited at least one real evidence label. Anything the model
|
||||
# asserted without a valid citation never reaches this list.
|
||||
claims: tuple[dict[str, Any], ...] = ()
|
||||
dropped_claims: int = 0
|
||||
needs_check: tuple[str, ...] = field(default=())
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
@ -41,10 +77,13 @@ class OllamaGenerateLlmClient:
|
|||
def __init__(
|
||||
self,
|
||||
base_url: str = "http://127.0.0.1:11434",
|
||||
model: str = "qwen2.5:0.5b-instruct",
|
||||
model: str = "qwen2.5:3b-instruct",
|
||||
transport: Any | None = None,
|
||||
timeout: int = 30,
|
||||
timeout: int = 120,
|
||||
connect_timeout: float = 2,
|
||||
num_ctx: int = 8192,
|
||||
num_predict: int = 512,
|
||||
keep_alive: str = "10m",
|
||||
) -> None:
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.model = model
|
||||
|
|
@ -53,17 +92,35 @@ class OllamaGenerateLlmClient:
|
|||
# generation still gets the full `timeout`.
|
||||
self.transport = transport or UrllibJsonTransport(connect_timeout=connect_timeout)
|
||||
self.timeout = timeout
|
||||
# Ollama defaults the context window to a couple of thousand tokens and
|
||||
# silently truncates past it. Setting it explicitly is what makes "the
|
||||
# model read the evidence we cited" true rather than assumed.
|
||||
self.num_ctx = num_ctx
|
||||
self.num_predict = num_predict
|
||||
# Keep the weights resident between cases so the second case in a batch
|
||||
# does not pay the load again.
|
||||
self.keep_alive = keep_alive
|
||||
|
||||
def summarize_evidence(self, evidence: list[Evidence]) -> str:
|
||||
return self.summarize_evidence_detailed(evidence).text
|
||||
|
||||
def summarize_evidence_detailed(self, evidence: list[Evidence]) -> LlmSummaryResult:
|
||||
selected = _select_prompt_evidence(evidence)
|
||||
labels = _evidence_labels(selected)
|
||||
payload = {
|
||||
"model": self.model,
|
||||
"system": SYSTEM_PROMPT,
|
||||
"prompt": _evidence_prompt(evidence),
|
||||
"prompt": _evidence_prompt(selected),
|
||||
"stream": False,
|
||||
"options": {"temperature": 0.1},
|
||||
# Constrain decoding to JSON so the citation field is a field, not a
|
||||
# string the reader has to guess at.
|
||||
"format": "json",
|
||||
"keep_alive": self.keep_alive,
|
||||
"options": {
|
||||
"temperature": 0.1,
|
||||
"num_ctx": self.num_ctx,
|
||||
"num_predict": self.num_predict,
|
||||
},
|
||||
}
|
||||
started = time.monotonic()
|
||||
response = self.transport.request_json(
|
||||
|
|
@ -73,14 +130,20 @@ class OllamaGenerateLlmClient:
|
|||
timeout=self.timeout,
|
||||
)
|
||||
elapsed_ms = int((time.monotonic() - started) * 1000)
|
||||
|
||||
raw = _response_text(response)
|
||||
claims, dropped, needs_check = _verified_claims(raw, set(labels))
|
||||
return LlmSummaryResult(
|
||||
text=_response_text(response),
|
||||
text=_render_summary(claims, needs_check),
|
||||
# Prefer the model the server says it ran: a tag can resolve to a
|
||||
# different blob than the one configured here.
|
||||
model=str(response.get("model") or self.model),
|
||||
prompt_id=PROMPT_ID,
|
||||
prompt_version=PROMPT_VERSION,
|
||||
latency_ms=elapsed_ms,
|
||||
claims=claims,
|
||||
dropped_claims=dropped,
|
||||
needs_check=needs_check,
|
||||
)
|
||||
|
||||
def _generate_url(self) -> str:
|
||||
|
|
@ -113,6 +176,11 @@ class InternalLlmAssistant:
|
|||
"prompt_id": result.prompt_id,
|
||||
"prompt_version": result.prompt_version,
|
||||
"latency_ms": result.latency_ms,
|
||||
# How much of what the model said survived citation checking. A
|
||||
# rising dropped count is the signal that the model or the prompt
|
||||
# has stopped working — without it, fabrication is invisible.
|
||||
"grounded_claims": len(result.claims),
|
||||
"dropped_claims": result.dropped_claims,
|
||||
}
|
||||
|
||||
source_urls = _source_urls(evidence)
|
||||
|
|
@ -178,13 +246,133 @@ def _source_evidence_ids(evidence: list[Evidence]) -> list[str]:
|
|||
return ids
|
||||
|
||||
|
||||
def _evidence_prompt(evidence: list[Evidence]) -> str:
|
||||
lines = ["Source evidence:"]
|
||||
for index, item in enumerate(evidence, start=1):
|
||||
lines.append(
|
||||
f"{index}. source={item.source.value}; reason={item.reason}; "
|
||||
f"confidence={item.confidence}; data={item.data}"
|
||||
def _prompt_priority(item: Evidence) -> int:
|
||||
"""Lower sorts first. Keeps the rows that carry risk when the cap bites."""
|
||||
data = item.data
|
||||
if item.source == EvidenceSource.FINGERPRINT:
|
||||
try:
|
||||
if float(data.get("similarity") or 0) >= 0.9:
|
||||
return 0
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
return 3
|
||||
if item.source == EvidenceSource.WEB_DETECTION:
|
||||
return 1 if data.get("match") in {"full", "partial", "page"} else 3
|
||||
if item.source == EvidenceSource.NAVER_SEARCH:
|
||||
return 1 if data.get("promoted") else 3
|
||||
if item.source == EvidenceSource.FACE_PERSON:
|
||||
return 2
|
||||
return 4
|
||||
|
||||
|
||||
def _select_prompt_evidence(evidence: list[Evidence]) -> list[Evidence]:
|
||||
ordered = sorted(
|
||||
(item for item in evidence if item.source != EvidenceSource.LLM_SUMMARY),
|
||||
key=lambda item: (_prompt_priority(item), -float(item.confidence or 0)),
|
||||
)
|
||||
return ordered[:MAX_PROMPT_EVIDENCE]
|
||||
|
||||
|
||||
def _evidence_labels(selected: list[Evidence]) -> dict[str, Evidence]:
|
||||
return {
|
||||
f"{EVIDENCE_LABEL_PREFIX}{index}": item
|
||||
for index, item in enumerate(selected, start=1)
|
||||
}
|
||||
|
||||
|
||||
def _evidence_prompt(selected: list[Evidence]) -> str:
|
||||
lines = ["Source evidence:"]
|
||||
for label, item in _evidence_labels(selected).items():
|
||||
fields = [f"source={item.source.value}", f"reason={item.reason}"]
|
||||
for key in PROMPT_DATA_FIELDS:
|
||||
value = item.data.get(key)
|
||||
if value not in (None, "", [], {}):
|
||||
fields.append(f"{key}={value}")
|
||||
lines.append(f"[{label}] " + "; ".join(fields))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _verified_claims(
|
||||
raw: str, valid_labels: set[str]
|
||||
) -> tuple[tuple[dict[str, Any], ...], int, tuple[str, ...]]:
|
||||
"""Keep only the claims that cite evidence the model was actually given.
|
||||
|
||||
Verification is a set-membership test on a JSON field, not an attempt to
|
||||
parse prose. An earlier design checked entities (URLs, numbers) inside free
|
||||
text, which passes any confident-sounding sentence that happens to mention a
|
||||
real URL — and so stamps "근거 확인됨" onto exactly the hallucinations it was
|
||||
meant to catch.
|
||||
"""
|
||||
document = _load_json_object(raw)
|
||||
if document is None:
|
||||
logger.warning("LLM returned unparseable output; dropping the summary")
|
||||
return (), 0, ()
|
||||
|
||||
claims: list[dict[str, Any]] = []
|
||||
dropped = 0
|
||||
for entry in document.get("claims") or []:
|
||||
if not isinstance(entry, dict):
|
||||
dropped += 1
|
||||
continue
|
||||
text = str(entry.get("text", "")).strip()
|
||||
cited = _normalized_labels(entry.get("evidence"))
|
||||
grounded = [label for label in cited if label in valid_labels]
|
||||
if not text or not grounded:
|
||||
# No text, or every cited label was invented. Either way there is
|
||||
# nothing the operator can check it against.
|
||||
dropped += 1
|
||||
continue
|
||||
claims.append({"text": text, "evidence": grounded})
|
||||
|
||||
needs_check = tuple(
|
||||
str(item).strip()
|
||||
for item in (document.get("needs_check") or [])
|
||||
if str(item).strip()
|
||||
)
|
||||
return tuple(claims), dropped, needs_check
|
||||
|
||||
|
||||
def _load_json_object(raw: str) -> dict[str, Any] | None:
|
||||
text = (raw or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
try:
|
||||
parsed = json.loads(text)
|
||||
except (ValueError, TypeError):
|
||||
# Some models wrap the object in prose despite format=json. Take the
|
||||
# outermost brace pair rather than giving up.
|
||||
start, end = text.find("{"), text.rfind("}")
|
||||
if start < 0 or end <= start:
|
||||
return None
|
||||
try:
|
||||
parsed = json.loads(text[start : end + 1])
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
return parsed if isinstance(parsed, dict) else None
|
||||
|
||||
|
||||
def _normalized_labels(value: Any) -> list[str]:
|
||||
if isinstance(value, str):
|
||||
value = [value]
|
||||
if not isinstance(value, (list, tuple)):
|
||||
return []
|
||||
return [str(item).strip().strip("[]").upper() for item in value if str(item).strip()]
|
||||
|
||||
|
||||
def _render_summary(
|
||||
claims: tuple[dict[str, Any], ...], needs_check: tuple[str, ...]
|
||||
) -> str:
|
||||
if not claims and not needs_check:
|
||||
return EMPTY_SUMMARY_TEXT
|
||||
|
||||
lines: list[str] = [SUMMARY_HEADING]
|
||||
for claim in claims:
|
||||
lines.append(f"- {claim['text']} [{', '.join(claim['evidence'])}]")
|
||||
if not claims:
|
||||
lines.append("- 인용이 확인된 근거가 없습니다.")
|
||||
if needs_check:
|
||||
lines.append(NEEDS_CHECK_HEADING)
|
||||
lines.extend(f"- {item}" for item in needs_check)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -12,19 +12,41 @@ output — wording, format contract, or language.
|
|||
from __future__ import annotations
|
||||
|
||||
PROMPT_ID = "operator-review-note-ko"
|
||||
PROMPT_VERSION = "ko-v2"
|
||||
PROMPT_VERSION = "ko-v3-structured"
|
||||
|
||||
# Marks rows written before prompt versioning existed. Never assume the text
|
||||
# below produced them.
|
||||
LEGACY_PROMPT_VERSION = "legacy"
|
||||
|
||||
# Evidence is labelled [E1]..[En] in the prompt and the model cites those labels
|
||||
# rather than the real evidence ids. The ids are 24-hex-char hashes; asking a
|
||||
# small model to echo them verbatim invites transcription errors that are
|
||||
# indistinguishable from fabricated citations.
|
||||
EVIDENCE_LABEL_PREFIX = "E"
|
||||
|
||||
# Asking for JSON rather than prose is what makes grounding checkable. A claim
|
||||
# carries the labels it rests on as a field, so verification is a set-membership
|
||||
# test on that field — not an attempt to parse Korean sentences and guess which
|
||||
# words a trailing citation covers.
|
||||
SYSTEM_PROMPT = (
|
||||
"You draft review notes for an internal image-rights operator, "
|
||||
"grounded ONLY in the provided source evidence. Reply in Korean. "
|
||||
"Produce two short parts: (1) '요약:' one or two sentences on what "
|
||||
"the evidence shows; (2) '확인 필요:' a brief list of what the "
|
||||
"operator should verify before deciding. Do NOT state or recommend "
|
||||
"a final decision (approve/hold/reject). Do NOT add any claim that "
|
||||
"is not supported by the source evidence; if evidence is thin, say "
|
||||
"so plainly."
|
||||
"You draft review notes for an internal image-rights operator, grounded "
|
||||
"ONLY in the labelled source evidence given to you.\n"
|
||||
"Reply with a JSON object and nothing else, in exactly this shape:\n"
|
||||
'{"claims": [{"text": "<한국어 문장>", "evidence": ["E1"]}], '
|
||||
'"needs_check": ["<한국어 구절>"]}\n'
|
||||
"Rules:\n"
|
||||
"- Write every 'text' value in Korean.\n"
|
||||
"- Every claim MUST cite at least one evidence label it rests on, using "
|
||||
"only the labels provided. Never invent a label.\n"
|
||||
"- Never assert anything the cited evidence does not support. If the "
|
||||
"evidence is thin, say so plainly in a claim citing what little exists.\n"
|
||||
"- Do NOT state or recommend a final decision (approve / hold / reject).\n"
|
||||
"- 'needs_check' lists what the operator should verify before deciding.\n"
|
||||
"- At most 4 claims and 4 needs_check items."
|
||||
)
|
||||
|
||||
# Rendered headings. Kept here so the prompt contract and the text the operator
|
||||
# reads move together.
|
||||
SUMMARY_HEADING = "요약:"
|
||||
NEEDS_CHECK_HEADING = "확인 필요:"
|
||||
EMPTY_SUMMARY_TEXT = "요약: 근거를 요약할 수 없습니다."
|
||||
|
|
|
|||
|
|
@ -20,6 +20,8 @@ from rights_filter.integrations.google_custom_search import (
|
|||
from rights_filter.integrations.naver_search import NaverOpenApiSearchClient, NaverSearchAdapter
|
||||
from rights_filter.integrations.search_policy import SearchApiPolicy
|
||||
|
||||
DEFAULT_OLLAMA_MODEL = "qwen2.5:3b-instruct"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProviderRuntime:
|
||||
|
|
@ -178,7 +180,7 @@ def build_provider_runtime(
|
|||
"llm",
|
||||
"Ollama local LLM",
|
||||
llm_assistant is not None,
|
||||
f"Ollama local API configured ({env.get('OLLAMA_MODEL', 'qwen2.5:0.5b-instruct')})",
|
||||
f"Ollama local API configured ({env.get('OLLAMA_MODEL', DEFAULT_OLLAMA_MODEL)})",
|
||||
llm_limit,
|
||||
"없음",
|
||||
"source-linked summary only",
|
||||
|
|
@ -244,8 +246,15 @@ def _google_custom_search_adapter(
|
|||
def _llm_assistant(env: Mapping[str, str], transport: Any | None) -> InternalLlmAssistant | None:
|
||||
client = OllamaGenerateLlmClient(
|
||||
base_url=env.get("OLLAMA_BASE_URL", "http://127.0.0.1:11434"),
|
||||
model=env.get("OLLAMA_MODEL", "qwen2.5:0.5b-instruct"),
|
||||
# 0.5B could not hold the output contract: it answered in English and
|
||||
# reprinted the evidence list instead of summarizing it. Reading evidence
|
||||
# and citing it needs a 3B-class model.
|
||||
model=env.get("OLLAMA_MODEL", DEFAULT_OLLAMA_MODEL),
|
||||
transport=transport,
|
||||
timeout=_bounded_int(env.get("OLLAMA_TIMEOUT"), default=120, minimum=5, maximum=600),
|
||||
num_ctx=_bounded_int(env.get("OLLAMA_NUM_CTX"), default=8192, minimum=1024, maximum=131072),
|
||||
num_predict=_bounded_int(env.get("OLLAMA_NUM_PREDICT"), default=512, minimum=64, maximum=4096),
|
||||
keep_alive=env.get("OLLAMA_KEEP_ALIVE", "10m"),
|
||||
)
|
||||
return InternalLlmAssistant(client)
|
||||
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ from rights_filter.server.store_constants import (
|
|||
DEFAULT_QUERY_COVERAGE_WARN_THRESHOLD,
|
||||
EVIDENCE_OPERATOR_STATUSES,
|
||||
MAX_COVERAGE_THRESHOLD,
|
||||
MAX_LLM_SUMMARIES_PER_SEED,
|
||||
MIN_COVERAGE_THRESHOLD,
|
||||
NON_CONTRIBUTING_OPERATOR_STATUSES,
|
||||
_bounded_int_env,
|
||||
|
|
@ -291,9 +292,42 @@ class CopyrighterStore(
|
|||
# both classify the same record as new and double-run analysis, external
|
||||
# searches, and audit writes for it.
|
||||
with self._write_lock:
|
||||
return self._seed_from_image_store_locked(image_store)
|
||||
created, queue_id = self._seed_from_image_store_locked(image_store)
|
||||
# LLM narration calls out to Ollama: seconds per case with a real model,
|
||||
# and a 2s connect timeout per case even when Ollama is not running.
|
||||
# Doing it under _write_lock meant a folder reload blocked every other
|
||||
# write for the whole batch — including record_decision, which takes the
|
||||
# same lock. So the operator's decision button stalled behind narration
|
||||
# of images they were not even looking at. Narration is only-if-missing
|
||||
# and therefore safe to finish on a later reload.
|
||||
if self._narrate_seeded_submissions(queue_id):
|
||||
# Provider coverage is derived from the evidence rows and is computed
|
||||
# inside the locked section — i.e. before narration now runs. Without
|
||||
# this refresh the llm provider reports "not_run" for summaries that
|
||||
# exist, which is the coverage panel lying about work it did.
|
||||
with self._write_lock:
|
||||
self._sync_submission_provider_state(queue_id=queue_id)
|
||||
return created
|
||||
|
||||
def _seed_from_image_store_locked(self, image_store: LocalSubmissionImageStore) -> int:
|
||||
def _narrate_seeded_submissions(self, queue_id: str) -> int:
|
||||
if self.provider_runtime.llm_assistant is None:
|
||||
return 0
|
||||
narrated = 0
|
||||
for submission in self._all("submissions", queue_id=queue_id):
|
||||
if narrated >= MAX_LLM_SUMMARIES_PER_SEED:
|
||||
logger.info(
|
||||
"LLM narration capped at %s per seed; remaining submissions "
|
||||
"will be narrated on the next reload",
|
||||
MAX_LLM_SUMMARIES_PER_SEED,
|
||||
)
|
||||
break
|
||||
if self._ensure_llm_summary(str(submission["id"]), only_if_missing=True):
|
||||
narrated += 1
|
||||
return narrated
|
||||
|
||||
def _seed_from_image_store_locked(
|
||||
self, image_store: LocalSubmissionImageStore
|
||||
) -> tuple[int, str]:
|
||||
self.purge_expired_face_crops()
|
||||
queue = self.ensure_queue(image_store.root)
|
||||
queue_id = str(queue["id"])
|
||||
|
|
@ -307,7 +341,6 @@ class CopyrighterStore(
|
|||
if "faceCrops" not in submission:
|
||||
self._sync_face_crops(str(submission["id"]), image_store)
|
||||
self._rescore_all_submissions(queue_id=queue_id)
|
||||
self._ensure_llm_summaries_for_existing_source_evidence(queue_id=queue_id)
|
||||
self._sync_submission_provider_state(queue_id=queue_id)
|
||||
|
||||
existing_submission_ids = {
|
||||
|
|
@ -333,7 +366,9 @@ class CopyrighterStore(
|
|||
continue
|
||||
records.append(record)
|
||||
if not records:
|
||||
return 0
|
||||
# Nothing new to analyze, but existing rows may still be missing a
|
||||
# narration — the caller runs that outside the lock.
|
||||
return 0, queue_id
|
||||
|
||||
repository = self._knowledge_repository()
|
||||
analyzer = InternalAnalyzer(
|
||||
|
|
@ -401,7 +436,6 @@ class CopyrighterStore(
|
|||
self._sync_face_crops(record["id"], image_store)
|
||||
self._auto_naver_search(record["id"], query_source_evidence, image_store)
|
||||
self._auto_google_custom_search(record["id"], query_source_evidence, image_store)
|
||||
self._ensure_llm_summary(record["id"])
|
||||
self._increment_knowledge_contribution_counts(record["id"], run.evidence)
|
||||
self._rescore_submission(record["id"])
|
||||
self.add_audit_event(
|
||||
|
|
@ -425,7 +459,7 @@ class CopyrighterStore(
|
|||
""",
|
||||
(int(datetime.now().timestamp()), now, queue_id),
|
||||
)
|
||||
return len(records)
|
||||
return len(records), queue_id
|
||||
|
||||
def _prune_missing_submission_files(
|
||||
self,
|
||||
|
|
|
|||
|
|
@ -48,6 +48,11 @@ DEFAULT_QUERY_COVERAGE_WARN_THRESHOLD = 40
|
|||
MIN_COVERAGE_THRESHOLD = 0
|
||||
MAX_COVERAGE_THRESHOLD = 100
|
||||
DEFAULT_FACE_CROP_RETENTION_DAYS = 90
|
||||
# Upper bound on LLM narrations generated by one seed/reload. A folder import can
|
||||
# bring in hundreds of submissions; narrating all of them in one request makes
|
||||
# the reload take as long as the model does, times the batch. The rest are picked
|
||||
# up by the next reload (narration is only-if-missing and therefore resumable).
|
||||
MAX_LLM_SUMMARIES_PER_SEED = 20
|
||||
PAYLOAD_REQUIRED_FIELDS = {
|
||||
"submissions": {
|
||||
"id",
|
||||
|
|
|
|||
198
tests/rights_filter/analysis/test_llm_grounding.py
Normal file
198
tests/rights_filter/analysis/test_llm_grounding.py
Normal file
|
|
@ -0,0 +1,198 @@
|
|||
"""Structured evidence reading: the model cites, and the citation is checked.
|
||||
|
||||
The assistant no longer returns free prose. It returns claims that each carry
|
||||
the evidence labels they rest on, and a claim whose labels were invented is
|
||||
dropped before it can reach the operator. These tests pin that boundary, the
|
||||
prompt input cap, and the field whitelist.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from rights_filter.analysis.llm_assistance import (
|
||||
MAX_PROMPT_EVIDENCE,
|
||||
InternalLlmAssistant,
|
||||
OllamaGenerateLlmClient,
|
||||
_evidence_prompt,
|
||||
_select_prompt_evidence,
|
||||
)
|
||||
from rights_filter.domain.records import Evidence, EvidenceSource
|
||||
|
||||
|
||||
class _ScriptedTransport:
|
||||
"""Returns a canned Ollama response and records the request payload."""
|
||||
|
||||
def __init__(self, document):
|
||||
self.sent = {}
|
||||
self._body = document if isinstance(document, str) else json.dumps(document)
|
||||
|
||||
def request_json(self, method, url, payload=None, timeout=None):
|
||||
self.sent = dict(payload or {})
|
||||
return {"response": self._body, "model": "qwen2.5:3b-instruct"}
|
||||
|
||||
|
||||
def _client(document) -> tuple[OllamaGenerateLlmClient, _ScriptedTransport]:
|
||||
transport = _ScriptedTransport(document)
|
||||
return OllamaGenerateLlmClient(transport=transport), transport
|
||||
|
||||
|
||||
def _web(match="full", url="https://example.test/a", **data):
|
||||
return Evidence(
|
||||
source=EvidenceSource.WEB_DETECTION,
|
||||
reason="Web detection match",
|
||||
confidence=0.9,
|
||||
data={"match": match, "result_url": url, "evidence_id": "ev-1", **data},
|
||||
)
|
||||
|
||||
|
||||
# --- citation verification ---------------------------------------------------
|
||||
|
||||
|
||||
def test_claim_citing_real_evidence_survives():
|
||||
client, _ = _client(
|
||||
{"claims": [{"text": "동일 이미지가 외부 페이지에 있습니다.", "evidence": ["E1"]}],
|
||||
"needs_check": ["출처 페이지의 라이선스 표기"]}
|
||||
)
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert result.dropped_claims == 0
|
||||
assert len(result.claims) == 1
|
||||
assert result.claims[0]["evidence"] == ["E1"]
|
||||
assert "동일 이미지가 외부 페이지에 있습니다." in result.text
|
||||
assert "출처 페이지의 라이선스 표기" in result.text
|
||||
|
||||
|
||||
def test_claim_citing_an_invented_label_is_dropped():
|
||||
# The failure mode that matters: a confident sentence attached to evidence
|
||||
# that was never shown to the model.
|
||||
client, _ = _client(
|
||||
{"claims": [
|
||||
{"text": "실제 근거.", "evidence": ["E1"]},
|
||||
{"text": "지어낸 근거에 붙은 단정.", "evidence": ["E9"]},
|
||||
]}
|
||||
)
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert result.dropped_claims == 1
|
||||
assert [claim["text"] for claim in result.claims] == ["실제 근거."]
|
||||
assert "지어낸" not in result.text
|
||||
|
||||
|
||||
def test_claim_without_any_citation_is_dropped():
|
||||
client, _ = _client({"claims": [{"text": "근거 없는 단정.", "evidence": []}]})
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert result.dropped_claims == 1
|
||||
assert result.claims == ()
|
||||
assert "근거 없는 단정" not in result.text
|
||||
|
||||
|
||||
def test_unparseable_output_yields_no_claims_and_no_prose():
|
||||
# A model that ignores the format contract must not have its prose shown as
|
||||
# if it had been checked.
|
||||
client, _ = _client("I think this image is fine, it looks authentic.")
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert result.claims == ()
|
||||
assert "authentic" not in result.text
|
||||
|
||||
|
||||
def test_json_wrapped_in_prose_is_still_read():
|
||||
client, _ = _client(
|
||||
'Sure! {"claims": [{"text": "근거 있음.", "evidence": ["E1"]}]} hope that helps'
|
||||
)
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert len(result.claims) == 1
|
||||
|
||||
|
||||
def test_partially_valid_citation_list_keeps_only_real_labels():
|
||||
client, _ = _client(
|
||||
{"claims": [{"text": "혼합 인용.", "evidence": ["E1", "E42"]}]}
|
||||
)
|
||||
|
||||
result = client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert result.claims[0]["evidence"] == ["E1"]
|
||||
|
||||
|
||||
# --- prompt input boundary ---------------------------------------------------
|
||||
|
||||
|
||||
def test_prompt_is_capped_and_keeps_the_rows_that_carry_risk():
|
||||
scoring = [_web(match="full", url=f"https://example.test/{index}") for index in range(4)]
|
||||
noise = [
|
||||
Evidence(EvidenceSource.SEARCH_SKIPPED, "quota reached", 0.0, {})
|
||||
for _ in range(MAX_PROMPT_EVIDENCE + 10)
|
||||
]
|
||||
|
||||
selected = _select_prompt_evidence(noise + scoring)
|
||||
|
||||
assert len(selected) == MAX_PROMPT_EVIDENCE
|
||||
assert all(item in selected for item in scoring)
|
||||
|
||||
|
||||
def test_prompt_excludes_unvetted_and_privacy_fields():
|
||||
evidence = _web(
|
||||
promotion_reason="이 값을 그대로 베끼면 요약이 증거 목록 복붙이 된다",
|
||||
privacy_note="동일 인물 판정은 아닙니다",
|
||||
crop_index=2,
|
||||
thumbnail_url="https://example.test/thumb.jpg",
|
||||
page_title="허용되는 제목",
|
||||
)
|
||||
|
||||
prompt = _evidence_prompt([evidence])
|
||||
|
||||
assert "허용되는 제목" in prompt
|
||||
assert "복붙" not in prompt
|
||||
assert "privacy_note" not in prompt
|
||||
assert "crop_index" not in prompt
|
||||
assert "thumbnail_url" not in prompt
|
||||
|
||||
|
||||
def test_prompt_labels_evidence_so_the_model_can_cite_it():
|
||||
prompt = _evidence_prompt(_select_prompt_evidence([_web(), _web(url="https://b.test")]))
|
||||
|
||||
assert "[E1]" in prompt
|
||||
assert "[E2]" in prompt
|
||||
|
||||
|
||||
def test_llm_summary_evidence_is_never_fed_back_into_the_prompt():
|
||||
prior = Evidence(EvidenceSource.LLM_SUMMARY, "prior summary", 0.0, {"summary": "이전 요약"})
|
||||
|
||||
selected = _select_prompt_evidence([prior, _web()])
|
||||
|
||||
assert all(item.source != EvidenceSource.LLM_SUMMARY for item in selected)
|
||||
|
||||
|
||||
# --- request shape -----------------------------------------------------------
|
||||
|
||||
|
||||
def test_request_pins_the_context_window_and_asks_for_json():
|
||||
client, transport = _client({"claims": []})
|
||||
|
||||
client.summarize_evidence_detailed([_web()])
|
||||
|
||||
assert transport.sent["format"] == "json"
|
||||
assert transport.sent["options"]["num_ctx"] == client.num_ctx
|
||||
assert transport.sent["keep_alive"] == client.keep_alive
|
||||
|
||||
|
||||
@pytest.mark.parametrize("field", ["grounded_claims", "dropped_claims"])
|
||||
def test_assistant_records_how_much_survived_verification(field):
|
||||
client, _ = _client(
|
||||
{"claims": [
|
||||
{"text": "진짜.", "evidence": ["E1"]},
|
||||
{"text": "가짜.", "evidence": ["E7"]},
|
||||
]}
|
||||
)
|
||||
|
||||
evidence = InternalLlmAssistant(client).summarize("SUB-1", [_web()])
|
||||
|
||||
assert evidence.data[field] == 1
|
||||
|
|
@ -3,7 +3,7 @@ import base64
|
|||
from rights_filter.analysis.preprocessing import ImagePayload
|
||||
from rights_filter.domain.records import Evidence, EvidenceSource
|
||||
from rights_filter.integrations.cloud_vision_web_detection import GoogleVisionRestClient
|
||||
from rights_filter.integrations.env_clients import build_provider_runtime
|
||||
from rights_filter.integrations.env_clients import DEFAULT_OLLAMA_MODEL, build_provider_runtime
|
||||
from rights_filter.integrations.google_custom_search import GoogleCustomSearchClient
|
||||
from rights_filter.integrations.naver_search import NaverOpenApiSearchClient
|
||||
from rights_filter.analysis.llm_assistance import OllamaGenerateLlmClient
|
||||
|
|
@ -130,8 +130,12 @@ def test_google_custom_search_client_uses_env_key_and_search_engine_id():
|
|||
assert "searchType=image" in call["url"]
|
||||
|
||||
|
||||
def test_ollama_generate_client_uses_local_endpoint_and_returns_response_text():
|
||||
transport = FakeJsonTransport({"response": "Source-linked summary"})
|
||||
def test_ollama_generate_client_uses_local_endpoint_and_renders_verified_claims():
|
||||
# The client no longer passes the model's prose through: it reads the
|
||||
# structured claims and renders only the ones whose citations check out.
|
||||
transport = FakeJsonTransport(
|
||||
{"response": '{"claims": [{"text": "출처 페이지가 확인됩니다.", "evidence": ["E1"]}]}'}
|
||||
)
|
||||
client = OllamaGenerateLlmClient(
|
||||
base_url="http://localhost:11434",
|
||||
model="qwen2.5:0.5b-instruct",
|
||||
|
|
@ -149,7 +153,8 @@ def test_ollama_generate_client_uses_local_endpoint_and_returns_response_text():
|
|||
summary = client.summarize_evidence(evidence)
|
||||
|
||||
call = transport.calls[0]
|
||||
assert summary == "Source-linked summary"
|
||||
assert "출처 페이지가 확인됩니다." in summary
|
||||
assert "[E1]" in summary
|
||||
assert call["method"] == "POST"
|
||||
assert call["url"] == "http://localhost:11434/api/generate"
|
||||
assert call["headers"] == {}
|
||||
|
|
@ -274,4 +279,6 @@ def test_provider_runtime_leaves_missing_env_providers_disabled():
|
|||
"GOOGLE_CUSTOM_SEARCH_CX": False,
|
||||
}
|
||||
assert runtime.provider_payloads["llm"]["enabled"] is True
|
||||
assert runtime.provider_payloads["llm"]["compliance"] == "Ollama local API configured (qwen2.5:0.5b-instruct)"
|
||||
assert runtime.provider_payloads["llm"]["compliance"] == (
|
||||
f"Ollama local API configured ({DEFAULT_OLLAMA_MODEL})"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -7482,7 +7482,11 @@ def test_sqlite_store_rerun_enrichment_uses_ollama_llm_when_configured(tmp_path:
|
|||
|
||||
|
||||
def test_sqlite_store_seed_auto_runs_ollama_llm_when_source_evidence_exists(tmp_path: Path):
|
||||
transport = FakeJsonTransport({"response": "Seed-time source summary"})
|
||||
# The assistant reads structured claims now; prose without citations is
|
||||
# dropped before it can be stored.
|
||||
transport = FakeJsonTransport(
|
||||
{"response": '{"claims": [{"text": "시드 시점 근거 요약.", "evidence": ["E1"]}]}'}
|
||||
)
|
||||
runtime = build_provider_runtime(
|
||||
env={"OLLAMA_BASE_URL": "http://localhost:11434", "OLLAMA_MODEL": "qwen2.5:0.5b-instruct"},
|
||||
transport=transport,
|
||||
|
|
|
|||
Loading…
Reference in a new issue