*"hj 6 d Z ddlZddlZddlZddlZddlmZmZ ej e Z
ej d Z ej dej ZdZdZdZd ed
efd Zd ed
ed
ee fdZd ed
ee fdZdeee d
efdZ G d d Zy)a
Process Reward Model (PRM) scorer.
Uses an OpenAI-compatible /v1/chat/completions endpoint as the judge,
so any external API (Anthropic, OpenAI, Together, etc.) works out of the box.
Public API:
scorer = PRMScorer(
prm_url="https://api.openai.com/v1",
prm_model="gpt-5.2",
api_key="your-openai-api-key",
prm_m=3,
)
result = await scorer.evaluate(response_text, instruction_text)
# result: {"score": float, "votes": list, "eval_text": str}
N)AnyOptionalz\\boxed\{([-+]?\d)\}zScore:\s*([-+]?\d)z[32mz[36mz[0mtextreturnc ddl }|j dd| |j } |j dd| } |j dd | } | S )
z7Replace XML-like tags that may trigger content filters.r Nz.*?z[tool_call block])flagsz<[a-zA-Z_][^>]{0,80}>z[tag]z]{0,80}>z[/tag])resubDOTALL)r _res //root/.hermes/SkillClaw/skillclaw/prm_scorer.py_sanitize_textr ( sQ 7702EtSVS]S]7 ^D 77+Wd ;D 77,h =D K
response_textinstruction_textc X d}t | }t | }d| d| d}d|dd|dgS )z0Construct the judge messages for PRM evaluation.a You are a quality reviewer for conversational responses.
You will be shown a user instruction and the assistant response to that instruction.
Based on instruction alignment and task completion quality, decide whether the response was helpful (+1), unhelpful (-1), or unclear (0).
Do NOT compare against any follow-up turn.
Only evaluate whether the response addresses the given instruction.
Use +1 when the response clearly follows and substantially completes the instruction.
Use -1 when the response is off-task, wrong, or fails to complete core requirements.
Use 0 when completion is ambiguous or evidence is insufficient.
Think briefly, then end your reply with exactly one of: Score: 1 / Score: -1 / Score: 0z
Instruction:
z
Response:
z[
Was the response helpful for this instruction? End with Score: 1, Score: -1, or Score: 0.system)rolecontentuser)r )r r r clean_instructionclean_responser s r
_build_prm_judge_promptr 4 s_ b ''78#M2N
*+ ,$% &5 5 & 1Ft3T UUr c t j | }|rt |d }|dv r|S t j | }|rt |d }|dv r|S y)zParse score from model output.
Accepts both formats:
- "Score: 1" / "Score: -1" / "Score: 0" (primary, avoids Azure filter)
- "\boxed{1}" / "\boxed{-1}" / "\boxed{0}" (legacy fallback)
) r r N) _SCORE_REfindallint _BOXED_RE)r matchesvals r
_parse_prm_scorer# M sc %G'"+ * J %G'"+ * J r scoresc | D cg c] }|| }}|syt j | }|j d d }t |j j |d dkD ryt
|d S c c}w )u3 Return the majority vote; ties or all-None → 0.0. r r ) collectionsCounter most_commonlistvaluescountfloat)r$ svalidcountertops r
_majority_voter2 c s} 01!-Q 0E 0 !!%(G
a
#C GNN ##CF+a/ Q=
1s
A<A<c
e Zd ZdZ ddedededededef d Z dd
ed ed ed
edef
dZ de
e dede e e ef fdZ
y) PRMScoreru{
Async PRM scorer using any OpenAI-compatible /v1/chat/completions API.
Works with external providers (Anthropic, OpenAI, Together, Fireworks, etc.)
or a self-hosted vLLM / LiteLLM proxy — anything that accepts
POST /v1/chat/completions with {"model", "messages", ...}.
Parameters
----------
prm_url:
Base URL of the API, e.g.
- ``"https://api.anthropic.com/v1"``
- ``"https://api.openai.com/v1"``
- ``"http://localhost:8081/v1"`` (local vLLM)
prm_model:
Model name to pass in the request body.
api_key:
Bearer token. If empty, no Authorization header is sent
(useful for unauthenticated local endpoints).
prm_m:
Number of parallel votes for majority voting.
temperature:
Sampling temperature.
max_new_tokens:
Maximum tokens to generate per vote.
Nprm_url prm_modelapi_keyprm_m temperaturemax_new_tokensc ||| _ n. ddlm} |j d }
d|i} |
| d< |di | | _ || _ || _ || _ || _ y # t $ r} t d | d } ~ ww xY w)Nr )OpenAIzLPRMScorer requires the 'openai' package. Install it with: pip install openai/r7 base_url ) _clientopenair< ImportErrorrstripr6 r8 r9 r: ) selfr5 r6 r7 r8 r9 r:
llm_clientr< er>
client_kwargss r
__init__zPRMScorer.__init__ s !%DL
y) ~~c*H-6,@M(0M* %!2M2DL"
& ,
y!"pqwxx
ys A A0 A++A0response instruction
session_idturn_numr c K t || }t j t | j D cg c] }| j || c} d{ }|D cg c] }|d } }t
| }
d} |
dk7 r |D ] \ } }
| | t |
k( s|
} n | D cg c] } | | nd
}} t j t d| d| d| j d | d
|
t |
|| d S c c}w 7 c c}w c c} w w) u^
Score one assistant response against the instruction for that turn.
Returns
-------
dict with keys:
"score" – float in {-1.0, 0.0, 1.0}
"votes" – list of per-vote scores (int | "fail")
"eval_text" – representative evaluation text (non-empty when score != 0)
Nr r& failz[PRMScorer] session=z turn=z model=z votes=u → score=)scorevotes eval_text)
r asynciogatherranger8 _query_oncer2 r loggerinfo_CYANr6 _RESET)rD rI rJ rK rL msgsiresultsrr$ finalrepresentativer. r
votes_displays r
evaluatezPRMScorer.evaluate s% " 'x= ERVR\R\L](^q)9)9$)B(^__ '(1!A$(( v& C<"
4=Q#e*_%)N
BHHAam7H
H g)* VH: F^^$GM?+eWVH
V
^TT# )__( IsG 3DC0DC5D C7* D D D!
C<.AD7
Dmessagesvote_idc K t j | j j j j
| j || j | j d { }|j d j j xs d}t | |fS 7 8# t $ r!}t j d|| Y d }~yd }~ww xY ww)N)modelrc r9 max_completion_tokensr rN z&[PRMScorer] query failed (vote %d): %s)NrN )rS to_threadr@ chat completionscreater6 r9 r: choicesmessager r# ExceptionrW warning)rD rc rd
completionr rF s r
rV zPRMScorer._query_once s & 0 0 !!--44nn! , ,&*&9&9 J !((+33;;ArG#G,g5 5 NNCWa P sA CA#B" 'B (7B" C B" " C +CCC C)zgpt-5.2rN g333333?i N)rN r )__name__
__module__ __qualname____doc__strr r- rH dictrb r* tupler rV r? r r
r4 r4 t s < # "-- - -
-
- -> $U$U $U $U
$U
$UL
$t*
s
uXVY]\_M_G`
r r4 ) ru rS r' loggingr typingr r getLoggerrr rW compiler
IGNORECASEr _GREENrY rZ rv r r* rw r r r# r- r2 r4 r? r r
r s " 8 $ BJJ. / BJJ,bmm < V3 V# V$t* V23 8C= , 4
. 5 "j j r