melmennaoui fcb865d638 fix(evaluation): preserve tool evidence in judge transcripts ...

Remove the implicit 500-byte tool-result cutoff so relevance judges can assess the same evidence recorded for the agent. Cover boundary lengths, tail and middle evidence, UTF-8, and large outputs.

Closes #4770

Assisted-By: docker-agent
Signed-off-by: melmennaoui <mohamed.elmennaoui@docker.com>

2026-10-29 23:02:31 +01:00

..

testdata

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

agent_image_test.go

test(evaluation): stop mutating version.Version from parallel tests

2026-09-10 18:18:27 +02:00

agent_image.go

test(evaluation): stop mutating version.Version from parallel tests

2026-09-10 18:18:27 +02:00

assertions_test.go

feat: wire assertion runner into runSingleEval execution pipeline

2026-09-01 16:48:52 +02:00

assertions.go

feat: wire assertion runner into runSingleEval execution pipeline

2026-09-01 16:48:52 +02:00

baseline_test.go

feat: include assertions and pass@k in baseline regression comparison

2026-09-01 18:01:16 +02:00

baseline.go

feat: include assertions and pass@k in baseline regression comparison

2026-09-01 18:01:16 +02:00

build_test.go

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

build.go

fix(eval): pin local agent image digest when building containers

2026-10-04 18:07:35 +02:00

container_env_test.go

refactor: centralize providers in an embedded catalog

2026-10-28 22:03:27 +01:00

dockerfile_template_test.go

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

Dockerfile.custom.template

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

Dockerfile.template

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

eval_judge_test.go

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

eval_test.go

fix(evaluation): preserve tool evidence in judge transcripts

2026-10-29 23:02:31 +01:00

eval.go

fix(evaluation): preserve tool evidence in judge transcripts

2026-10-29 23:02:31 +01:00

flavors_test.go

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

judge_container_test.go

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

judge_container.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge_evaluator_test.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge_evaluator.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge_factory_test.go

feat: route evaluators through the models gateway, add OpenAI Decisions

2026-10-07 12:36:48 +02:00

judge_factory.go

feat: route evaluators through the models gateway, add OpenAI Decisions

2026-10-07 12:36:48 +02:00

judge_llm_test.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge_llm.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge_model_test.go

eval: default judge model to openai/gpt-5.6-terra

2026-10-05 10:39:24 +02:00

judge_test.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

judge.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

mock_script_test.go

test(evaluation): reuse an immutable runtime launcher

2026-10-17 16:32:22 +02:00

names.go

Add evaluation framework with Docker isolation

2026-01-08 09:22:17 +01:00

progress_lifecycle_test.go

refactor(evaluation): scope progress rendering to its run

2026-10-21 19:17:29 +02:00

progress_test.go

refactor(evaluation): scope progress rendering to its run

2026-10-21 19:17:29 +02:00

progress.go

refactor(evaluation): scope progress rendering to its run

2026-10-21 19:17:29 +02:00

repeat_test.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

repeat.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

save_evaluation_test.go

fix: preserve parallel tool batches when replaying evaluator events

2026-09-22 17:59:59 +02:00

save_replay_test.go

fix: preserve parallel tool batches when replaying evaluator events

2026-09-22 17:59:59 +02:00

save_test.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

save.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00

scoring.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

structured_output_test.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

structured_output.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

termination.go

fix(eval): preserve budget termination outcomes

2026-07-28 15:47:36 +02:00

types.go

feat(eval): add evaluator judge backend

2026-10-07 10:30:07 +02:00