Arnaud Héritier a01b9102d5 test(eval): avoid writing executable runtimes during parallel tests ...

Create the shared shell fixture before the parent parallel barrier and use per-case symlinks and data files. Preserve digest pinning and fallback assertions without concurrent writes to executable fixture inodes.

Signed-off-by: Arnaud Héritier <aheritier@users.noreply.github.com>

2026-10-06 09:30:58 +00:00

..

agent_image_test.go

test(evaluation): stop mutating version.Version from parallel tests

2026-09-10 18:18:27 +02:00

agent_image.go

test(evaluation): stop mutating version.Version from parallel tests

2026-09-10 18:18:27 +02:00

assertions_test.go

feat: wire assertion runner into runSingleEval execution pipeline

2026-09-01 16:48:52 +02:00

assertions.go

feat: wire assertion runner into runSingleEval execution pipeline

2026-09-01 16:48:52 +02:00

baseline_test.go

feat: include assertions and pass@k in baseline regression comparison

2026-09-01 18:01:16 +02:00

baseline.go

feat: include assertions and pass@k in baseline regression comparison

2026-09-01 18:01:16 +02:00

build_test.go

test(eval): avoid writing executable runtimes during parallel tests

2026-10-06 09:30:58 +00:00

build.go

fix(eval): pin local agent image digest when building containers

2026-10-04 18:07:35 +02:00

dockerfile_template_test.go

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

Dockerfile.custom.template

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

Dockerfile.template

feat(eval): pin injected docker-agent image to host CLI version by default

2026-09-04 19:00:51 +00:00

eval_test.go

refactor: migrate UUID generation and parsing to Go 1.27 stdlib

2026-09-23 15:44:22 +02:00

eval.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

flavors_test.go

fix(eval): skip POSIX flavor runtime fixture on Windows

2026-10-02 11:33:13 +02:00

judge_model_test.go

eval: default judge model to openai/gpt-5.6-terra

2026-10-05 10:39:24 +02:00

judge_test.go

eval: include judge reasons for passed relevance criteria

2026-04-02 14:59:39 +02:00

judge.go

feat: upgrade relevance judge prompt with rubric, CoT, and anti-bias rules

2026-09-01 09:27:26 +02:00

names.go

Add evaluation framework with Docker isolation

2026-01-08 09:22:17 +01:00

progress.go

feat: wire assertion runner into runSingleEval execution pipeline

2026-09-01 16:48:52 +02:00

repeat_test.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

repeat.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

save_evaluation_test.go

fix: preserve parallel tool batches when replaying evaluator events

2026-09-22 17:59:59 +02:00

save_replay_test.go

fix: preserve parallel tool batches when replaying evaluator events

2026-09-22 17:59:59 +02:00

save_test.go

test(evaluation): stop mutating version.Version from parallel tests

2026-09-10 18:18:27 +02:00

save.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

scoring.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00

structured_output_test.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

structured_output.go

feat(eval): support --flavor and capture structured tool output

2026-10-02 11:15:38 +02:00

termination.go

fix(eval): preserve budget termination outcomes

2026-07-28 15:47:36 +02:00

types.go

feat(eval): report repeat stability and cost outliers

2026-10-02 11:49:05 +02:00