-
Notifications
You must be signed in to change notification settings - Fork 69
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
[Loop Eval][Replay] Mine loop-control bug candidates with repeated prompt replay
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:loop-controlGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorkind:evaluationBenchmark adapters, eval runners, result schemas, and metricsBenchmark adapters, eval runners, result schemas, and metricskind:testingAutomated checks, release gates, regression tests, and validationAutomated checks, release gates, regression tests, and validationpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[Loop Eval][Discover] Run tokenmaxxing/context optimization research loop
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:loop-controlGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorarea:tokenmaxxing-srTokenmaxxing telemetry, context/cache evidence, SR policy, and routing metadataTokenmaxxing telemetry, context/cache evidence, SR policy, and routing metadatakind:evaluationBenchmark adapters, eval runners, result schemas, and metricsBenchmark adapters, eval runners, result schemas, and metricskind:researchResearch questions, hypotheses, taxonomy, and policy designResearch questions, hypotheses, taxonomy, and policy designpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[Loop Eval][Deliver] Close a real loop-control bug/use case with verification
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:loop-controlGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorkind:evaluationBenchmark adapters, eval runners, result schemas, and metricsBenchmark adapters, eval runners, result schemas, and metricskind:testingAutomated checks, release gates, regression tests, and validationAutomated checks, release gates, regression tests, and validationpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M4][evaluation] Compare final acceptance against Terminal, SWE, and internal evidence
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:release-opsMilestones, release gates, final acceptance, docs, and issue closureMilestones, release gates, final acceptance, docs, and issue closurekind:evaluationBenchmark adapters, eval runners, result schemas, and metricsBenchmark adapters, eval runners, result schemas, and metricspriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M4][testing] Add evidence index consistency checks
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:release-opsMilestones, release gates, final acceptance, docs, and issue closureMilestones, release gates, final acceptance, docs, and issue closurekind:testingAutomated checks, release gates, regression tests, and validationAutomated checks, release gates, regression tests, and validationpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M4][engineering] Generate GA evidence index from ga-eval artifacts
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:release-opsMilestones, release gates, final acceptance, docs, and issue closureMilestones, release gates, final acceptance, docs, and issue closurekind:engineeringProduct/runtime implementation, docs wiring, and operational workProduct/runtime implementation, docs wiring, and operational workpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M3][testing] Add internal corpus smoke tasks for verifier, compression, and self-improve
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:self-improveLearning signals, skill proposals, replay gates, and post-adoption impactLearning signals, skill proposals, replay gates, and post-adoption impactkind:testingAutomated checks, release gates, regression tests, and validationAutomated checks, release gates, regression tests, and validationpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M3][engineering] Add maker/checker policy for eval tasks
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:loop-controlGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorkind:engineeringProduct/runtime implementation, docs wiring, and operational workProduct/runtime implementation, docs wiring, and operational workpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M3][engineering] Run GA eval tasks in managed worktrees
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:loop-controlGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorGoal, horizon, attempt, verifier, decision, HIL, and completion behaviorkind:engineeringProduct/runtime implementation, docs wiring, and operational workProduct/runtime implementation, docs wiring, and operational workpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M3][evaluation] Implement internal corpus runner
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:self-improveLearning signals, skill proposals, replay gates, and post-adoption impactLearning signals, skill proposals, replay gates, and post-adoption impactkind:evaluationBenchmark adapters, eval runners, result schemas, and metricsBenchmark adapters, eval runners, result schemas, and metricspriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M3][research] Define internal real-loop corpus taxonomy and manifest
area:evaluation-harnessBenchmark tasks, adapters, artifacts, and reproducible result recordsBenchmark tasks, adapters, artifacts, and reproducible result recordsarea:self-improveLearning signals, skill proposals, replay gates, and post-adoption impactLearning signals, skill proposals, replay gates, and post-adoption impactkind:researchResearch questions, hypotheses, taxonomy, and policy designResearch questions, hypotheses, taxonomy, and policy designpriority:p1-gaRequired for GA, but not blocking the first closed evidence sliceRequired for GA, but not blocking the first closed evidence sliceStatus: Open.[GA-M2][testing] Add offline route replay fixtures for cost and quality guardrails
area:tokenmaxxing-srTokenmaxxing telemetry, context/cache evidence, SR policy, and routing metadataTokenmaxxing telemetry, context/cache evidence, SR policy, and routing metadatakind:testingAutomated checks, release gates, regression tests, and validationAutomated checks, release gates, regression tests, and validationpriority:p0-blockerBlocks the first credible GA evidence slice or core GA claimBlocks the first credible GA evidence slice or core GA claimStatus: Open.