# Nymrel: Astra / Fable native workflow pilot

All six workflows passed all five tasks. This pilot hit a ceiling: cross-model refinement showed no quality gain over either one pass or same-model refinement. Two-pass workflows took more component time. The result does not establish how pairing performs on harder studio work.

Five small, sanitized Python repair tasks; 37 held-out examples across the five tasks per workflow (222 executed example checks across all six workflows); 30 native CLI calls. This tests bounded offline repairs, not production development or isolated base-model ability.

| Workflow | Complete passes | Total component time, five tasks |
|---|---:|---:|
| Astra | 5/5 | 49.64 s |
| Fable native workflow | 5/5 | 43.34 s |
| Astra → Astra | 5/5 | 107.52 s |
| Fable → Fable | 5/5 | 103.35 s |
| Astra → Fable | 5/5 | 110.82 s |
| Fable → Astra | 5/5 | 106.55 s |

Two-pass times include both calls. Initial answers feed two refinements, so the outcomes are correlated. These totals represent each workflow separately; adding all six rows would double-count reused first passes.

| Task | A | F | AA | FF | AF | FA |
|---|---|---|---|---|---|---|
| 01_state | PASS | PASS | PASS | PASS | PASS | PASS |
| 02_data | PASS | PASS | PASS | PASS | PASS | PASS |
| 03_validation | PASS | PASS | PASS | PASS | PASS | PASS |
| 04_performance | PASS | PASS | PASS | PASS | PASS | PASS |
| 05_maintainability | PASS | PASS | PASS | PASS | PASS | PASS |

## Experiment overhead

The 30 scored calls totaled 335.3 seconds. Claim-to-collection wall time was 33.7 minutes, including 28.1 minutes outside scored calls. That includes setup, method reviews, fixes and orchestration, and excludes earlier startup and later reporting/publication. This was not an end-to-end efficiency win on five small tasks. The setup is reusable, but future savings remain unmeasured. See setup-overhead.json.

## Scope and limits

- Same tasks, fresh contexts, unlabeled refinements, hidden checks, no grader feedback or performance-based retries.
- Same-model second passes control for additional work. Five cases provide descriptive results, not statistical significance.
- Fable native calls can include Haiku background work; all reported model usage is included. Astra identity is verified in runtime context, with provider-resolved identity unavailable.
- Both primary routes request high effort. CLI context overhead and provider token definitions differ; compute is not equalized.
- Membership and AST budgets are narrow proxies. Neither establishes production speed or universal code quality.
- Setup, protocol reviews, and safety review are additional overhead. Main-session and subagent tokens are not fully attributable; scored-call timing is not end-to-end studio savings. Subscription use does not establish cash cost.
- The original first-call receipt check rejected Windows CRLF serialization. A documented amendment normalizes text validation, preserves the original evidence, and reuses the call once. Tasks, order, answers and grader are unchanged.

## Offline reproduction

The evidence bundle includes every task, held-out check, oracle, emitted candidate and grade. On the public gist, download artifact-data.json and extract_evidence.py into the same directory, then run `python extract_evidence.py` to extract and verify the files in a new nymrel-pilot-evidence folder. Change into that folder. After reviewing candidate safety, rerun a grade with:

```text
python cases/evaluate.py cases/01_state outputs/01_state/A.json
```

The evaluator uses restricted Python and time limits, not an OS sandbox for hostile code. Only run these inspected cooperative submissions. Published cases become contaminated for future model evaluations.

See summary.json for provider-specific usage, per-call metadata, failures, and component durations. Protocol.md records the native-call method. The artifact manifest verifies the exported files.