Goal and constraints
The objective was to fine-tune Qwen3.6-27B on the existing V1 Svelte dataset so it could be compared directly with the earlier Nemotron V1 model. The job ran locally on Morningstar: one NVIDIA RTX 5090 with 31.84 GiB of usable VRAM under WSL.
A full-parameter run was not practical in that memory envelope. The chosen approach was text-only 4-bit NF4 QLoRA with bfloat16 compute, gradient checkpointing, tiled MLP execution, and adapters limited to the final language layers.
Final recipe
- Base checkpoint
- unsloth/Qwen3.6-27B at revision c7d3d676ae77b6ae969f875aec68ed2daad7118c
- Dataset
- 900 V1 training examples and 100 validation examples
- Sequence length
- 7,680 tokens; no accepted example was truncated
- Token distribution
- 943 minimum, 3,343 median, 5,382 p95, 7,420 maximum
- LoRA
- Rank 8, alpha 16, no dropout, final 16 of 64 language layers
- Target modules
- q/k/v/o projections plus gate/up/down MLP projections
- Trainable parameters
- 9,961,472, or 0.037% of the 26.9B-parameter model
- Optimization
- Batch 1, gradient accumulation 8, one epoch, 5e-5 cosine learning rate
Failures and recovery
The first configurations proved that VRAM, not raw arithmetic throughput, was the hard local constraint. The failures were recoverable and were used to converge on the final recipe.
| Attempt | Result | What changed |
|---|---|---|
| Initial full adapter recipe | OOM | Reached the early optimizer steps, then exhausted VRAM during checkpointed backward execution. |
| Second memory pass | OOM at step 10 | Confirmed that the failure was repeatable on long sequence groups. |
| WSL offload experiment | Driver failure | Activation offload triggered a CUDA “device not ready” path, so that approach was removed. |
| Final 12-layer experiment | OOM at step 10 | Reducing adapter layers alone was not sufficient. |
| Preallocated checkpoint buffer | Passed | A fixed gradient-checkpoint buffer removed allocator fragmentation and allowed the stronger final-16-layer recipe. |
| Production run | Completed | After step 101 stalled, checkpoint 100 was verified and resumed; all remaining steps and final evaluation completed. |
The successful production path took about 12 hours 23 minutes from launch through adapter export. Including the earlier configuration experiments, the end-to-end effort was about 15 hours.
Training result
| Checkpoint | Eval loss | Interpretation |
|---|---|---|
| Step 50 | 0.3233 | First full held-out evaluation |
| Step 100 | 0.3084 | 4.6% improvement from step 50 |
| Step 113 | 0.3082 | Final result; stable rather than overfitting sharply |
The exported adapter loaded successfully over the local Qwen3.6-27B NVFP4 serving checkpoint in vLLM. A direct Svelte 5 completion worked before benchmarking began.
Benchmark result
All models received the same frozen ten-prompt suite with greedy decoding and reasoning disabled. Answers were preserved verbatim, then evaluated through compiler/static checks, a fixed six-category rubric, and order-swapped pairwise judgments.
| Model | Static clean | Rubric mean | Pairwise result |
|---|---|---|---|
| Qwen3.6-27B base | 0 / 10 | 2.33 / 5 | Beat the Qwen adapter 5–4; one case was order-sensitive |
| Qwen3.6-27B V1 adapter | 0 / 10 | 2.22 / 5 | Beat Nemotron V1 5–3; two cases were order-sensitive |
| Nemotron V1 adapter | 2 / 10 | 1.83 / 5 | More compile-clean outputs, but weaker judged implementations overall |
The Qwen adapter's failures were substantive: several answers mixed legacy export let or $: syntax with runes, two hallucinated nonexistent Svelte reactivity imports, one included a Markdown fence, and one stopped after a single completion token with an empty answer. Its responses were often visually richer, which explains why the pairwise judge preferred it over Nemotron more often despite the static failures.
The Fable judge is useful but not independent of the training-data family, and order sensitivity remains visible. Compiler/static results are the stronger signal for code correctness. On that signal, this V1 Qwen adapter is not ready to replace the base model.
Inspect every before-and-after component
The live Morningstar report contains every raw answer, static result, rubric rationale, pairwise verdict, local source view, and Siva Playground link. The previously missing playgrounds were backfilled after Siva came online; all 40 repaired URLs returned HTTP 200.
- Open the complete live benchmark report
- Jump to Qwen base — before fine-tuning
- Jump to Qwen V1 adapter — after fine-tuning
- Jump to Nemotron base
- Jump to Nemotron V1 adapter
Representative direct playground pair
These two links open the same keyboard-command-palette task before and after the Qwen fine-tune:
Conclusion and next move
The 5090 is capable of fine-tuning a 27B model locally when the recipe is aggressively memory-aware. The run proved the infrastructure, recovery, export, serving, and benchmark path end to end. The weakness was not feasibility; it was the training target quality and behavioral consistency of the V1 dataset when applied to this already-strong Qwen base.
Do not repeat this recipe unchanged. First filter or repair examples that teach legacy Svelte, Markdown-wrapped answers, invalid imports, or mixed runes mode. Then run a shorter checkpoint sweep against the same frozen benchmark before committing to another full overnight epoch.
Repository validation after the run passed 31 tests, Svelte check with zero diagnostics, ESLint, Prettier, shell syntax checks, Python compilation, and Git whitespace checks.