LOOM
How usable can a 30B-class local model become on an 8 GB M1 Mac if the surrounding runtime, storage and expert-access path are engineered aggressively?
A runtime study of a 30B-class local model on an 8 GB Apple Silicon M1.
LOOM investigated whether aggressive runtime, storage and expert-access engineering could make a 30B-class local model practically useful on hardware far below its conventional memory envelope. The retained runtime became materially faster and more usable. The project nevertheless closed because the remaining constraint was model capability rather than throughput.
Separate runtime performance from model capability.
The study treated serving, expert residency, routed-expert access, storage placement, lossless sidecar layout, quantization, context handling and speculative decoding as bounded engineering variables. Candidates were promoted only when they preserved frozen behavioral references; negative results were retained rather than hidden.
A staged lineage with explicit GO / NO_GO decisions.
UNLOCKED UOPT-003 S40
Matched measurements below compare the retained S40 profile against the S32 baseline where a controlled baseline is available. They describe this project configuration, not a general claim about 30B models or Apple Silicon.
- Decode
- 6.660 tok/s7.557 tok/s+13.46%
- 1,155-token cold prefill
- 11.306 tok/s
- 1,155-token cold TTFT
- 161.676 s102.242 s−36.76%
- Expert hit rate
- 81.745%89.831%
- Expert misses
- 90,22950,260
Performance gains were retained only with behavioral checks intact.
- 0 / 6
- Refusal gate
- 0 / 6
- Held-out degeneration gate
- 8 / 8
- Benign capability gate
What the experiments established.
Lossless layout changes were high leverage.
UOPT-002 reduced routed-expert miss I/O through an expert-major sidecar while preserving the mathematical content of the model.
Expert residency materially changed runtime behavior.
UOPT-003 S40 improved decode and cold-start behavior without changing model weights and became the final retained profile.
Lower-bit expert quantization crossed the quality boundary.
UOPT-004 and UOPT-005 reduced footprint but failed the frozen functional arithmetic reference, so smaller degraded candidates were not promoted.
Storage placement can dominate apparent performance.
A diagnostic experiment isolated an external-volume sidecar as the bottleneck rather than the model or runtime logic.
Draftless speculation did not engage.
UOPT-006 ngram-simple and ngram-mod produced 0 drafted and 0 accepted tokens and did not reach the immediate 5% decode-improvement gate.
The final bottleneck was capability.
Additional speed ideas remained possible, but they were deliberately not pursued because they would not move the practical intelligence ceiling of the retained model.
One storage experiment changed the apparent performance by an order of magnitude.
This was a diagnostic comparison, not the project baseline: moving only a temporary sidecar from an external archive volume to internal storage isolated storage placement as the cause of pathological performance.
- Decode
- 0.69 tok/s→9.63 tok/s
- TTFT
- 148.277 s→5.605 s
The correct optimization target changed.
LOOM did not end because of an unresolved runtime failure. By closure, runtime engineering had succeeded enough that further throughput work would have optimized the wrong constraint. The retained 30B model still did not meet the practical capability required for the intended workloads.
Reassess the model target before investing in additional runtime optimization.
A research record, not a model distribution repository.
The public archive preserves durable source, configuration, patches and experiment documentation. Large or machine-local artifacts are intentionally excluded so the repository records engineering provenance without redistributing models or local evidence bundles.
Open research archive ↗- Repository is closed, archived and read-only by default.
- Model GGUF files, expert sidecars, runtime builds, caches and local benchmark evidence remain outside Git.
- Historical experiment records preserve measurements, rollback notes and implementation provenance.
- A future restart should reassess the model/hardware premise before continuing the historical optimization frontier.