Merge pull request 'docs: llama.cpp provenance (patched, not stock) + mermaid diagrams' (#8 ) from docs/inference-stack into main

docs: document llama.cpp provenance (patched, not stock) + mermaid diagrams (#4 )
2026-09-29 08:37:05 +01:00 · 2026-09-29 07:36:55 +00:00
4 changed files with 569 additions and 5 deletions
@@ -22,7 +22,36 @@ path; for inference we wrote our own Go GGUF converter.

 Details: [`docs/audit-open1b-init.md`](docs/audit-open1b-init.md),
 [`docs/open1b-local-inference.md`](docs/open1b-local-inference.md),
-[`docs/benchmarks.md`](docs/benchmarks.md).
+[`docs/benchmarks.md`](docs/benchmarks.md),
+[`docs/inference-stack.md`](docs/inference-stack.md).
+
+## How it works
+
+**Audit — replay one published commitment and compare.**
+
+```mermaid
+flowchart LR
+  KIT["Gensyn audit kit"] --> SHA{"sha256\nvs kit.json"}
+  SHA --> WHL["repop + pretrain wheels"]
+  WHL --> REPLAY["pretrain-audit-replay\n--from-init --device cpu"]
+  REPLAY --> H1["regenerated\ninit state hash"]
+  KIT --> H2["published\ncommitment"]
+  H1 --> CMP{"equal?"}
+  H2 --> CMP
+  CMP -->|yes| OK["MATCH"]
+  CMP -->|no| BAD["MISMATCH"]
+```
+
+**Inference — convert in Go, serve on one Arc (the engine is a patched llama.cpp).**
+
+```mermaid
+flowchart LR
+  HF["HF checkpoint\nmodel.safetensors (fp32)"] --> GO["tools/hf2gguf\n(pure Go)"]
+  GO --> GGUF["GGUF (f16)\nbyte-identical to reference"]
+  GGUF --> LC["llama.cpp Vulkan\nupstream + open1b.patch"]
+  LC --> GPU["Intel Arc B580"]
+  GPU --> API["llama-server HTTP"]
+```

 ## Repository layout

@@ -31,6 +60,8 @@ docs/
  audit-open1b-init.md      # init-unit audit report (raw evidence)
  open1b-local-inference.md # local bring-up report
  benchmarks.md             # speed benchmarks + reference comparison
+  inference-stack.md        # how llama.cpp was built/launched (NOT stock)
+  llama-cpp-open1b.patch    # local Open1B patch applied to llama.cpp
 tools/hf2gguf/              # pure-Go safetensors → GGUF converter (dump/convert/verify)
 ```

@@ -54,8 +85,9 @@ the vendor harness on a plain CPU regenerates the `1b_repop_v2` initial state ha
 `tools/hf2gguf` reads the HF checkpoint (`config.json`, `model.safetensors`,
 `tokenizer.json`) and writes a GGUF file, using
 [`github.com/cymertek/go-gguf`](https://github.com/cymertek/go-gguf) for GGUF
-read/write. The reference GGUF produced by Gensyn's converter is used **read-only
-as the schema oracle**, so the output is validated byte-for-byte.
+read/write. A reference GGUF produced by llama.cpp's `convert_hf_to_gguf.py`
+**with our [Open1B patch](docs/llama-cpp-open1b.patch)** is used **read-only as
+the schema oracle**, so the output is validated byte-for-byte.

 ```bash
 cd tools/hf2gguf
@@ -76,6 +108,10 @@ two files share SHA-256 `a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249

 ## Running the model locally

+> **Engine caveat:** open-1b needs an `open1b` architecture that upstream llama.cpp
+> does not have. We run a build with a local patch — see
+> [`docs/inference-stack.md`](docs/inference-stack.md).
+
 Serve the (or our Go-built) GGUF on one Arc B580 with llama.cpp's Vulkan backend:

 ```bash
@@ -0,0 +1,91 @@
+# Inference stack — how llama.cpp was built and launched
+
+This documents the exact engine used to run **open-1b** locally, and answers the
+fair question: *is the llama.cpp build stock?* **No.**
+
+## Engine provenance — not stock
+
+| | |
+|---|---|
+| **Upstream base** | `ggerganov/llama.cpp` @ `c32d1dabe` — "tests : increase tolerance for Add fusion tests (#28691)", 2026-09-10 |
+| **Local changes** | uncommitted patches adding the **`open1b`** architecture (9 files, +299 lines) |
+| **Exported patch** | [`llama-cpp-open1b.patch`](llama-cpp-open1b.patch) |
+| **Build** | `-DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release` |
+| **Binary** | `build-vulkan/bin/llama-server` — `0.4.0-dev (build 10893, commit c32d1dabe)` |
+
+**Why the patch is required.** Upstream llama.cpp has no `open1b` architecture. open-1b
+is Llama-3-derived but differs in four ways that the loader/graph must know about:
+
+- gain-free per-head RMSNorm on Q and K before RoPE (`qk_norm`, no learnable gain);
+- RMSNorm on the **token-embedding output** (`embedding_norm`);
+- **hybrid sliding-window attention** (512-token window, full causal on the last layer of each group of 5 and on the final layer);
+- **untied** input embedding / LM head, and interleaved-pair RoPE (`LLAMA_ROPE_TYPE_NORM`).
+
+## Patch inventory
+
+| File | Change |
+|---|---|
+| `src/models/open1b.cpp` | **new** — `llama_model_open1b`: hparams, tensor mapping, graph |
+| `src/models/models.h` | declare `llama_model_open1b` |
+| `src/llama-arch.h` / `.cpp` | add `LLM_ARCH_OPEN1B` (enum + name) |
+| `src/llama-model.cpp` | dispatch `LLM_ARCH_OPEN1B`; RoPE type `NORM` |
+| `gguf-py/gguf/constants.py` | `MODEL_ARCH.OPEN1B`, arch name, tensor list |
+| `gguf-py/gguf/tensor_mapping.py` | HF → GGUF tensor name map |
+| `conversion/__init__.py` | register the converter |
+| `conversion/open1b.py` | **new** — `Open1BModel` HF→GGUF converter |
+
+## Build
+
+```bash
+git clone https://github.com/ggerganov/llama.cpp
+cd llama.cpp
+git checkout c32d1dabe
+git apply /path/to/llama-cpp-open1b.patch
+
+cmake -B build-vulkan -DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
+cmake --build build-vulkan --config Release -j
+```
+
+## Launch
+
+```bash
+setsid --fork ./build-vulkan/bin/llama-server \
+  -m models/open-1b-sft-f16-go.gguf \
+  --device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \
+  --spec-type ngram-map-k,ngram-cache \
+  -t 16 --parallel 1 --alias open-1b \
+  --host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null &
+```
+
+| Flag | Why |
+|---|---|
+| `--device Vulkan2` | a single Intel Arc B580 (Vulkan1/2/3 = the three Arc; Vulkan0 = AMD iGPU) |
+| `-ngl 99` | offload all layers |
+| `-fa off` | Flash-Attention crashes on these Arc/Vulkan builds; KV thus stays f16 |
+| `-c 32768` | requested, but capped to the model's **4096**-token context |
+| `--spec-type ngram-map-k,ngram-cache` | ~×2.4 decode on small dense models |
+| `--jinja` | use the model's chat template |
+
+## Pipeline
+
+```mermaid
+flowchart LR
+  HF["HF checkpoint\nmodel.safetensors (fp32)"] --> GO["tools/hf2gguf\n(pure Go)"]
+  GO --> GGUF["open-1b-sft-f16-go.gguf\n(f16, byte-identical to reference)"]
+  GGUF --> LC["llama.cpp Vulkan\nupstream c32d1dabe\n+ open1b.patch"]
+  LC --> GPU["Intel Arc B580\n(Vulkan2)"]
+  GPU --> API["llama-server\nHTTP /completion"]
+```
+
+## Caveats
+
+- **The patch is ours and uncommitted.** It lives only in the working tree of the
+  build source; `llama-cpp-open1b.patch` is its export. For reproducibility it should
+  be committed and, ideally, proposed upstream.
+- **The "reference" GGUF is not an independent vendor artifact.** It was produced by
+  llama.cpp's `convert_hf_to_gguf.py` **with this same Open1B patch**. Our Go converter
+  reproduces it byte-for-byte — a genuine independent reimplementation of the GGUF
+  serialization — but "byte-identical to the reference" means *identical to our patched
+  Python path*, not to a Gensyn-published file.
+- Patch quality: written for bring-up and validated by a working inference run; not
+  reviewed upstream.
@@ -0,0 +1,437 @@
+diff --git a/conversion/__init__.py b/conversion/__init__.py
+index 4d58bcd10..b869b4dc8 100644
+--- a/conversion/__init__.py
+++ b/conversion/__init__.py
+@@ -201,6 +201,7 @@ TEXT_MODEL_MAP: dict[str, str] = {
+     "OlmoeForCausalLM": "olmo",
+     "MuseGlimmerAssistantModel": "muse_glimmer",
+     "MuseGlimmerForConditionalGeneration": "muse_glimmer",
+    "Open1BForCausalLM": "open1b",
+     "OpenELMForCausalLM": "openelm",
+     "OrionForCausalLM": "orion",
+     "PLMForCausalLM": "plm",
+diff --git a/conversion/open1b.py b/conversion/open1b.py
+new file mode 100644
+index 000000000..7ace1adc7
+--- /dev/null
+++ b/conversion/open1b.py
+@@ -0,0 +1,52 @@
+from __future__ import annotations
+
+from typing import Iterable, TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from torch import Tensor
+
+from .base import ModelBase, TextModel, gguf, logger
+
+
+@ModelBase.register("Open1BForCausalLM")
+@ModelBase.example("Gensyn/open-1b-sft")
+class Open1BModel(TextModel):
+    model_arch = gguf.MODEL_ARCH.OPEN1B
+
+    def get_vocab_base_pre(self, tokenizer) -> str:
+        # open-1b's tokenizer.json declares a `<Split>` step whose pattern is
+        # serialized with the "String" variant (a literal `\d`, not a regex),
+        # so it is a no-op and the effective pre-tokenizer is plain byte-level
+        # BPE with the GPT-2 regex. That is exactly llama.cpp's "gpt-2" pre.
+        return "gpt-2"
+
+    def set_gguf_parameters(self):
+        super().set_gguf_parameters()
+
+        sliding_window = self.hparams["sliding_window"]
+        self.gguf_writer.add_sliding_window(sliding_window)
+        logger.info(f"gguf: sliding window = {sliding_window}")
+
+        # open-1b is full-causal on the last layer of every `swa_full_every`
+        # group and on the final layer; the runtime applies that exact pattern.
+        swa_full_every = self.hparams["swa_full_every"]
+        self.gguf_writer.add_sliding_window_pattern(swa_full_every)
+        logger.info(f"gguf: sliding window pattern period = {swa_full_every}")
+
+    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
+        # LSQ per-channel weight scales are not materialised: the released
+        # master weights are used directly, equivalent to quantized_forward=false.
+        if name.endswith(".weight_scale"):
+            return
+
+        # the checkpoint stores a fused gate+up projection; split it for ggml
+        if name.endswith("ffn.w_gate_up.weight"):
+            assert bid is not None
+            n_ff = data_torch.shape[0] // 2
+            gate = data_torch[:n_ff].contiguous()
+            up   = data_torch[n_ff:].contiguous()
+            yield (self.format_tensor_name(gguf.MODEL_TENSOR.FFN_GATE, bid), gate)
+            yield (self.format_tensor_name(gguf.MODEL_TENSOR.FFN_UP,   bid), up)
+            return
+
+        yield from super().modify_tensors(data_torch, name, bid)
+diff --git a/gguf-py/gguf/constants.py b/gguf-py/gguf/constants.py
+index d3a639f37..146d0cebe 100644
+--- a/gguf-py/gguf/constants.py
+++ b/gguf-py/gguf/constants.py
+@@ -630,6 +630,7 @@ class MODEL_ARCH(IntEnum):
+     NANBEIGE         = auto()
+     QWEN3TTS         = auto()
+     POCKETTTS        = auto()
+    OPEN1B           = auto()
+ 
+ 
+ class VISION_PROJECTOR_TYPE(IntEnum):
+@@ -1385,6 +1386,7 @@ MODEL_ARCH_NAMES: dict[MODEL_ARCH, str] = {
+     MODEL_ARCH.NANBEIGE:         "nanbeige",
+     MODEL_ARCH.QWEN3TTS:         "qwen3tts",
+     MODEL_ARCH.POCKETTTS:        "pockettts",
+    MODEL_ARCH.OPEN1B:           "open1b",
+ }
+ 
+ VISION_PROJECTOR_TYPE_NAMES: dict[VISION_PROJECTOR_TYPE, str] = {
+@@ -5528,6 +5530,21 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
+         MODEL_TENSOR.FFN_DOWN,
+         MODEL_TENSOR.FFN_UP,
+     ],
+    MODEL_ARCH.OPEN1B: [
+        MODEL_TENSOR.TOKEN_EMBD,
+        MODEL_TENSOR.TOKEN_EMBD_NORM,
+        MODEL_TENSOR.OUTPUT_NORM,
+        MODEL_TENSOR.OUTPUT,
+        MODEL_TENSOR.ATTN_NORM,
+        MODEL_TENSOR.ATTN_Q,
+        MODEL_TENSOR.ATTN_K,
+        MODEL_TENSOR.ATTN_V,
+        MODEL_TENSOR.ATTN_OUT,
+        MODEL_TENSOR.FFN_NORM,
+        MODEL_TENSOR.FFN_GATE,
+        MODEL_TENSOR.FFN_UP,
+        MODEL_TENSOR.FFN_DOWN,
+    ],
+ }
+ 
+ # tensors that will not be serialized
+diff --git a/gguf-py/gguf/tensor_mapping.py b/gguf-py/gguf/tensor_mapping.py
+index d2dfeece5..ac5ba7c75 100644
+--- a/gguf-py/gguf/tensor_mapping.py
+++ b/gguf-py/gguf/tensor_mapping.py
+@@ -35,6 +35,7 @@ class TensorNameMap:
+             "model.transformer.wte",                     # llada
+             "embed_tokens",                              # qwen3-embedding
+             "model.embed",                               # talkie
+            "embedding.tok_embeddings",                  # open1b
+         ),
+ 
+         # Masked embeddings
+@@ -63,6 +64,7 @@ class TensorNameMap:
+             "model.layers.0.pre_norm",    # rwkv7
+             "backbone.norm",              # wavtokenizer
+             "model.embedding_norm",       # lfm2
+            "emb_norm",                   # open1b
+         ),
+ 
+         # Position embeddings
+@@ -86,6 +88,7 @@ class TensorNameMap:
+             "model.lm_head",             # dflash
+             "model.transformer.ff_out",  # llada
+             "head.decoder",              # modern-bert
+            "embedding.output",          # open1b
+         ),
+         MODEL_TENSOR.DENSE_2_OUT: (
+             "dense_2_out",  # embeddinggemma
+@@ -116,6 +119,7 @@ class TensorNameMap:
+             "model.transformer.ln_f",                  # llada
+             "final_norm",                              # modern-bert
+             "model.norm",                              # cogvlm
+            "norm_out",                                # open1b
+         ),
+ 
+         # Rope frequencies
+@@ -2761,6 +2765,29 @@ class TensorNameMap:
+ 
+     # architecture-specific block mappings
+     arch_block_mappings_cfg: dict[MODEL_ARCH, dict[MODEL_TENSOR, tuple[str, ...]]] = {
+        MODEL_ARCH.OPEN1B: {
+            MODEL_TENSOR.ATTN_NORM: (
+                "blocks.{bid}.norm1",
+            ),
+            MODEL_TENSOR.FFN_NORM: (
+                "blocks.{bid}.norm2",
+            ),
+            MODEL_TENSOR.ATTN_Q: (
+                "blocks.{bid}.attn.wq",
+            ),
+            MODEL_TENSOR.ATTN_K: (
+                "blocks.{bid}.attn.wk",
+            ),
+            MODEL_TENSOR.ATTN_V: (
+                "blocks.{bid}.attn.wv",
+            ),
+            MODEL_TENSOR.ATTN_OUT: (
+                "blocks.{bid}.attn.wo",
+            ),
+            MODEL_TENSOR.FFN_DOWN: (
+                "blocks.{bid}.ffn.w_down",
+            ),
+        },
+         MODEL_ARCH.ARCTIC: {
+             MODEL_TENSOR.FFN_NORM: (
+                 "model.layers.{bid}.residual_layernorm",
+diff --git a/src/llama-arch.cpp b/src/llama-arch.cpp
+index b5efb7206..9951cf5ce 100644
+--- a/src/llama-arch.cpp
+++ b/src/llama-arch.cpp
+@@ -156,6 +156,7 @@ static const std::map<llm_arch, const char *> LLM_ARCH_NAMES = {
+     { LLM_ARCH_NANBEIGE,         "nanbeige"         },
+     { LLM_ARCH_QWEN3TTS,         "qwen3tts"         },
+     { LLM_ARCH_POCKETTTS,        "pockettts"        },
+    { LLM_ARCH_OPEN1B,           "open1b"           },
+     { LLM_ARCH_UNKNOWN,          "(unknown)"        },
+ };
+ 
+diff --git a/src/llama-arch.h b/src/llama-arch.h
+index f1d173a57..02a0bee15 100644
+--- a/src/llama-arch.h
+++ b/src/llama-arch.h
+@@ -161,6 +161,7 @@ enum llm_arch {
+     LLM_ARCH_QWEN3TTS,
+     LLM_ARCH_POCKETTTS,
+     LLM_ARCH_MINIMAX_01,
+    LLM_ARCH_OPEN1B,
+     LLM_ARCH_UNKNOWN,
+ };
+ 
+diff --git a/src/llama-model.cpp b/src/llama-model.cpp
+index d10b60afd..b12c43d46 100644
+--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
+@@ -108,6 +108,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params
+             return new llama_model_qwen2vl(params);
+         case LLM_ARCH_QWEN2MOE:
+             return new llama_model_qwen2moe(params);
+        case LLM_ARCH_OPEN1B:
+            return new llama_model_open1b(params);
+         case LLM_ARCH_QWEN3:
+             return new llama_model_qwen3(params);
+         case LLM_ARCH_QWEN3MOE:
+@@ -2943,6 +2945,8 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {
+         case LLM_ARCH_DOTS3NOTE:
+         case LLM_ARCH_NANBEIGE:
+         case LLM_ARCH_POCKETTTS:
+        // open-1b pairs consecutive head values (interleaved-pair RoPE)
+        case LLM_ARCH_OPEN1B:
+         // HY_V4 rotates consecutive pairs, matching the reference implementation
+         case LLM_ARCH_HY_V4:
+             return LLAMA_ROPE_TYPE_NORM;
+diff --git a/src/models/models.h b/src/models/models.h
+index 87195fddd..d2e30faab 100644
+--- a/src/models/models.h
+++ b/src/models/models.h
+@@ -583,6 +583,20 @@ struct llama_model_qwen3 : public llama_model_base {
+ };
+ 
+ 
+struct llama_model_open1b : public llama_model_base {
+    llama_model_open1b(const struct llama_model_params & params) : llama_model_base(params) {}
+    void load_arch_hparams(llama_model_loader & ml) override;
+    void load_arch_tensors(llama_model_loader & ml) override;
+
+    template <bool iswa>
+    struct graph : public llm_graph_context {
+        graph(const llama_model & model, const llm_graph_params & params);
+    };
+
+    std::unique_ptr<llm_graph_context> build_arch_graph(const llm_graph_params & params) const override;
+};
+
+
+ struct llama_model_qwen3moe : public llama_model_base {
+     llama_model_qwen3moe(const struct llama_model_params & params) : llama_model_base(params) {}
+     void load_arch_hparams(llama_model_loader & ml) override;
+diff --git a/src/models/open1b.cpp b/src/models/open1b.cpp
+new file mode 100644
+index 000000000..9d8489ffd
+--- /dev/null
+++ b/src/models/open1b.cpp
+@@ -0,0 +1,182 @@
+#include "models.h"
+
+// Gensyn open-1b: a 1.6B decoder-only LM, Llama-3-derived with these deltas:
+//   - gain-free per-head RMSNorm on Q and K before RoPE
+//   - RMSNorm on the token-embedding output
+//   - hybrid sliding-window attention (last layer of each group + final layer full)
+//   - untied input embedding and LM head
+//   - interleaved-pair (GGML_ROPE_TYPE_NORM) RoPE
+// All attention/FFN linears were trained with int8 W8A8 LSQ QAT; the released
+// master weights are used directly here (equivalent to quantized_forward=false).
+
+void llama_model_open1b::load_arch_hparams(llama_model_loader & ml) {
+    ml.get_key(LLM_KV_ATTENTION_LAYERNORM_RMS_EPS, hparams.f_norm_rms_eps);
+
+    // hybrid sliding-window attention: the last layer of each group of
+    // `sliding_window_pattern` layers is full-causal, the rest slide; the
+    // final layer is always full-causal.
+    const bool found_swa = ml.get_key(LLM_KV_ATTENTION_SLIDING_WINDOW, hparams.n_swa, false);
+    if (found_swa && hparams.n_swa > 0) {
+        hparams.swa_type = LLAMA_SWA_TYPE_STANDARD;
+
+        uint32_t swa_period = 5;
+        ml.get_key_or_arr(LLM_KV_ATTENTION_SLIDING_WINDOW_PATTERN, swa_period, false);
+        hparams.set_swa_pattern(swa_period);
+        // open-1b forces the final layer to full causal attention
+        hparams.is_swa_impl[hparams.n_layer() - 1] = 0;
+    } else {
+        hparams.swa_type = LLAMA_SWA_TYPE_NONE;
+    }
+
+    type = LLM_TYPE_1B;
+}
+
+void llama_model_open1b::load_arch_tensors(llama_model_loader &) {
+    LLAMA_LOAD_LOCALS;
+
+    tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+    tok_norm = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD_NORM, "weight", 0), {n_embd}, 0);
+
+    // output
+    output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+    output      = create_tensor(tn(LLM_TENSOR_OUTPUT,      "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+    // if output is NULL, init from the input tok embed
+    if (output == NULL) {
+        output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+    }
+
+    for (int i = 0; i < n_layer; ++i) {
+        auto & layer = layers[i];
+
+        layer.attn_norm = create_tensor(tn(LLM_TENSOR_ATTN_NORM, "weight", i), {n_embd}, 0);
+
+        create_tensor_qkv(layer, i, n_embd, n_embd_head_k * n_head, n_embd_gqa, n_embd_gqa, 0);
+        layer.wo = create_tensor(tn(LLM_TENSOR_ATTN_OUT, "weight", i), {n_embd_head_k * n_head, n_embd}, 0);
+
+        layer.ffn_norm = create_tensor(tn(LLM_TENSOR_FFN_NORM, "weight", i), {n_embd}, 0);
+        layer.ffn_gate = create_tensor(tn(LLM_TENSOR_FFN_GATE, "weight", i), {n_embd, n_ff}, 0);
+        layer.ffn_up   = create_tensor(tn(LLM_TENSOR_FFN_UP,   "weight", i), {n_embd, n_ff}, 0);
+        layer.ffn_down = create_tensor(tn(LLM_TENSOR_FFN_DOWN, "weight", i), {n_ff, n_embd}, 0);
+    }
+}
+
+std::unique_ptr<llm_graph_context> llama_model_open1b::build_arch_graph(const llm_graph_params & params) const {
+    if (hparams.swa_type != LLAMA_SWA_TYPE_NONE) {
+        return std::make_unique<graph<true>>(*this, params);
+    }
+    return std::make_unique<graph<false>>(*this, params);
+}
+
+template <bool iswa>
+llama_model_open1b::graph<iswa>::graph(const llama_model & model, const llm_graph_params & params) : llm_graph_context(params) {
+    const int64_t n_embd_head = hparams.n_embd_head_k();
+
+    GGML_ASSERT(n_embd_head == hparams.n_embd_head_v());
+    GGML_ASSERT(n_embd_head == n_rot);
+
+    ggml_tensor * cur;
+    ggml_tensor * inpL;
+
+    inpL = build_inp_embd(model.tok_embd);
+    inpL = build_norm(inpL, model.tok_norm, NULL, LLM_NORM_RMS, -1);
+    cb(inpL, "inp_norm", -1);
+
+    // inp_pos - contains the positions
+    ggml_tensor * inp_pos = build_inp_pos();
+
+    using inp_attn_type = std::conditional_t<iswa, llm_graph_input_attn_kv_iswa, llm_graph_input_attn_kv>;
+    inp_attn_type * inp_attn = nullptr;
+    if constexpr (iswa) {
+        inp_attn = build_attn_inp_kv_iswa();
+    } else {
+        inp_attn = build_attn_inp_kv();
+    }
+
+    ggml_tensor * inp_out_ids = build_inp_out_ids();
+
+    for (int il = 0; il < n_layer; ++il) {
+        float freq_base_l  = freq_base;
+        float freq_scale_l = freq_scale;
+
+        if constexpr (iswa) {
+            freq_base_l  = model.get_rope_freq_base (cparams, il);
+            freq_scale_l = model.get_rope_freq_scale(cparams, il);
+        }
+
+        res->t_layer_inp[il] = inpL;
+
+        ggml_tensor * inpSA = inpL;
+
+        // norm
+        cur = build_norm(inpL, model.layers[il].attn_norm, NULL, LLM_NORM_RMS, il);
+        cb(cur, "attn_norm", il);
+
+        // self-attention
+        {
+            auto [Qcur, Kcur, Vcur] = build_qkv(model.layers[il], cur,
+                    n_embd_head, n_head, n_head_kv, il);
+
+            // gain-free per-head RMSNorm on Q and K before RoPE
+            Qcur = build_norm(Qcur, NULL, NULL, LLM_NORM_RMS, il);
+            Qcur = ggml_rope_ext(
+                    ctx0, Qcur, inp_pos, nullptr,
+                    n_rot, rope_type, n_ctx_orig, freq_base_l, freq_scale_l,
+                    ext_factor, attn_factor, beta_fast, beta_slow);
+            cb(Qcur, "Qcur", il);
+
+            Kcur = build_norm(Kcur, NULL, NULL, LLM_NORM_RMS, il);
+            Kcur = ggml_rope_ext(
+                    ctx0, Kcur, inp_pos, nullptr,
+                    n_rot, rope_type, n_ctx_orig, freq_base_l, freq_scale_l,
+                    ext_factor, attn_factor, beta_fast, beta_slow);
+            cb(Kcur, "Kcur", il);
+            cb(Vcur, "Vcur", il);
+
+            // default scale 1/sqrt(head_dim), as in the reference SDPA
+            cur = build_attn(inp_attn,
+                    model.layers[il].wo, NULL, model.layers[il].wo_s,
+                    Qcur, Kcur, Vcur, nullptr, nullptr, nullptr, 1.0f/sqrtf(float(n_embd_head)), il);
+        }
+        if (il == n_layer - 1 && inp_out_ids) {
+            cur   = ggml_get_rows(ctx0,   cur, inp_out_ids);
+            inpSA = ggml_get_rows(ctx0, inpSA, inp_out_ids);
+        }
+        ggml_tensor * ffn_inp = ggml_add(ctx0, cur, inpSA);
+        cb(ffn_inp, "ffn_inp", il);
+
+        // feed-forward network
+        cur = build_norm(ffn_inp, model.layers[il].ffn_norm, NULL, LLM_NORM_RMS, il);
+        cb(cur, "ffn_norm", il);
+
+        cur = build_ffn(cur,
+                model.layers[il].ffn_up,   NULL, NULL,
+                model.layers[il].ffn_gate, NULL, NULL,
+                model.layers[il].ffn_down, NULL, NULL,
+                NULL,
+                LLM_FFN_SILU, LLM_FFN_PAR, il);
+        cb(cur, "ffn_out", il);
+
+        cur = ggml_add(ctx0, cur, ffn_inp);
+
+        cur = build_cvec(cur, il);
+        cb(cur, "l_out", il);
+
+        // input for next layer
+        inpL = cur;
+    }
+    cur = inpL;
+
+    cur = build_norm(cur, model.output_norm, NULL, LLM_NORM_RMS, -1);
+    cb(cur, "result_norm", -1);
+    res->t_embd = cur;
+
+    // lm_head
+    cur = build_lora_mm(model.output, cur, model.output_s);
+    cb(cur, "result_output", -1);
+    res->t_logits = cur;
+
+    ggml_build_forward_expand(gf, cur);
+}
+
+template struct llama_model_open1b::graph<false>;
+template struct llama_model_open1b::graph<true>;
@@ -25,7 +25,7 @@ Reference SHA-256 (and our output — identical):
 | Item | Path | Notes |
 |------|------|-------|
 | HF checkpoint (fp32) | `models/open-1b-sft/model.safetensors` | 6,432,081,672 B; dtype **F32**; header 34560 B |
-| Reference GGUF (f16) | `models/open-1b-sft-f16.gguf` | 3,220,153,856 B; vendor converter; **read-only oracle** |
+| Reference GGUF (f16) | `models/open-1b-sft-f16.gguf` | 3,220,153,856 B; produced by llama.cpp `convert_hf_to_gguf.py` + our local Open1B patch; **read-only oracle** |
 | HF config | `models/open-1b-sft/config.json` | `model_type=open1b`, 24 layers, hidden 2048, heads 16/4, head_dim 128, ffn 5632, vocab 128256, rope_theta 5e5, rms_eps 1e-5, swa 512 every 5, qk_norm gain-free, embedding_norm, untied |
 | Vendor Python | `models/open-1b-sft/modeling_open1b.py` | read only, never executed |
 | Engine | `/root/projects/inference/build-vulkan/bin/llama-server` | symlink → `/mnt/2TB/projects/build-vulkan`; shared prod tree, untouched |
@@ -298,7 +298,7 @@ Both the test server (:8085) and a diagnostic server (:8086) were stopped;
   | `\n\n\n` | `\n1. **Earth**: …` (coherent) |
   | ` ` | `\nSure, here are two names…` (coherent) |

-   Because our GGUF is byte-identical to the vendor reference, this is a
+   Because our GGUF is byte-identical to the reference GGUF, this is a
   property of the released checkpoint/GGUF (likely SFT formatting), not of our
   conversion. Workarounds for consumers: use `ignore_eos` + trim, seed the
   assistant turn, or fix the template's trailing newline.
Author	SHA1	Message	Date
vad	4fc62ef8d1	Merge pull request 'docs: llama.cpp provenance (patched, not stock) + mermaid diagrams' (#8 ) from docs/inference-stack into main	2026-09-29 08:37:05 +01:00
vad	217bd951fe	docs: document llama.cpp provenance (patched, not stock) + mermaid diagrams (#4 )	2026-09-29 07:36:55 +00:00