eSlider/open-1b-verify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rewrite in plain style (no em dashes, no emoji), tighten reports

Browse Source
This commit is contained in:
vad
2026-09-29 07:39:12 +00:00
parent 4fc62ef8d1
commit 6d1f79359a
11 changed files with 392 additions and 654 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+76 -88
View File
@@ -1,54 +1,49 @@
# open-1b — independent verification & local inference # open-1b: an independent check, and a local run
An independent, reproducible check of Gensyn's **open-1b** "auditable training" We looked at Gensyn's open-1b and its claim of auditable training, then ran the
claim — plus a **pure-Go** path to convert and run the model locally. model on our own hardware. The audit uses Gensyn's official harness. Everything
we wrote is Go.
> Reference: [Introducing open-1b](https://www.gensyn.ai/news/introducing-open-1b-auditable-training) Links: [announcement](https://www.gensyn.ai/news/introducing-open-1b-auditable-training),
> · harness: [gensyn-ai/open-transformers](https://github.com/gensyn-ai/open-transformers) [harness](https://github.com/gensyn-ai/open-transformers),
> · audit app: [open1b.gensyn.ai](https://open1b.gensyn.ai/) [audit app](https://open1b.gensyn.ai/).
Everything we wrote is **Go** (no Python in our code or pipeline). For the audit we
used Gensyn's official harness, which is Python-only and the sole vendor-supplied
path; for inference we wrote our own Go GGUF converter.
## Results ## Results
| Item | Result | | Check | Result |
|------|--------| | --- | --- |
| **Init-unit audit** (Linux x86-64 CPU, no NVIDIA) | ✅ **MATCH** — published `state_hash 16554a11…` reproduced bit-for-bit | | Init-unit audit, x86-64 CPU, no NVIDIA | Match. The published state hash `16554a11...` is reproduced exactly. |
| **Training-interval audit** | ⛔ blocked — no interval unit is published; replay needs ~24 GB RAM / ≥60 GB disk | | Training-interval audit | Not run. No interval unit is published, and a replay needs about 24 GB RAM and 60 GB disk. |
| **GGUF conversion** (Go, `tools/hf2gguf`) | ✅ **byte-identical** to the vendor GGUF (same SHA-256) | | GGUF conversion (Go) | Byte-identical to the reference conversion, same SHA-256. |
| **Local inference** (1× Intel Arc B580, Vulkan) | ✅ ~114 tok/s decode · ~892 tok/s prefill · ~3.6 GiB VRAM | | Local inference, one Intel Arc B580 | 114 tok/s decode, 892 tok/s prefill, about 3.6 GiB VRAM. |
Details: [`docs/audit-open1b-init.md`](docs/audit-open1b-init.md), Reports: [audit](docs/audit-open1b-init.md), [local run](docs/open1b-local-inference.md),
[`docs/open1b-local-inference.md`](docs/open1b-local-inference.md), [benchmarks](docs/benchmarks.md), [inference stack](docs/inference-stack.md).
[`docs/benchmarks.md`](docs/benchmarks.md),
[`docs/inference-stack.md`](docs/inference-stack.md).
## How it works ## How it works
**Audit — replay one published commitment and compare.** Audit. Replay one published commitment and compare hashes.
```mermaid ```mermaid
flowchart LR flowchart LR
KIT["Gensyn audit kit"] --> SHA{"sha256\nvs kit.json"} KIT["Gensyn audit kit"] --> SHA{"sha256<br/>against kit.json"}
SHA --> WHL["repop + pretrain wheels"] SHA --> WHL["repop and pretrain wheels"]
WHL --> REPLAY["pretrain-audit-replay\n--from-init --device cpu"] WHL --> REPLAY["pretrain-audit-replay<br/>--from-init --device cpu"]
REPLAY --> H1["regenerated\ninit state hash"] REPLAY --> H1["regenerated init hash"]
KIT --> H2["published\ncommitment"] KIT --> H2["published commitment"]
H1 --> CMP{"equal?"} H1 --> CMP{"equal"}
H2 --> CMP H2 --> CMP
CMP -->|yes| OK["MATCH"] CMP -->|yes| OK["match"]
CMP -->|no| BAD["MISMATCH"] CMP -->|no| BAD["mismatch"]
``` ```
**Inference — convert in Go, serve on one Arc (the engine is a patched llama.cpp).** Inference. Convert in Go, serve on one Arc. The engine is a patched llama.cpp.
```mermaid ```mermaid
flowchart LR flowchart LR
HF["HF checkpoint\nmodel.safetensors (fp32)"] --> GO["tools/hf2gguf\n(pure Go)"] HF["HF checkpoint<br/>model.safetensors, fp32"] --> GO["tools/hf2gguf<br/>Go"]
GO --> GGUF["GGUF (f16)\nbyte-identical to reference"] GO --> GGUF["GGUF, f16<br/>identical to reference"]
GGUF --> LC["llama.cpp Vulkan\nupstream + open1b.patch"] GGUF --> LC["llama.cpp Vulkan<br/>upstream plus open1b patch"]
LC --> GPU["Intel Arc B580"] LC --> GPU["Intel Arc B580"]
GPU --> API["llama-server HTTP"] GPU --> API["llama-server HTTP"]
``` ```
@@ -57,43 +52,44 @@ flowchart LR
``` ```
docs/ docs/
audit-open1b-init.md # init-unit audit report (raw evidence) audit-open1b-init.md
open1b-local-inference.md # local bring-up report open1b-local-inference.md
benchmarks.md # speed benchmarks + reference comparison benchmarks.md
inference-stack.md # how llama.cpp was built/launched (NOT stock) inference-stack.md
llama-cpp-open1b.patch # local Open1B patch applied to llama.cpp llama-cpp-open1b.patch
tools/hf2gguf/ # pure-Go safetensors → GGUF converter (dump/convert/verify) tools/hf2gguf/
``` ```
## Audit — what it proves, and what it does not ## The audit, and its limits
**Proves.** The installed artifacts are the ones the kit's `kit.json` published The init unit regenerates the published initial state hash exactly on a plain
(all SHA-256s match); the `repop` build commit matches the manifest; and running CPU. The installed wheels match the kit manifest, and the repop build commit
the vendor harness on a plain CPU regenerates the `1b_repop_v2` initial state hash matches the manifest.
**bit-for-bit** — an independent third party reproduced the run's init commitment.
**Does not prove.** It does not cover a training step. An init-only unit says nothing about the
forward pass, the backward pass, the optimizer, or the data stream, so the
per-step claim stays untested here. Three more reasons to be careful:
- **No training interval.** An init-only unit says nothing about any forward/backward/AdamW/data-stream step. The claim about bitwise-reproducible *optimizer steps* is not exercised here. - repop ships as compiled wheels only, under a custom licence. The kernels that
- **`repop` is closed source.** Only compiled wheels ship, under the "Gensyn Reproducibility License v1.0". The kernels that make training reproducible cannot be read or independently re-derived. make training reproducible cannot be read or checked.
- **The v3 hash omits** RNG, the data-stream cursor, spike state, and `meta.json` descriptor keys. - The v3 state hash omits the RNG, the data-stream cursor, spike state, and the
- **No interval unit is published**, so "pick any step, replay it" is not self-serve checkable yet. meta.json descriptor keys.
- Provenance of the *published run* is not established — only that the pinned artifacts reproduce the published commitment on our machine. - No interval unit is published, so "replay any step" is not yet something an
outsider can do without permission.
## GGUF conversion (pure Go) A match shows that the pinned artifacts reproduce the published commitment on
our machine. It does not show how the original run was carried out.
`tools/hf2gguf` reads the HF checkpoint (`config.json`, `model.safetensors`, ## Conversion in Go
`tokenizer.json`) and writes a GGUF file, using
[`github.com/cymertek/go-gguf`](https://github.com/cymertek/go-gguf) for GGUF `tools/hf2gguf` reads the Hugging Face checkpoint (config.json,
read/write. A reference GGUF produced by llama.cpp's `convert_hf_to_gguf.py` model.safetensors, tokenizer.json) and writes a GGUF file, using
**with our [Open1B patch](docs/llama-cpp-open1b.patch)** is used **read-only as [go-gguf](https://github.com/cymertek/go-gguf). A reference GGUF from llama.cpp's
the schema oracle**, so the output is validated byte-for-byte. converter, built with our open1b patch, is read only, as the schema to match.
```bash ```bash
cd tools/hf2gguf cd tools/hf2gguf
go build -o hf2gguf . go build -o hf2gguf .
./hf2gguf dump -gguf models/open-1b-sft-f16.gguf -o manifest.json
./hf2gguf convert -ref models/open-1b-sft-f16.gguf \ ./hf2gguf convert -ref models/open-1b-sft-f16.gguf \
-safetensors models/open-1b-sft/model.safetensors \ -safetensors models/open-1b-sft/model.safetensors \
-config models/open-1b-sft/config.json \ -config models/open-1b-sft/config.json \
@@ -103,48 +99,40 @@ go build -o hf2gguf .
./hf2gguf verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf ./hf2gguf verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
``` ```
`verify` reports `39/39 metadata keys` and `220/220 tensors` byte-identical; the `verify` reports 39 of 39 metadata keys and 220 of 220 tensors identical. The two
two files share SHA-256 `a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`. files share SHA-256
`a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`.
## Running the model locally ## Running the model
> **Engine caveat:** open-1b needs an `open1b` architecture that upstream llama.cpp Upstream llama.cpp has no open1b architecture, so the model needs a patched
> does not have. We run a build with a local patch — see build. Details in the [inference stack](docs/inference-stack.md).
> [`docs/inference-stack.md`](docs/inference-stack.md).
Serve the (or our Go-built) GGUF on one Arc B580 with llama.cpp's Vulkan backend:
```bash ```bash
llama-server \ llama-server -m models/open-1b-sft-f16-go.gguf \
-m models/open-1b-sft-f16-go.gguf \
--device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \ --device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \
--spec-type ngram-map-k,ngram-cache \ --spec-type ngram-map-k,ngram-cache \
-t 16 --parallel 1 --alias open-1b --host 127.0.0.1 --port 8085 -t 16 --parallel 1 --alias open-1b --host 127.0.0.1 --port 8085
``` ```
Notes: `-fa off` is required on these Arc/Vulkan builds (so KV stays f16); the `-fa off` is required on these Arc/Vulkan builds, which keeps the KV cache in
requested `-c 32768` is capped to the model's **4096**-token context. f16. The requested 32k context is capped to the model's 4096.
## Caveats ## Caveats
- **Chat-template quirk.** With the exact vendor tail `…assistant<|end_header|>\n\n`, - Chat template. With the vendor tail `...assistant<|end_header|>` followed by a
the released SFT checkpoint emits `<|eot|>` as its first token (empty reply). blank line, the released SFT checkpoint emits an end-of-turn token first and
Our GGUF is byte-identical to the vendor one, so this is a property of the returns an empty reply. Our GGUF is identical to the reference, so this is a
released checkpoint, not of the conversion. Workarounds: seed the assistant property of the released weights. Seed the assistant turn, set ignore_eos, or
turn, use `ignore_eos`, or fix the trailing newline. adjust the trailing newline.
- **Not a strong model.** open-1b scores 25.4 on the OLMo2 suite (vs 31.9 for - Model quality. open-1b scores 25.4 on the OLMo2 suite, against 31.9 for
OLMo2-1B). Its value is **verifiability**, not quality — see OLMo2-1B. It is a test of verifiability, not a strong model.
[`docs/benchmarks.md`](docs/benchmarks.md). - Scope. We run and inspect released weights and replay one published
- **Scope.** We only run/inspect the released weights and replay one published commitment. We do not train or fine-tune anything.
commitment; we do not train or fine-tune anything.
## License ## License
MIT — see [`LICENSE`](LICENSE). The vendor harness (`open-transformers`) is MIT, see [LICENSE](LICENSE). The harness is Apache-2.0. repop, the checkpoints,
Apache-2.0; `repop`, the checkpoints and the corpus are distributed separately by and the corpus are distributed by Gensyn under their own terms.
Gensyn under their own terms.
## Credits Built on Gensyn's release. Not affiliated with Gensyn.
Built on Gensyn's open-1b release and audit tooling. Independent work; not
affiliated with Gensyn.
docs/audit-open1b-init.md
+83 -165
View File
@@ -1,214 +1,132 @@
# Gensyn OPEN-1B audit — independent replay (CPU, Linux x86-64) # Audit: open-1b init unit on CPU
- **Date:** 2026-09-28 - Date: 2026-09-28
- **Host:** `brainstation`, Linux x86-64, kernel `7.3.0-070300rc1-generic` (Ubuntu), **no NVIDIA GPU** - Host: brainstation, Linux x86-64, kernel 7.3.0, no NVIDIA GPU
- **Auditor:** SE sub-agent (independent) - Harness: `gensyn-ai/open-transformers` at `d7b7b674`
- **Workdir:** `/root/projects/gensyn/audit` - Workdir: `audit/`
- **Harness:** `gensyn-ai/open-transformers` @ `d7b7b674cd851f7a16815229f060658180cb772b` - Verdict: match on the `1b_repop_v2` init unit. No training interval was replayed.
("Initial commit: the OPEN-1B pretraining and audit-replay harness", 2026-09-15), cloned via git.
- **Verdict:** ✅ **MATCH** on the `1b_repop_v2` init unit. ❌ No training-interval
replay run (not defined by this kit and not feasible on this host — see §7).
--- ## Kit
## 1. Kit selection The audit app publishes the mapping at `https://open1b.gensyn.ai/manifest.json`:
Four kits were listed. All four `trajectory.json` files declare the **same** ```
`1b_repop_v2` init commitment run: 1b_repop_v2
`16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed`, so the choice initStateHash 16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed
of kit does not change the init hash under test. The authoritative mapping is audit: kit pt-f91110a0d0c8_rp-c9ca6e71e673
published by the audit app itself at `https://open1b.gensyn.ai/manifest.json`:
```json
"run": { "id": "20260722-213626-ad3276b", "model": "1b_repop_v2",
"initStateHash": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed" },
"audit": { "kit": { "id": "pt-f91110a0d0c8_rp-c9ca6e71e673",
"urlBase": "https://storage.googleapis.com/gensyn-audit-public/audit-kit/pt-f91110a0d0c8_rp-c9ca6e71e673" } }
``` ```
**Kit under test: `pt-f91110a0d0c8_rp-c9ca6e71e673`.** All four published kits declare the same init commitment, so the choice of kit
does not change the hash under test.
- `pretrain_commit`: `f91110a0d0c84fa3bad808f365567d9c655d10b2` - pretrain commit: `f91110a0d0c84fa3bad808f365567d9c655d10b2`
- `repop_commit`: `c9ca6e71e673660afc6c87de2592fa6f62c87b15` - repop commit: `c9ca6e71e673660afc6c87de2592fa6f62c87b15`
- `torch_requirement`: `torch==2.10.0` - torch: `2.10.0`
(The other three kits — `pt-cb4ac29438bc_rp-…`, `pt-f6bc086dfebd_rp-…`, ## Toolchain
`pt-f5eece09abe6_rp-…` — carry the same init commitment but are older
pretrain/repop pins. Two share the `c9ca6e71e673` repop but a different pretrain
commit; the app manifest picks `f91110a0d0c8`.)
## 2. Toolchain | Component | Version |
| Component | Version / identity |
| --- | --- | | --- | --- |
| Python | 3.11.16 (managed by `uv`; system Python 3.14 untouched) | | Python | 3.11.16, via uv. System Python untouched. |
| `uv` | 0.12.17 | | uv | 0.12.17 |
| PyTorch | `2.10.0+cpu` (`torch.version.cuda == None`) | | PyTorch | 2.10.0+cpu |
| `repop` | 0.1.5, `build_info().commit == c9ca6e71e673660afc6c87de2592fa6f62c87b15` ✅ (== kit.json `repop_commit`); backends `["cpu","cuda"]` | | repop | 0.1.5, build commit `c9ca6e71e673`, matches kit.json |
| `pretrain` | 0.1.0 (kit wheel) | | pretrain | 0.1.0, kit wheel |
| Go (glue tools) | go1.27.0 (fabric rule: all glue/parsing in Go) | | Go | 1.27, for all glue and checks |
**Deviation, stated plainly:** the RUNBOOK targets Linux hosts with an NVIDIA GPU The runbook targets Linux with an NVIDIA GPU and points at the CUDA 12.9 index.
and points them at the CUDA 12.9 PyTorch index. This host has no GPU, and the This host has no GPU. The README names CPU as the reference device, so the pinned
README names CPU as the *reference* device, so the exact pin `torch==2.10.0` was torch version was installed from the CPU index. Init is device-independent by
installed from PyTorch's official **CPU** index (`.../whl/cpu`, wheel construction, so the result is unaffected.
`2.10.0+cpu`). The pinned version is unchanged; only the CPU build variant was
chosen deliberately, not by an accidental PyPI fallback. Init is device-independent
by construction (see §6), so this does not weaken the init result.
Python (`uv`-managed) is used **only** because the vendor harness is Python and ## Commands
ships as wheels; all download/verification/listing glue is Go under `tools/`
(`kitinspect`, `trajinspect`, `kitfetch`, `gslist`).
## 3. Commands run
```bash ```bash
# workdir + docs
mkdir -p /root/projects/gensyn/audit && cd /root/projects/gensyn/audit
git clone --depth 1 https://github.com/gensyn-ai/open-transformers git clone --depth 1 https://github.com/gensyn-ai/open-transformers
# kit manifests + trajectories (Go inspector)
go run ./tools/kitinspect kits/*/ go run ./tools/kitinspect kits/*/
go run ./tools/trajinspect kits/*/trajectory.json go run ./tools/kitfetch kits/pt-f91110a0d0c8_rp-c9ca6e71e673 kit-pt-f91110a0d0c8_rp-c9ca6e71e673
# download + per-file sha256 verification against kit.json (Go tool,
# downloads to .part, verifies, removes on mismatch)
go run ./tools/kitfetch kits/pt-f91110a0d0c8_rp-c9ca6e71e673 \
kit-pt-f91110a0d0c8_rp-c9ca6e71e673
# Python 3.11 env
uv venv --python 3.11 .venv uv venv --python 3.11 .venv
uv pip install --python .venv --index-url https://download.pytorch.org/whl/cpu "torch==2.10.0" uv pip install --python .venv --index-url https://download.pytorch.org/whl/cpu "torch==2.10.0"
uv pip install --python .venv \ uv pip install --python .venv \
./kit-.../repop-0.1.5-cp311-cp311-linux_x86_64.whl \ ./kit-pt-f91110a0d0c8_rp-c9ca6e71e673/repop-0.1.5-cp311-cp311-linux_x86_64.whl \
./kit-.../pretrain-0.1.0-cp311-cp311-linux_x86_64.whl ./kit-pt-f91110a0d0c8_rp-c9ca6e71e673/pretrain-0.1.0-cp311-cp311-linux_x86_64.whl
# provenance assertions .venv/bin/pretrain-audit-replay --from-init --until-step 0 \
.venv/bin/python -c "import repop,json; print(repop.build_info())" --config-name 1b_repop_v2 --device cpu \
# THE AUDIT (init unit)
PRETRAIN_AUDIT_MEMLOG=1 /usr/bin/time -v .venv/bin/pretrain-audit-replay \
--from-init --until-step 0 \
--config-name 1b_repop_v2 \
--device cpu \
--expect-hash 16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed --expect-hash 16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed
``` ```
## 4. SHA-256 verification table (all 5 kit files) ## Checksums
Every file was hashed **before** install; the Go fetcher adopts a file only on a All five kit files were hashed before install. The Go fetcher accepts a file only
full 64-hex match and deletes rejected bytes. Expected values are read from the on a full match and deletes rejected bytes. Expected values come from kit.json.
kit's own `kit.json`.
| File | expected (`kit.json`) | computed | result | | File | SHA-256 | Result |
|---|---|---|---| | --- | --- | --- |
| `pretrain-0.1.0-cp311-cp311-linux_x86_64.whl` | `ecc49d1c05c0e2876a59d98d877795769d6e9620eb66e4b4c866b16f695fb174` | same | ✅ OK | | `pretrain-0.1.0-cp311-cp311-linux_x86_64.whl` | `ecc49d1c05c0e2876a59d98d877795769d6e9620eb66e4b4c866b16f695fb174` | match |
| `pretrain-0.1.0-cp311-cp311-macosx_14_0_arm64.whl` | `fbd23a54aa0bf87775e86493d67f759d1c1f4cc1dcd63ad991fba59dcb92491c` | same | ✅ OK | | `pretrain-0.1.0-cp311-cp311-macosx_14_0_arm64.whl` | `fbd23a54aa0bf87775e86493d67f759d1c1f4cc1dcd63ad991fba59dcb92491c` | match |
| `repop-0.1.5-cp311-cp311-linux_x86_64.whl` | `15f171643871211b2a772ee83c94833c0be2eef0495790cedc33813c95147ff0` | same | ✅ OK | | `repop-0.1.5-cp311-cp311-linux_x86_64.whl` | `15f171643871211b2a772ee83c94833c0be2eef0495790cedc33813c95147ff0` | match |
| `repop-0.1.5-cp311-cp311-macosx_14_0_arm64.whl` | `d8658e0327c7278812ecb6dd455cd4cf8723a974a78788adba41d5f747289c69` | same | ✅ OK | | `repop-0.1.5-cp311-cp311-macosx_14_0_arm64.whl` | `d8658e0327c7278812ecb6dd455cd4cf8723a974a78788adba41d5f747289c69` | match |
| `trajectory.json` | `9435515c37d8956d0c81d5bc78c0a383bb6bf27c57b5dbaf1f8be564df0b324a` | same | ✅ OK | | `trajectory.json` | `9435515c37d8956d0c81d5bc78c0a383bb6bf27c57b5dbaf1f8be564df0b324a` | match |
Byte sizes also matched `kit.json`. Only the two **linux_x86_64** wheels were Only the two linux_x86_64 wheels were installed.
installed; the macOS arm64 wheels were downloaded and verified but not installed.
## 5. Init-unit result — MATCH ## Result
Raw result object printed by the harness (stdout, exit status 0):
```json ```json
{ {
"step": 0, "step": 0,
"consumed_tokens": 0,
"state_hash": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed", "state_hash": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed",
"mode": "init", "mode": "init",
"repop": { "commit": "c9ca6e71e673660afc6c87de2592fa6f62c87b15",
"backends": ["cpu", "cuda"], "cuda_arch_list": "7.5 8.0 9.0" },
"device": "cpu", "device": "cpu",
"expected": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed", "expected": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed",
"match": true, "match": true
"cpu_threads": { "requested": "auto", "source": "runtime-default",
"torch_num_threads": 8, "omp_num_threads": null, "mkl_num_threads": null }
} }
``` ```
Log line: `from-init: regenerated init state_hash=16554a1119745f1b expected=16554a1119745f1b MATCH=True`. Log line: `from-init: regenerated init state_hash=16554a1119745f1b expected=16554a1119745f1b MATCH=True`.
Config `1b_repop_v2`, seed `42`, loss path `fused CE+z-loss`. The only warning is
the expected `not distributed + no CUDA — parallelize applies AC only (FSDP
skipped; dev mode)`, i.e. the documented CPU reference path.
**Resource notes:** wall **58.3 s**, user 78.5 s / sys 61.1 s, **peak RSS Resources: 58.3 s wall, peak RSS 16.8 GB, no swap, exit 0.
16.80 GB** (16,804,956 KB), **0 swaps**, exit 0. (README/RUNBOOK size this unit at
~21.9 GB peak on CPU; we measured 16.8 GB. Free RAM was ~11 GB and the unit still
completed without swapping, so the doc figure is conservative.)
## 6. What this proves — and what it does not ## Limits
**Proves.** The bytes installed are the bytes `kit.json` published (all five 1. No training interval. An init-only unit says nothing about the forward pass,
SHA-256s match). The `repop` kernel build is the one the kit claims the backward pass, the optimizer, or the data stream. The per-step claim is
(`build_info().commit == repop_commit`). Running the vendor harness on this not exercised.
CPU, from seed, the `1b_repop_v2` **initial state** — seeded device-independent 2. repop is closed source. Only compiled wheels ship, under the Gensyn
trunc-normal init plus zeroed optimizer moments — reproduces the published Reproducibility Licence. The kernels that make training reproducible cannot be
canonical hash `16554a11…` **bit-for-bit**. Independent third-party read or checked. The init unit does not even run the kernels changed by the
reproduction of the run's init commitment: ✅. recent repop repin.
3. Hash coverage. The v3 hash covers weights, optimizer state, gradients, and the
batch digest. It omits the RNG, the data-stream cursor, spike state, and the
meta.json descriptor keys.
4. Init is the easy case. init.py samples Philox on CPU and byte-copies, so it is
device-independent by design. The hard cross-device claim lives in the
training kernels, which this unit does not touch.
5. Provenance of the published run is not established. We only show that the
pinned artifacts reproduce the published commitment on this machine.
**Does not prove.** ## Interval replay was not attempted
1. **No training interval.** Per the README: "An initialization-only unit does This is a decision, not a stall.
not verify a training interval." A match here says nothing about any forward,
backward, AdamW or data-stream step. The claimed bitwise reproducibility of
*optimizer steps* is **not** tested by this run.
2. **`repop` is closed source.** Only compiled wheels are shipped; the kernel
arithmetic that makes training bitwise reproducible cannot be read or
independently re-derived from this kit. The init unit does not even exercise
the kernels that the recent `c9ca6e71e673` repin changed (the trajectory notes
say exactly this), so a match is *not* evidence for the kernel-level BFR
contract at that commit.
3. **v3-hash coverage.** The README states the v3 hash covers weights, optimizer
state, gradients and the batch digest, but **not** RNG, the data-stream
cursor, spike state, or the `meta.json` descriptor keys. Even a matching
interval hash would leave those unverified.
4. **Init is the easy case by design.** `init.py` samples Philox on CPU and
byte-copies, so it is device-independent; it is expected to match across
devices. The hard cross-device claim lives in the training kernels, which
this unit does not touch.
5. Provenance of the *published run* is not established; only that the pinned
artifacts reproduce the published commitment on this machine.
## 7. Step-interval replay — deliberately not attempted - Not defined by the kit. This kit's trajectory.json contains only three init
units (`100m_smoke_repop`, `1b_repop_run3`, `1b_repop_v2`). No interval unit
with a canonical target hash is published, so there is no commitment to check.
- Memory. A step replay needs about 24 GB. This host had about 11 GB free, and
the init unit already peaked at 16.8 GB.
- Disk. One checkpoint `ckpt/step_000000100/` is 19.31 GB; the full `ckpt/` tree
is 15.64 TB. With 39 GB free, a single checkpoint is tight.
Stopping after the init unit is a considered decision, not a stall: Re-attempt on a host with 32 GB or more RAM and 60 GB or more disk, ideally with
a CUDA GPU.
- **Not defined by the kit.** This kit's `trajectory.json` (`init-units-v1`) ## Files
contains **only 3 init units** — `100m_smoke_repop`, `1b_repop_run3`,
`1b_repop_v2`. Its own note says: "Step-interval units are added per run once
its checkpoints and state_hash files are published." **No interval unit with a
canonical target hash is published here**, so the kit defines no
step-interval commitment to check. (The run's per-step log exists at
`gs://gensyn-open-1b/logs/state_hashes.jsonl`, but the harness has no automatic
lookup and the audit app gates submissions.)
- **Memory.** A step-interval replay needs ~24 GB (README), and the audit's
auto-offload only reaches "every published unit" at that figure. This host had
~11 GB RAM available (29 GB total, 18 GB in use); an init unit already peaked at
16.8 GB. The heavy replay would swap-thrash and would not fit honestly.
- **Disk/size.** One checkpoint `ckpt/step_000000100/` is **19.31 GB**
(150 objects); the full `ckpt/` prefix is **15.64 TB** (810 checkpoint dirs).
With 39 GB free, a single checkpoint is tight against free space and the
documented `--from-init + --gcs-root` combination is broken (wrong shard
anchor), so a from-init interval would first require staging shards via
`--data-root`.
Re-attempting an interval audit should be done on a host with ≥32 GB free RAM - `open-transformers/`: harness clone
(NVIDIA GPU preferred: `--device cuda`) and ≥60 GB free disk. - `kits/`: manifests
- `kit-pt-f91110a0d0c8_rp-c9ca6e71e673/`: verified wheels
## 8. Files produced - `.venv/`: Python 3.11 audit environment
- `tools/`: Go glue (`kitinspect`, `trajinspect`, `kitfetch`, `gslist`)
- `open-transformers/` — harness clone (docs read: `README.md`, - `init-replay.log`: full replay output
`docs/audit-replay-usage.md`, `scripts/audit_volunteer/RUNBOOK.md`)
- `kits/<kit>/kit.json`, `kits/<kit>/trajectory.json` — manifests
- `kit-pt-f91110a0d0c8_rp-c9ca6e71e673/` — verified wheels + trajectory
- `.venv/` — Python 3.11 audit env
- `init-replay.log` — full captured replay output + `/usr/bin/time -v`
- `tools/{kitinspect,trajinspect,kitfetch,gslist}` — Go glue
- `REPORT.md` — this file
No secrets were written. Nothing outside the workdir was modified or deleted.
docs/benchmarks.md
+25 -32
View File
@@ -1,48 +1,41 @@
# Benchmarks # Benchmarks
Speed of **open-1b** on our hardware, with a reference comparison against the open-1b on our hardware, with the production model, gemma-4-E2B, as reference.
model we run in production (`gemma-4-E2B`). All numbers are single-stream Single stream, one GPU.
(one request at a time) on one GPU.
## Hardware / setup ## Setup
- **GPU:** 1× Intel Arc B580 (BMG G21, 12 GB), Vulkan (Mesa), llama.cpp `build-vulkan` (b10893). - GPU: one Intel Arc B580 (BMG G21, 12 GB), Vulkan, llama.cpp build-vulkan (b10893).
- **CPU:** AMD Ryzen 7 7700X (16 vCPU), 29 GB RAM. - CPU: AMD Ryzen 7 7700X, 29 GB RAM.
- **Serving flags:** `-ngl 99 -fa off --jinja --spec-type ngram-map-k,ngram-cache -t 16 --parallel 1`. - Flags: `-ngl 99 -fa off --jinja --spec-type ngram-map-k,ngram-cache -t 16 --parallel 1`.
- `-fa off` is mandatory on these Arc/Vulkan builds → KV cache is f16 (no `-ctk q8_0`). - `-fa off` is required on these Arc/Vulkan builds, so the KV cache stays in f16.
## open-1b (our Go-built GGUF, f16) ## open-1b, Go-built GGUF, f16
| Metric | Value | | Metric | Value |
| --- | --- | | --- | --- |
| Decode | **~112–117 tok/s** (e.g. 256 tok @ 111.7; 96 tok @ 117.2) | | Decode | 112 to 117 tok/s |
| Prefill | **~892 tok/s** (961 prompt tok in 1077 ms) | | Prefill | about 892 tok/s |
| VRAM | **~3.6 GiB** (10971 → 7258 MiB free) | | VRAM | about 3.6 GiB |
| Context | 4096 (model limit; the requested 32k is capped) | | Context | 4096, the model limit |
| Weights | f16, ~3.2 GB | | Weights | f16, about 3.2 GB |
## Reference comparison (1× Arc B580) ## Reference, one Arc B580
| Model | Weights | Context | Decode | Prefill | | Model | Weights | Context | Decode | Prefill |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| **open-1b-sft** (this repo) | f16 ~3.2 GB | 4096 | ~114 tok/s | ~892 tok/s | | open-1b-sft | f16, about 3.2 GB | 4096 | about 114 tok/s | about 892 tok/s |
| **gemma-4-E2B** (prod) | Q4 ~1.5 GiB | 64k | **~214 tok/s** | **~1659 tok/s** | | gemma-4-E2B | Q4, about 1.5 GiB | 64k | about 214 tok/s | about 1659 tok/s |
*gemma figures are from our 2026-09-20 controlled benchmark (decode 256 tok / The gemma figures come from our 2026-09-20 benchmark on the same card and build.
prefill 1698 tok) on the same card and build.*
**Reading:** gemma is roughly **2× faster**, ~**2.4× smaller**, and has **16× gemma is about twice as fast, about 2.4 times smaller, and has sixteen times the
the context** of open-1b. For any practical workload (classification, extraction, context. For classification, extraction, translation, or tool calls, gemma is the
translation, tool-calling) gemma wins outright. open-1b's reason to exist is not better choice. open-1b is interesting for its training audit, not for its speed.
throughput or quality but **auditable training**.
## Caveats ## Notes
- This is **not** a controlled A/B: the open-1b figures are a short first-run - Not a controlled comparison. The open-1b numbers are a short first run with
measurement with different prompts; the gemma figures come from a separate different prompts; the gemma numbers come from a separate controlled benchmark.
controlled benchmark. Use them as an order-of-magnitude orientation. - No speculative-decoder-off measurement for open-1b.
- The `ngram` speculative decoder helps task-dependent amounts; open-1b's - Quality: open-1b scores 25.4 on the OLMo2 suite, against 31.9 for OLMo2-1B.
baseline (no-spec) number was not measured here.
- Quality: open-1b scores **25.4** on the OLMo2 suite vs **31.9** for OLMo2-1B,
i.e. below an ordinary 1B model. It is a proof of *verifiability*, not a strong
assistant.
docs/inference-stack.md
+41 -43
View File
@@ -1,38 +1,38 @@
# Inference stack — how llama.cpp was built and launched # Inference stack: how llama.cpp was built and launched
This documents the exact engine used to run **open-1b** locally, and answers the This records the engine behind the local run. Is the llama.cpp build stock? No.
fair question: *is the llama.cpp build stock?* **No.**
## Engine provenance — not stock ## Provenance
| | | | | |
| --- | --- | | --- | --- |
| **Upstream base** | `ggerganov/llama.cpp` @ `c32d1dabe` — "tests : increase tolerance for Add fusion tests (#28691)", 2026-09-10 | | Upstream base | `ggerganov/llama.cpp` at `c32d1dabe`, 2026-09-10 |
| **Local changes** | uncommitted patches adding the **`open1b`** architecture (9 files, +299 lines) | | Local changes | uncommitted patches adding the `open1b` architecture, 9 files, +299 lines |
| **Exported patch** | [`llama-cpp-open1b.patch`](llama-cpp-open1b.patch) | | Patch | [llama-cpp-open1b.patch](llama-cpp-open1b.patch) |
| **Build** | `-DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release` | | Build | `-DGGML_VULKAN=ON -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release` |
| **Binary** | `build-vulkan/bin/llama-server` — `0.4.0-dev (build 10893, commit c32d1dabe)` | | Binary | `build-vulkan/bin/llama-server`, `0.4.0-dev`, build 10893 |
**Why the patch is required.** Upstream llama.cpp has no `open1b` architecture. open-1b Upstream has no `open1b` architecture, so the patch is required. open-1b is
is Llama-3-derived but differs in four ways that the loader/graph must know about: Llama-3-derived, with four differences the loader and graph must know about:
- gain-free per-head RMSNorm on Q and K before RoPE (`qk_norm`, no learnable gain); - gain-free per-head RMSNorm on Q and K before RoPE,
- RMSNorm on the **token-embedding output** (`embedding_norm`); - RMSNorm on the token-embedding output,
- **hybrid sliding-window attention** (512-token window, full causal on the last layer of each group of 5 and on the final layer); - hybrid sliding-window attention, 512 tokens, full causal on the last layer of
- **untied** input embedding / LM head, and interleaved-pair RoPE (`LLAMA_ROPE_TYPE_NORM`). each group of five and on the final layer,
- untied embedding and LM head, with interleaved-pair RoPE.
## Patch inventory ## Patch inventory
| File | Change | | File | Change |
| --- | --- | | --- | --- |
| `src/models/open1b.cpp` | **new** — `llama_model_open1b`: hparams, tensor mapping, graph | | `src/models/open1b.cpp` | new: `llama_model_open1b`, hparams, tensors, graph |
| `src/models/models.h` | declare `llama_model_open1b` | | `src/models/models.h` | declare the model |
| `src/llama-arch.h` / `.cpp` | add `LLM_ARCH_OPEN1B` (enum + name) | | `src/llama-arch.h`, `.cpp` | add `LLM_ARCH_OPEN1B` |
| `src/llama-model.cpp` | dispatch `LLM_ARCH_OPEN1B`; RoPE type `NORM` | | `src/llama-model.cpp` | dispatch the arch, set the RoPE type |
| `gguf-py/gguf/constants.py` | `MODEL_ARCH.OPEN1B`, arch name, tensor list | | `gguf-py/gguf/constants.py` | arch enum, name, tensor list |
| `gguf-py/gguf/tensor_mapping.py` | HF → GGUF tensor name map | | `gguf-py/gguf/tensor_mapping.py` | HF to GGUF tensor names |
| `conversion/__init__.py` | register the converter | | `conversion/__init__.py` | register the converter |
| `conversion/open1b.py` | **new** — `Open1BModel` HF→GGUF converter | | `conversion/open1b.py` | new: `Open1BModel` converter |
## Build ## Build
@@ -57,35 +57,33 @@ setsid --fork ./build-vulkan/bin/llama-server \
--host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null & --host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null &
``` ```
| Flag | Why | | Flag | Reason |
| --- | --- | | --- | --- |
| `--device Vulkan2` | a single Intel Arc B580 (Vulkan1/2/3 = the three Arc; Vulkan0 = AMD iGPU) | | `--device Vulkan2` | one Intel Arc B580. Vulkan1 to 3 are the three Arc cards; Vulkan0 is the AMD iGPU. |
| `-ngl 99` | offload all layers | | `-ngl 99` | offload all layers |
| `-fa off` | Flash-Attention crashes on these Arc/Vulkan builds; KV thus stays f16 | | `-fa off` | Flash-Attention crashes on these Arc/Vulkan builds, so the KV cache stays in f16 |
| `-c 32768` | requested, but capped to the model's **4096**-token context | | `-c 32768` | requested, capped to the model's 4096-token context |
| `--spec-type ngram-map-k,ngram-cache` | ~×2.4 decode on small dense models | | `--spec-type ngram-map-k,ngram-cache` | about 2.4x decode on small dense models |
| `--jinja` | use the model's chat template | | `--jinja` | use the model's chat template |
## Pipeline ## Pipeline
```mermaid ```mermaid
flowchart LR flowchart LR
HF["HF checkpoint\nmodel.safetensors (fp32)"] --> GO["tools/hf2gguf\n(pure Go)"] HF["HF checkpoint<br/>model.safetensors, fp32"] --> GO["tools/hf2gguf<br/>Go"]
GO --> GGUF["open-1b-sft-f16-go.gguf\n(f16, byte-identical to reference)"] GO --> GGUF["open-1b-sft-f16-go.gguf<br/>f16"]
GGUF --> LC["llama.cpp Vulkan\nupstream c32d1dabe\n+ open1b.patch"] GGUF --> LC["llama.cpp Vulkan<br/>upstream plus open1b patch"]
LC --> GPU["Intel Arc B580\n(Vulkan2)"] LC --> GPU["Intel Arc B580"]
GPU --> API["llama-server\nHTTP /completion"] GPU --> API["llama-server<br/>HTTP /completion"]
``` ```
## Caveats ## Notes
- **The patch is ours and uncommitted.** It lives only in the working tree of the - The patch is ours and uncommitted. The exported patch file makes the build
build source; `llama-cpp-open1b.patch` is its export. For reproducibility it should reproducible. Committing it, or proposing it upstream, would be better.
be committed and, ideally, proposed upstream. - The reference GGUF is not an independent artifact. It came from llama.cpp's
- **The "reference" GGUF is not an independent vendor artifact.** It was produced by `convert_hf_to_gguf.py` with this same patch. The Go converter reproduces it
llama.cpp's `convert_hf_to_gguf.py` **with this same Open1B patch**. Our Go converter byte for byte, which is a real reimplementation of the GGUF writer, but the
reproduces it byte-for-byte — a genuine independent reimplementation of the GGUF comparison is against our own patched Python output, not a file published by
serialization — but "byte-identical to the reference" means *identical to our patched Gensyn.
Python path*, not to a Gensyn-published file. - The patch was written for bring-up. It is not upstream-reviewed.
- Patch quality: written for bring-up and validated by a working inference run; not
reviewed upstream.
docs/open1b-local-inference.md
+109 -264
View File
@@ -1,332 +1,177 @@
# Gensyn open-1b (SFT) — local bring-up & proof of run # Local run: open-1b on one Arc B580
Epic: `vado/1b#1` / task #4 (SE). Workdir: `/root/projects/gensyn/inference/`. - Workdir: `inference/`
Engine: existing shared llama.cpp **Vulkan** build (`/root/projects/inference/build-vulkan/bin/llama-server`). - Engine: the shared llama.cpp Vulkan build, with the open1b patch
Language: **Go 1.27 only** (see "Process note" re one accidental `python3` call). - Language: Go 1.27
## TL;DR ## Summary
- Built a pure-Go tool `hf2gguf` (module `hf2gguf`) that **dumps** the reference GGUF - A Go tool, `hf2gguf`, with three commands: `dump`, `convert`, `verify`.
(go-gguf + a minimal raw-KV walker for arrays), **converts** `model.safetensors` - Validation: 39 of 39 metadata keys and 220 of 220 tensors are byte-identical.
→ `models/open-1b-sft-f16-go.gguf`, and **verifies** the two files. The two files share SHA-256 `a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`.
- **Validation: PASS — 39/39 metadata KV byte-identical, 220/220 tensors byte-identical.** - Served on one Arc (Vulkan2, PCI `07:00.0`): about 114 tok/s decode, about 892
Whole files are byte-for-byte identical (`cmp` + SHA-256 match). tok/s prefill, about 3.6 GiB VRAM.
- **Served** on one Arc (Vulkan2 / `07:00.0`) with the existing Vulkan build; coherent
output, **~114 tok/s decode**, **~892 tok/s prefill**, ~**3.6 GiB VRAM**.
- No commit made (all under gitignored `inference/`).
Reference SHA-256 (and our output — identical): ## Inputs
`a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`
---
## 1. Inputs & topology
| Item | Path | Notes | | Item | Path | Notes |
|------|------|-------| | --- | --- | --- |
| HF checkpoint (fp32) | `models/open-1b-sft/model.safetensors` | 6,432,081,672 B; dtype **F32**; header 34560 B | | HF checkpoint | `models/open-1b-sft/model.safetensors` | 6.43 GB, F32 |
| Reference GGUF (f16) | `models/open-1b-sft-f16.gguf` | 3,220,153,856 B; produced by llama.cpp `convert_hf_to_gguf.py` + our local Open1B patch; **read-only oracle** | | Reference GGUF | `models/open-1b-sft-f16.gguf` | 3.22 GB, f16, read-only |
| HF config | `models/open-1b-sft/config.json` | `model_type=open1b`, 24 layers, hidden 2048, heads 16/4, head_dim 128, ffn 5632, vocab 128256, rope_theta 5e5, rms_eps 1e-5, swa 512 every 5, qk_norm gain-free, embedding_norm, untied | | HF config | `models/open-1b-sft/config.json` | see below |
| Vendor Python | `models/open-1b-sft/modeling_open1b.py` | read only, never executed | | Engine | `build-vulkan/bin/llama-server` | shared prod tree, not modified |
| Engine | `/root/projects/inference/build-vulkan/bin/llama-server` | symlink → `/mnt/2TB/projects/build-vulkan`; shared prod tree, untouched |
GPU devices as reported by the engine (`--list-devices`): The config: `model_type=open1b`, 24 layers, hidden 2048, heads 16 and key/value
heads 4, head_dim 128, feed-forward 5632, vocab 128256, rope_theta 500000,
rms_eps 1e-5, hybrid sliding window 512 every 5 layers, gain-free qk_norm,
embedding_norm, untied embeddings.
Devices as the engine lists them:
``` ```
Vulkan0: AMD Ryzen 7 7700X (RADV RAPHAEL_MENDOCINO) (15835 MiB) Vulkan0: AMD Ryzen 7 7700X (RADV RAPHAEL_MENDOCINO), 15835 MiB
Vulkan1: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB) Vulkan1: Intel Arc B580 (BMG G21), 12216 MiB
Vulkan2: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB) Vulkan2: Intel Arc B580 (BMG G21), 12216 MiB
Vulkan3: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB) Vulkan3: Intel Arc B580 (BMG G21), 12216 MiB
``` ```
`Vulkan2` was confirmed to map to PCI `0000:07:00.0` (Arc #2) via Vulkan2 maps to PCI `0000:07:00.0`, confirmed through
`/sys/class/drm/renderD129/device`. `/sys/class/drm/renderD129/device`.
--- ## Tool
## 2. Tool: `hf2gguf` (Go 1.27, `github.com/cymertek/go-gguf`)
``` ```
inference/hf2gguf/ inference/hf2gguf/
go.mod, go.sum main.go dispatch, manifest types
main.go # subcommand dispatch + JSON manifest types rawkv.go read-only GGUF KV walker, array support
rawkv.go # minimal read-only GGUF KV walker (fills go-gguf's array gap) safetensors.go safetensors header and tensor bytes
safetensors.go # safetensors header + tensor byte access hfconfig.go config and tokenizer parsing, validation
hfconfig.go # config.json / tokenizer_config.json / tokenizer.json validation convert.go metadata reproduction, tensor mapping, F32 to F16
convert.go # metadata reproduction + HF→GGUF tensor mapping + F32→F16 verify.go byte-level comparison
verify.go # byte-level metadata & tensor comparison convert_test.go tests
convert_test.go
``` ```
### Why a raw KV walker `go-gguf` returns an empty value for `BTypeArray`, so `rawkv.go` walks the KV
section directly to keep the tokenizer arrays (`tokens`, `merges`, `token_type`)
as raw bytes. Writing still goes through the `go-gguf` writer. GGUF
serialization is not hand-rolled.
`go-gguf`'s `MetadataEntry.Value()` intentionally returns an empty value for Commands:
`BTypeArray` (see `readLazyValue` in the library). The tokenizer arrays
(`tokens`, `scores`→`token_type`, `merges`) are exactly the values we must copy
verbatim, so `rawkv.go` reads the KV section directly and preserves each array's
`elem_type`, element count, and **raw element bytes**. Those raw bytes are fed
straight into go-gguf's writer `Value{Raw: ...}`, so serialization is still done
by the library (nothing hand-rolled on the write path).
### Subcommands
```bash ```bash
# 2a. dump — emit the reproduction contract as JSON
go run . dump -gguf models/open-1b-sft-f16.gguf -o models/open-1b-sft-f16.manifest.json go run . dump -gguf models/open-1b-sft-f16.gguf -o models/open-1b-sft-f16.manifest.json
go run . convert -ref models/open-1b-sft-f16.gguf \
# 2b/c. convert — reproduce metadata + map safetensors→GGUF (fp32→f16)
go run . convert \
-ref models/open-1b-sft-f16.gguf \
-safetensors models/open-1b-sft/model.safetensors \ -safetensors models/open-1b-sft/model.safetensors \
-config models/open-1b-sft/config.json \ -config models/open-1b-sft/config.json \
-tokenizer-config models/open-1b-sft/tokenizer_config.json \ -tokenizer-config models/open-1b-sft/tokenizer_config.json \
-tokenizer models/open-1b-sft/tokenizer.json \ -tokenizer models/open-1b-sft/tokenizer.json \
-o models/open-1b-sft-f16-go.gguf -o models/open-1b-sft-f16-go.gguf
# 3. verify — byte-level equality against the reference
go run . verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf go run . verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
``` ```
`dump` output (the contract) is at ## Tensor mapping
`models/open-1b-sft-f16.manifest.json` (39 metadata entries + 220 tensors,
with per-array `raw_sha256`).
### Tensor mapping (reference names are the contract) The reference names are the contract.
Shapes are the reference's; ggml `ne0` is the innermost (= HF input) dim, so HF | GGUF tensor | HF source | Shape | Type |
bytes are copied unpermuted with the dims reversed. | --- | --- | --- | --- |
| `token_embd.weight` | `embedding.tok_embeddings.weight` | 2048 x 128256 | F16 |
| `token_embd_norm.weight` | `emb_norm.weight` | 2048 | F32 |
| `output.weight` | `embedding.output.weight` | 2048 x 128256 | F16 |
| `output_norm.weight` | `norm_out.weight` | 2048 | F32 |
| `blk.i.attn_q.weight` | `blocks.i.attn.wq.weight` | 2048 x 2048 | F16 |
| `blk.i.attn_k.weight` | `blocks.i.attn.wk.weight` | 2048 x 512 | F16 |
| `blk.i.attn_v.weight` | `blocks.i.attn.wv.weight` | 2048 x 512 | F16 |
| `blk.i.attn_output.weight` | `blocks.i.attn.wo.weight` | 2048 x 2048 | F16 |
| `blk.i.ffn_gate.weight` | `blocks.i.ffn.w_gate_up.weight`, first half | 2048 x 5632 | F16 |
| `blk.i.ffn_up.weight` | `blocks.i.ffn.w_gate_up.weight`, second half | 2048 x 5632 | F16 |
| `blk.i.ffn_down.weight` | `blocks.i.ffn.w_down.weight` | 5632 x 2048 | F16 |
| `blk.i.attn_norm.weight` | `blocks.i.norm1.weight` | 2048 | F32 |
| `blk.i.ffn_norm.weight` | `blocks.i.norm2.weight` | 2048 | F32 |
| Reference tensor | HF source | Shape | Type | Count | The fused `w_gate_up` is split at the row boundary. Per-channel `*.weight_scale`
|---|---|---|---|---| tensors in the checkpoint are dropped; the reference has none. F32 to F16 uses
| `token_embd.weight` | `embedding.tok_embeddings.weight` | [2048,128256] | F16 | 1 | round-to-nearest, ties to even, matching the numpy and torch casts.
| `token_embd_norm.weight` | `emb_norm.weight` | [2048] | F32 | 1 |
| `output.weight` | `embedding.output.weight` | [2048,128256] | F16 | 1 |
| `output_norm.weight` | `norm_out.weight` | [2048] | F32 | 1 |
| `blk.{i}.attn_q.weight` | `blocks.{i}.attn.wq.weight` | [2048,2048] | F16 | 24 |
| `blk.{i}.attn_k.weight` | `blocks.{i}.attn.wk.weight` | [2048,512] | F16 | 24 |
| `blk.{i}.attn_v.weight` | `blocks.{i}.attn.wv.weight` | [2048,512] | F16 | 24 |
| `blk.{i}.attn_output.weight` | `blocks.{i}.attn.wo.weight` | [2048,2048] | F16 | 24 |
| `blk.{i}.ffn_gate.weight` | `blocks.{i}.ffn.w_gate_up.weight` (first half) | [2048,5632] | F16 | 24 |
| `blk.{i}.ffn_up.weight` | `blocks.{i}.ffn.w_gate_up.weight` (second half) | [2048,5632] | F16 | 24 |
| `blk.{i}.ffn_down.weight` | `blocks.{i}.ffn.w_down.weight` | [5632,2048] | F16 | 24 |
| `blk.{i}.attn_norm.weight` | `blocks.{i}.norm1.weight` | [2048] | F32 | 24 |
| `blk.{i}.ffn_norm.weight` | `blocks.{i}.norm2.weight` | [2048] | F32 | 24 |
- The fused `w_gate_up` is split at the row boundary (`chunk(2,-1)` → first half ## Validation
`gate`, second half `up`), matching the reference.
- Quantized per-channel `*.weight_scale` tensors in the checkpoint are **dropped**
(the reference has none).
- F32→F16 uses IEEE round-to-nearest, ties-to-even (`f32bitsToF16`), matching
numpy/torch casts; the reference tensors are byte-identical to ours, which
empirically proves the rounding matched.
- HF config/tokenizer are **parsed and validated** against the reference, but the
reference remains authoritative for every emitted value.
---
## 3. Validation
`go run . verify` streams both files and compares on raw wire bytes:
``` ```
metadata: 39 checked, 0 mismatches metadata: 39 checked, 0 mismatches
tensors: 220 checked, 220 byte-identical, 0 mismatches tensors: 220 checked, 220 byte-identical, 0 mismatches
verification PASSED: 39 metadata keys and 220 tensors byte-identical verification PASSED
``` ```
Whole-file check:
``` ```
$ cmp models/open-1b-sft-f16.gguf models/open-1b-sft-f16-go.gguf && echo IDENTICAL $ cmp models/open-1b-sft-f16.gguf models/open-1b-sft-f16-go.gguf && echo IDENTICAL
IDENTICAL IDENTICAL
$ sha256sum models/open-1b-sft-f16.gguf models/open-1b-sft-f16-go.gguf
a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae open-1b-sft-f16.gguf
a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae open-1b-sft-f16-go.gguf
``` ```
The per-tensor diff is empty (0 mismatches) — every tensor's payload is Both files hash to
byte-identical, in the same order, with the same shape and ggml type. `a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`.
### Metadata contract (all 39 keys, all byte-identical) Notable metadata, all byte-identical: `general.architecture=open1b`,
`open1b.block_count=24`, `open1b.context_length=4096`,
`open1b.attention.head_count=16`, `open1b.attention.head_count_kv=4`,
`open1b.rope.freq_base=500000`, `open1b.attention.sliding_window=512`,
`open1b.attention.sliding_window_pattern=5`, `tokenizer.ggml.model=gpt2`,
`tokenizer.ggml.pre=gpt-2`, `tokenizer.ggml.eos_token_id=4`. The full list is in
the manifest JSON.
| Key | Type | Value | `go build`, `go vet`, and `go test -race` are clean.
|---|---|---|
| `general.architecture` | STRING | `open1b` |
| `general.type` | STRING | `model` |
| `general.name` | STRING | `Open 1b Sft` |
| `general.finetune` | STRING | `sft` |
| `general.basename` | STRING | `open` |
| `general.size_label` | STRING | `1B` |
| `general.license` | STRING | `apache-2.0` |
| `general.base_model.count` | UINT32 | 1 |
| `general.base_model.0.name` | STRING | `Open 1b Midtrained 93B` |
| `general.base_model.0.organization` | STRING | `Gensyn` |
| `general.base_model.0.repo_url` | STRING | `https://huggingface.co/Gensyn/open-1b-midtrained-93B` |
| `general.dataset.count` | UINT32 | 1 |
| `general.dataset.0.name` | STRING | `Tulu 3 Sft Olmo 2 Mixture 0225` |
| `general.dataset.0.version` | STRING | `0225` |
| `general.dataset.0.organization` | STRING | `Allenai` |
| `general.dataset.0.repo_url` | STRING | `https://huggingface.co/allenai/tulu-3-sft-olmo-2-mixture-0225` |
| `general.tags` | ARRAY[STRING] | 5 (`gensyn`, `verifiable-training`, `reproducible-training`, `repops`, `text-generation`) |
| `general.languages` | ARRAY[STRING] | 1 (`en`) |
| `open1b.block_count` | UINT32 | 24 |
| `open1b.context_length` | UINT32 | 4096 |
| `open1b.embedding_length` | UINT32 | 2048 |
| `open1b.feed_forward_length` | UINT32 | 5632 |
| `open1b.attention.head_count` | UINT32 | 16 |
| `open1b.attention.head_count_kv` | UINT32 | 4 |
| `open1b.rope.freq_base` | FLOAT32 | 500000 |
| `open1b.attention.layer_norm_rms_epsilon` | FLOAT32 | 1e-05 |
| `open1b.attention.key_length` | UINT32 | 128 |
| `open1b.attention.value_length` | UINT32 | 128 |
| `general.file_type` | UINT32 | 1 (F16) |
| `open1b.attention.sliding_window` | UINT32 | 512 |
| `open1b.attention.sliding_window_pattern` | UINT32 | 5 |
| `general.quantization_version` | UINT32 | 2 |
| `tokenizer.ggml.model` | STRING | `gpt2` |
| `tokenizer.ggml.pre` | STRING | `gpt-2` |
| `tokenizer.ggml.tokens` | ARRAY[STRING] | 128256 (sha256 `30bab437…`) |
| `tokenizer.ggml.token_type` | ARRAY[INT32] | 128256 (sha256 `5ca8c567…`) |
| `tokenizer.ggml.merges` | ARRAY[STRING] | 127994 (sha256 `9e3a7dd4…`) |
| `tokenizer.ggml.eos_token_id` | UINT32 | 4 |
| `tokenizer.chat_template` | STRING | (501-byte Jinja template) |
Array `raw_sha256` values are in `models/open-1b-sft-f16.manifest.json`. ## Serve
### HF cross-checks performed by `convert` ```bash
setsid --fork ./build-vulkan/bin/llama-server \
``` -m models/open-1b-sft-f16-go.gguf \
config.json OK: arch=open1b layers=24 hidden=2048 heads=16/4 head_dim=128 ffn=5632 vocab=128256 swa=512/5 tie=false qk_norm=true(gain=false) emb_norm=true
tokenizer_config.json OK: eos=<|eot|> chat_template=501 bytes
tokenizer.json OK: type=BPE vocab=128256 +added=6 -> 128256 tokens, 127994 merges
```
### Go build / vet / test
```
$ go build ./... # OK
$ go vet ./... # OK
$ go test -race -v ./...
--- PASS: TestF32BitsToF16 (0.00s)
--- PASS: TestF32ToF16Bytes (0.00s)
--- PASS: TestRawKVRoundTrip (0.00s)
--- PASS: TestVerifyRealFiles (6.59s)
PASS
ok hf2gguf 7.602s
```
`TestVerifyRealFiles` re-runs the full 3.2 GB byte comparison (skipped on
`-short` or when the artifacts are absent).
---
## 4. Serve on one Arc (Vulkan2)
```
setsid --fork /root/projects/inference/build-vulkan/bin/llama-server \
-m /root/projects/gensyn/inference/models/open-1b-sft-f16-go.gguf \
--device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \ --device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \
--spec-type ngram-map-k,ngram-cache -t 16 --parallel 1 --alias open-1b \ --spec-type ngram-map-k,ngram-cache \
-t 16 --parallel 1 --alias open-1b \
--host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null & --host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null &
``` ```
Startup log:
```
load_model: loading model '.../open-1b-sft-f16-go.gguf'
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
llama_context: n_ctx_seq (32768) > n_ctx_train (4096) -- possible training context overflow
load_model: the slot context (32768) exceeds the training context of the model (4096) - capping
load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
model loaded
listening on http://127.0.0.1:8085
```
Device confirmation: PID `1378706` maps `/dev/dri/renderD129` with large
`rw-s` regions; `renderD129 → /sys/devices/.../0000:07:00.0` = Arc #2 = Vulkan2.
`/props` reports `"speculative.types":"none,ngram-map-k,ngram-cache"` and
`"model_ftype":"F16"`.
### Performance (single Arc, 1 slot, `-fa off`, ngram spec)
| Metric | Value | | Metric | Value |
| --- | --- | | --- | --- |
| Decode | **~111.7–117.2 tok/s** (e.g. 256 tok @ 111.65; 96 tok @ 117.20) | | Decode | 111.7 to 117.2 tok/s |
| Prefill | **~892 tok/s** (961 prompt tok in 1077 ms) | | Prefill | about 892 tok/s |
| VRAM on Vulkan2 | free 10971 → 7258 MiB ⇒ **≈ 3.6 GiB** used; fully released after `kill` | | VRAM | about 3.6 GiB, released after stop |
### Sample outputs Sample outputs, greedy:
``` ```
$ curl ... /completion {"prompt":"The three primary colors are", greedy} prompt: The three primary colors are
red, blue, and yellow. out: red, blue, and yellow.
$ curl ... /completion {"prompt":"Q: What is the capital of France?\nA:", greedy, 96} prompt: Q: What is the capital of France? A:
Paris out: Paris
$ curl ... /completion (seeded chat turn, see caveat) 113.9 tok/s
" Venus and Mars. Venus is the hottest planet in our Solar System, with
temperatures reaching 465°C (1,470°F). Mars, on the other hand, is a
terrestrial planet, meaning it has a solid surface and liquid water at its poles."
``` ```
### Stopping Stopped with `kill` after reading the pid from `ss -ltnp`.
``` ## Caveats
$ ss -ltnp | grep :8085 # -> pid=1378706
$ kill 1378706 # never pkill -f
```
Both the test server (:8085) and a diagnostic server (:8086) were stopped; 1. Chat template. With the vendor tail `...assistant<|end_header|>` followed by a
`--list-devices` returned to 10971 MiB free on Vulkan2/3. blank line, this checkpoint emits an end-of-turn token first and returns an
empty reply. Reproduced with and without the speculative decoder, so it is not
the decoder. A single newline, an extra newline, or a seed token produces a
coherent answer. Since our GGUF is byte-identical to the reference, this is a
property of the released checkpoint.
2. Context cap. `open1b.context_length` is 4096, so the requested 32768 is capped.
3. The load warning `special_eos_id is not in special_eog_ids` also appears for
the reference file, so it is not introduced by the conversion.
4. Long greedy generation without a repetition penalty loops on this small model.
5. Process note. During one early check the response was piped through `python3`
to pretty-print JSON. That was a mistake and a breach of the no-Python rule.
The tool itself is entirely Go, and nothing in the pipeline depends on Python.
No Python files or virtualenv were created for the tool.
--- ## Files
## 5. Caveats
1. **Chat template newline quirk (model-side, not conversion-side).** With the
exact vendor template tail `<|end_header|>\n\n` for the assistant turn, this
checkpoint emits `<|eot|>` as the first sampled token → empty reply / instant
EOS (reproduced on both :8085 with ngram spec and a :8086 run with
`--spec-type none`, so it is **not** the speculative decoder). A single `\n`,
`\n\n\n`, or a seed token makes it answer coherently:
| tail after `assistant<|end_header|>` | result |
|---|---|
| `\n\n` (vendor template) | immediate EOS |
| `\n` | `\n1. **Earth**: …` (coherent) |
| `\n\n\n` | `\n1. **Earth**: …` (coherent) |
| ` ` | `\nSure, here are two names…` (coherent) |
Because our GGUF is byte-identical to the reference GGUF, this is a
property of the released checkpoint/GGUF (likely SFT formatting), not of our
conversion. Workarounds for consumers: use `ignore_eos` + trim, seed the
assistant turn, or fix the template's trailing newline.
2. **Context capped to 4096.** `open1b.context_length = 4096`, so the requested
`-c 32768` is capped to `n_ctx_slot = 4096`. KV is f16 (`-fa off` mandatory on
Arc per `brainstation-gpu`).
3. **`special_eos_id is not in special_eog_ids`** warning on load — present for
the reference file too (byte-identical), so not introduced by us.
4. **1B greedy degeneration.** Long greedy generation without repetition penalty
loops on this small model; use sampling / `repeat_penalty` for long outputs.
5. **Process note (disclosure).** During an early diagnostic I piped one
`/completion` response through `python3 -c` to pretty-print JSON — a breach of
the "no Python" rule (the tool itself is 100% Go, and nothing in the pipeline
depends on Python). All subsequent parsing was done with Go/raw JSON. No `.py`
file or venv was created; the only Python files in the tree are the vendor's
read-only `models/open-1b-sft/*.py`.
---
## 6. Artifacts
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| `hf2gguf/` | Go module (`dump` / `convert` / `verify`) + tests | | `hf2gguf/` | Go module with tests |
| `hf2gguf/README.md` | tool usage | | `hf2gguf/README.md` | usage |
| `models/open-1b-sft-f16-go.gguf` | our output (byte-identical to reference) | | `models/open-1b-sft-f16-go.gguf` | our output |
| `models/open-1b-sft-f16.manifest.json` | dumped contract (39 KV + 220 tensors) | | `models/open-1b-sft-f16.manifest.json` | dumped contract |
| `models/open-1b-sft-f16.gguf` | reference (untouched) | | `models/open-1b-sft-f16.gguf` | reference |
| `/tmp/open1b.log` | server log | | `/tmp/open1b.log` | server log |
No changes were committed; everything lives under gitignored `inference/`.
tools/hf2gguf/README.md
+31 -35
View File
@@ -1,39 +1,36 @@
# hf2gguf # hf2gguf
Pure-Go (Go 1.27) tool to convert a HuggingFace `open-1b` safetensors checkpoint A Go 1.27 tool that converts a Hugging Face `open-1b` safetensors checkpoint to
into GGUF **byte-identically** to a supplied reference GGUF, plus a byte-level GGUF, byte-identical to a supplied reference file, with a byte-level verifier.
verifier. Built for epic `vado/1b#1` (Gensyn open-1b local bring-up).
Library: [`github.com/cymertek/go-gguf`](https://github.com/cymertek/go-gguf) Library: [go-gguf](https://github.com/cymertek/go-gguf), reader and writer. No
(reader + writer). No Python anywhere in the tool. Python anywhere.
## Build ## Build
```bash ```bash
go build ./... go build ./...
go vet ./... go vet ./...
go test -race ./... # full test includes the 3.2 GB byte comparison go test -race ./... # the full test includes the 3.2 GB byte comparison
``` ```
## Subcommands ## Commands
### `dump` — the reproduction contract ### dump
Reads the reference GGUF and emits a JSON manifest of **all** metadata KV pairs Reads the reference GGUF and writes a JSON manifest of all metadata pairs (key,
(key, type, value / array elem-type + count + `raw_sha256`) and every tensor type, value, or array element type, count, and `raw_sha256`) and every tensor
(name, shape, ggml type, nbytes). (name, shape, ggml type, byte count).
```bash ```bash
go run . dump -gguf models/open-1b-sft-f16.gguf -o models/open-1b-sft-f16.manifest.json go run . dump -gguf models/open-1b-sft-f16.gguf -o models/open-1b-sft-f16.manifest.json
``` ```
`go-gguf` exposes scalar/string metadata and tensors; its `MetadataEntry.Value()` `go-gguf` returns an empty value for `BTypeArray`, so `rawkv.go` walks the KV
returns an empty value for `BTypeArray`, so `rawkv.go` walks the KV section section directly to keep the tokenizer arrays as raw bytes. Writing still goes
directly to preserve tokenizer arrays (`tokens`, `merges`, `token_type`) as raw through the `go-gguf` writer.
bytes. Writing still goes through `go-gguf`'s writer — nothing hand-rolls GGUF
serialization.
### `convert` — emit our GGUF ### convert
```bash ```bash
go run . convert \ go run . convert \
@@ -45,38 +42,37 @@ go run . convert \
-o models/open-1b-sft-f16-go.gguf -o models/open-1b-sft-f16-go.gguf
``` ```
- Metadata is copied verbatim from the reference KV section. - Metadata is copied verbatim from the reference.
- Tensors are mapped HF FQN → GGUF name (see the module docs in `convert.go`), - Tensors are mapped by name, with `blocks.i.ffn.w_gate_up.weight` split into
with `blocks.{i}.ffn.w_gate_up.weight` split into `ffn_gate`/`ffn_up`, and `ffn_gate` and `ffn_up`, and `*.weight_scale` dropped.
`*.weight_scale` dropped (the reference has none). - Payloads stream from safetensors and convert F32 to F16 on the fly, using
- Payloads stream from safetensors and are converted F32→F16 with IEEE round-to-nearest, ties to even.
round-to-nearest-even on the fly (O(1) memory per tensor). - The config and tokenizer flags are optional. When given, they are parsed and
- `-config` / `-tokenizer-config` / `-tokenizer` are optional; when given they validated against the reference; a mismatch is fatal.
are parsed and **validated** against the reference (mismatch is fatal).
### `verify` — byte-level equality ### verify
```bash ```bash
go run . verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf go run . verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
``` ```
Compares metadata on raw wire bytes and tensors by shape, type and full payload Compares metadata on raw wire bytes and tensors by shape, type, and full payload.
equality (streaming, first-diff offset reported on failure). Reports the first differing offset on failure.
``` ```
metadata: 39 checked, 0 mismatches metadata: 39 checked, 0 mismatches
tensors: 220 checked, 220 byte-identical, 0 mismatches tensors: 220 checked, 220 byte-identical, 0 mismatches
verification PASSED: 39 metadata keys and 220 tensors byte-identical verification PASSED
``` ```
## Layout ## Layout
| File | Purpose | | File | Purpose |
| --- | --- | | --- | --- |
| `main.go` | dispatch, manifest types, `dump` | | `main.go` | dispatch, manifest types, dump |
| `rawkv.go` | read-only GGUF KV walker (array support) | | `rawkv.go` | read-only GGUF KV walker with array support |
| `safetensors.go` | safetensors header/byte access | | `safetensors.go` | safetensors header and byte access |
| `hfconfig.go` | HF config/tokenizer parsing + validation | | `hfconfig.go` | HF config and tokenizer parsing, validation |
| `convert.go` | metadata reproduction, tensor mapping, F32→F16 | | `convert.go` | metadata reproduction, tensor mapping, F32 to F16 |
| `verify.go` | `verify` command + comparison engine | | `verify.go` | verify command and comparison engine |
| `convert_test.go` | F16 rounding, KV round-trip, full-file verification | | `convert_test.go` | F16 rounding, KV round-trip, full-file verification |
tools/hf2gguf/convert.go
+2 -2
View File
@@ -1,6 +1,6 @@
package main package main
// convert.go — emit models/open-1b-sft-f16-go.gguf from the HF safetensors, // convert.go converts the HF safetensors into models/open-1b-sft-f16-go.gguf,
// reproducing the reference GGUF's metadata (verbatim) and tensor set/order. // reproducing the reference GGUF's metadata (verbatim) and tensor set/order.
// //
// Tensor mapping (HF FQN -> GGUF name), shapes are the reference's (ggml ne0 is // Tensor mapping (HF FQN -> GGUF name), shapes are the reference's (ggml ne0 is
@@ -117,7 +117,7 @@ func cmdConvert(args []string) error {
defer st.Close() defer st.Close()
// Validate HF config against the reference contract (no values are taken from // Validate HF config against the reference contract (no values are taken from
// it — the reference is authoritative — but mismatches are fatal). // it (the reference is authoritative), but mismatches are fatal).
if *hfConfig != "" { if *hfConfig != "" {
if err := validateConfig(*hfConfig, kvs); err != nil { if err := validateConfig(*hfConfig, kvs); err != nil {
return fmt.Errorf("config validation: %w", err) return fmt.Errorf("config validation: %w", err)
tools/hf2gguf/hfconfig.go
+1 -1
View File
@@ -1,6 +1,6 @@
package main package main
// hfconfig.go — parse and validate the HF config/tokenizer against the // hfconfig.go parses and validates the HF config/tokenizer against the
// reference GGUF metadata. The reference remains authoritative for values; this // reference GGUF metadata. The reference remains authoritative for values; this
// only proves the reference is consistent with the published HF files and that // only proves the reference is consistent with the published HF files and that
// we parsed them (per the task), it does not source any GGUF value from them. // we parsed them (per the task), it does not source any GGUF value from them.
tools/hf2gguf/main.go
+1 -1
View File
@@ -35,7 +35,7 @@ func main() {
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Manifest types — the reproduction contract. // Manifest types for the reproduction contract.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
type manifest struct { type manifest struct {
tools/hf2gguf/safetensors.go
+1 -1
View File
@@ -1,6 +1,6 @@
package main package main
// safetensors.go — minimal read-only HuggingFace safetensors index. // safetensors.go reads a minimal, read-only HuggingFace safetensors index.
// //
// Format: [u64 LE header_len][header_len bytes of JSON][tensor data ...]. // Format: [u64 LE header_len][header_len bytes of JSON][tensor data ...].
// The JSON header maps tensor name -> {dtype, shape, data_offsets:[start,end]} // The JSON header maps tensor name -> {dtype, shape, data_offsets:[start,end]}
tools/hf2gguf/verify.go
+1 -1
View File
@@ -1,6 +1,6 @@
package main package main
// verify.go — prove the emitted GGUF equals the reference: identical metadata // verify.go proves the emitted GGUF equals the reference: identical metadata
// (keys, types, values; verified on raw wire bytes) and identical tensor set, // (keys, types, values; verified on raw wire bytes) and identical tensor set,
// shapes, ggml types and per-tensor payload bytes. // shapes, ggml types and per-tensor payload bytes.