eSlider/open-1b-verify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: public-ready English repo — README, docs/benchmarks, MIT license, gofmt

Browse Source
This commit is contained in:
vad
2026-09-28 16:43:51 +00:00
parent 1d73cf05d5
commit e9adbd1591
13 changed files with 198 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/audit-open1b-init.md
+214
View File
@@ -0,0 +1,214 @@
# Gensyn OPEN-1B audit — independent replay (CPU, Linux x86-64)
- **Date:** 2026-09-28
- **Host:** `brainstation`, Linux x86-64, kernel `7.3.0-070300rc1-generic` (Ubuntu), **no NVIDIA GPU**
- **Auditor:** SE sub-agent (independent)
- **Workdir:** `/root/projects/gensyn/audit`
- **Harness:** `gensyn-ai/open-transformers` @ `d7b7b674cd851f7a16815229f060658180cb772b`
("Initial commit: the OPEN-1B pretraining and audit-replay harness", 2026-09-15), cloned via git.
- **Verdict:** ✅ **MATCH** on the `1b_repop_v2` init unit. ❌ No training-interval
replay run (not defined by this kit and not feasible on this host — see §7).
---
## 1. Kit selection
Four kits were listed. All four `trajectory.json` files declare the **same**
`1b_repop_v2` init commitment
`16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed`, so the choice
of kit does not change the init hash under test. The authoritative mapping is
published by the audit app itself at `https://open1b.gensyn.ai/manifest.json`:
```json
"run": { "id": "20260722-213626-ad3276b", "model": "1b_repop_v2",
"initStateHash": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed" },
"audit": { "kit": { "id": "pt-f91110a0d0c8_rp-c9ca6e71e673",
"urlBase": "https://storage.googleapis.com/gensyn-audit-public/audit-kit/pt-f91110a0d0c8_rp-c9ca6e71e673" } }
```
**Kit under test: `pt-f91110a0d0c8_rp-c9ca6e71e673`.**
- `pretrain_commit`: `f91110a0d0c84fa3bad808f365567d9c655d10b2`
- `repop_commit`: `c9ca6e71e673660afc6c87de2592fa6f62c87b15`
- `torch_requirement`: `torch==2.10.0`
(The other three kits — `pt-cb4ac29438bc_rp-…`, `pt-f6bc086dfebd_rp-…`,
`pt-f5eece09abe6_rp-…` — carry the same init commitment but are older
pretrain/repop pins. Two share the `c9ca6e71e673` repop but a different pretrain
commit; the app manifest picks `f91110a0d0c8`.)
## 2. Toolchain
| Component | Version / identity |
|---|---|
| Python | 3.11.16 (managed by `uv`; system Python 3.14 untouched) |
| `uv` | 0.12.17 |
| PyTorch | `2.10.0+cpu` (`torch.version.cuda == None`) |
| `repop` | 0.1.5, `build_info().commit == c9ca6e71e673660afc6c87de2592fa6f62c87b15` ✅ (== kit.json `repop_commit`); backends `["cpu","cuda"]` |
| `pretrain` | 0.1.0 (kit wheel) |
| Go (glue tools) | go1.27.0 (fabric rule: all glue/parsing in Go) |
**Deviation, stated plainly:** the RUNBOOK targets Linux hosts with an NVIDIA GPU
and points them at the CUDA 12.9 PyTorch index. This host has no GPU, and the
README names CPU as the *reference* device, so the exact pin `torch==2.10.0` was
installed from PyTorch's official **CPU** index (`.../whl/cpu`, wheel
`2.10.0+cpu`). The pinned version is unchanged; only the CPU build variant was
chosen deliberately, not by an accidental PyPI fallback. Init is device-independent
by construction (see §6), so this does not weaken the init result.
Python (`uv`-managed) is used **only** because the vendor harness is Python and
ships as wheels; all download/verification/listing glue is Go under `tools/`
(`kitinspect`, `trajinspect`, `kitfetch`, `gslist`).
## 3. Commands run
```bash
# workdir + docs
mkdir -p /root/projects/gensyn/audit && cd /root/projects/gensyn/audit
git clone --depth 1 https://github.com/gensyn-ai/open-transformers
# kit manifests + trajectories (Go inspector)
go run ./tools/kitinspect kits/*/
go run ./tools/trajinspect kits/*/trajectory.json
# download + per-file sha256 verification against kit.json (Go tool,
# downloads to .part, verifies, removes on mismatch)
go run ./tools/kitfetch kits/pt-f91110a0d0c8_rp-c9ca6e71e673 \
kit-pt-f91110a0d0c8_rp-c9ca6e71e673
# Python 3.11 env
uv venv --python 3.11 .venv
uv pip install --python .venv --index-url https://download.pytorch.org/whl/cpu "torch==2.10.0"
uv pip install --python .venv \
./kit-.../repop-0.1.5-cp311-cp311-linux_x86_64.whl \
./kit-.../pretrain-0.1.0-cp311-cp311-linux_x86_64.whl
# provenance assertions
.venv/bin/python -c "import repop,json; print(repop.build_info())"
# THE AUDIT (init unit)
PRETRAIN_AUDIT_MEMLOG=1 /usr/bin/time -v .venv/bin/pretrain-audit-replay \
--from-init --until-step 0 \
--config-name 1b_repop_v2 \
--device cpu \
--expect-hash 16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed
```
## 4. SHA-256 verification table (all 5 kit files)
Every file was hashed **before** install; the Go fetcher adopts a file only on a
full 64-hex match and deletes rejected bytes. Expected values are read from the
kit's own `kit.json`.
| File | expected (`kit.json`) | computed | result |
|---|---|---|---|
| `pretrain-0.1.0-cp311-cp311-linux_x86_64.whl` | `ecc49d1c05c0e2876a59d98d877795769d6e9620eb66e4b4c866b16f695fb174` | same | ✅ OK |
| `pretrain-0.1.0-cp311-cp311-macosx_14_0_arm64.whl` | `fbd23a54aa0bf87775e86493d67f759d1c1f4cc1dcd63ad991fba59dcb92491c` | same | ✅ OK |
| `repop-0.1.5-cp311-cp311-linux_x86_64.whl` | `15f171643871211b2a772ee83c94833c0be2eef0495790cedc33813c95147ff0` | same | ✅ OK |
| `repop-0.1.5-cp311-cp311-macosx_14_0_arm64.whl` | `d8658e0327c7278812ecb6dd455cd4cf8723a974a78788adba41d5f747289c69` | same | ✅ OK |
| `trajectory.json` | `9435515c37d8956d0c81d5bc78c0a383bb6bf27c57b5dbaf1f8be564df0b324a` | same | ✅ OK |
Byte sizes also matched `kit.json`. Only the two **linux_x86_64** wheels were
installed; the macOS arm64 wheels were downloaded and verified but not installed.
## 5. Init-unit result — MATCH
Raw result object printed by the harness (stdout, exit status 0):
```json
{
"step": 0,
"consumed_tokens": 0,
"state_hash": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed",
"mode": "init",
"repop": { "commit": "c9ca6e71e673660afc6c87de2592fa6f62c87b15",
"backends": ["cpu", "cuda"], "cuda_arch_list": "7.5 8.0 9.0" },
"device": "cpu",
"expected": "16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed",
"match": true,
"cpu_threads": { "requested": "auto", "source": "runtime-default",
"torch_num_threads": 8, "omp_num_threads": null, "mkl_num_threads": null }
}
```
Log line: `from-init: regenerated init state_hash=16554a1119745f1b expected=16554a1119745f1b MATCH=True`.
Config `1b_repop_v2`, seed `42`, loss path `fused CE+z-loss`. The only warning is
the expected `not distributed + no CUDA — parallelize applies AC only (FSDP
skipped; dev mode)`, i.e. the documented CPU reference path.
**Resource notes:** wall **58.3 s**, user 78.5 s / sys 61.1 s, **peak RSS
16.80 GB** (16,804,956 KB), **0 swaps**, exit 0. (README/RUNBOOK size this unit at
~21.9 GB peak on CPU; we measured 16.8 GB. Free RAM was ~11 GB and the unit still
completed without swapping, so the doc figure is conservative.)
## 6. What this proves — and what it does not
**Proves.** The bytes installed are the bytes `kit.json` published (all five
SHA-256s match). The `repop` kernel build is the one the kit claims
(`build_info().commit == repop_commit`). Running the vendor harness on this
CPU, from seed, the `1b_repop_v2` **initial state** — seeded device-independent
trunc-normal init plus zeroed optimizer moments — reproduces the published
canonical hash `16554a11…` **bit-for-bit**. Independent third-party
reproduction of the run's init commitment: ✅.
**Does not prove.**
1. **No training interval.** Per the README: "An initialization-only unit does
not verify a training interval." A match here says nothing about any forward,
backward, AdamW or data-stream step. The claimed bitwise reproducibility of
*optimizer steps* is **not** tested by this run.
2. **`repop` is closed source.** Only compiled wheels are shipped; the kernel
arithmetic that makes training bitwise reproducible cannot be read or
independently re-derived from this kit. The init unit does not even exercise
the kernels that the recent `c9ca6e71e673` repin changed (the trajectory notes
say exactly this), so a match is *not* evidence for the kernel-level BFR
contract at that commit.
3. **v3-hash coverage.** The README states the v3 hash covers weights, optimizer
state, gradients and the batch digest, but **not** RNG, the data-stream
cursor, spike state, or the `meta.json` descriptor keys. Even a matching
interval hash would leave those unverified.
4. **Init is the easy case by design.** `init.py` samples Philox on CPU and
byte-copies, so it is device-independent; it is expected to match across
devices. The hard cross-device claim lives in the training kernels, which
this unit does not touch.
5. Provenance of the *published run* is not established; only that the pinned
artifacts reproduce the published commitment on this machine.
## 7. Step-interval replay — deliberately not attempted
Stopping after the init unit is a considered decision, not a stall:
- **Not defined by the kit.** This kit's `trajectory.json` (`init-units-v1`)
contains **only 3 init units** — `100m_smoke_repop`, `1b_repop_run3`,
`1b_repop_v2`. Its own note says: "Step-interval units are added per run once
its checkpoints and state_hash files are published." **No interval unit with a
canonical target hash is published here**, so the kit defines no
step-interval commitment to check. (The run's per-step log exists at
`gs://gensyn-open-1b/logs/state_hashes.jsonl`, but the harness has no automatic
lookup and the audit app gates submissions.)
- **Memory.** A step-interval replay needs ~24 GB (README), and the audit's
auto-offload only reaches "every published unit" at that figure. This host had
~11 GB RAM available (29 GB total, 18 GB in use); an init unit already peaked at
16.8 GB. The heavy replay would swap-thrash and would not fit honestly.
- **Disk/size.** One checkpoint `ckpt/step_000000100/` is **19.31 GB**
(150 objects); the full `ckpt/` prefix is **15.64 TB** (810 checkpoint dirs).
With 39 GB free, a single checkpoint is tight against free space and the
documented `--from-init + --gcs-root` combination is broken (wrong shard
anchor), so a from-init interval would first require staging shards via
`--data-root`.
Re-attempting an interval audit should be done on a host with ≥32 GB free RAM
(NVIDIA GPU preferred: `--device cuda`) and ≥60 GB free disk.
## 8. Files produced
- `open-transformers/` — harness clone (docs read: `README.md`,
`docs/audit-replay-usage.md`, `scripts/audit_volunteer/RUNBOOK.md`)
- `kits/<kit>/kit.json`, `kits/<kit>/trajectory.json` — manifests
- `kit-pt-f91110a0d0c8_rp-c9ca6e71e673/` — verified wheels + trajectory
- `.venv/` — Python 3.11 audit env
- `init-replay.log` — full captured replay output + `/usr/bin/time -v`
- `tools/{kitinspect,trajinspect,kitfetch,gslist}` — Go glue
- `REPORT.md` — this file
No secrets were written. Nothing outside the workdir was modified or deleted.
docs/benchmarks.md
+48
View File
@@ -0,0 +1,48 @@
# Benchmarks
Speed of **open-1b** on our hardware, with a reference comparison against the
model we run in production (`gemma-4-E2B`). All numbers are single-stream
(one request at a time) on one GPU.
## Hardware / setup
- **GPU:** 1× Intel Arc B580 (BMG G21, 12 GB), Vulkan (Mesa), llama.cpp `build-vulkan` (b10893).
- **CPU:** AMD Ryzen 7 7700X (16 vCPU), 29 GB RAM.
- **Serving flags:** `-ngl 99 -fa off --jinja --spec-type ngram-map-k,ngram-cache -t 16 --parallel 1`.
- `-fa off` is mandatory on these Arc/Vulkan builds → KV cache is f16 (no `-ctk q8_0`).
## open-1b (our Go-built GGUF, f16)
| Metric | Value |
|---|---|
| Decode | **~112–117 tok/s** (e.g. 256 tok @ 111.7; 96 tok @ 117.2) |
| Prefill | **~892 tok/s** (961 prompt tok in 1077 ms) |
| VRAM | **~3.6 GiB** (10971 → 7258 MiB free) |
| Context | 4096 (model limit; the requested 32k is capped) |
| Weights | f16, ~3.2 GB |
## Reference comparison (1× Arc B580)
| Model | Weights | Context | Decode | Prefill |
|---|---|---|---|---|
| **open-1b-sft** (this repo) | f16 ~3.2 GB | 4096 | ~114 tok/s | ~892 tok/s |
| **gemma-4-E2B** (prod) | Q4 ~1.5 GiB | 64k | **~214 tok/s** | **~1659 tok/s** |
*gemma figures are from our 2026-09-20 controlled benchmark (decode 256 tok /
prefill 1698 tok) on the same card and build.*
**Reading:** gemma is roughly **2× faster**, ~**2.4× smaller**, and has **16×
the context** of open-1b. For any practical workload (classification, extraction,
translation, tool-calling) gemma wins outright. open-1b's reason to exist is not
throughput or quality but **auditable training**.
## Caveats
- This is **not** a controlled A/B: the open-1b figures are a short first-run
measurement with different prompts; the gemma figures come from a separate
controlled benchmark. Use them as an order-of-magnitude orientation.
- The `ngram` speculative decoder helps task-dependent amounts; open-1b's
baseline (no-spec) number was not measured here.
- Quality: open-1b scores **25.4** on the OLMo2 suite vs **31.9** for OLMo2-1B,
i.e. below an ordinary 1B model. It is a proof of *verifiability*, not a strong
assistant.
docs/issues/01-epic.md
-26
View File
@@ -1,26 +0,0 @@
# Epic: Gensyn open-1b — верификация auditable training и локальный inference
**Цель.** Независимо проверить заявление Gensyn о *bitwise auditable training* (open-1b) на нашем железе и запустить модель локально; зафиксировать факты, ограничения и решения.
## Контекст
- Анонс 2026-09-15: https://www.gensyn.ai/news/introducing-open-1b-auditable-training (+ PRNewswire 302878850)
- Harness: https://github.com/gensyn-ai/open-transformers (Apache-2.0)
- Audit CLI: https://github.com/gensyn-ai/gensyn-audit
- Audit app: https://open1b.gensyn.ai/ (manifest: https://open1b.gensyn.ai/manifest.json)
- Модель: HF `Gensyn/open-1b-base`, `-midtrained-93B`, `-sft`
- Модель: 1.61B total / 1.08B non-emb, 24 слоя, 400B токенов, 80 957 шагов, 48×H100
- 2dph info-leaf (PO, 2026-09-28): `6703767319c6e2f9fa6d63465934d72e`
## Дети эпика
- `#AUDIT-INIT` — Аудит init-unit open-1b (CPU, Linux x86-64)
- `#AUDIT-INTERVAL` — Аудит тренировочного интервала open-1b (заблокировано)
- `#INFERENCE` — Локальный запуск open-1b на Arc B580 (llama.cpp Vulkan)
- `#REPORT` — Сводный отчёт и запись фактов в 2dph
## Критерий закрытия эпика
Все дочерние issues закрыты с доказательствами (тест/лог/хэш). Заблокированный
интервальный аудит может остаться открытым, если блокер явно задокументирован
(нет опубликованного interval-unit + ресурсы хоста).
docs/issues/02-audit-init.md
-25
View File
@@ -1,25 +0,0 @@
Epic: #EPIC
## Цель
Независимо воспроизвести опубликованный **init-хэш** open-1b официальным harness'ом
на CPU (Linux x86-64, без NVIDIA).
## Критерии приёмки
- [x] kit `pt-f91110a0d0c8_rp-c9ca6e71e673` скачан; sha256 **всех** файлов сверены с `kit.json`
- [x] `repop.build_info().commit == kit.json.repop_commit` (`c9ca6e71e673`)
- [x] `pretrain-audit-replay --from-init --until-step 0 --config-name 1b_repop_v2 --device cpu` → `match=true`
- [x] отчёт с raw-выводом и таблицей sha256
## Результат (2026-09-28)
✅ **MATCH**. `state_hash = 16554a1119745f1b9bf4ac60b03226e5e9ca1247a2bd56169cd4c822fd457aed`,
wall 58.3 s, peak RSS 16.8 GB, exit 0.
Отчёт: `audit/REPORT.md`.
## Ограничение
Init-unit по README **не** проверяет тренировочный интервал — это отдельная задача
`#AUDIT-INTERVAL`.
docs/issues/03-audit-interval.md
-29
View File
@@ -1,29 +0,0 @@
Epic: #EPIC
## Цель
Воспроизвести **тренировочный интервал** (replay шага против опубликованного hash) —
именно это доказывает заявление «bitwise auditable training», в отличие от init-unit.
## Критерии приёмки
- [ ] получен interval-unit с canonical target hash (published commitment на шаг)
- [ ] скачан чекпоинт + шарды, потреблённые интервалом
- [ ] `pretrain-audit-replay --checkpoint … --until-step N --gcs-root … --expect-hash …` → `MATCH`
- [ ] отчёт с raw-выводом
## Блокер (2026-09-28)
- ⛔ В kit `pt-f91110a0d0c8_rp-c9ca6e71e673` `trajectory.json` содержит **только init-units**
(`100m_smoke_repop`, `1b_repop_run3`, `1b_repop_v2`). Interval-units «added per run once its
checkpoints and state_hash files are published» — **не опубликованы**. Self-serve кит
не определяет commit на шаг.
- ⛔ Ресурсы хоста: нужно ~24 ГБ RAM (есть ~11 ГБ) и ≥60 ГБ диска (есть 39 ГБ);
один чекпоинт `ckpt/step_000000100/` = 19.31 ГБ.
- Решение по разблокировке (владелец — PO/пользователь): запросить у audit-app interval-unit,
либо выполнять на хосте с ≥32 ГБ RAM / ≥60 ГБ диска (предпочтительно NVIDIA `--device cuda`).
## Примечание
Per-step log доступен на `gs://gensyn-open-1b/logs/state_hashes.jsonl`; harness не умеет
автоподстановку, а audit-app гейтит сабмиты — это тоже часть блокера.
docs/issues/04-inference.md
-23
View File
@@ -1,23 +0,0 @@
Epic: #EPIC
## Цель
Поднять open-1b локально и доказать работоспособность на нашем железе
(Intel Arc B580, Vulkan; без NVIDIA/CUDA).
## Критерии приёмки
- [ ] llama.cpp собран с Vulkan; бэкенд активен на Arc B580
- [ ] веса HF (`Gensyn/open-1b-sft`, fallback `-base`) скачаны, provenance = точная ревизия
- [ ] конвертация в GGUF; собран/выбран квант
- [ ] `llama-cli`/`llama-server` даёт связный ответ; замер tok/s и VRAM
- [ ] отчёт с точными командами и caveat'ами
## Прочее
- Сервисы — только docker compose (не systemd).
- Обвязка — Go; Python только через `uv` (канонический `convert_hf_to_gguf.py`).
## Статус
🔄 выполняется фоновым SE-субагентом (2026-09-28).
docs/issues/05-report.md
-16
View File
@@ -1,16 +0,0 @@
Epic: #EPIC
## Цель
Свести результаты в отчёт и записать проверенные факты в 2dph (brain), пометив
что подтверждено, а что остаётся ограничением.
## Критерии приёмки
- [ ] сводный отчёт (что доказано / что нет) в репо
- [ ] info-leaf в 2dph с итогом (source: `po/gensyn`)
- [ ] явно зафиксированы ограничения: `repop` closed-source, v3-hash не покрывает RNG/cursor/spike/meta, interval-unit не опубликован
## Статус
info-leaf записан: `6703767319c6e2f9fa6d63465934d72e`. Отчёт — по завершении #INFERENCE и #AUDIT-INTERVAL.
docs/open1b-local-inference.md
+332
View File
@@ -0,0 +1,332 @@
# Gensyn open-1b (SFT) — local bring-up & proof of run
Epic: `vado/1b#1` / task #4 (SE). Workdir: `/root/projects/gensyn/inference/`.
Engine: existing shared llama.cpp **Vulkan** build (`/root/projects/inference/build-vulkan/bin/llama-server`).
Language: **Go 1.27 only** (see "Process note" re one accidental `python3` call).
## TL;DR
- Built a pure-Go tool `hf2gguf` (module `hf2gguf`) that **dumps** the reference GGUF
(go-gguf + a minimal raw-KV walker for arrays), **converts** `model.safetensors`
→ `models/open-1b-sft-f16-go.gguf`, and **verifies** the two files.
- **Validation: PASS — 39/39 metadata KV byte-identical, 220/220 tensors byte-identical.**
Whole files are byte-for-byte identical (`cmp` + SHA-256 match).
- **Served** on one Arc (Vulkan2 / `07:00.0`) with the existing Vulkan build; coherent
output, **~114 tok/s decode**, **~892 tok/s prefill**, ~**3.6 GiB VRAM**.
- No commit made (all under gitignored `inference/`).
Reference SHA-256 (and our output — identical):
`a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`
---
## 1. Inputs & topology
| Item | Path | Notes |
|------|------|-------|
| HF checkpoint (fp32) | `models/open-1b-sft/model.safetensors` | 6,432,081,672 B; dtype **F32**; header 34560 B |
| Reference GGUF (f16) | `models/open-1b-sft-f16.gguf` | 3,220,153,856 B; vendor converter; **read-only oracle** |
| HF config | `models/open-1b-sft/config.json` | `model_type=open1b`, 24 layers, hidden 2048, heads 16/4, head_dim 128, ffn 5632, vocab 128256, rope_theta 5e5, rms_eps 1e-5, swa 512 every 5, qk_norm gain-free, embedding_norm, untied |
| Vendor Python | `models/open-1b-sft/modeling_open1b.py` | read only, never executed |
| Engine | `/root/projects/inference/build-vulkan/bin/llama-server` | symlink → `/mnt/2TB/projects/build-vulkan`; shared prod tree, untouched |
GPU devices as reported by the engine (`--list-devices`):
```
Vulkan0: AMD Ryzen 7 7700X (RADV RAPHAEL_MENDOCINO) (15835 MiB)
Vulkan1: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB)
Vulkan2: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB)
Vulkan3: Intel(R) Arc(tm) B580 Graphics (BMG G21) (12216 MiB)
```
`Vulkan2` was confirmed to map to PCI `0000:07:00.0` (Arc #2) via
`/sys/class/drm/renderD129/device`.
---
## 2. Tool: `hf2gguf` (Go 1.27, `github.com/cymertek/go-gguf`)
```
inference/hf2gguf/
go.mod, go.sum
main.go # subcommand dispatch + JSON manifest types
rawkv.go # minimal read-only GGUF KV walker (fills go-gguf's array gap)
safetensors.go # safetensors header + tensor byte access
hfconfig.go # config.json / tokenizer_config.json / tokenizer.json validation
convert.go # metadata reproduction + HF→GGUF tensor mapping + F32→F16
verify.go # byte-level metadata & tensor comparison
convert_test.go
```
### Why a raw KV walker
`go-gguf`'s `MetadataEntry.Value()` intentionally returns an empty value for
`BTypeArray` (see `readLazyValue` in the library). The tokenizer arrays
(`tokens`, `scores`→`token_type`, `merges`) are exactly the values we must copy
verbatim, so `rawkv.go` reads the KV section directly and preserves each array's
`elem_type`, element count, and **raw element bytes**. Those raw bytes are fed
straight into go-gguf's writer `Value{Raw: ...}`, so serialization is still done
by the library (nothing hand-rolled on the write path).
### Subcommands
```bash
# 2a. dump — emit the reproduction contract as JSON
go run . dump -gguf models/open-1b-sft-f16.gguf -o models/open-1b-sft-f16.manifest.json
# 2b/c. convert — reproduce metadata + map safetensors→GGUF (fp32→f16)
go run . convert \
-ref models/open-1b-sft-f16.gguf \
-safetensors models/open-1b-sft/model.safetensors \
-config models/open-1b-sft/config.json \
-tokenizer-config models/open-1b-sft/tokenizer_config.json \
-tokenizer models/open-1b-sft/tokenizer.json \
-o models/open-1b-sft-f16-go.gguf
# 3. verify — byte-level equality against the reference
go run . verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
```
`dump` output (the contract) is at
`models/open-1b-sft-f16.manifest.json` (39 metadata entries + 220 tensors,
with per-array `raw_sha256`).
### Tensor mapping (reference names are the contract)
Shapes are the reference's; ggml `ne0` is the innermost (= HF input) dim, so HF
bytes are copied unpermuted with the dims reversed.
| Reference tensor | HF source | Shape | Type | Count |
|---|---|---|---|---|
| `token_embd.weight` | `embedding.tok_embeddings.weight` | [2048,128256] | F16 | 1 |
| `token_embd_norm.weight` | `emb_norm.weight` | [2048] | F32 | 1 |
| `output.weight` | `embedding.output.weight` | [2048,128256] | F16 | 1 |
| `output_norm.weight` | `norm_out.weight` | [2048] | F32 | 1 |
| `blk.{i}.attn_q.weight` | `blocks.{i}.attn.wq.weight` | [2048,2048] | F16 | 24 |
| `blk.{i}.attn_k.weight` | `blocks.{i}.attn.wk.weight` | [2048,512] | F16 | 24 |
| `blk.{i}.attn_v.weight` | `blocks.{i}.attn.wv.weight` | [2048,512] | F16 | 24 |
| `blk.{i}.attn_output.weight` | `blocks.{i}.attn.wo.weight` | [2048,2048] | F16 | 24 |
| `blk.{i}.ffn_gate.weight` | `blocks.{i}.ffn.w_gate_up.weight` (first half) | [2048,5632] | F16 | 24 |
| `blk.{i}.ffn_up.weight` | `blocks.{i}.ffn.w_gate_up.weight` (second half) | [2048,5632] | F16 | 24 |
| `blk.{i}.ffn_down.weight` | `blocks.{i}.ffn.w_down.weight` | [5632,2048] | F16 | 24 |
| `blk.{i}.attn_norm.weight` | `blocks.{i}.norm1.weight` | [2048] | F32 | 24 |
| `blk.{i}.ffn_norm.weight` | `blocks.{i}.norm2.weight` | [2048] | F32 | 24 |
- The fused `w_gate_up` is split at the row boundary (`chunk(2,-1)` → first half
`gate`, second half `up`), matching the reference.
- Quantized per-channel `*.weight_scale` tensors in the checkpoint are **dropped**
(the reference has none).
- F32→F16 uses IEEE round-to-nearest, ties-to-even (`f32bitsToF16`), matching
numpy/torch casts; the reference tensors are byte-identical to ours, which
empirically proves the rounding matched.
- HF config/tokenizer are **parsed and validated** against the reference, but the
reference remains authoritative for every emitted value.
---
## 3. Validation
`go run . verify` streams both files and compares on raw wire bytes:
```
metadata: 39 checked, 0 mismatches
tensors: 220 checked, 220 byte-identical, 0 mismatches
verification PASSED: 39 metadata keys and 220 tensors byte-identical
```
Whole-file check:
```
$ cmp models/open-1b-sft-f16.gguf models/open-1b-sft-f16-go.gguf && echo IDENTICAL
IDENTICAL
$ sha256sum models/open-1b-sft-f16.gguf models/open-1b-sft-f16-go.gguf
a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae open-1b-sft-f16.gguf
a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae open-1b-sft-f16-go.gguf
```
The per-tensor diff is empty (0 mismatches) — every tensor's payload is
byte-identical, in the same order, with the same shape and ggml type.
### Metadata contract (all 39 keys, all byte-identical)
| Key | Type | Value |
|---|---|---|
| `general.architecture` | STRING | `open1b` |
| `general.type` | STRING | `model` |
| `general.name` | STRING | `Open 1b Sft` |
| `general.finetune` | STRING | `sft` |
| `general.basename` | STRING | `open` |
| `general.size_label` | STRING | `1B` |
| `general.license` | STRING | `apache-2.0` |
| `general.base_model.count` | UINT32 | 1 |
| `general.base_model.0.name` | STRING | `Open 1b Midtrained 93B` |
| `general.base_model.0.organization` | STRING | `Gensyn` |
| `general.base_model.0.repo_url` | STRING | `https://huggingface.co/Gensyn/open-1b-midtrained-93B` |
| `general.dataset.count` | UINT32 | 1 |
| `general.dataset.0.name` | STRING | `Tulu 3 Sft Olmo 2 Mixture 0225` |
| `general.dataset.0.version` | STRING | `0225` |
| `general.dataset.0.organization` | STRING | `Allenai` |
| `general.dataset.0.repo_url` | STRING | `https://huggingface.co/allenai/tulu-3-sft-olmo-2-mixture-0225` |
| `general.tags` | ARRAY[STRING] | 5 (`gensyn`, `verifiable-training`, `reproducible-training`, `repops`, `text-generation`) |
| `general.languages` | ARRAY[STRING] | 1 (`en`) |
| `open1b.block_count` | UINT32 | 24 |
| `open1b.context_length` | UINT32 | 4096 |
| `open1b.embedding_length` | UINT32 | 2048 |
| `open1b.feed_forward_length` | UINT32 | 5632 |
| `open1b.attention.head_count` | UINT32 | 16 |
| `open1b.attention.head_count_kv` | UINT32 | 4 |
| `open1b.rope.freq_base` | FLOAT32 | 500000 |
| `open1b.attention.layer_norm_rms_epsilon` | FLOAT32 | 1e-05 |
| `open1b.attention.key_length` | UINT32 | 128 |
| `open1b.attention.value_length` | UINT32 | 128 |
| `general.file_type` | UINT32 | 1 (F16) |
| `open1b.attention.sliding_window` | UINT32 | 512 |
| `open1b.attention.sliding_window_pattern` | UINT32 | 5 |
| `general.quantization_version` | UINT32 | 2 |
| `tokenizer.ggml.model` | STRING | `gpt2` |
| `tokenizer.ggml.pre` | STRING | `gpt-2` |
| `tokenizer.ggml.tokens` | ARRAY[STRING] | 128256 (sha256 `30bab437…`) |
| `tokenizer.ggml.token_type` | ARRAY[INT32] | 128256 (sha256 `5ca8c567…`) |
| `tokenizer.ggml.merges` | ARRAY[STRING] | 127994 (sha256 `9e3a7dd4…`) |
| `tokenizer.ggml.eos_token_id` | UINT32 | 4 |
| `tokenizer.chat_template` | STRING | (501-byte Jinja template) |
Array `raw_sha256` values are in `models/open-1b-sft-f16.manifest.json`.
### HF cross-checks performed by `convert`
```
config.json OK: arch=open1b layers=24 hidden=2048 heads=16/4 head_dim=128 ffn=5632 vocab=128256 swa=512/5 tie=false qk_norm=true(gain=false) emb_norm=true
tokenizer_config.json OK: eos=<|eot|> chat_template=501 bytes
tokenizer.json OK: type=BPE vocab=128256 +added=6 -> 128256 tokens, 127994 merges
```
### Go build / vet / test
```
$ go build ./... # OK
$ go vet ./... # OK
$ go test -race -v ./...
--- PASS: TestF32BitsToF16 (0.00s)
--- PASS: TestF32ToF16Bytes (0.00s)
--- PASS: TestRawKVRoundTrip (0.00s)
--- PASS: TestVerifyRealFiles (6.59s)
PASS
ok hf2gguf 7.602s
```
`TestVerifyRealFiles` re-runs the full 3.2 GB byte comparison (skipped on
`-short` or when the artifacts are absent).
---
## 4. Serve on one Arc (Vulkan2)
```
setsid --fork /root/projects/inference/build-vulkan/bin/llama-server \
-m /root/projects/gensyn/inference/models/open-1b-sft-f16-go.gguf \
--device Vulkan2 -ngl 99 -c 32768 -fa off --jinja \
--spec-type ngram-map-k,ngram-cache -t 16 --parallel 1 --alias open-1b \
--host 127.0.0.1 --port 8085 >/tmp/open1b.log 2>&1 </dev/null &
```
Startup log:
```
load_model: loading model '.../open-1b-sft-f16-go.gguf'
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
llama_context: n_ctx_seq (32768) > n_ctx_train (4096) -- possible training context overflow
load_model: the slot context (32768) exceeds the training context of the model (4096) - capping
load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
model loaded
listening on http://127.0.0.1:8085
```
Device confirmation: PID `1378706` maps `/dev/dri/renderD129` with large
`rw-s` regions; `renderD129 → /sys/devices/.../0000:07:00.0` = Arc #2 = Vulkan2.
`/props` reports `"speculative.types":"none,ngram-map-k,ngram-cache"` and
`"model_ftype":"F16"`.
### Performance (single Arc, 1 slot, `-fa off`, ngram spec)
| Metric | Value |
|---|---|
| Decode | **~111.7–117.2 tok/s** (e.g. 256 tok @ 111.65; 96 tok @ 117.20) |
| Prefill | **~892 tok/s** (961 prompt tok in 1077 ms) |
| VRAM on Vulkan2 | free 10971 → 7258 MiB ⇒ **≈ 3.6 GiB** used; fully released after `kill` |
### Sample outputs
```
$ curl ... /completion {"prompt":"The three primary colors are", greedy}
red, blue, and yellow.
$ curl ... /completion {"prompt":"Q: What is the capital of France?\nA:", greedy, 96}
Paris
$ curl ... /completion (seeded chat turn, see caveat) 113.9 tok/s
" Venus and Mars. Venus is the hottest planet in our Solar System, with
temperatures reaching 465°C (1,470°F). Mars, on the other hand, is a
terrestrial planet, meaning it has a solid surface and liquid water at its poles."
```
### Stopping
```
$ ss -ltnp | grep :8085 # -> pid=1378706
$ kill 1378706 # never pkill -f
```
Both the test server (:8085) and a diagnostic server (:8086) were stopped;
`--list-devices` returned to 10971 MiB free on Vulkan2/3.
---
## 5. Caveats
1. **Chat template newline quirk (model-side, not conversion-side).** With the
exact vendor template tail `<|end_header|>\n\n` for the assistant turn, this
checkpoint emits `<|eot|>` as the first sampled token → empty reply / instant
EOS (reproduced on both :8085 with ngram spec and a :8086 run with
`--spec-type none`, so it is **not** the speculative decoder). A single `\n`,
`\n\n\n`, or a seed token makes it answer coherently:
| tail after `assistant<|end_header|>` | result |
|---|---|
| `\n\n` (vendor template) | immediate EOS |
| `\n` | `\n1. **Earth**: …` (coherent) |
| `\n\n\n` | `\n1. **Earth**: …` (coherent) |
| ` ` | `\nSure, here are two names…` (coherent) |
Because our GGUF is byte-identical to the vendor reference, this is a
property of the released checkpoint/GGUF (likely SFT formatting), not of our
conversion. Workarounds for consumers: use `ignore_eos` + trim, seed the
assistant turn, or fix the template's trailing newline.
2. **Context capped to 4096.** `open1b.context_length = 4096`, so the requested
`-c 32768` is capped to `n_ctx_slot = 4096`. KV is f16 (`-fa off` mandatory on
Arc per `brainstation-gpu`).
3. **`special_eos_id is not in special_eog_ids`** warning on load — present for
the reference file too (byte-identical), so not introduced by us.
4. **1B greedy degeneration.** Long greedy generation without repetition penalty
loops on this small model; use sampling / `repeat_penalty` for long outputs.
5. **Process note (disclosure).** During an early diagnostic I piped one
`/completion` response through `python3 -c` to pretty-print JSON — a breach of
the "no Python" rule (the tool itself is 100% Go, and nothing in the pipeline
depends on Python). All subsequent parsing was done with Go/raw JSON. No `.py`
file or venv was created; the only Python files in the tree are the vendor's
read-only `models/open-1b-sft/*.py`.
---
## 6. Artifacts
| Path | What |
|---|---|
| `hf2gguf/` | Go module (`dump` / `convert` / `verify`) + tests |
| `hf2gguf/README.md` | tool usage |
| `models/open-1b-sft-f16-go.gguf` | our output (byte-identical to reference) |
| `models/open-1b-sft-f16.manifest.json` | dumped contract (39 KV + 220 tensors) |
| `models/open-1b-sft-f16.gguf` | reference (untouched) |
| `/tmp/open1b.log` | server log |
No changes were committed; everything lives under gitignored `inference/`.