docs: foreground hf2gguf, add advantages vs Python, honest limits
This commit is contained in:
@@ -1,13 +1,55 @@
|
||||
# open-1b: an independent check, and a local run
|
||||
# open-1b: an independent check, a local run, and a Go GGUF converter
|
||||
|
||||
We looked at Gensyn's open-1b and its claim of auditable training, then ran the
|
||||
model on our own hardware. The audit uses Gensyn's official harness. Everything
|
||||
we wrote is Go.
|
||||
We checked Gensyn's open-1b and its claim of auditable training, then ran the
|
||||
model on our own hardware. The audit uses Gensyn's official harness. The
|
||||
conversion and tooling are Go.
|
||||
|
||||
Links: [announcement](https://www.gensyn.ai/news/introducing-open-1b-auditable-training),
|
||||
[harness](https://github.com/gensyn-ai/open-transformers),
|
||||
[audit app](https://open1b.gensyn.ai/).
|
||||
|
||||
## hf2gguf
|
||||
|
||||
`tools/hf2gguf` is the tool at the center of this repo. It converts a Hugging
|
||||
Face open-1b checkpoint to GGUF and checks the result byte for byte, with no
|
||||
Python, no torch, and no pip. For open-1b it reproduces llama.cpp's converter
|
||||
output exactly, down to the same SHA-256.
|
||||
|
||||
Why build it this way:
|
||||
|
||||
| | hf2gguf | llama.cpp `convert_hf_to_gguf.py` |
|
||||
| --- | --- | --- |
|
||||
| Runtime | one Go binary | Python 3 with torch, numpy, transformers, gguf-py |
|
||||
| Setup | `go build` | pip or uv, wheels, pinned versions |
|
||||
| Startup | immediate | imports torch and its chain |
|
||||
| Memory | streams one tensor at a time | loads tensors through torch |
|
||||
| Output check | built-in byte-level `verify` | none |
|
||||
| Output | byte-identical to a reference file | the reference |
|
||||
| Scope | open-1b only | many architectures |
|
||||
| Quantization | none, F16 only | many quant types |
|
||||
|
||||
It is deliberately narrow. It maps one Llama-derived architecture and copies the
|
||||
GGUF metadata from a reference file, so it is not a general converter. Its value
|
||||
is that it is small, readable, dependency-free, and able to prove the file it
|
||||
writes is identical to the reference.
|
||||
|
||||
```bash
|
||||
cd tools/hf2gguf
|
||||
go build -o hf2gguf .
|
||||
./hf2gguf convert -ref models/open-1b-sft-f16.gguf \
|
||||
-safetensors models/open-1b-sft/model.safetensors \
|
||||
-config models/open-1b-sft/config.json \
|
||||
-tokenizer-config models/open-1b-sft/tokenizer_config.json \
|
||||
-tokenizer models/open-1b-sft/tokenizer.json \
|
||||
-o models/open-1b-sft-f16-go.gguf
|
||||
./hf2gguf verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
|
||||
```
|
||||
|
||||
`verify` reports 39 of 39 metadata keys and 220 of 220 tensors identical. The two
|
||||
files share SHA-256
|
||||
`a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`.
|
||||
See [tools/hf2gguf/README.md](tools/hf2gguf/README.md).
|
||||
|
||||
## Results
|
||||
|
||||
| Check | Result |
|
||||
@@ -51,13 +93,13 @@ flowchart LR
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
tools/hf2gguf/ # the Go converter: dump, convert, verify
|
||||
docs/
|
||||
audit-open1b-init.md
|
||||
open1b-local-inference.md
|
||||
benchmarks.md
|
||||
inference-stack.md
|
||||
llama-cpp-open1b.patch
|
||||
tools/hf2gguf/
|
||||
```
|
||||
|
||||
## The audit, and its limits
|
||||
@@ -80,29 +122,6 @@ per-step claim stays untested here. Three more reasons to be careful:
|
||||
A match shows that the pinned artifacts reproduce the published commitment on
|
||||
our machine. It does not show how the original run was carried out.
|
||||
|
||||
## Conversion in Go
|
||||
|
||||
`tools/hf2gguf` reads the Hugging Face checkpoint (config.json,
|
||||
model.safetensors, tokenizer.json) and writes a GGUF file, using
|
||||
[go-gguf](https://github.com/cymertek/go-gguf). A reference GGUF from llama.cpp's
|
||||
converter, built with our open1b patch, is read only, as the schema to match.
|
||||
|
||||
```bash
|
||||
cd tools/hf2gguf
|
||||
go build -o hf2gguf .
|
||||
./hf2gguf convert -ref models/open-1b-sft-f16.gguf \
|
||||
-safetensors models/open-1b-sft/model.safetensors \
|
||||
-config models/open-1b-sft/config.json \
|
||||
-tokenizer-config models/open-1b-sft/tokenizer_config.json \
|
||||
-tokenizer models/open-1b-sft/tokenizer.json \
|
||||
-o models/open-1b-sft-f16-go.gguf
|
||||
./hf2gguf verify -ref models/open-1b-sft-f16.gguf -got models/open-1b-sft-f16-go.gguf
|
||||
```
|
||||
|
||||
`verify` reports 39 of 39 metadata keys and 220 of 220 tensors identical. The two
|
||||
files share SHA-256
|
||||
`a18530417c732bb115a3501a7e1388ea7a54bd2a96c70be571d1249aa210fbae`.
|
||||
|
||||
## Running the model
|
||||
|
||||
Upstream llama.cpp has no open1b architecture, so the model needs a patched
|
||||
|
||||
+40
-4
@@ -1,10 +1,46 @@
|
||||
# hf2gguf
|
||||
|
||||
A Go 1.27 tool that converts a Hugging Face `open-1b` safetensors checkpoint to
|
||||
GGUF, byte-identical to a supplied reference file, with a byte-level verifier.
|
||||
A Go 1.27 tool that converts a Hugging Face open-1b checkpoint to GGUF and
|
||||
verifies the result byte for byte. No Python, no torch, no pip. Library:
|
||||
[go-gguf](https://github.com/cymertek/go-gguf).
|
||||
|
||||
Library: [go-gguf](https://github.com/cymertek/go-gguf), reader and writer. No
|
||||
Python anywhere.
|
||||
For open-1b it reproduces llama.cpp's `convert_hf_to_gguf.py` output exactly,
|
||||
down to the same SHA-256. The output is deterministic and the check is built in,
|
||||
so the conversion can be trusted without trusting a Python environment.
|
||||
|
||||
## Why Go, and why this shape
|
||||
|
||||
| | hf2gguf | llama.cpp `convert_hf_to_gguf.py` |
|
||||
| --- | --- | --- |
|
||||
| Runtime | one Go binary | Python 3 with torch, numpy, transformers, gguf-py |
|
||||
| Setup | `go build` | pip or uv, wheels, pinned versions |
|
||||
| Startup | immediate | imports torch and its chain |
|
||||
| Memory | streams one tensor at a time, O(1) per tensor | loads tensors through torch |
|
||||
| Output check | built-in byte-level `verify` | none |
|
||||
| Output | byte-identical to a reference file | the reference |
|
||||
| Scope | open-1b only | many architectures |
|
||||
| Quantization | none, F16 only | many quant types |
|
||||
| Dependencies | go-gguf | a large Python tree |
|
||||
|
||||
The advantages that matter here:
|
||||
|
||||
- No Python runtime. Nothing to install or pin, and it runs where Python is
|
||||
unavailable or unwanted.
|
||||
- Deterministic output, checked byte for byte against a reference.
|
||||
- Small and readable. One binary, one small module, no large dependency tree.
|
||||
- Streaming. Tensors are converted one at a time; memory does not grow with the
|
||||
model size.
|
||||
- Easy to build and ship. `go build` and a single file in CI images.
|
||||
|
||||
## Limits
|
||||
|
||||
- Narrow by design. It maps one Llama-derived architecture and does not claim to
|
||||
convert arbitrary models.
|
||||
- Metadata is copied from a reference GGUF, not derived from the config alone.
|
||||
The config and tokenizer are parsed and validated, but the reference stays
|
||||
authoritative.
|
||||
- F16 only. There is no quantization to Q4, Q8, or other ggml types.
|
||||
- It depends on a third-party GGUF writer for serialization.
|
||||
|
||||
## Build
|
||||
|
||||
|
||||
Reference in New Issue
Block a user