eSlider/ollama-intel-gpu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Rework README for better GitHub presentation

Browse Source 
Rewrite README with clear value proposition, architecture diagram,
troubleshooting section, and streamlined structure. Update CHANGELOG
to reflect full history of Vulkan-to-SYCL migration.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Andrey
2026-02-12 17:31:33 +00:00
parent c56646e7e7
commit 971852d3af
2 changed files with 130 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+23 -27
View File
@@ -1,40 +1,36 @@
# Changelog
## 2026-02-12
## 2026-02-12 — Switch to SYCL backend
### Fix: Ollama not reachable from host via Docker port mapping
### GPU backend: Vulkan -> SYCL
The bundled IPEX-LLM `/start-ollama.sh` entrypoint hardcodes
`OLLAMA_HOST='127.0.0.1:11434'` and `OLLAMA_KEEP_ALIVE=10m`, overriding any
values set through Docker Compose environment variables.
- Replaced Vulkan GPU backend with custom-built SYCL backend for ~2x inference
speed on Intel GPUs
- Multi-stage Dockerfile: builds `libggml-sycl.so` from upstream llama.cpp
(commit `a5bb8ba4`) using Intel oneAPI 2025.1.1
- Added `patch-sycl.py` to fix two ollama-specific API divergences:
- `graph_compute` signature (`int batch_size` parameter)
- `GGML_TENSOR_FLAG_COMPUTE` removal (critical — without this patch all
compute nodes are skipped, producing garbage output)
- Bundled oneAPI runtime libraries (SYCL, oneMKL, oneDNN, TBB, Level-Zero)
into the runtime image
- Added a custom `start-ollama.sh` that respects environment variables
(`${OLLAMA_HOST:-0.0.0.0:11434}`, `${OLLAMA_KEEP_ALIVE:-24h}`) instead of
hardcoding them
- Mounted the script into the container as a read-only volume
(`./start-ollama.sh:/start-ollama.sh:ro`)
- Fixed `LD_LIBRARY_PATH` env var syntax in docker-compose.yml (`:` -> `=`)
### Ollama upgrade: 0.9.3 -> 0.15.6
### Updated Intel GPU runtime stack to latest releases
- Upgraded from IPEX-LLM bundled ollama 0.9.3 to official ollama v0.15.6
- Switched from IPEX-LLM portable zip to official ollama binary
- Removed CUDA/MLX/Vulkan runners from image to reduce size
### Intel GPU runtime stack
- **level-zero**: v1.22.4 -> v1.28.0
- Loader based on oneAPI Level Zero Specification v1.15.31
- Memory leak fixes, expanded multidriver teardown support
- **intel-graphics-compiler (IGC)**: v2.11.7 (build 19146) -> v2.28.4 (build 20760)
- Built with LLVM 16.0.6, opaque pointers support
- **intel-graphics-compiler (IGC)**: v2.11.7 -> v2.28.4
- **compute-runtime**: 25.18.33578.6 -> 26.05.37020.3
- Built with IGC v2.28.4 and level-zero v1.27.0
- Panther Lake production support, Wildcat Lake pre-release
- **libigdgmm**: 22.7.0 -> 22.9.0
- **ipex-llm ollama** (nightly): 2.3.0b20250612 -> 2.3.0b20250725
- Latest available nightly Ubuntu ollama portable zip
### Docker Compose adjustments
### Docker Compose
- Disabled persistent webui volume for stateless restarts
- Device mapping changed to full `/dev/dri` access for SYCL/Level-Zero
- Added `ONEAPI_DEVICE_SELECTOR=level_zero:0` and `ZES_ENABLE_SYSMAN=1`
- Removed `OLLAMA_VULKAN=1`
- Disabled web UI authentication (`WEBUI_AUTH=False`)
### README
- Formatting and heading structure improvements
- Updated tested GPU model to Intel Core Ultra 5 155H
README.md
+107 -82
View File
@@ -1,20 +1,14 @@
# Ollama for Intel GPU
# Ollama for Intel GPU (SYCL)
[![GitHub license](https://img.shields.io/github/license/mattcurf/ollama-intel-gpu)](
> Run LLMs on Intel GPUs at full speed — no NVIDIA required.
Run LLM models on your local Intel GPU using Ollama with Docker.
Includes [Open WebUI](https://github.com/open-webui/open-webui) for a
browser-based chat interface.
A Docker-based setup that pairs [Ollama](https://github.com/ollama/ollama) **v0.15.6** with a custom-built **SYCL backend** for Intel GPU acceleration, plus [Open WebUI](https://github.com/open-webui/open-webui) for a browser chat interface. Three commands to go from zero to local AI.
## Screenshot
**Why this exists:** Ollama's official release ships only a Vulkan backend for Intel GPUs, leaving significant performance on the table. This repo builds the `ggml-sycl` backend from source with Intel oneAPI, unlocking oneMKL, oneDNN, and Level-Zero direct GPU access.
![screenshot](doc/screenshot.png)
## Prerequisites
* Ubuntu 24.04 or newer
* Docker and Docker Compose
* Intel GPU (tested with Intel Core Ultra 7 155H integrated Arc Graphics — Meteor Lake)
---
## Quick start
@@ -24,101 +18,132 @@ cd ollama-intel-gpu
docker compose up
```
Then open http://localhost:3000 in your browser.
Open **http://localhost:3000** — pull a model and start chatting.
> If you have multiple GPUs (integrated + discrete), set
> `ONEAPI_DEVICE_SELECTOR=level_zero:0` in the docker-compose environment
> to select the intended device.
The first `docker compose up` builds the SYCL backend from source (~2 min on a modern CPU). Subsequent starts are instant.
## GPU backend: SYCL vs Vulkan
> **Multiple GPUs?** Set `ONEAPI_DEVICE_SELECTOR=level_zero:0` in `docker-compose.yml` to pick the right device.
Ollama can accelerate inference on Intel GPUs via two backends.
This repo defaults to **SYCL** (built from upstream llama.cpp's ggml-sycl
with Intel oneAPI) for best Intel GPU performance.
---
### Performance comparison (llama-2-7b Q4_0, llama.cpp benchmarks)
## Tested hardware
| Intel GPU | Vulkan tok/s | SYCL tok/s | SYCL advantage |
|---------------------|-------------|------------|----------------|
| MTL iGPU (155H) | ~8-11 | **16** | +45-100% |
| ARL-H iGPU | ~10-12 | **17** | +40-70% |
| Arc A770 | ~30-35 | **55** | +57-83% |
| Flex 170 | ~30-35 | **50** | +43-67% |
| Data Center Max 1550| — | **73** | — |
| Intel GPU | Status |
|-----------|--------|
| Core Ultra 7 155H integrated Arc (Meteor Lake) | Verified |
| Arc A-series (A770, A750, A380) | Expected compatible |
| Data Center Flex / Max | Expected compatible |
### Why SYCL is faster
**Requirements:** Ubuntu 24.04+, Docker with Compose, Intel GPU with Level-Zero driver support.
* **oneDNN** — Intel's Deep Neural Network Library for optimized GEMM (matrix multiply)
* **oneMKL** — Intel Math Kernel Library for optimized math operations
* **Level-zero direct access** — lower-overhead GPU communication than Vulkan
* **Intel-specific MUL_MAT kernels** — hand-tuned for MTL, ARL, Arc, Flex, PVC architectures
* **FP16 compute path** — optional `GGML_SYCL_F16=ON` for faster compute
* **Multi-GPU support** — `--split-mode layer` across multiple Intel GPUs
---
### Why you might still use Vulkan
## SYCL vs Vulkan performance
* Shipped in official ollama releases — no build step required
* Cross-vendor (Intel, AMD, NVIDIA)
* Simpler deployment, smaller image
Both backends run on Intel GPUs. This repo defaults to SYCL for the speed advantage.
To switch to Vulkan, see the `Dockerfile.vulkan` (if provided) or use the
official ollama Docker image with `OLLAMA_VULKAN=1`.
| Intel GPU | Vulkan | SYCL | Gain |
|---|---|---|---|
| MTL iGPU (155H) | ~8-11 tok/s | **~16 tok/s** | +45-100% |
| ARL-H iGPU | ~10-12 tok/s | **~17 tok/s** | +40-70% |
| Arc A770 | ~30-35 tok/s | **~55 tok/s** | +57-83% |
| Flex 170 | ~30-35 tok/s | **~50 tok/s** | +43-67% |
| Data Center Max 1550 | — | **~73 tok/s** | — |
## Architecture
*Benchmarks: llama-2-7b Q4_0, llama.cpp, community-reported.*
The Docker image builds in two stages:
**What makes SYCL faster:**
1. **Build stage** (`intel/oneapi-basekit:2025.1.1`) — clones ollama v0.15.6
source, fetches the matching `ggml-sycl` backend from upstream llama.cpp
(commit `a5bb8ba4`, the exact ggml version ollama vendors), patches two
ollama-specific API divergences (`batch_size` parameter, `GGML_TENSOR_FLAG_COMPUTE`
removal), and compiles `libggml-sycl.so` with `icpx` + oneAPI.
2. **Runtime stage** (`ubuntu:24.04`) — minimal image with Intel GPU drivers,
the official ollama binary, and the SYCL runner + oneAPI runtime libraries.
- **oneMKL / oneDNN** — Intel's optimized math and neural network libraries
- **Level-Zero** — direct GPU communication, lower overhead than Vulkan
- **Intel-tuned kernels** — MUL_MAT hand-optimized per architecture (MTL, ARL, Arc, Flex, PVC)
### Key components
**When Vulkan makes sense:** no build step, cross-vendor support (AMD/NVIDIA), smaller image. Use the official Ollama Docker image with `OLLAMA_VULKAN=1`.
| Component | Source | Purpose |
|-----------|--------|---------|
| ollama binary | Official v0.15.6 release | Go server, API, model management |
| ggml-sycl backend | llama.cpp @ `a5bb8ba4` | `libggml-sycl.so` compiled with oneAPI |
| oneAPI runtime | Intel oneAPI 2025.1.1 | SYCL runtime, oneMKL, oneDNN, TBB |
| GPU drivers | Intel compute-runtime 26.05 | Level-zero, IGC, OpenCL ICD |
| patch-sycl.py | This repo | Patches ggml-sycl for ollama API compat |
| Web UI | Open WebUI | Browser-based chat interface |
---
## How it works
Ollama ships the `ggml-sycl.h` header but intentionally excludes the SYCL implementation from its vendored ggml. This repo fills that gap:
```
┌─────────────────────────────────────────────────────────┐
│ Stage 1: Build (intel/oneapi-basekit:2025.1.1) │
│ │
│ ollama v0.15.6 source ──┐ │
│ ├── cmake + icpx ── libggml-sycl.so
│ ggml-sycl @ a5bb8ba4 ──┘ │
│ ▲ │
│ └── patch-sycl.py (2 API fixes) │
├─────────────────────────────────────────────────────────┤
│ Stage 2: Runtime (ubuntu:24.04) │
│ │
│ ollama binary (official v0.15.6) │
│ + libggml-sycl.so + oneAPI runtime libs │
│ + Intel GPU drivers (Level-Zero, IGC, compute-runtime) │
│ + Open WebUI (separate container) │
└─────────────────────────────────────────────────────────┘
```
The `ggml-sycl` source is fetched from the **exact llama.cpp commit** (`a5bb8ba4`) that ollama vendors, ensuring ABI compatibility. Two small patches are applied by `patch-sycl.py`:
1. **`graph_compute` signature** — ollama adds an `int batch_size` parameter not present upstream
2. **`GGML_TENSOR_FLAG_COMPUTE` removal** — ollama drops this enum; without the patch, every compute node gets skipped, producing garbage output
---
## Configuration
Key environment variables in `docker-compose.yml`:
Environment variables in `docker-compose.yml`:
| Variable | Default | Description |
|----------|---------|-------------|
|---|---|---|
| `OLLAMA_HOST` | `0.0.0.0` | Listen address |
| `OLLAMA_KEEP_ALIVE` | `24h` | Keep models loaded in memory |
| `OLLAMA_NUM_PARALLEL` | `1` | Parallel request handling |
| `OLLAMA_MAX_LOADED_MODELS` | `1` | Max models in memory |
| `ONEAPI_DEVICE_SELECTOR` | `level_zero:0` | Select Intel GPU device |
| `OLLAMA_KEEP_ALIVE` | `24h` | How long models stay loaded in memory |
| `OLLAMA_NUM_PARALLEL` | `1` | Concurrent request slots |
| `OLLAMA_MAX_LOADED_MODELS` | `1` | Max models in VRAM simultaneously |
| `ONEAPI_DEVICE_SELECTOR` | `level_zero:0` | Which Intel GPU to use |
| `ZES_ENABLE_SYSMAN` | `1` | Enable Level-Zero system management |
| `OLLAMA_DEBUG` | `1` | Verbose logging (disable in production) |
## How the SYCL build works
---
Ollama intentionally excludes `ggml-sycl` from its vendored ggml source tree
(it keeps the header `ggml-sycl.h` but not the implementation). This repo
rebuilds it by:
## Project structure
1. Cloning the ollama source (for the ggml build system and headers)
2. Fetching `ggml-sycl` from the **exact llama.cpp commit** that ollama
vendors (`a5bb8ba4`) to ensure ABI compatibility
3. Applying two patches via `patch-sycl.py`:
- **`graph_compute` signature**: ollama adds an `int batch_size` parameter
- **`GGML_TENSOR_FLAG_COMPUTE`**: ollama removes this enum value, so the
skip-check in the compute loop must be removed (otherwise ALL nodes
get skipped, producing garbage output)
4. Building with Intel oneAPI `icpx` compiler, linking oneMKL and oneDNN
```
.
├── Dockerfile # Multi-stage build: oneAPI SYCL → minimal runtime
├── docker-compose.yml # ollama + Open WebUI services
├── patch-sycl.py # Patches ggml-sycl for ollama API compatibility
├── start-ollama.sh # Custom entrypoint (legacy, from IPEX-LLM era)
└── doc/
└── screenshot.png
```
---
## Troubleshooting
**SYCL device not detected** — Ensure `/dev/dri` is accessible. Check `docker compose logs ollama-intel-gpu` for `SYCL0` in the device list.
**"failed to sample token"** — Usually means an ABI mismatch between ggml-sycl and ollama's vendored ggml. The `GGML_COMMIT` ARG in the Dockerfile must match the ggml version ollama vendors.
**Model too large for VRAM** — Intel integrated GPUs share system memory. Increase `shm_size` in `docker-compose.yml` or use a smaller quantization (Q4_0, Q4_K_M).
**Slow first inference** — SYCL JIT-compiles GPU kernels on first run. Subsequent inferences are faster.
---
## References
* [Intel GPU driver installation](https://dgpu-docs.intel.com/driver/client/overview.html)
* [llama.cpp SYCL backend docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md)
* [Intel oneAPI base toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html)
* [ollama GitHub](https://github.com/ollama/ollama)
* [Open WebUI](https://github.com/open-webui/open-webui)
- [Ollama](https://github.com/ollama/ollama)
- [Open WebUI](https://github.com/open-webui/open-webui)
- [llama.cpp SYCL backend](https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md)
- [Intel oneAPI base toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html)
- [Intel GPU driver installation](https://dgpu-docs.intel.com/driver/client/overview.html)
---
## License
See [LICENSE](LICENSE) for details.