eSlider/ollama-intel-gpu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Stop building the ollama image manually from scratch and use the official intel image

Browse Source
This commit is contained in:
eleiton
2025-03-11 22:41:02 +01:00
parent 8f6f5ee0fd
commit 316eb23905
3 changed files with 22 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/serve.sh
+6 -12
View File
@@ -1,15 +1,9 @@
#!/bin/sh
# Reference: https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/ollama_quickstart.md#3-run-ollama-serve
export OLLAMA_NUM_GPU=999
export no_proxy=localhost,127.0.0.1
export ZES_ENABLE_SYSMAN=1
cd /llm/scripts/
source ipex-llm-init --gpu --device Arc
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
# [optional] under most circumstances, the following environment variable may improve performance, but sometimes this may also cause performance degradation
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
# [optional] if you want to run on single GPU, use below command to limit GPU may improve performance
export ONEAPI_DEVICE_SELECTOR=level_zero:0
/usr/local/lib/python3.12/dist-packages/bigdl/cpp/libs/ollama serve
mkdir -p /llm/ollama
cd /llm/ollama
init-ollama
./ollama serve