eSlider/ollama-intel-gpu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Revert to ipex-llm version of ollama for gpu acceleration

Browse Source
This commit is contained in:
Matt Curfman
2024-08-16 22:41:15 -07:00
parent aec98f6f30
commit 40313a7364
4 changed files with 17 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -2
View File
@@ -30,6 +30,9 @@ $ docker-compose -f docker-compose-wsl2.yml up
Then launch your web browser to http://localhost:3000 to launch the web ui. Create a local OpenWeb UI credential, then click the settings icon in the top right of the screen, then select 'Models', then click 'Show', then download a model like 'llama3.1:8b-instruct-q8_0' for Intel ARC A770 16GB VRAM
# Known issues
* Little effort has been made to prune the packages pulled into the Ollama docker image for Intel GPU
# References
* https://github.com/ollama/ollama/issues/1590
* https://github.com/ollama/ollama/pull/3278
* https://dgpu-docs.intel.com/driver/client/overview.html
* https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/ollama_quickstart.html