3 Commits

Author	SHA1	Message	Date
Daniel Hiltgen	d74ce6bd4f	Detect very old CUDA GPUs and fall back to CPU ... If we try to load the CUDA library on an old GPU, it panics and crashes the server. This checks the compute capability before we load the library so we can gracefully fall back to CPU mode.	2024-01-06 21:40:29 -08:00
Jeffrey Morgan	df32537312	gpu: read memory info from all cuda devices (#1802) ... * gpu: read memory info from all cuda devices * add `LOOKUP_SIZE` constant * better constant name * address comments	2024-01-05 11:25:58 -05:00
Daniel Hiltgen	35934b2e05	Adapted rocm support to cgo based llama.cpp	2023-12-19 09:05:46 -08:00