Daniel Hiltgen 667a2ba18a Detect lack of AVX and fallback to CPU mode ...

We build the GPU libraries with AVX enabled to ensure that if not all
layers fit on the GPU we get better performance in a mixed mode.
If the user is using a virtualization/emulation system that lacks AVX
this used to result in an illegal instruction error and crash before this
fix.  Now we will report a warning in the server log, and just use
CPU mode to ensure we don't crash.

2024-01-26 11:36:03 -08:00

..

cpu_common.go

Mechanical switch from log to slog

2024-01-18 14:12:57 -08:00

gpu_darwin.go

Fix up the CPU fallback selection

2024-01-11 15:27:06 -08:00

gpu_info_cpu.c

calculate overhead based number of gpu devices (#1875)

2024-01-09 15:53:33 -05:00

gpu_info_cuda.c

Fix crash on cuda ml init failure

2024-01-26 09:18:33 -08:00

gpu_info_cuda.h

More logging for gpu management

2024-01-24 10:32:36 -08:00

gpu_info_rocm.c

Ignore AMD integrated GPUs

2024-01-26 09:21:35 -08:00

gpu_info_rocm.h

More logging for gpu management

2024-01-24 10:32:36 -08:00

gpu_info.h

Ignore AMD integrated GPUs

2024-01-26 09:21:35 -08:00

gpu_test.go

Merge pull request #1819 from dhiltgen/multi_variant

2024-01-11 14:00:48 -08:00

gpu.go

Detect lack of AVX and fallback to CPU mode

2024-01-26 11:36:03 -08:00

types.go

Support multiple variants for a given llm lib type

2024-01-10 17:27:51 -08:00