4 Commits

Author	SHA1	Message	Date
Sam	1bdab9fdb1	llm: introduce k/v context quantization (vRAM improvements) (#6279)	2024-12-03 15:57:19 -08:00
Daniel Hiltgen	df011054fa	Jetpack support for Go server (#7217) ... This adds support for the Jetson JetPack variants into the Go runner	2024-11-12 10:31:52 -08:00
Daniel Hiltgen	16f4eabe2d	Refine default thread selection for NUMA systems (#7322) ... Until we have full NUMA support, this adjusts the default thread selection algorithm to count up the number of performance cores across all sockets.	2024-10-30 15:05:45 -07:00
Daniel Hiltgen	05cd82ef94	Rename gpu package discover (#7143) ... Cleaning up go package naming	2024-10-16 17:45:00 -07:00