llm: avoid loading model if system memory is too small (#5637)

* llm: avoid loading model if system memory is too small

* update log

* Instrument swap free space

On linux and windows, expose how much swap space is available
so we can take that into consideration when scheduling models

* use `systemSwapFreeMemory` in check

---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
This commit is contained in:
Jeffrey Morgan
2024-07-11 16:42:57 -07:00
committed by GitHub
parent 791650ddef
commit c4cf8ad559
6 changed files with 22 additions and 13 deletions

View File

@@ -10,6 +10,7 @@ import (
type memInfo struct {
TotalMemory uint64 `json:"total_memory,omitempty"`
FreeMemory uint64 `json:"free_memory,omitempty"`
FreeSwap uint64 `json:"free_swap,omitempty"`
}
// Beginning of an `ollama info` command