matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-11 00:07:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

lock on llm.lock(); decrease batch size

Browse Source
This commit is contained in:
Michael Yang
2023-07-20 09:29:43 -07:00
parent f62a882760
commit c490416189
2 changed files with 11 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
api/types.go
View File

@@ -177,7 +177,7 @@ func DefaultOptions() Options {
UseNUMA: false,
NumCtx: 2048,
NumBatch: 512,
NumBatch: 32,
NumGPU: 1,
LowVRAM: false,
F16KV: true,