matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-10 07:46:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
764e199d6703d80da4d245381efa4a3a412813b2
ollama37/sample
History
Parth Sareen 0682dae027 sample: improve ollama engine sampler performance (#9374) 
This change bring in various interface cleanups along with greatly improving the performance of the sampler.

Tested with llama3.2 on local machine.
Improves performance from ~ 70 tokens/s -> 135 tokens/s with topK(40) enabled.
Without topK performance is ~ 110 tokens/s
2025-03-07 12:37:48 -08:00
..
samplers_benchmark_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
samplers_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
samplers.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
transforms_test.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00
transforms.go
sample: improve ollama engine sampler performance (#9374)
2025-03-07 12:37:48 -08:00