matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-11 00:07:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

gpu: read memory info from all cuda devices (#1802)

Browse Source 
* gpu: read memory info from all cuda devices

* add `LOOKUP_SIZE` constant

* better constant name

* address comments
This commit is contained in:
Jeffrey Morgan
2024-01-05 11:25:58 -05:00
committed by GitHub
parent 3367b5f3df
commit df32537312
2 changed files with 29 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
gpu/gpu_info_cuda.h
View File

@@ -21,6 +21,7 @@ typedef struct cuda_handle {
nvmlReturn_t (*shutdownFn)(void);
nvmlReturn_t (*getHandle)(unsigned int, nvmlDevice_t *);
nvmlReturn_t (*getMemInfo)(nvmlDevice_t, nvmlMemory_t *);
nvmlReturn_t (*getCount)(unsigned int *);
} cuda_handle_t;
typedef struct cuda_init_resp {