matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-19 20:27:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
f25016b7d5865725ea4e5c7fd45eae4ca0efdb2e
ollama37/docs/favicon-dark.svg
Shang Chieh Tseng ef14fb5b26 Sync with upstream ollama/ollama and restore Tesla K80 (compute 3.7) support 
This commit represents a complete rework after pulling the latest changes from
official ollama/ollama repository and re-applying Tesla K80 compatibility patches.

## Key Changes

### CUDA Compute Capability 3.7 Support (Tesla K80)
- Added sm_37 (compute 3.7) to CMAKE_CUDA_ARCHITECTURES in CMakeLists.txt
- Updated CMakePresets.json to include compute 3.7 in "CUDA 11" preset
- Using 37-virtual (PTX with JIT compilation) for maximum compatibility

### Legacy Toolchain Compatibility
- **NVIDIA Driver**: 470.256.02 (last version supporting Kepler/K80)
- **CUDA Version**: 11.4.4 (last CUDA 11.x supporting compute 3.7)
- **GCC Version**: 10.5.0 (required by CUDA 11.4 host_config.h)

### CPU Architecture Trade-offs
Due to GCC 10.5 limitation, sacrificed newer CPU optimizations:
- Alderlake CPU variant enabled WITHOUT AVX_VNNI (requires GCC 11+)
- Still supports: SSE4.2, AVX, F16C, AVX2, BMI2, FMA
- Performance impact: ~3-7% on newer CPUs (acceptable for K80 compatibility)

### Build System Updates
- Modified ml/backend/ggml/ggml/src/ggml-cuda/CMakeLists.txt for compute 3.7
- Added -Wno-deprecated-gpu-targets flag to suppress warnings
- Updated ml/backend/ggml/ggml/src/CMakeLists.txt for Alderlake without AVX_VNNI

### Upstream Sync
Merged latest llama.cpp changes including:
- Enhanced KV cache management with ISWA and hybrid memory support
- Improved multi-modal support (mtmd framework)
- New model architectures (Gemma3, Llama4, Qwen3, etc.)
- GPU backend improvements for CUDA, Metal, and ROCm
- Updated quantization support and GGUF format handling

### Documentation
- Updated CLAUDE.md with comprehensive build instructions
- Documented toolchain constraints and CPU architecture trade-offs
- Removed outdated CI/CD workflows (tesla-k80-*.yml)
- Cleaned up temporary development artifacts

## Rationale

This fork maintains Tesla K80 GPU support (compute 3.7) which was dropped in
official Ollama due to legacy driver/CUDA requirements. The toolchain constraint
creates a deadlock:
- K80 → Driver 470 → CUDA 11.4 → GCC 10 → No AVX_VNNI

We accept the loss of cutting-edge CPU optimizations to enable running modern
LLMs on legacy but still capable Tesla K80 hardware (12GB VRAM per GPU).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 14:03:05 +08:00

4 lines
6.7 KiB
XML
Raw Blame History

<svg width="28" height="28" viewBox="0 0 28 28" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M5.92343 3.10209C6.16484 3.19868 6.38278 3.35777 6.58061 3.56799C6.91031 3.91571 7.18861 4.41342 7.40096 5.00318C7.61443 5.59635 7.75302 6.25316 7.80555 6.91224C8.50922 6.5075 9.29019 6.26103 10.0956 6.18953L10.1526 6.18498C11.125 6.10544 12.0862 6.28384 12.9244 6.72361C13.0373 6.78384 13.1479 6.84861 13.2563 6.91679C13.3122 6.27021 13.4486 5.62817 13.6587 5.04863C13.871 4.45773 14.1493 3.96117 14.4779 3.61231C14.6616 3.41034 14.8862 3.25137 15.1362 3.14641C15.4235 3.03277 15.7286 3.01232 16.0259 3.09868C16.474 3.22822 16.8585 3.51685 17.1614 3.93617C17.4386 4.3191 17.6465 4.81 17.7884 5.39863C18.0455 6.45998 18.0902 7.85655 17.9169 9.54061L17.9762 9.58607L18.0052 9.60766C18.8513 10.2622 19.4403 11.1951 19.7522 12.2781C20.2383 13.9678 19.9936 15.8632 19.1553 16.9235L19.1352 16.9473L19.1374 16.9507C19.6035 17.8166 19.8863 18.7314 19.9466 19.678L19.9489 19.7121C20.0204 20.9223 19.7253 22.1404 19.0391 23.337L19.0312 23.3484L19.0424 23.3756C19.57 24.6904 19.7354 26.0142 19.532 27.3369L19.5253 27.3813C19.4938 27.5744 19.3881 27.7472 19.2316 27.8612C19.0751 27.9753 18.8805 28.0219 18.6903 27.9903C18.5962 27.9753 18.5059 27.9417 18.4246 27.8912C18.3433 27.8408 18.2725 27.7744 18.2165 27.696C18.1604 27.6178 18.1201 27.529 18.0979 27.4347C18.0756 27.3406 18.0718 27.2428 18.0868 27.1472C18.2735 25.9733 18.098 24.7961 17.5503 23.5984C17.4992 23.4871 17.4766 23.3645 17.4844 23.242C17.4922 23.1194 17.5302 23.0009 17.5951 22.8972L17.5995 22.8904C18.2746 21.8404 18.554 20.8109 18.4936 19.7996C18.4422 18.9143 18.1304 18.045 17.5995 17.2166C17.4963 17.0556 17.4595 16.8597 17.4971 16.6713C17.5348 16.4829 17.6438 16.3171 17.8007 16.2098L17.8108 16.203C18.0823 16.0223 18.3327 15.561 18.459 14.9303C18.5983 14.1846 18.562 13.4159 18.3528 12.6872C18.1237 11.8917 17.7046 11.2281 17.1178 10.7747C16.4528 10.2588 15.5721 10.0099 14.4578 10.0815C14.3121 10.0911 14.1669 10.0561 14.041 9.98097C13.9151 9.90581 13.8142 9.79399 13.7515 9.65993C13.4005 8.90426 12.8886 8.36336 12.2504 8.02814C11.6378 7.71723 10.9518 7.58657 10.27 7.65087C8.8785 7.76337 7.65132 8.56108 7.28584 9.56675C7.23414 9.70829 7.14113 9.8304 7.01928 9.91672C6.89744 10.003 6.75257 10.0494 6.60408 10.0497C5.41154 10.052 4.48836 10.3361 3.8133 10.8485C3.22989 11.2917 2.83201 11.911 2.62189 12.6531C2.43175 13.3515 2.40571 14.0857 2.54589 14.7962C2.67107 15.4303 2.91583 15.9553 3.19636 16.2382L3.2053 16.2462C3.44224 16.4814 3.49254 16.8485 3.32713 17.1382C2.92477 17.845 2.62412 18.8984 2.57495 19.9109C2.51906 21.0677 2.78283 22.0722 3.37854 22.7927L3.39642 22.8143C3.48632 22.9208 3.54413 23.0514 3.56297 23.1905C3.58179 23.3295 3.56083 23.4711 3.5026 23.5984C2.85883 25.0029 2.66101 26.1574 2.87448 27.0665C2.91284 27.2556 2.87714 27.4524 2.77496 27.6149C2.67279 27.7776 2.5122 27.8933 2.32739 27.9374C2.14257 27.9817 1.94812 27.9506 1.78542 27.8512C1.62269 27.7517 1.50458 27.5917 1.45618 27.4051C1.18459 26.2483 1.36901 24.9233 1.98483 23.4302L2.00048 23.3904L1.99153 23.3768C1.68885 22.9222 1.46298 22.4195 1.32318 21.8893L1.31759 21.8677C1.14793 21.2062 1.08119 20.5218 1.11977 19.8393C1.16895 18.8052 1.43048 17.7462 1.81495 16.8962L1.82836 16.8666L1.82612 16.8644C1.49865 16.3894 1.25612 15.7814 1.122 15.1087L1.11642 15.0814C0.931624 14.1431 0.967232 13.1737 1.22036 12.2519C1.51318 11.2122 2.08877 10.319 2.93707 9.67356C3.00413 9.62243 3.07454 9.57129 3.14495 9.52357C2.96724 7.827 3.01195 6.42134 3.27013 5.35317C3.41207 4.76455 3.62107 4.27365 3.89825 3.89071C4.20001 3.47254 4.58448 3.18391 5.03266 3.05323C5.32996 2.96686 5.63619 2.98732 5.92343 3.10209ZM10.5237 13.4315C11.5698 13.4315 12.5354 13.7871 13.2575 14.403C13.9616 15.0019 14.3807 15.8064 14.3807 16.6076C14.3807 17.6166 13.9269 18.403 13.1144 18.9052C12.4214 19.3314 11.4927 19.5382 10.4287 19.5382C9.30097 19.5382 8.33755 19.2439 7.64237 18.7041C6.95278 18.17 6.56608 17.42 6.56608 16.6076C6.56608 15.8042 7.0109 14.9973 7.74632 14.3962C8.49291 13.786 9.47867 13.4315 10.5237 13.4315ZM10.5237 14.4496C9.7483 14.4428 8.99346 14.7031 8.38226 15.1883C7.86702 15.6087 7.57532 16.1371 7.57532 16.6087C7.57532 17.095 7.81002 17.5507 8.25708 17.8973C8.76561 18.2916 9.51332 18.52 10.4287 18.52C11.3217 18.52 12.075 18.353 12.588 18.0359C13.1054 17.7178 13.3703 17.2564 13.3703 16.6076C13.3703 16.1269 13.0954 15.5962 12.607 15.1803C12.066 14.7201 11.3329 14.4496 10.5237 14.4496ZM11.2636 15.8246L11.268 15.8292C11.4021 16.0007 11.3742 16.2496 11.2054 16.386L10.8791 16.6473V17.1541C10.8785 17.267 10.8339 17.375 10.7551 17.4544C10.6764 17.5339 10.5698 17.5783 10.4589 17.578C10.3479 17.5783 10.2413 17.5339 10.1626 17.4544C10.0838 17.375 10.0392 17.267 10.0386 17.1541V16.6314L9.73573 16.3837C9.69577 16.3511 9.66255 16.3109 9.63798 16.2652C9.61342 16.2195 9.59799 16.1694 9.59258 16.1176C9.58718 16.0658 9.59191 16.0135 9.60651 15.9636C9.62111 15.9137 9.64527 15.8672 9.67762 15.8269C9.74358 15.7452 9.83858 15.6934 9.94196 15.6825C10.0453 15.6717 10.1487 15.7027 10.2297 15.7689L10.47 15.9644L10.7159 15.7667C10.7966 15.7018 10.899 15.6716 11.0013 15.6825C11.1036 15.6933 11.1977 15.7443 11.2636 15.8246ZM5.6306 13.644C6.16484 13.644 6.59961 14.0871 6.59961 14.6337C6.5999 14.8957 6.4979 15.1471 6.31599 15.3327C6.13408 15.5183 5.88717 15.6229 5.62949 15.6235C5.37218 15.6226 5.12572 15.518 4.9441 15.3327C4.76246 15.1474 4.66048 14.8965 4.66048 14.6349C4.65988 14.3729 4.76161 14.1213 4.94329 13.9355C5.125 13.7497 5.37292 13.6449 5.6306 13.644ZM15.3609 13.644C15.8973 13.644 16.331 14.0871 16.331 14.6337C16.3313 14.8957 16.2293 15.1471 16.0474 15.3327C15.8655 15.5183 15.6186 15.6229 15.3609 15.6235C15.1036 15.6226 14.8571 15.518 14.6755 15.3327C14.4938 15.1474 14.3919 14.8965 14.3919 14.6349C14.3913 14.3729 14.493 14.1213 14.6747 13.9355C14.8564 13.7497 15.1032 13.6449 15.3609 13.644ZM5.40372 4.47705L5.40037 4.47932C5.27088 4.53657 5.16032 4.63046 5.08184 4.74977L5.07625 4.75659C4.92201 4.97136 4.7879 5.28727 4.68731 5.70203C4.49731 6.48839 4.44589 7.55541 4.54872 8.86335C5.02931 8.7179 5.55349 8.62699 6.1179 8.59404L6.12908 8.5929L6.15031 8.55426C6.20172 8.46108 6.25649 8.37131 6.31572 8.28268C6.45319 7.40655 6.34031 6.35998 6.03296 5.50545C5.88319 5.09182 5.70102 4.76682 5.52666 4.5816C5.49067 4.5431 5.45055 4.5088 5.40707 4.47932L5.40372 4.47705ZM15.657 4.52251L15.6548 4.52364C15.6113 4.55312 15.5712 4.58741 15.5352 4.62591C15.3609 4.81114 15.1776 5.13727 15.0289 5.5509C14.7048 6.45316 14.5964 7.56905 14.7719 8.47358L14.8367 8.58381L14.8456 8.59972H14.8792C15.4339 8.59988 15.9856 8.68101 16.5176 8.84062C16.6138 7.56337 16.5601 6.51907 16.3746 5.74749C16.274 5.33272 16.1399 5.01682 15.9845 4.80205L15.98 4.79523C15.9017 4.67548 15.7911 4.58121 15.6615 4.52364L15.657 4.52251Z" fill="black"/>
</svg>
Reference in New Issue View Git Blame Copy Permalink