matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-10 07:46:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
ollama37/docs/favicon.svg
Shang Chieh Tseng ef14fb5b26 Sync with upstream ollama/ollama and restore Tesla K80 (compute 3.7) support 
This commit represents a complete rework after pulling the latest changes from
official ollama/ollama repository and re-applying Tesla K80 compatibility patches.

## Key Changes

### CUDA Compute Capability 3.7 Support (Tesla K80)
- Added sm_37 (compute 3.7) to CMAKE_CUDA_ARCHITECTURES in CMakeLists.txt
- Updated CMakePresets.json to include compute 3.7 in "CUDA 11" preset
- Using 37-virtual (PTX with JIT compilation) for maximum compatibility

### Legacy Toolchain Compatibility
- **NVIDIA Driver**: 470.256.02 (last version supporting Kepler/K80)
- **CUDA Version**: 11.4.4 (last CUDA 11.x supporting compute 3.7)
- **GCC Version**: 10.5.0 (required by CUDA 11.4 host_config.h)

### CPU Architecture Trade-offs
Due to GCC 10.5 limitation, sacrificed newer CPU optimizations:
- Alderlake CPU variant enabled WITHOUT AVX_VNNI (requires GCC 11+)
- Still supports: SSE4.2, AVX, F16C, AVX2, BMI2, FMA
- Performance impact: ~3-7% on newer CPUs (acceptable for K80 compatibility)

### Build System Updates
- Modified ml/backend/ggml/ggml/src/ggml-cuda/CMakeLists.txt for compute 3.7
- Added -Wno-deprecated-gpu-targets flag to suppress warnings
- Updated ml/backend/ggml/ggml/src/CMakeLists.txt for Alderlake without AVX_VNNI

### Upstream Sync
Merged latest llama.cpp changes including:
- Enhanced KV cache management with ISWA and hybrid memory support
- Improved multi-modal support (mtmd framework)
- New model architectures (Gemma3, Llama4, Qwen3, etc.)
- GPU backend improvements for CUDA, Metal, and ROCm
- Updated quantization support and GGUF format handling

### Documentation
- Updated CLAUDE.md with comprehensive build instructions
- Documented toolchain constraints and CPU architecture trade-offs
- Removed outdated CI/CD workflows (tesla-k80-*.yml)
- Cleaned up temporary development artifacts

## Rationale

This fork maintains Tesla K80 GPU support (compute 3.7) which was dropped in
official Ollama due to legacy driver/CUDA requirements. The toolchain constraint
creates a deadlock:
- K80 → Driver 470 → CUDA 11.4 → GCC 10 → No AVX_VNNI

We accept the loss of cutting-edge CPU optimizations to enable running modern
LLMs on legacy but still capable Tesla K80 hardware (12GB VRAM per GPU).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 14:03:05 +08:00

4 lines
6.5 KiB
XML
Raw Permalink Blame History

<svg width="10" height="12" viewBox="0 0 10 12" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M2.95258 0.545107C3.06058 0.587607 3.15808 0.657607 3.24658 0.750107C3.39408 0.903107 3.51858 1.1221 3.61358 1.3816C3.70908 1.6426 3.77108 1.9316 3.79458 2.2216C4.10938 2.04351 4.45876 1.93506 4.81908 1.9036L4.84458 1.9016C5.27958 1.8666 5.70958 1.9451 6.08458 2.1386C6.13508 2.1651 6.18458 2.1936 6.23308 2.2236C6.25808 1.9391 6.31908 1.6566 6.41308 1.4016C6.50808 1.1416 6.63258 0.923107 6.77958 0.769607C6.86173 0.680739 6.96222 0.610791 7.07408 0.564607C7.20258 0.514607 7.33908 0.505607 7.47208 0.543607C7.67258 0.600607 7.84458 0.727607 7.98008 0.912107C8.10408 1.0806 8.19708 1.2966 8.26058 1.5556C8.37558 2.0226 8.39558 2.6371 8.31808 3.3781L8.34458 3.3981L8.35758 3.4076C8.73608 3.6956 8.99958 4.1061 9.1391 4.5826C9.3566 5.3261 9.2471 6.1601 8.87208 6.6266L8.86308 6.6371L8.86408 6.6386C9.0726 7.0196 9.1991 7.4221 9.2261 7.8386L9.2271 7.8536C9.2591 8.3861 9.1271 8.9221 8.82008 9.4486L8.81658 9.4536L8.82158 9.4656C9.0576 10.0441 9.1316 10.6266 9.0406 11.2086L9.0376 11.2281C9.0235 11.3131 8.97624 11.3891 8.90623 11.4393C8.83621 11.4895 8.74913 11.51 8.66408 11.4961C8.62196 11.4895 8.58156 11.4747 8.54518 11.4525C8.50881 11.4303 8.47717 11.4011 8.45209 11.3666C8.42701 11.3322 8.40897 11.2931 8.39902 11.2516C8.38906 11.2102 8.38738 11.1672 8.39408 11.1251C8.47758 10.6086 8.39908 10.0906 8.15408 9.56361C8.13122 9.51465 8.12107 9.46071 8.12456 9.40679C8.12806 9.35287 8.14509 9.3007 8.17408 9.2551L8.17608 9.2521C8.47808 8.7901 8.60308 8.3371 8.57608 7.8921C8.55308 7.5026 8.41358 7.1201 8.17608 6.7556C8.12989 6.68474 8.11343 6.59855 8.13027 6.51566C8.14711 6.43276 8.1959 6.35982 8.26608 6.3126L8.27058 6.3096C8.39208 6.2301 8.50408 6.0271 8.56058 5.7496C8.62292 5.42147 8.60664 5.08324 8.51308 4.7626C8.41058 4.4126 8.22308 4.1206 7.96058 3.9211C7.66308 3.6941 7.26908 3.5846 6.77058 3.6161C6.70539 3.62034 6.64044 3.60493 6.5841 3.57186C6.52777 3.53879 6.48265 3.48959 6.45458 3.4306C6.29758 3.0981 6.06858 2.8601 5.78308 2.7126C5.50898 2.5758 5.20211 2.51831 4.89708 2.5466C4.27458 2.5961 3.72558 2.9471 3.56208 3.3896C3.53895 3.45188 3.49734 3.50561 3.44283 3.54359C3.38832 3.58156 3.32351 3.60198 3.25708 3.6021C2.72358 3.6031 2.31058 3.7281 2.00858 3.9536C1.74758 4.1486 1.56958 4.4211 1.47558 4.7476C1.39052 5.05493 1.37887 5.37795 1.44158 5.6906C1.49758 5.9696 1.60708 6.2006 1.73258 6.3251L1.73658 6.3286C1.84258 6.4321 1.86508 6.5936 1.79108 6.7211C1.61108 7.0321 1.47658 7.4956 1.45458 7.9411C1.42958 8.4501 1.54758 8.8921 1.81408 9.2091L1.82208 9.2186C1.8623 9.26547 1.88816 9.32293 1.89659 9.38412C1.90501 9.4453 1.89563 9.50761 1.86958 9.56361C1.58158 10.1816 1.49308 10.6896 1.58858 11.0896C1.60574 11.1728 1.58977 11.2594 1.54406 11.3309C1.49835 11.4025 1.42651 11.4534 1.34383 11.4728C1.26115 11.4923 1.17416 11.4786 1.10137 11.4349C1.02857 11.3911 0.975731 11.3207 0.954081 11.2386C0.832581 10.7296 0.915081 10.1466 1.19058 9.4896L1.19758 9.4721L1.19358 9.4661C1.05817 9.2661 0.957121 9.04489 0.894581 8.8116L0.892081 8.8021C0.816181 8.51105 0.786321 8.2099 0.803581 7.9096C0.825581 7.4546 0.942581 6.9886 1.11458 6.6146L1.12058 6.6016L1.11958 6.6006C0.973081 6.3916 0.864581 6.1241 0.804581 5.8281L0.802081 5.8161C0.719411 5.40323 0.735341 4.97667 0.848581 4.5711C0.979581 4.1136 1.23708 3.7206 1.61658 3.4366C1.64658 3.4141 1.67808 3.3916 1.70958 3.3706C1.63008 2.6241 1.65008 2.0056 1.76558 1.5356C1.82908 1.2766 1.92258 1.0606 2.04658 0.892107C2.18158 0.708107 2.35358 0.581107 2.55408 0.523607C2.68708 0.485607 2.82408 0.494607 2.95258 0.545107ZM5.01058 5.0901C5.47858 5.0901 5.91058 5.2466 6.23358 5.5176C6.54858 5.7811 6.73608 6.1351 6.73608 6.4876C6.73608 6.9316 6.53308 7.2776 6.16958 7.4986C5.85958 7.6861 5.44408 7.7771 4.96808 7.7771C4.46358 7.7771 4.03258 7.6476 3.72158 7.4101C3.41308 7.1751 3.24008 6.8451 3.24008 6.4876C3.24008 6.1341 3.43908 5.7791 3.76808 5.5146C4.10208 5.2461 4.54308 5.0901 5.01058 5.0901ZM5.01058 5.5381C4.6637 5.53508 4.32601 5.64963 4.05258 5.8631C3.82208 6.0481 3.69158 6.2806 3.69158 6.4881C3.69158 6.7021 3.79658 6.9026 3.99658 7.0551C4.22408 7.2286 4.55858 7.3291 4.96808 7.3291C5.36758 7.3291 5.70458 7.2556 5.93408 7.1161C6.16558 6.9761 6.28408 6.7731 6.28408 6.4876C6.28408 6.2761 6.16108 6.0426 5.94258 5.8596C5.70058 5.6571 5.37258 5.5381 5.01058 5.5381ZM5.34158 6.1431L5.34358 6.1451C5.40358 6.2206 5.39108 6.3301 5.31558 6.3901L5.16958 6.5051V6.7281C5.16931 6.77775 5.14937 6.82526 5.11413 6.86022C5.07889 6.89518 5.03122 6.91474 4.98158 6.9146C4.93194 6.91474 4.88427 6.89518 4.84903 6.86022C4.81378 6.82526 4.79384 6.77775 4.79358 6.7281V6.4981L4.65808 6.3891C4.6402 6.37478 4.62534 6.35706 4.61435 6.33697C4.60336 6.31687 4.59646 6.2948 4.59404 6.27202C4.59162 6.24924 4.59374 6.22621 4.60027 6.20426C4.6068 6.1823 4.61761 6.16186 4.63208 6.1441C4.66159 6.10818 4.70409 6.08535 4.75034 6.08057C4.79658 6.0758 4.84285 6.08947 4.87908 6.1186L4.98658 6.2046L5.09658 6.1176C5.13268 6.08908 5.1785 6.07579 5.22426 6.08055C5.27003 6.08531 5.31212 6.10776 5.34158 6.1431ZM2.82158 5.1836C3.06058 5.1836 3.25508 5.3786 3.25508 5.6191C3.25521 5.73439 3.20958 5.84501 3.1282 5.92667C3.04682 6.00832 2.93636 6.05434 2.82108 6.0546C2.70597 6.05421 2.59571 6.0082 2.51446 5.92667C2.4332 5.84513 2.38758 5.73471 2.38758 5.6196C2.38731 5.50432 2.43282 5.39365 2.5141 5.3119C2.59539 5.23014 2.7063 5.184 2.82158 5.1836ZM7.17458 5.1836C7.41458 5.1836 7.60858 5.3786 7.60858 5.6191C7.60871 5.73439 7.56308 5.84501 7.4817 5.92667C7.40032 6.00832 7.28986 6.05434 7.17458 6.0546C7.05947 6.05421 6.94921 6.0082 6.86796 5.92667C6.7867 5.84513 6.74108 5.73471 6.74108 5.6196C6.74081 5.50432 6.78632 5.39365 6.8676 5.3119C6.94889 5.23014 7.0593 5.184 7.17458 5.1836ZM2.72008 1.1501L2.71858 1.1511C2.66065 1.17629 2.61119 1.2176 2.57608 1.2701L2.57358 1.2731C2.50458 1.3676 2.44458 1.5066 2.39958 1.6891C2.31458 2.0351 2.29158 2.5046 2.33758 3.0801C2.55258 3.0161 2.78708 2.9761 3.03958 2.9616L3.04458 2.9611L3.05408 2.9441C3.07708 2.9031 3.10158 2.8636 3.12808 2.8246C3.18958 2.4391 3.13908 1.9786 3.00158 1.6026C2.93458 1.4206 2.85308 1.2776 2.77508 1.1961C2.75898 1.17916 2.74103 1.16407 2.72158 1.1511L2.72008 1.1501ZM7.30708 1.1701L7.30608 1.1706C7.28663 1.18357 7.26868 1.19866 7.25258 1.2156C7.17458 1.2971 7.09258 1.4406 7.02608 1.6226C6.88108 2.0196 6.83258 2.5106 6.91108 2.9086L6.94008 2.9571L6.94408 2.9641H6.95908C7.20723 2.96417 7.45408 2.99987 7.69208 3.0701C7.73508 2.5081 7.71108 2.0486 7.62808 1.7091C7.58308 1.5266 7.52308 1.3876 7.45358 1.2931L7.45158 1.2901C7.41654 1.23741 7.36707 1.19593 7.30908 1.1706L7.30708 1.1701Z" fill="white"/>
</svg>
Reference in New Issue View Git Blame Copy Permalink