mirror of
https://github.com/dogkeeper886/ollama37.git
synced 2025-12-09 23:37:06 +00:00
ef14fb5b26
This commit represents a complete rework after pulling the latest changes from official ollama/ollama repository and re-applying Tesla K80 compatibility patches. ## Key Changes ### CUDA Compute Capability 3.7 Support (Tesla K80) - Added sm_37 (compute 3.7) to CMAKE_CUDA_ARCHITECTURES in CMakeLists.txt - Updated CMakePresets.json to include compute 3.7 in "CUDA 11" preset - Using 37-virtual (PTX with JIT compilation) for maximum compatibility ### Legacy Toolchain Compatibility - **NVIDIA Driver**: 470.256.02 (last version supporting Kepler/K80) - **CUDA Version**: 11.4.4 (last CUDA 11.x supporting compute 3.7) - **GCC Version**: 10.5.0 (required by CUDA 11.4 host_config.h) ### CPU Architecture Trade-offs Due to GCC 10.5 limitation, sacrificed newer CPU optimizations: - Alderlake CPU variant enabled WITHOUT AVX_VNNI (requires GCC 11+) - Still supports: SSE4.2, AVX, F16C, AVX2, BMI2, FMA - Performance impact: ~3-7% on newer CPUs (acceptable for K80 compatibility) ### Build System Updates - Modified ml/backend/ggml/ggml/src/ggml-cuda/CMakeLists.txt for compute 3.7 - Added -Wno-deprecated-gpu-targets flag to suppress warnings - Updated ml/backend/ggml/ggml/src/CMakeLists.txt for Alderlake without AVX_VNNI ### Upstream Sync Merged latest llama.cpp changes including: - Enhanced KV cache management with ISWA and hybrid memory support - Improved multi-modal support (mtmd framework) - New model architectures (Gemma3, Llama4, Qwen3, etc.) - GPU backend improvements for CUDA, Metal, and ROCm - Updated quantization support and GGUF format handling ### Documentation - Updated CLAUDE.md with comprehensive build instructions - Documented toolchain constraints and CPU architecture trade-offs - Removed outdated CI/CD workflows (tesla-k80-*.yml) - Cleaned up temporary development artifacts ## Rationale This fork maintains Tesla K80 GPU support (compute 3.7) which was dropped in official Ollama due to legacy driver/CUDA requirements. The toolchain constraint creates a deadlock: - K80 → Driver 470 → CUDA 11.4 → GCC 10 → No AVX_VNNI We accept the loss of cutting-edge CPU optimizations to enable running modern LLMs on legacy but still capable Tesla K80 hardware (12GB VRAM per GPU). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
92 lines
4.1 KiB
Plaintext
92 lines
4.1 KiB
Plaintext
---
|
|
title: Windows
|
|
---
|
|
|
|
Ollama runs as a native Windows application, including NVIDIA and AMD Radeon GPU support.
|
|
After installing Ollama for Windows, Ollama will run in the background and
|
|
the `ollama` command line is available in `cmd`, `powershell` or your favorite
|
|
terminal application. As usual the Ollama [API](/api) will be served on
|
|
`http://localhost:11434`.
|
|
|
|
## System Requirements
|
|
|
|
- Windows 10 22H2 or newer, Home or Pro
|
|
- NVIDIA 452.39 or newer Drivers if you have an NVIDIA card
|
|
- AMD Radeon Driver https://www.amd.com/en/support if you have a Radeon card
|
|
|
|
Ollama uses unicode characters for progress indication, which may render as unknown squares in some older terminal fonts in Windows 10. If you see this, try changing your terminal font settings.
|
|
|
|
## Filesystem Requirements
|
|
|
|
The Ollama install does not require Administrator, and installs in your home directory by default. You'll need at least 4GB of space for the binary install. Once you've installed Ollama, you'll need additional space for storing the Large Language models, which can be tens to hundreds of GB in size. If your home directory doesn't have enough space, you can change where the binaries are installed, and where the models are stored.
|
|
|
|
### Changing Install Location
|
|
|
|
To install the Ollama application in a location different than your home directory, start the installer with the following flag
|
|
|
|
```powershell
|
|
OllamaSetup.exe /DIR="d:\some\location"
|
|
```
|
|
|
|
### Changing Model Location
|
|
|
|
To change where Ollama stores the downloaded models instead of using your home directory, set the environment variable `OLLAMA_MODELS` in your user account.
|
|
|
|
1. Start the Settings (Windows 11) or Control Panel (Windows 10) application and search for _environment variables_.
|
|
|
|
2. Click on _Edit environment variables for your account_.
|
|
|
|
3. Edit or create a new variable for your user account for `OLLAMA_MODELS` where you want the models stored
|
|
|
|
4. Click OK/Apply to save.
|
|
|
|
If Ollama is already running, Quit the tray application and relaunch it from the Start menu, or a new terminal started after you saved the environment variables.
|
|
|
|
## API Access
|
|
|
|
Here's a quick example showing API access from `powershell`
|
|
|
|
```powershell
|
|
(Invoke-WebRequest -method POST -Body '{"model":"llama3.2", "prompt":"Why is the sky blue?", "stream": false}' -uri http://localhost:11434/api/generate ).Content | ConvertFrom-json
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
Ollama on Windows stores files in a few different locations. You can view them in
|
|
the explorer window by hitting `<Ctrl>+R` and type in:
|
|
|
|
- `explorer %LOCALAPPDATA%\Ollama` contains logs, and downloaded updates
|
|
- _app.log_ contains most resent logs from the GUI application
|
|
- _server.log_ contains the most recent server logs
|
|
- _upgrade.log_ contains log output for upgrades
|
|
- `explorer %LOCALAPPDATA%\Programs\Ollama` contains the binaries (The installer adds this to your user PATH)
|
|
- `explorer %HOMEPATH%\.ollama` contains models and configuration
|
|
- `explorer %TEMP%` contains temporary executable files in one or more `ollama*` directories
|
|
|
|
## Uninstall
|
|
|
|
The Ollama Windows installer registers an Uninstaller application. Under `Add or remove programs` in Windows Settings, you can uninstall Ollama.
|
|
|
|
<Note>
|
|
If you have [changed the OLLAMA_MODELS location](#changing-model-location), the installer will not remove your downloaded models
|
|
</Note>
|
|
|
|
## Standalone CLI
|
|
|
|
The easiest way to install Ollama on Windows is to use the `OllamaSetup.exe`
|
|
installer. It installs in your account without requiring Administrator rights.
|
|
We update Ollama regularly to support the latest models, and this installer will
|
|
help you keep up to date.
|
|
|
|
If you'd like to install or integrate Ollama as a service, a standalone
|
|
`ollama-windows-amd64.zip` zip file is available containing only the Ollama CLI
|
|
and GPU library dependencies for Nvidia. If you have an AMD GPU, also download
|
|
and extract the additional ROCm package `ollama-windows-amd64-rocm.zip` into the
|
|
same directory. This allows for embedding Ollama in existing applications, or
|
|
running it as a system service via `ollama serve` with tools such as
|
|
[NSSM](https://nssm.cc/).
|
|
|
|
<Note>
|
|
If you are upgrading from a prior version, you should remove the old directories first.
|
|
</Note>
|