matt/ollama37
1
0
Fork 0
mirror of https://github.com/dogkeeper886/ollama37.git synced 2025-12-19 20:27:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Separate NVIDIA driver and CUDA toolkit installation steps

Browse Source 
- Split Step 3 into two distinct steps:
  - Step 3: NVIDIA Driver 470 installation via .run file
  - Step 4: CUDA 11.4 Toolkit installation via local installer
- Add libglvnd-devel dependency requirement
- Add text mode (init 3) requirement for driver installation
- Specify exact driver version (470.256.02) and download URL
- Specify exact CUDA installer (11.4.0 with 470.42.01 driver)
- Add note to deselect driver during CUDA installation
- Separate environment configuration:
  - PATH in /etc/profile.d/cuda-11.4.sh
  - Dynamic linker in /etc/ld.so.conf.d/cuda-11-4.conf
- Update all subsequent step numbers (5-7)
- Update all cross-references throughout document
This commit is contained in:
Shang Chieh Tseng
2025-10-28 16:55:38 +08:00
parent 35c4d078f7
commit e6e91af024
1 changed files with 136 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

205
docs/manual-build.md
View File

@@ -23,10 +23,11 @@ For native builds on **Rocky Linux 9**, you'll need to follow these steps in ord
**Installation Steps:** **Installation Steps:**
1. **Install GCC 10** - Required for kernel compilation and ollama37 source builds 1. **Install GCC 10** - Required for kernel compilation and ollama37 source builds
2. **Compile Custom Kernel** (Linux 5.14.x) - Required for NVIDIA 470 compatibility 2. **Compile Custom Kernel** (Linux 5.14.x) - Required for NVIDIA 470 compatibility
3. **Install NVIDIA Driver 470 & CUDA 11.4** - Tesla K80 GPU support 3. **Install NVIDIA Driver 470** - Tesla K80 GPU driver support
4. **Install CMake 4.0** - Build system 4. **Install CUDA 11.4 Toolkit** - CUDA development environment
5. **Install Go 1.24.2** - Go compiler 5. **Install CMake 4.0** - Build system
6. **Compile Ollama37** (Optional - if not using pre-built binaries) 6. **Install Go 1.25.3** - Go compiler
7. **Compile Ollama37** (Optional - if not using pre-built binaries)
**Quick Native Build (after prerequisites):** **Quick Native Build (after prerequisites):**
@@ -36,8 +37,8 @@ git clone https://github.com/dogkeeper886/ollama37
cd ollama37 cd ollama37
# If compiling from source (requires GCC 10): # If compiling from source (requires GCC 10):
cmake -B build CC=/usr/local/bin/gcc CXX=/usr/local/bin/g++ cmake -B build
cmake --build build -j$(nproc) CC=/usr/local/bin/gcc CXX=/usr/local/bin/g++ cmake --build build -j$(nproc)
go build -o ollama . go build -o ollama .
# If using pre-built binary (GCC 10 not required): # If using pre-built binary (GCC 10 not required):
@@ -179,12 +180,13 @@ dnf -y install ncurses-devel
# Example: cp /usr/src/kernels/5.14.0-570.52.1.el9_6.x86_64/.config .config # Example: cp /usr/src/kernels/5.14.0-570.52.1.el9_6.x86_64/.config .config
cp /usr/src/kernels/$(uname -r)/.config .config cp /usr/src/kernels/$(uname -r)/.config .config
``` ```
2. **Open menuconfig to adjust settings:** 2. **Open menuconfig to adjust settings:**
```bash ```bash
make menuconfig make menuconfig
``` ```
4. **Required Configuration Changes:** 3. **Required Configuration Changes:**
Navigate and **DISABLE** the following options: Navigate and **DISABLE** the following options:
@@ -214,26 +216,26 @@ dnf -y install ncurses-devel
> - Trusted keys: Conflicts with out-of-tree driver compilation > - Trusted keys: Conflicts with out-of-tree driver compilation
> - BTF debug: Can cause build failures and is unnecessary for production use > - BTF debug: Can cause build failures and is unnecessary for production use
5. **Save configuration:** 4. **Save configuration:**
- Press `<Save>` - Press `<Save>`
- Confirm default filename `.config` - Confirm default filename `.config`
- Press `<Exit>` to quit menuconfig - Press `<Exit>` to quit menuconfig
#### Compile Kernel #### Compile Kernel
2. **Compile kernel (using all CPU cores):** 1. **Compile kernel (using all CPU cores):**
```bash ```bash
make -j$(nproc) make -j$(nproc)
``` ```
> **Estimated time**: 30-60 minutes depending on CPU performance > **Estimated time**: 30-60 minutes depending on CPU performance
4. **Install kernel modules:** 2. **Install kernel modules:**
```bash ```bash
make modules_install make modules_install
``` ```
5. **Install kernel:** 3. **Install kernel:**
```bash ```bash
make install make install
``` ```
@@ -275,7 +277,7 @@ Can't read private key
--- ---
### Step 3: NVIDIA Driver 470 & CUDA 11.4 Installation ### Step 3: NVIDIA Driver 470 Installation
**Prerequisites:** **Prerequisites:**
- Rocky Linux 9 system running custom kernel 5.14.x (from Step 2) - Rocky Linux 9 system running custom kernel 5.14.x (from Step 2)
@@ -289,64 +291,125 @@ Can't read private key
dnf -y update dnf -y update
``` ```
2. **Install EPEL Repository:** 2. **Install required dependencies:**
```bash ```bash
dnf -y install epel-release dnf -y install epel-release
dnf -y install libglvnd-devel.x86_64
``` ```
3. **Add NVIDIA CUDA Repository:** 3. **Switch to text mode (runlevel 3):**
```bash ```bash
dnf -y config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo init 3
``` ```
> **Note**: This will exit the graphical interface. You'll need to log in via text console.
4. **Install NVIDIA Driver (Version 470):** 4. **Download NVIDIA Driver 470.256.02:**
```bash ```bash
dnf -y module install nvidia-driver:470-dkms cd /tmp
wget https://us.download.nvidia.com/tesla/470.256.02/NVIDIA-Linux-x86_64-470.256.02.run
``` ```
> **Note**: If the module install fails, you may need to install directly: 5. **Install NVIDIA Driver:**
> ```bash
> dnf -y install nvidia-driver-470 nvidia-driver-470-dkms
> ```
5. **Install CUDA Toolkit 11.4:**
```bash ```bash
dnf -y install cuda-11-4 chmod +x NVIDIA-Linux-x86_64-470.256.02.run
sh NVIDIA-Linux-x86_64-470.256.02.run
``` ```
> **Installation prompts:**
> - Accept the license agreement
> - If asked about DKMS, select "Yes" to register with DKMS
> - If asked about 32-bit compatibility libraries, select based on your needs
> - If asked about X configuration, select "Yes" if you use graphical interface
6. **Set up CUDA Environment Variables:** 6. **Reboot to load NVIDIA driver:**
```bash
# Create /etc/profile.d/cuda-11.4.sh
cat > /etc/profile.d/cuda-11.4.sh << 'EOF'
#!/bin/sh
# cuda-11.4.sh - CUDA 11.4 environment configuration for Tesla K80 support
export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
EOF
# Apply changes
source /etc/profile.d/cuda-11.4.sh
```
7. **Reboot to load NVIDIA driver:**
```bash ```bash
reboot reboot
``` ```
**Verification:** **Verification:**
```bash ```bash
# Check CUDA compiler
nvcc --version
# Should show: Cuda compilation tools, release 11.4
# Check driver and GPU # Check driver and GPU
nvidia-smi nvidia-smi
# Should show Tesla K80 GPU(s) with driver version 470.x # Should show Tesla K80 GPU(s) with driver version 470.256.02
``` ```
--- ---
### Step 4: CMake 4.0 Installation ### Step 4: CUDA 11.4 Toolkit Installation
**Prerequisites:**
- NVIDIA Driver 470 installed and verified (from Step 3)
- Root privileges
**Steps:**
1. **Download CUDA 11.4.0 installer:**
```bash
cd /tmp
wget https://developer.download.nvidia.com/compute/cuda/11.4.0/local_installers/cuda_11.4.0_470.42.01_linux.run
```
2. **Run CUDA installer:**
```bash
sh cuda_11.4.0_470.42.01_linux.run
```
> **Installation prompts:**
> - Accept the license agreement
> - **IMPORTANT**: Deselect "Driver" option (driver already installed in Step 3)
> - Keep selected: CUDA Toolkit, CUDA Samples, CUDA Demo Suite, CUDA Documentation
> - Confirm installation
3. **Set up CUDA Environment Variables:**
Create two configuration files:
**a) PATH configuration in `/etc/profile.d/`:**
```bash
cat > /etc/profile.d/cuda-11.4.sh << 'EOF'
#!/bin/sh
# cuda-11.4.sh - CUDA 11.4 PATH configuration for Tesla K80 support
export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
EOF
# Apply PATH changes
source /etc/profile.d/cuda-11.4.sh
```
**b) Dynamic linker configuration:**
The CUDA installer creates `/etc/ld.so.conf.d/cuda-11-4.conf` automatically with the following content:
```
/usr/local/cuda-11.4/lib64
/usr/local/cuda-11.4/targets/x86_64-linux/lib
```
If the file doesn't exist or needs to be recreated:
```bash
cat > /etc/ld.so.conf.d/cuda-11-4.conf << 'EOF'
/usr/local/cuda-11.4/lib64
/usr/local/cuda-11.4/targets/x86_64-linux/lib
EOF
# Update dynamic linker cache
ldconfig
```
**Verification:**
```bash
# Check CUDA compiler
nvcc --version
# Should show: Cuda compilation tools, release 11.4, V11.4.48
# Check driver and CUDA compatibility
nvidia-smi
# Should show Tesla K80 GPU(s) with driver version 470.256.02 and CUDA Version: 11.4
```
---
### Step 5: CMake 4.0 Installation
1. **Install OpenSSL Development Libraries:** 1. **Install OpenSSL Development Libraries:**
```bash ```bash
@@ -393,47 +456,49 @@ nvidia-smi
--- ---
### Step 5: Go 1.24.2 Installation ### Step 6: Go 1.25.3 Installation
1. **Download Go Distribution:** 1. **Download Go Distribution:**
```bash ```bash
cd /usr/local cd /usr/local
wget https://go.dev/dl/go1.24.2.linux-amd64.tar.gz wget https://go.dev/dl/go1.25.3.linux-amd64.tar.gz
``` ```
2. **Extract the Archive:** 2. **Extract the Archive:**
```bash ```bash
tar xvf go1.24.2.linux-amd64.tar.gz tar xvf go1.25.3.linux-amd64.tar.gz
``` ```
3. **Post Install Configuration:** 3. **Post Install Configuration:**
```bash ```bash
cat > /etc/profile.d/go-1.24.2.sh << 'EOF' cat > /etc/profile.d/go.conf << 'EOF'
#!/bin/sh #!/bin/sh
# go-1.24.2.sh - Go 1.24.2 environment configuration # go.conf - Go environment configuration
export PATH=/usr/local/go/bin${PATH:+:${PATH}} export PATH=/usr/local/go/bin${PATH:+:${PATH}}
EOF EOF
source /etc/profile.d/go-1.24.2.sh # Apply the configuration
source /etc/profile.d/go.conf
``` ```
4. **Verify Installation:** 4. **Verify Installation:**
```bash ```bash
go version go version
# Should output: go version go1.24.2 linux/amd64 # Should output: go version go1.25.3 linux/amd64
``` ```
--- ---
### Step 6: Ollama37 Compilation (Optional - For Custom Builds) ### Step 7: Ollama37 Compilation (Optional - For Custom Builds)
**Prerequisites:** **Prerequisites:**
All components installed as per the guides above: All components installed as per the guides above:
- GCC 10 (from Step 1) - GCC 10 (from Step 1)
- Rocky Linux 9 with custom kernel 5.14.x (from Step 2) - Rocky Linux 9 with custom kernel 5.14.x (from Step 2)
- CUDA Toolkit 11.4 (from Step 3) - NVIDIA Driver 470 (from Step 3)
- CMake 4.0 (from Step 4) - CUDA Toolkit 11.4 (from Step 4)
- Go 1.24.2 (from Step 5) - CMake 4.0 (from Step 5)
- Go 1.25.3 (from Step 6)
- Git - Git
**Compilation Steps:** **Compilation Steps:**
@@ -452,13 +517,13 @@ All components installed as per the guides above:
3. **CMake Configuration:** 3. **CMake Configuration:**
Set compiler variables and configure the build system: Set compiler variables and configure the build system:
```bash ```bash
cmake -B build CC=/usr/local/bin/gcc CXX=/usr/local/bin/g++ cmake -B build
``` ```
4. **CMake Build:** 4. **CMake Build:**
Compile the C++ components (parallel build): Compile the C++ components (parallel build):
```bash ```bash
cmake --build build -j$(nproc) CC=/usr/local/bin/gcc CXX=/usr/local/bin/g++ cmake --build build -j$(nproc)
``` ```
> **Note:** `-j$(nproc)` enables parallel compilation using all available CPU cores. You can specify a number like `-j4` to limit the number of parallel jobs. > **Note:** `-j$(nproc)` enables parallel compilation using all available CPU cores. You can specify a number like `-j4` to limit the number of parallel jobs.
@@ -653,17 +718,19 @@ watch -n 1 nvidia-smi
### Path 1: Pre-built Binary (Easier) ### Path 1: Pre-built Binary (Easier)
1. ❌ Skip GCC 10 installation (not needed for pre-built binaries) 1. ❌ Skip GCC 10 installation (not needed for pre-built binaries)
2. Compile custom kernel 5.14.x 2. Compile custom kernel 5.14.x
3. Install NVIDIA Driver 470 & CUDA 11.4 3. Install NVIDIA Driver 470
4. Install CMake 4.0 4. Install CUDA 11.4 Toolkit
5. Install Go 1.24.2 5. Install CMake 4.0
6. Download and run pre-built ollama37 binary 6. Install Go 1.25.3
7. Download and run pre-built ollama37 binary
### Path 2: Compile from Source (Advanced - Requires All Steps) ### Path 2: Compile from Source (Advanced - Requires All Steps)
1. ✅ **Install GCC 10** (required for kernel and ollama37 compilation) 1. ✅ **Install GCC 10** (required for kernel and ollama37 compilation)
2. Compile custom kernel 5.14.x (uses GCC 10) 2. Compile custom kernel 5.14.x (uses GCC 10)
3. Install NVIDIA Driver 470 & CUDA 11.4 3. Install NVIDIA Driver 470
4. Install CMake 4.0 4. Install CUDA 11.4 Toolkit
5. Install Go 1.24.2 5. Install CMake 4.0
6. Compile ollama37 from source (uses GCC 10) 6. Install Go 1.25.3
7. Compile ollama37 from source (uses GCC 10)
Choose the path that best fits your requirements and technical expertise. Choose the path that best fits your requirements and technical expertise.