mirror of
https://github.com/dogkeeper886/ollama-k80-lab.git
synced 2025-12-10 07:46:59 +00:00
23 lines
1.4 KiB
Markdown
23 lines
1.4 KiB
Markdown
# ollama-k80-lab
|
|
|
|
[](https://opensource.org/licenses/MIT)
|
|
|
|
## Overview
|
|
|
|
This project explores running Ollama, a local LLM runner, with a NVIDIA K80 GPU and investigates its integration with Dify, a powerful framework for building LLM-powered applications. The goal is to assess performance, explore limitations, and demonstrate the potential of this combination for local LLM experimentation and deployment.
|
|
|
|
## Motivation
|
|
|
|
* **Local LLM Exploration:** Ollama makes it incredibly easy to run Large Language Models locally. This project aims to leverage that ease with the power of a GPU.
|
|
* **K80 Utilization:** The NVIDIA K80, while older, remains a viable GPU for LLM inference. This project aims to demonstrate its capability for running smaller to medium sized LLMs.
|
|
* **Dify Integration:** Dify provides a robust framework for building LLM applications (chatbots, agents, etc.). We want to see how seamlessly Ollama and Dify can work together, allowing us to rapidly prototype and deploy LLM-powered solutions.
|
|
* **Cost-Effective Experimentation:** Running LLMs locally avoids the costs associated with cloud-based APIs, enabling broader access and experimentation.
|
|
|
|
## Contributing
|
|
|
|
Contributions are welcome! Please feel free to submit pull requests or open issues.
|
|
|
|
## License
|
|
|
|
This project is licensed under the [MIT License](LICENSE).
|