added python rag news summary

Signed-off-by: Matt Williams <m@technovangelist.com>
This commit is contained in:
Matt Williams
2023-10-17 16:41:28 -07:00
parent 08b0e04f40
commit c5c8b4b16a
4 changed files with 225 additions and 0 deletions

View File

@@ -0,0 +1,22 @@
# News Summarizer
This example goes through a series of steps:
1. You choose a topic area (e.g., "news", "NVidia", "music", etc.).
2. Gets the most recent articles on that topic from various sources.
3. Uses Ollama to summarize each article.
4. Creates chunks of sentences from each article.
5. Uses Sentence Transformers to generate embeddings for each of those chunks.
6. You enter a question regarding the summaries shown.
7. Uses Sentence Transformers to generate an embedding for that question.
8. Uses the embedded question to find the most similar chunks.
9. Feeds all that to Ollama to generate a good answer to your question based on these news articles.
This example lets you pick from a few different topic areas, then summarize the most recent x articles for that topic. It then creates chunks of sentences from each article and then generates embeddings for each of those chunks.
You can run the example like this:
```bash
python3 -m pip install -r requirements.txt
python3 summ.py
```