Jesse Gross 493385eb3e ollamarunner: Don't truncate a SameBatch ...

When truncating inputs to the the context window at the beginning of
a sequence, we remove the minimum amount possible. However, this
may cause us to truncate to the middle of a set of inputs that
the model specified should not be split up. To avoid this, we
need to remove the rest of the partial batch.

2025-04-02 10:40:38 -07:00

7.5 KiB

Raw Blame History

View Raw