Jesse Gross 282bfaaa95 ollamarunner: Use a separate context per multimodal input ...

Currently there is a single context per sequence, shared all by
all multimodal inputs. Since we build a vision encoder graph per
image, with a large number of inputs we can eventually hit the
maximum number of graph nodes per context.

This changes to use a separate context for each image, ensuring
that available resource limits are consistent.

2025-03-14 15:38:54 -07:00

7.4 KiB

Raw Blame History

View Raw