mirror of
https://github.com/dogkeeper886/ollama37.git
synced 2025-12-12 08:47:01 +00:00
Allow models to force a new batch
This is useful for a few things: - Work around bugs, such as having 2 images in one batch - Keep the image in a single batch for fully connected attention - Improve performance by not evaluating embeddings multiple times
This commit is contained in:
committed by
Michael Yang
parent
a8e83a7654
commit
06007c0a18
@@ -15,6 +15,12 @@ type Input struct {
|
||||
// stored in Multimodal, used for caching and comparing
|
||||
// equality.
|
||||
MultimodalHash uint64
|
||||
|
||||
// BatchBreak forces a new batch to be started with this
|
||||
// input. For example, this can be used to align images
|
||||
// with batches. Note that batches may be divided in additional
|
||||
// locations as well.
|
||||
BatchBreak bool
|
||||
}
|
||||
|
||||
// MultimodalIndex is a multimodal element (such as an image)
|
||||
|
||||
Reference in New Issue
Block a user