Jeffrey Morgan 15c2d8fe14 server: parallelize embeddings in API web handler instead of in subprocess runner (#6220) ...

For simplicity, perform parallelization of embedding requests in the API handler instead of offloading this to the subprocess runner. This keeps the scheduling story simpler as it builds on existing parallel requests, similar to existing text completion functionality.

2024-08-11 11:57:10 -07:00

122 KiB

Vendored

Raw Blame History

View Raw