Fix multi-sequence embeddings

[iamlemec]

Fixes multi-sequence (batch) embeddings by handling n_seq_max and kv_unified flags. See discussion in #2051.

[LimePencil]

@abetlen any updates yet?

[freckletonj]

confirming this is still an issue

[mlisovyi]

Shouldn't n_seq_max be also used in Llama.embed() ? One should add or p_batch == n_seq_max to the batch-evaluation condition in the loop here. Otherwise one runs into a danger of collecting a batch that will consist of more sequences as the configured maximum (if individual inputs are short or the configured n_seq_max is small) and this will also lead to the same llama_decode returned -1 error

[mlisovyi]

Also, would it make sense to expose those parameters in ModelSettings with some meiningful defaults to allow setting them in the server run?

[abetlen]

@iamlemec thank you, this should have been fixed in v0.3.23