Potential race condition

[xingjianan]

System Info

We are using lorax 0.12.1 and we notice that sometimes different requests of different adapters could affect each other.

We noticed that sometimes one request's input could pollute into another request(that request could be sent to a total different adapter)'s output with a special token N/ . Not sure if you can pour some insights into this issue.

Our base model is mistralai/Mistral-7B-Instruct-v0.2

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

Still trying to reproduce

Expected behavior

Requests should be isolated

[thincal]

So is this issue reproduced again?