The seed inference parameter for reproducibility

[PaulBellow]

Ah, okay, it’s new, and I haven’t used it yet personally.

This thread might have some answers?

Seed param and reproducible output do not work

I’ve figured out that issue with not reproducing same output with defined seed is related with calculating question embeddings which tend to produce varying results. So for testing purposes to achieve comparable results you need to retrieve once and then reuse (mock) embeddings for defined set of test questions. More about this issue you can find in thread below: