Sounds like you’ve made lots of design choices already. Are you using Python and Langchain as well? And I’d be interested to know what your hardware setup is (memory, graphics card, CPU), and which LLM models you’ve determined you can run on that.
1 Like