Hello, my problem concerns the use of Pickle with FAISS. Previously, I could normally dump the vectorstore like this:
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(docs, embeddings)
with open("base_documents.pkl", "wb") as f:
pickle.dump(vectorstore, f)
However, now I encounter an error:
cannot pickle '_thread.RLock' object
Has anyone else encountered a problem like this? I use it in RetrievalQAWithSourcesChain as a retriever.
Hi and welcome to the Developer Forum!
What OpenAI service is this regarding? I see embedding-3-small but the rest of it does not look like an OpenAI service I am familiar with.
Seems to be Langchain, have you tried on their forum? Happy for you to see if anyone from here may have come across this issue.
have you tried to use the faiss serialize to bytes method?
you can’t just pickle random class instances 
It worked before soooo… 
Yea im trying this now but still looking how to save and load this vectorestore