FAISS cannot pickle '_thread.RLock' object

API
1

Hello, my problem concerns the use of Pickle with FAISS. Previously, I could normally dump the vectorstore like this:

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(docs, embeddings)
with open("base_documents.pkl", "wb") as f:
            pickle.dump(vectorstore, f)

However, now I encounter an error:

cannot pickle '_thread.RLock' object

Has anyone else encountered a problem like this? I use it in RetrievalQAWithSourcesChain as a retriever.

1 Like
2

Hi and welcome to the Developer Forum!

What OpenAI service is this regarding? I see embedding-3-small but the rest of it does not look like an OpenAI service I am familiar with.

Seems to be Langchain, have you tried on their forum? Happy for you to see if anyone from here may have come across this issue.

1 Like
3

have you tried to use the faiss serialize to bytes method?

you can’t just pickle random class instances :laughing:

4

It worked before soooo… :smiley:

Yea im trying this now but still looking how to save and load this vectorestore