I am looking for a way to enable caching on AsyncAzureOpenAI. Currently, I am exploring the redisvl library using the sample app provided by microsoft (github: microsoft/sample-app-aoai-chatGPT)
Does anyone have experience or suggestions on how to introduce caching at the client declaration level? I’m aiming for something similar to what LangChain offers for LLM caching. Any insights or examples would be greatly appreciated!
Langchain example:
from langchain_community.cache import RedisSemanticCache
from langchain_openai import OpenAIEmbeddings
set_llm_cache(
RedisSemanticCache(redis_url="redis://localhost:6379", embedding=OpenAIEmbeddings())
)
# First time not cahced so slow, but second time it is, so it goes faster
llm.invoke("Tell me a joke")