Changing the template and adding customzation in pandas_dataframe_agent of OpenAI(gpt-4 model) in langchain

Prompting
1

Hi @all,
Hope all of you are doing great.

I am developing a chatbot/question-answer agent using GPT-4 on pandas dataframe in langchain. I am able to make it work with following code:

gpt4_agent = create_pandas_dataframe_agent(
    ChatOpenAI(temperature=0,
               openai_api_key=OPENAI_API_KEY,
               model_name = 'gpt-4'),[sub_listing_df,company_df], verbose=True
)

But, I think that I can make it more optimized by changing some paramters, and especially prompt template: Here is the prompt template being used by default:

display(Markdown(gpt4_agent.agent.llm_chain.prompt.template))
----------------------
You are working with {num_dfs} pandas dataframes in Python named df1, df2, etc. You should use the tools below to answer the question posed of you:

python_repl_ast: A Python shell. Use this to execute python commands. Input should be a valid python command. When using this tool, sometimes output is abbreviated - make sure it does not look abbreviated before using it in your answer.

Use the following format:

Question: the input question you must answer Thought: you should always think about what to do Action: the action to take, should be one of [python_repl_ast] Action Input: the input to the action Observation: the result of the action ... (this Thought/Action/Action Input/Observation can repeat N times) Thought: I now know the final answer Final Answer: the final answer to the original input question

This is the result of print(df.head()) for each dataframe: {dfs_head}

Begin! Question: {input} {agent_scratchpad}

How can I change the prompt template?

Thanks for any help

2

Hi @muhammadali
At present, Pandas agent within LangChain offers the capability to append only prefixes and suffixes to prompts. However, if you require a more tailored prompt customization to suit your specific needs, you can delve into the codebase provided in the following GitHub repository:

(https://github.com/langchain-ai/langchain/blob/master/libs/experimental/langchain_experimental/agents/agent_toolkits/pandas/base.py)

Integrate the base.py file into your local codebase
you can customize _get_multi_prompt and _get_single_prompt for prompt based on your usecase