Long data misread from OpenAI Assistant (gpt-4o-mini)

API
1

Hi everyone,

I’m developing an AI app that let users interact with EVM Blockchain. Currently my flow is let the AI call tools to get the transaction data contain these field:
{
to: string,
value: number,
data: string
}

Usually the data is long, in the hex format for example 0x4d8160ba000000000000000000000000f50d05a1402d0adafa880d36050736f9f6ee7dee000000000000000000000000000000000000000000000000016345785d8a000000000000000000000000000000000000000000000000000000000000000001400000000000000000000000006352a56caadc4f1e25cd6c75970fa768a3304e6400000000000000000000000000000000000000000000000000000000000001600000000000000000000000003c499c542cef5e3811e1192ce70d8cc03d5c3359000000000000000000000000000000000000000000000000000000000008f0d7000000000000000000000000d7afc6f4d11d1b81107e300171e148b764f558ac000000000000000000000000ef4fb24ad0916217251f553c0596f8edc630eb660000000000000000000000000000000000000000000000000000000000000c4000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000aa490411a320000000000000000000000001e82ad8a12068a85fcb96368463b434e77b21201000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000001c0000000000000000000000000f50d05a1402d0adafa880d36050736f9f6ee7dee0000000000000000000000003c499c542cef5e3811e1192ce70d8cc03d5c33590000000000000000000000001e82ad8a12068a85fcb96368463b434e77b21201000000000000000000000000663dc15d3c1ac63ff12e45ab68fea3f0a883c251…"

This data has to be right, being wrong at only one character lead the transaction failed. How can I assure that the assistant get the data right before submit the data to user, and also to the tool that actually submit the transaction.

I have included some prompts like this but it still misread the data wrong sometimes

- "Always log and display the exact output from tool calls. Do not modify, reformat, or apply any internal logic to the data (e.g., if the tool returns 0x1, log 0x1 exactly, never alter it to 0x2)."
        - "Never use your internal knowledge or assumptions to generate or modify transaction data. Always use the raw data received from the corresponding tool call."
        - "Always use the exact output from any tool call when presenting calldata. Do not reformat, re-case, or infer any changes from your own knowledge. For instance, if the approve_erc20 tool returns 0x095ea7b300000000..., then that string must be shown exactly, without modification."
        - Never use internal logic or assumptions to change the data. Always rely on the raw tool output.
          This should cover not only approvals but every on-chain operation that outputs calldata.
        - "Do not alter any part of the calldata—even minor case changes (e.g., converting 0xd7afc6f4d11d1b81107e300171e148b764f558ac to 0xd7aFc6f4d11d1B81107E300171E148b764f558aC) are forbidden."

Can anyone help me on this? Thank you!

2

Anything that requires 100% accuracy for financial transaction audit trails should not be passed through an LLM.

There is no way around this, you need to build a traditional code deterministic system to handle that side of it, and even that system needs to have finance system level checking to ensure that memory, disk and network errors do not corrupt the data.

You should use an LLM to allow flexible interfacing and intelligent analysis, they should not be used at any critical path point in a financial transaction

2 Likes