How can I integrate AI assistant in my website but not a chatbot?

ChatGPT
, , ,
17

Hello, I’m trying to build an AI assistant for our website that feels native to the product.

I initially built this using the OpenAI Agent SDK and ChatKit, but I’ve run into several challenges while moving toward a production-ready experience. I’m now looking for advice from anyone who has solved similar problems.

Challenges I’ve faced with Agent SDK + ChatKit:

1. Production reliability
The agent works well in testing, but with real users I’m seeing inconsistent behavior:

  • Unpredictable tool calls

  • Responses that don’t always match the conversation context

  • Cases where the agent fails to follow without a recovery path

2. UI Customisation limiation
My goal is to create an AI assistant that feels like a natural part of the product experience, not a chat box added on top.

Even with ChatKit customization options, I’m finding it difficult to make the experience fully match our product UI and workflows.

Has anyone successfully extended ChatKit beyond the default chat interface?

3. Limited support and debugging experience
I’ve struggled to find reliable support resources when running into issues.

I tried using the documentation AI assistant, but it frequently fails with errors like “The docs agent couldn’t complete the request. Please retry.” I also tested some official examples and ran into similar issues.

OAI-docs
OAI-docs1920×1035 164 KB

4. Knowledge base management and improving over time
Our knowledge base is maintained by multiple team members, which creates challenges:

  • Outdated policies mixed with updated information

  • Multiple documents containing slightly different answers

  • No clear visibility into why the agent selected a specific answer

I’m looking for ways to build systems where teams can understand agent decisions, identify knowledge gaps, and continuously improve responses.

5. Action-oriented AI assistant
Beyond answering questions, I want the assistant to take actions, such as:

  • Navigating users to specific areas of the product

  • Triggering workflows

I’d really appreciate any advice you can share on building a reliable system and resolving these issues.