Using the Openai SDK to create an AI that will be integrated into an organization that will assist with various daily tasks

API
1

Hi @_j

I am just getting my feet wet with llms and I am new to programming . I need to create an Ai system that will help people in an organization do various tasks and Id like know if the way I am approaching this is correct.

Currently:

I have used the openai SDK to create agents which uses @function_tool. I have used a vector database to house the company database and I have written a tool that uses the llm to connect and query the vector database. The users will be using a management agent that uses handoffs for the other agents I have created for example:

  1. Look through vector database
  2. Search the web for necessary details.

I am doing this through a Flask Api end point with a React front end interface.

What I plan to do:

Eventually I will need to have different ai managers that have different roles in the business (I am not aware of the roles as yet) where access is granted to the different ai models based on the position of employees.

I plan to create a role base access control system in python which will then allow access to the different API calls which will have the different AI managers, which then all feeds into the react application for the user.

2

Implied question

Is my current architecture for an org-wide, multi-agent LLM assistant (OpenAI SDK + function tools + vector DB + Flask API + React UI, with planned role-based “AI managers” and RBAC) a sound approach—and what should I change or add to make it secure, scalable, and practical in a real organization?

Implied need

A sanity check and concrete guidance on architecture and best practices—especially around agent handoffs, retrieval/RAG design, role-based access control, data/security boundaries, and operational concerns (monitoring, evals, cost).

Final ask (in the author’s voice)

Given this setup and plan, does this architecture make sense for a real organization? What would you change or recommend—particularly for agent handoffs/tooling patterns, retrieval/RAG design, RBAC/authorization boundaries, security (prompt injection/data leakage), and monitoring/evaluation/cost control? If you’ve shipped something similar, I’d really value the patterns or pitfalls you’d highlight before I go further.

It sounds like you would want to start with a centralized chatbot, instead of a concept of ‘agent handoff’ that doesn’t have a clear path for success.

The easiest implementation would be to have information retrieval as functions, either there or not depending on rights, that the AI can call upon and receive back texts. You would want to decide early if knowledge retrieval should be automatic injection based on user inputs, threshold-based, or if the knowledge is better accessed and placed by AI written function queries to a tool for semantic search.

“Search the web” isn’t something that comes magically, and OpenAI’s internal tool or special web search model doesn’t return enough information for it to be “knowledge”, but rather is “click here” search results. A better result for the patient user, allowed to use high cost services, would be queries sent to a deep research model, which can make more extensive answers that are more suitable as a function return to provide information that can be distilled with your own system message application in force.

Segmentation of roles? Up to you. Hold back the vector database that has ‘company downsizing and layoff plans’ from the general employees?

3

Thank you for the explanation.

Id just like to make sure I understand.

1)When you refer to having a centralized chatbot, do you mean having one ai model that will be used by the entire organization?

  1. When you refer to having the information retrieval as functions, do you mean connecting to a vector database through written function?

  2. What will work best from my understanding of the organization , it would work best when the knowledge retrieval is automated on the user query. So for that will I need to just have a function that does that a vector search with embeddings and provides a result?

  3. When I need to search the web, I wrote a function with a google api to use a custom engine as I know the websites I will be using to get the information I need. Is that correct?

  4. Yes, so essentially to control the access of the organizations data.

I appreciate your help