Hi there,
I’m are building an AI Agent that uses skill.md files.
Each skill.md contains:
- Skill metadata
- skill_name
- description
- examples
- business domain
- tags
- Execution instructions
- SQL query itself
- Snowflake execution details
Workflow:
- User asks a question.
- LLM reviews available skill metadata.
- LLM selects the most relevant skill.
- Full skill.md is loaded.
- SQL executes on Snowflake.
- Results are returned to UI.
Requirements:
- New skills must be dynamically discoverable at runtime.
- No application redeployment when adding new skills.
- Minimize latency.
- Avoid fetching all skill files for every user request.
- Enterprise-grade architecture.
- Skills may be updated at any time.
- Skill selection must scale to hundreds or thousands of skills.(if needed)
- Agent runs in Kubernetes.
- Snowflake is the data source, skill.md files loookup is in oracle database
- MCP can be used if necessary but should not become a bottleneck.
Current Ideas:
Approach A:
- Store skills in a Git repository.
- Use MCP to fetch skills.
- Rejected due to dependency on external Git-based workflow.
Approach B:
- Store skills in oracle DB lookup tables.
- using MCP Call to Load all skills into memory during pod startup.
- LLM uses metadata to select skill.
- Full skill loaded from memory when selected.
- Problem: Pod restart required when new skills are added.
Approach C:
- Invoke MCP and fetch skills on every user request.
- Rejected due to performance and latency concerns.
Tasks:
- Design an enterprise-grade architecture for dynamic skill management.
- Explain how large-scale AI agents typically manage dynamic skills.
- Recommend:
- Skill registry design
- Metadata catalog design
- Cache strategy
- Refresh strategy
- Versioning strategy
- Skill discovery architecture
- Skill selection architecture
- MCP usage strategy
- Kubernetes deployment pattern
- Explain whether polling, event-driven refresh, or cache invalidation is best.
- Explain how new skills can become available immediately without pod restarts.
- Provide reference architecture diagrams using Mermaid.
- Provide scalability considerations for:
- Describe best practices used by enterprise AI platforms.
- Provide a final recommended architecture with pros, cons, implementation steps, and operational considerations.
Tech Stack :
Python 3.13, Langchain, LangGraph, FastAPI, Pydantic V2, Angular frontend, LLM - Azure OpenAI models - Gpt - 5 - Mini
IMO skills should be treated like code and version controlled. How you fetch it is up to you, but the underlying structure should live in a repository. Then how you expose it can be done many different ways, depending on who needs to access it and what their privileges are. It can be pushed into a S3 bucket or Google Drive or you can have MCP server around it. For devs they can probably just use repo directly, but if you are a knowledge worker with no GH seat/license you probably wanna be able to fetch it differently.
Hi, I agree that IMO skills should be version-controlled and managed centrally. However, I’d prefer not to bundle the skill files directly into the application codebase, since every skill change would require a new deployment. I also want to avoid introducing dependencies on external cloud storage or Git repositories.
Since I’m already fetching data from the database, my current approach is to store the skills there and retrieve the relevant ones through MCP calls. The main challenge is handling updates in production without restarting the pod whenever a new skill is added or an existing one is modified.
One option is to implement periodic polling to check for updates, but I’m wondering if there’s a more efficient, production-ready approach, such as event-driven updates, change notifications, or dynamic cache refresh. How would you recommend managing skill versioning and making updated skills available at runtime without requiring pod restarts?
Oh I never said you should bundle it in the same codebase as the application. I jus said that you should treat it as code and version control it in a repo. You can have a monorepo just for skills, and everyone in your org can commit there, or you can have multiple skills repos. Personally I find it easier to have a monorepo for skills.
Now that you bring it up though, there are cases where skills together with application codebase should live together, but that’s up to you to decide, based on the good old fashioned separation of concerns principle. If your skills are only ever used and relevant for that specific application, there is nothing wrong with having them there.
At the end of the day I just think of all this as all other software engineering principles.
I’d avoid treating this as a choice between loading everything at startup and fetching on every request. A middle path: keep a small per-pod index in memory (skill_id, name, description, tags, version, updated_at) and load the full skill body lazily, cached by skill_id plus version.
Refreshing then becomes cheap. Every N seconds each pod runs one indexed query such as “select id, version from skills where updated_at > :last_seen” and only reloads what changed, so no pod restart is needed. If you want push instead of poll, have the code that writes a skill also publish a “skill changed” event (Redis pub/sub, Kafka, or an outbox table) and invalidate just those entries. Keep the poll as a safety net, since events can be dropped.
For selection at scale, don’t hand hundreds of descriptions to the LLM. Pre-filter the index with embeddings or keyword search down to the top 5-10 candidates, then let the model choose. Finally, pin the skill version for the duration of a single request so a mid-flight update can’t change the SQL being executed.