Technical feasibility of an AI tutoring assistant

Hi All!

I am exploring the idea of a web-based AI tutoring assistant using the GPT API. Users would be able to ask questions, upload their own learning materials and interactively complete worksheets within the application.

The system should retrieve relevant information from uploaded documents, provide understandable explanations, ask follow-up questions and generate individual practice exercises. It should also support interactive worksheet processing, for example by allowing users to complete tasks step by step, receive hints, check answers and get feedback on their solution process.

I would be interested in your practical assessment:

  • How would you design the technical architecture of such a system? (e.g. which model also regarding pricing?

  • Where do you see the greatest technical challenges?

  • How could hallucinations and incorrect feedback be reduced?

  • Which challenges would arise regarding document parsing, privacy, latency, costs and scalability?

Experiences from similar projects would be especially helpful and sharing helpful insights.

Just to say you could shortcut a lot of the work and build it as a plugin on top of Discourse.

(for reference this site is a Discourse instance.)

This would take care of:

  • User accounts and authentication
  • Provision of a database and cache layer (industry standard Postgres and Redis)
  • Provision of extensible job scheduling system (industry standard Sidekiq)
  • Provision of an extensible webserver and front end framework (Industry standard Ruby on Rails and EmberJS)
  • Provision of a reverse proxy and scripted certificate automation.
  • Reusable, user profile, chat and topic system

out of the box and as an ongoing open source project a lot of maintenance would be done for you to the core functionality.

It allows you to get a website up in minutes with all the capabilities of an app server that you can customise.

EmberJS gives you an app like experience which is very modern and touch device compatible.

I’ve been writing Discourse plugins for nearly a decade and can highly recommend it as a base platform. It’s a very well maintained and modern platform.

I’ve had many clients who’ve built on top of it for just the reasons stated above.

Out of the box it already has a lot of AI capabilities, and fun fact, I wrote its first AI chatbot for Discourse as a plugin a few years ago:

Here’s an install guide for Discourse:

It’s very easy to self host.

Thank you! I will have a look on it. But still, I need to use an Open AI API KEY, however Im wondering if it’s feasible to use them or if I will end up spending more for tokens than earrn from customers. Do you have experience in building such business cases?

If you are an individual get a ChatGPT Pro sub and use it in Codex to build your solution.

If a business organisation open a Business account and use an OpenAI key with Codex.

I build business solutions as a job. Feel free to PM me privately or contact me via my website linked on my user card (click my avatar). Take a look at my GitHub account for work stretching years.

Best of luck with the project!

This is feasible, but I would not design it as just a chatbot with document upload. I would design it as a stateful tutoring workflow.

The main pieces I would use are:

  1. Document ingestion

Uploaded material needs to be parsed into structured learning content, not just dumped into a prompt. Plain text documents are easier. PDFs, scans, handwritten work, diagrams, maths worksheets, tables, charts, and screenshots need OCR and layout extraction. For that part, you probably want a multimodal model or a document processing pipeline that can preserve layout, images, equations, and source references.

  1. Retrieval

Use retrieval for the uploaded learning materials, but chunk by learning unit rather than arbitrary token size where possible. Store source spans so the assistant can say “this answer comes from page 4, section 2” instead of making unsupported claims.

  1. Tutor state

You need to track the learner’s current topic, skill level, mistakes, completed exercises, hints already given, and what concept they are struggling with. Without state, the assistant becomes a Q&A bot rather than a tutor.

  1. Worksheet state

Interactive worksheets should be represented as structured tasks. Each question should have the prompt, expected answer or rubric, current student answer, attempt history, hint history, feedback, and completion status. The model should not just improvise whether an answer is correct every time.

  1. Answer checking

For exact answers, use deterministic checks where possible. For open answers, use a rubric. For maths or code, use a solver, unit tests, or step validation where possible. The model can explain feedback, but it should not be the only judge if correctness can be checked another way.

  1. Model use

I would not send every request to the most expensive model. Use cheaper models for routing, simple Q&A, summarisation, and formatting. Use a stronger text model for difficult tutoring explanations. Use a multimodal model when the input includes images, diagrams, scans, handwritten work, worksheet screenshots, or visual layout.

  1. Hallucination control

The biggest reduction comes from not letting the model answer from memory when it should answer from course material. Ground answers in retrieved source text, show citations or page references, use rubrics for marking, and make the system say “I do not have enough information from the uploaded material” when retrieval is weak.

  1. Privacy

Treat uploaded learning material and student work as private data. Separate users and courses properly, avoid leaking documents between users, store only what is needed, and decide early whether uploaded content can be retained, deleted, exported, or used only for the session.

  1. Latency and cost

The expensive part will be document parsing, multimodal analysis, long context, and repeated feedback loops. Cache parsed documents, cache embeddings, store worksheet state, and avoid reprocessing the same upload every time the student asks a question.

So the short version is: yes, it is technically feasible, but the hard part is not calling the GPT API. The hard part is turning learning material into structured state, tracking the learner’s progress, checking answers reliably, and using the model only where reasoning or explanation is actually needed.

Building it on top of an existing platform like Discourse could help with accounts, posts, permissions, and community features. But the tutoring core would still need its own ingestion, retrieval, worksheet state, answer checking, and feedback workflow.

Thank you for the answer! My biggest concern is the quality of the answers. My goal is to implement it first for latin translations as a pilot project. I already realized that the output and tutoring is not always in same quality.

You were writing about checking answers with a solver. For latin translations the correct answers can vary pretty broadly.

Are there any suitable solutions for that?

Latin is a really interesting use case. A friend of mine is an archivist and a Latin expert. He was not impressed with the Latin translation capability of frontier models at the time we had a chat about it. He found it sometimes too “literal” and missed nuance and “idiomaticism”.

What do you think of the quality of latin translations on ChatGPT out of interest?

Found these papers btw. A little more complimentary:

https://arxiv.org/pdf/2504.10660

In this latter paper they adopt some fine tuning to improve the LLM output.

Thanks a lot! I will have a look at it!

Regarding the question about the quality in latin, in my opinion Chat Gpt is not that bad. Especially when it comes to vocabs and so on the AI can give a lot of translation advices. Also, when translating simple textes (lets say clasess 5-9) then the output is still pretty good!

But when it comes to classic literatures such as Cicero and so on the result varies from my point of view.

But I have to admit, that there are so many ways to translate it properly so that even latin teacher would translate it in a way differently. So Latin is a quiet flexible language…

When using it as AI Tutor I find it more difficult that the AI asks the right questions and follows a structured plan to explain it understandable rather than just give the solution already

well hats off to you for working in this space - Latin education is imho really worth preserving!

This is definitely feasible with the current API stack, but I’d treat it as an orchestration problem rather than just a model selection problem. I’d keep the LLM focused on reasoning while using RAG for course materials, structured worksheet state, and a separate evaluation pipeline for answer checking. The biggest challenge isn’t generating explanations, it’s providing consistently accurate, pedagogically useful feedback. Strong prompt design, citations back to uploaded content, confidence checks, and human review for edge cases will go a long way. I’d also invest early in observability (logging, evals, and user feedback), since that’s usually what differentiates a demo from a production-ready tutoring assistant.

Thats true! I mean connecting my application with an API Key is not the challenge. But how do I ensure the quality? Regarding the RAG: How would you build up a feasible architecture in order to get a functional prototype done?

For RAG you need a vector database, preferably, e.g. postgres with pgvector (recommdended), or a specialised vector database (overkill)

You will then need jobs to scan your content and populate the database.

But why do you need RAG from the get-go?

Stepping back a little I think you don’t want to take on too much at once.

I would start with a functional project definition and take things step by step.

Split up your requirements into Epics, and tackle the most important one first.

Build your features iteratively.

Begin with your interactive bot discussion. Then you can tag things on.

Like I said, maybe just start with Discourse and use its inbuilt capabilities and then decide what you need to extend and improve later.

You can use the core AI plugin, but FYI my Chatbot allows users to discuss the contents of the pdfs they upload … and it implements RAG. Have a look at the code or use an AI coding agent to discuss the code with you.