I am trying to create a InterviewerGPT Agent (with RAG included) wherein the agent will act as an interviewer – rather than a Q&A response bot.
Such agents have applicability in cases like job interviews, diagnosis, flowcharts, etc.
I’m curious if anyone has had success in using the “system” parameter (or custom instructions) to get ChatGPT to behave in this manner.
@cdonvd0s @curt.kennedy @bill.french – any thoughts on this?
More specifics (this is a hobby project of mine) : I’ve built a RAG with about 500 Pubmed papers about a very rare kidney disease (apparently only a couple of thousand people on Earth have this disease and I am helping their foundation with various gen AI uses cases).
The problem is: Instead of just Q&A on this RAG (which is easy) – I need the chatbot to act like an Interviewer, so that it can be used to have a conversation with someone wondering if they have the disease. So basically, the bot asks the questions (instead of the other way around).
I put up a quick demo on a Saturday morning (for more context)
Thoughts?
On the face of it, it seems fairly trivial to embed the interview questions and expected ideal answers along with programmatic flow control to ask a set of questions in order, build prompts from that flow… is the point of this to give a score at the end, or just to take the data? Also, is it expected to function as a pre-screening element for a larger recruitment pipeline?
I fully believe in what you’re doing here champ, helping people with rare diseases earns many points in my book.
I’ve been working on a similar project, that I’m not willing to share publicly yet, but do throw a personal message my way if you’re interested.
In short, I think you’re best bet is to use standard diagnostic interview forms in the system prompt to steer the conversation.
The flow might be quite complicated – so I am wondering if all of it can be jammed into the system value in one go.
For this specific case, yes – it would be like a score (or a suggested based on the score). I have other use cases, where the interviewer wants to kickoff other workflows based on the path taken (e.g. I have a large financial planning firm that wants to ask 8 questions and then take the user down some path)
This would be a wonderful use case too. In this specific case, it would be a pre-screening to urge the user to maybe get further testing.
For example, here is the pre-screening flow for this specific case:
Oh dang - love this – we should talk (the old-style forms are turning out to be a pain in a lot of cases)
Love it – will do.
Yeah – that’s what I was thinking at a high level – so rather than CONTEXT, we put in INSTRUCTIONS straight into the system.
Quick question: Do you have some examples of the standard diagnostic interview forms
In the context of generative AI, it prompts the question -
What comes after forms as we know them?
The forms providers are busy making their legacy approach more productive with AGI. This signals that disruption lies ahead for them, but what will it look like?
CustomGPT appears to have all the ingredients to demonstrate a better way to collect information. If your support team would help me get through ticket #5316, I might be able to demonstrate this next week. 
In my experience, relying solely on a system prompt and allowing it to operate autonomously is often an inconsistent method. This process can be enhanced through a more agent-driven approach, as outlined below:
Agent 1 is provided with the flowchart you posted in a Mermaid chart format. The fundamental question that must be presented to the user is: “What leads you to believe you have ___ disease?” Moreover, Agent 1 should have access to a specific function (function calling) that allows for interactions similar to consulting a doctor about symptoms. Except it will be consulting another agent.
Agent 2’s role is to translate the inquiry into three separate questions that may either assist in answering Agent 1’s question or facilitate further investigation. Agent 2 will direct these questions to Agent 3, utilizing a multithreaded approach. (This is done so that there is enough text in the question to match relevant chunks)
Agent 3, having access to research paper embeddings, will extract relevant information from these sources.
Agent 2, armed with the data obtained from Agent 3, will formulate additional questions for Agent 1.
Agent 1 will then assess whether further questions are required, or if a conclusion can be reached.
This approach necessitates a specific level of technical expertise for successful implementation.
For prototyping without writing code, you can try superagent.sh
Love it. Completely agree. Separation of Concerns ftw
Also agree, trying to pile everything into one prompt will yield less and less performant results as the number of tasks in the prompt spread the attention heads in too many directions at once, missing key points, hallucination and general low quality context referencing.
If the use case can stand the additional cost (latency can be handled with parallelism/streaming), then multiple prompts covering multiple topics and tasks is the way to go.
Fully agree here,
You can mitigate some of the cost by running some agents locally, not all tasks need to be done using state of the art technology, I’m personally using the core_web_sm model from spacy to find similar text, it’s 26mb you can run it on a laptop.
The system prompt should only be used to initiate and explain the interview.
Love it – YES !
Thank you – will check this out – I can easily create the 3 RAG-based agents – so putting it all together is the part that will need some coordination.
Yes - that is what I am seeing – also, after many turns, it is difficult for it to keep track (so it start repeating the questions)
Yeah – will try the multi-agent system first (cost is not a factor right now)
Thank you – this is working good in my initial tests. Will play around more with it (This was supposed to be my Saturday morning project – LOL – got too excited – back to work!)
Hey, good to know that someone trying to resolve the hiring issues.
I’m also doing similar one to gather the information based on their resume but challenges are that system is going in a cycle while asking questions based on their content.
I built a medical follow up plan conductor based on the similar idea, in medical scenario, medical professionals need to collect patient’s info as they are on a follow up plan due to prospective researches or on the purpose of given treatments.
We tried to build something like this over 5 years ago for recruiters using Dialog Flow. It was a little early, ChatGPT API may be the right tool.
You can build it as a custom LLM but that may not work well (we tried it for Python). Foundation models are too big and generic to customize. But here are some thoughts.
This is just the prototype.
Hi @bill.french, @alden et al - I am willing to volunteer to help test your products/projects and to showcase/promote them on the social platform I am building and to my linkedin followers.
I am interested in helping people to liberate their data from social platforms and to have a safe space to work on their life story, including a bot to interview people. I could see a bot that could help people with some issues that toxic social has worsened such as self image/self-harm. I haven’t been able to find funding and am considering going open source. The fundamental featureset has validated demand.
Interesting project!
The idea of using technology to capture personal histories will certainly help people if it gets off the ground.
Regarding angel funding, it’s a common misconception that all investors will see the broader societal value in every pitch. While your project has a societal impact, many investors prioritize financial returns, exit strategies, and profit margins.
Considering you’ve mentioned the idea of open-sourcing the project, this could indeed be a strategic move. By going open-source, you can garner community support, contributions, and possibly find collaborators who share your vision and can help bring it to fruition.
If you’re open to advise, I’d say try connecting with organizations or groups dedicated to mental health and storytelling. They might provide you with insights or partnerships that can fuel your journey. Additionally, consider creating a separate topic on this forum dedicated to your project. It’ll not only provide a dedicated space for discussions but also make your initiative more searchable and accessible to those interested.
I hope that helps 
Thanks for input. The validation does speak to financial side and demand. Best wishes.
“DiagGPT: An LLM-based Chatbot with Automatic Topic Management for Task-Oriented Dialogue” by Lang Cao (pdf) (DOI:10.48550/arXiv.2308.08043)
Found that just today in doing some research related to an SQL builder in this topic on the Discourse Meta forum.
While much of what is there is not of interest to you, there are enough touch points that the topic is worth a read for anyone with more than a passing interest in your pursuit.
Had to make me look up something.
Always happy to help
en_core_web_sm is a small English model trained on written web text (blogs, news, comments), that includes vocabulary, syntax and entities. The newest version is only 12mb (not 26, that’s an earlier version)
spaCy is an open-source library, not a platform, not an API and it’s not a out-of-the-box chat bot engine, or a company. While it can power conversational apps, it isn’t specifically for chatbots but offers underlying text processing capabilities. It’s built on recent research and aims for practicality, making different design choices than teaching and research platforms like NLTK or CoreNLP.
