Hello, my name is Glenn. It’s a pleasure to "meet’ you. I am in dire need of some direction / advice. I am an English teacher living in Japan. I have an idea to increase the effectiveness of speaking practice for my students. I typically use google docs, with the voice to text function, and give students a target text to read / memorize. We then compare the original text to the text they generated with their speech. I want to automate this concept, specifically I would like to have an app that will compare the base text to the students’ text, and then generate a percentage of accuracy. My intuition tells me this isn’t such a big problem, I just lack the skills to proceed. What do you suggest?
2 Likes
Thanks for the regex code I can use to prevent certain words or phrases. But I think that does not solve my problem.
The problem is that davinci partially twists the meaning.
davinci gives answers to the same task with the correct sense one time and with the wrong sense the next time.
Here again the example with the wrong sense:
Answer from a customer to initial e-mail: “I have no need for your offer at the moment.”
davinci: “I am glad to read that you have no interest in our offer at the moment.”
Nobody is glad that a customer has no interest at the moment, but all words davinci used would fit in another context.
Like here:
Answer from a customer to initial e-mail: “I am glad you wrote, I happen to have a need for your offer right now.”
davinci: “I am glad to read that you have interest in our offer at the moment.”
You can see, except for the “no”, all the words in the sentence are good words that davinci would be allowed to use in the answer.
The only difference is the missing “no” which gives the sentence a different meaning.
And “no” shouldn’t be deleted from the vocabulary with regex either.
I also want to repeat my questions regarding semantic search:
What can I achieve with semantic search? What does it do? How do I apply it?
Hi Glenn,
Thanks for sharing your project, just so that I understand your currently workflow.
Are you currently using the different google doc and compare for the difference similar to the image below?
2 Likes
A simple solution is the include a description on what kind of answer you are looking for in the prompt to lead GPT towards the answers you are looking for. But if you already try that and if it doesn’t work, you can try more expensive approach.
I have created a spreadsheet you can use to test it out this process. See How to use custom data with GPT3 without fine tuning
1. Create two categories of phrases, one for approval and one for disapproval. In this example Row 2 is consider as approval and Row 3 is disapproval, you can add more text if you need.
2. Use Cosine Similarly to validate the GPT response with your embeddings, notice on row 3 we have detected that the reply is not what you are looking for because it belongs in the disapproval category.
3. So we can ask GPT to paraphrase it to something better
Technical Summary
We are using a text classification approach to determine what type of results we are getting, and using a text generation approach (GPT) to get the results until we are satisfy.
Obviously this approach will have a higher cost so you might want to put a max number of retry to limit your API usage.
Let me know if you have any other questions, I’ll be glad to help.
2 Likes
Nelson, good morning. Thank you so much for your reply. So, currently I will either write a short story, or dialog, then have my students practice reading / speaking the text. Next, I will separate students into teams and have them compete against each other by using speech to text function in a google doc. I compare each of the competitors resulting text to the original text and judge the winner by whoever was the most accurate. So, what I am looking for is a program that will do the comparison for me automatically. In my mind I see two large text boxes, one text box has the target text that the students have to use, the other text box is where they will use a speech to text function to read the target text. Then the “program” will compare the student’s text to the original and give a percentage of accuracy. I really appreciate your time Nelson, I can’t tell you how much it means to me. Thank you.
2 Likes
Thank you for sharing your idea with me. I think this can be build using OpenAI Whisper for speech to text and GPT is not required. If you can send me a sample of the original text and a student recording, I’ll be happy to take a look.
3 Likes
Thank you very much, I’ll take a closer look and also try to get a better result with changes to the prompt.
The more I look into the subject of finetuning, the more doubts arise with me whether it makes sense at all.
As far as I know, you can not finetune davinci-003, but only the basic models.
Another user has also written the following:
I do not know how he could fintune davinci-002, but the point is anyway significantly more severe compared to the base model.
One might invest a lot of money in finetuning and get no ROI, because the foundation on which one builds is much too bad to get a good result.
What is your opinion about that?
1 Like
Hi @read2text
It shouldn’t be too difficult, I was able to implement Whisper into one of the project I am working on in a couple days.The source code for Whisper is located here GitHub - openai/whisper: Robust Speech Recognition via Large-Scale Weak Supervision
Just make sure you follow the instruction on the readme and remember to install ffmpeg. Also you might want to run it on a machine with a minimum of 4 CPU. Let me know if you run into any issues.
3 Likes
I agree, most people who tried the fine tuning API do run into these kind of issues.
For me, so far doing prepossessing of the prompt and validating of the output seems to be working well on most use cases.
I was hoping to use AI to help build my database for a dating app that I would be able to create on app sheet. Is it possible to do this ny creating the tables, expressions, and data model? Ive created a detailed process flow, woreframed up the UI but nothing has been done on the backend. Im really green in IT but this has been an awesome learning experience!
Thanks for sharing your project @cryptofrye.
Feel free to reach out to me here on PM or linkedin on what you are trying to build.
I’m interested in learning more on what you have in mind in terms of building the database.
Thanks.
hi nelson I am new to ai a senior trying to use it could like some training other than youtube can you help me I have been trying to write a book for 7 years but need help past the first few pages and introduction
thanks a old senior who wish to be a writer!
Tim
Sure, welcome the community.
Feel free to PM me with any questions you have and I’ll be happy to help.
Hey, I’m not sure if this is too advanced for you. I have only played around with ChatGPT and got very limited programmer language.
I’m looking to make an Astrology Guide.
However, I have made my own System, which is not Astrology, but similar.
So I need an AI to be able to receive questions, and fetch answers from my system, and generate logical sounding sentences with explanations.
I know normal Astrology Systems use a Swiss Ephemeris: Astrology Ephemeris Files for 9000 years - Astrodienst
However, my system use a different way to decipher the solar system.
Is it possible to use GPT-3 for this purpose, or is it too advanced?
Any recommendations of how to put it together?
Disclaimer:
I know Astrology is seen as pseudoscience, which I understand. However, my system will be recognized by Science in the future, it follows the scientific method and is replicable to get same results. 
Hi @MasterKnutA
Thank you for sharing your project, sounds very interesting.
I assume your currently system doesn’t use NLP to request information, and if that is the case, is this the type of solution you are looking for?
- User asks a question thru a chat interface.
- Modify the question to a protocol that your system will understand using GPT (ex: sql, api)
- Request information from your system
- Use GPT prompt engineering to modify the result from your system to a user friendly response
- Repeat steps 1-4
1 Like
I am a complete programming novice but have seen tremendous use for chat gpt in the legal field. as an attorney, I want to create a chatbot that can help with legal research, negotiations, case valuations, document production, automation, simulation of cases. I want to find a way for the chatbot to be easily updated. I would love to discuss the project in more detail
Hi @arta,
That makes a lot of sense, I think it’s a very good use case, let’s connect.
@nelson let me know the best way to do so. [my username]@gmail.com
Hi Nelson,
Few years back I have developed a mobile app that can give accurate result based on search query instead of giving millions of search results like google. This was not AI based but with human interaction. After reading your post, I am willing to discuss with you on how we can further enhance this app. Let me know how we can proceed.
Hi @Absha
Thanks for sharing your project with me. I’m interested to know how a human interaction search query will work as well. Feel free to reach out to me via PM or linkedin when you are ready.
Thanks.
https://www.linkedin.com/in/nelsonchu247/