I think you’ll find it a very hard challenge to detect cheating in this way. While there may be some obvious cases, there will be too many in the gray areas to make this worthwhile.
It’s better to change how we think about education and student evaluation in the era of digital companions than to try to force long-outdated paradigms on students of the future.
You may be right in that the method I mentioned may not work in catching all the ways students may try and cheat using the system. As of right now, I haven’t given it much thought and am much more focused on proving out the idea. If it proves to be valuable to students, I think the cheating problem can be figured out with enough time, people, and money.
im confused, isnt there already a specialized model out that assesses the probability of the order of words in a given document calculating the likelihood of ai generation? Am i missing something?
AI detectors worked to an extent with GPT-3.5 but not so much with GPT-4, and the sheer number of new models coming out from various companies and open source make it very difficult to reliably detect AI content with accuracy.
So, you want the Universities to fund You in order to be replaced by You? Really?
I am happy to help you with this. Being an educator and trying to bypass universities
Please contact me. I am trying to create something along these lines too. I have connections at universities but also D2c
A couple of the dev team have built this at my university. They used Azure GPT modules with data lakes.
I would caution you from going into ed-tech because, it doesnt scale easily. Selling to education is VERY different from any other sector. The users (students) are far removed from the organisational buyer (probably a faculty Dean, plus some additional senior leadership team members). The organisational buyer/s already have a solution to the problem you’re trying to solve - the library/internet/Google/resources on the LMS.
Lastly, I realised this was already mentioned, but your primary barrier to sales will be that no university will be prepared to have an unsupervised system hand out answers to students which could adversely impact their grades as this would leave the university (and in turn, your company) liable to litigation.
Hello!
I enjoyed your discussion and learned much.
I recently started working in IT in an institution, not as a software developer, but as a planner and facilitator. We offer trainings, seminars etc., many of them hybrid via Moodle.
We now have a project with Moodle in which we would like to integrate ChatGPT (preferably GPT4, or would 3.5 be enough?). If the learners are stuck on a question, for example, then they could ask ChatGPT and it would help (a bit). In contrast to your start-up idea we have a pilot project that will be tested with about 10-15 learners. And it is not a university, so the number of learners/subscribers is limited/manageable.
@elmstedt
Is it possible to integrate ChatGPT into Moodle fluently?
What do you need to consider technically?
Governance
@codie Why exactly are you for Azure GPT?
We have two service providers, one is currently working on Dialogflow from Google. But since that is not always optimal, because it is rigid and the dialog tree is predefined, we would like to integrate ChatGPT as well. ChatGPT is more flexible.
I read something from about the block_openai_chat. Is that usable? Is this only available in version 3.5?
Thanks a lot!
We’re already building something similar.
Thanks for the feedback. Do the devs that built this work at the university?
And if selling to Universities is so hard, would you say selling directly to students would be a better plan for now?
Yes, the devs work at the university. Selling direct to students would have far fewer barriers and is a much simpler sales process as your user and your buyer are the same person.
Yeah, to keep moving forward and making progress, I’ve decided to go straight to students which comes with its own challenges, mostly coming up with ways to get data from their courses like scraping and gathering data from public APIs. However, I’m still not giving up on eventually selling to Universities and continuing to go through the slow sales process. For scale and reach, it makes more sense to sell to Universities and onboard 10,000+ customers all at once while being free for the students to use.
They have a 99% Uptime Service Level Agreement, no cap limit on the total account bill (so you don’t need to get permission to scale your business), they have excellent customer support, startup Azure credits for AI companies, they have 40%-70%ish percent faster response time compared to OpenAI, and they have the exact same API as OpenAI so it is dead simple to migrate. Not to mention all of the other cognitive services you can tap into.
I work for Paradiso LMS. We have already implemented such feature in our LMS from last 3 months. If anyone is interested in a demo , please check out ww.paradisosolutions.com or paradiso.ai . Or you can email at sales@paradisosolutions.com . If anyone has questions or want to discuss, feel free to reach out.
@elmstedt What exactly do you mean with preventing cheating attempts? I think chatGPT should not be available during exams, and in the case of an exam in an LMS like Moodle you can open a separate room for the exam and or block/disable the interface to the chatbot.
@sanfran2502
How exactly did you train chatGPT afterwards spoecifically. There is a OpenAI Chat Block for Moodle, but it seems, as if you have to train it manually via question-answer pairs, don’t you? I would like to train it speifically on a topic with hundres/thousands of questions.
I am studying the docs at the moment. GPT-4 is the general model as far as I know, ChatGPT is the pre-trained chatbot. Do I have to use the OpenAI API for using/chosing GPT-4 or can I use ChatGPT directly? Are both pretrained or do I have to train GPT-4 on my own. Is it possible to integrate ChatGPT directly into one’s project or do you need to use GPT (OpenAI API)?
I didn’t say anything about preventing cheating. I said,
The main concern for many schools have had with generative AI like ChatGPT is students using the technology to write papers and do assignments.
I am just saying none of the tools designed to detect AI-generated writing are any good.
You should read the two posts above mine to understand the context of what I have written.
As was once the issue with calculators in school.
“You won’t have one in your pocket all your life”.
Jokes on you, Mr. Hall. It also plays video games and now has opinions.
The AI detector seemed to always be a temporary barricade for the zombies. Brainless drones charging and complaining about something they don’t bother to understand.
I know, I know. Not the same.
But, I’m very excited to see how some schools adapt to this new technology, and incorporate it into their curriculum. Kids are going to be so dang smart, or delusional. But, man, I can’t imagine how often a parent will hear “But ChatGPT told me so!”
@RonaldGRuckus The curricula will have to be adapted, the needed skills shift. metacognitive, creative, critical thinking skills and others AI is not (yet) capable of will be in focus for humans.