Categorization of website by list

API
1

Hi I’m trying to use the API to read a website’s content and then determine which categories from a preexisting list of niches I want it to fit into. I have tried training OpenAI and yet it seems to still make up niches I don’t have on my list. I’m not sure what’s the best way to go about doing this using OpenAI.

I also want it to read the website content which is another issue. I am figuring that using Python to extract the website contents and giving OpenAI is the best way of doing that? Any insights to either issue would be helpful!

2

Hi Ludwig and welcome to the Forum! Do you mind sharing your current prompt?

I have been using GPT models quite a bit for classification tasks, using pre-defined lists of categories. Happy to take a look at your prompt and help refine further.

In general, I can recommend fine-tuning a GPT 3.5 model for that purpose. It will create more consistency in your responses. That said, with some targeted prompt refinement you may also achieve it without going down that route.

1 Like