PHP: Fine-Tuning + Mac Terminal = Frustration

I browsed a post or two here, such as this one. The linked video that “really helps” no longer exists.

I’ll hunt around on YouTube. I’ve also posted to the Discord mess.

The story goes …

I’ve uninstalled OpenAI on my Mac and am going to restart with a pRam reset.

  1. I’m then going to pip3 install openai.
  2. Next, I’ll cd to a folder with a .py file in it.
  3. That file will ask about France and the API will respond with “Paris” or whatever.

Great. It did a thing! I suppose that means I’m good to go, right?

Wrong.

I have solid fine-tuning data that I spent all day yesterday building. Carefully.

And no matter what I do, I can’t upload it to the API.

The commands on this page return OpenAI “can’t find function”, “syntax error”, “OpenAI doesn’t even know how it’s installed so anyhow good luck and enjoy your breakfast maybe just pour a soda onto your keyboard” type responses.

What gives?

If anybody has a PHP file that can do the work via a form, I’d love to run that instead.

In the end, my hope is to run the following command …

openai api fine_tunes.create -t <TRAIN_FILE_ID_OR_PATH> -m curie --suffix <CUSTOM_NAME>

Again, a PHP curl script would be HUGELY welcome.

UPDATE: Here’s what I get from the pip3 install (note the errors)

pip3 install openai
Defaulting to user installation because normal site-packages is not writeable
Collecting openai
Using cached openai-0.26.4.tar.gz (55 kB)
WARNING: Value for prefixed-purelib does not match. Please report this to https://github.com/pypa/pip/issues/10151
distutils: /private/var/folders/_y/vglg_5w11zxbnzybjf_knjmh0000gp/T/pip-build-env-pyrh4i9a/normal/lib/python3.9/site-packages
sysconfig: /Library/Python/3.9/site-packages
WARNING: Value for prefixed-platlib does not match. Please report this to https://github.com/pypa/pip/issues/10151
distutils: /private/var/folders/_y/vglg_5w11zxbnzybjf_knjmh0000gp/T/pip-build-env-pyrh4i9a/normal/lib/python3.9/site-packages
sysconfig: /Library/Python/3.9/site-packages
WARNING: Additional context:

… and it just gets uglier.

2 Likes

Update #2

Seems it’s installed now and I’ve updated the file locations using a Terminal prompt supplied by ChatGPT.

Now, I’m faced with the following:

OpenAI error: 

    missing `pandas` 

This feature requires additional dependencies:

    $ pip install openai[datalib]

Update #3

Pandas is installed. Excellent.

Now I’m told my .json file isn’t valid. Hmmm.

I copied my .json files contents into here (a known, excellent resource): https://jsoneditoronline.org/

Copied left to right (which reformats), then copied right to left. Then selected all, pasted into my doc, saved and tried again.

I’m told …

ERROR in read_any_format validator: Your file '` does not appear to be in valid JSON format. Please ensure your file is formatted as a valid JSON file.*

1 Like

Update #4

ANNNNND Working. Okay. Cool.

My summary:

  1. pip3 uninstall openai
  2. restart + pram reset
  3. pip3 upgrade
  4. pip3 warned about directories, sssooo
  • nano ~/.bash_profile
  • export PATH=$PATH:/Users/robindean/Library/Python/3.9/bin
  • close terminal, then open new terminal
  • source ~/.bash_profile
  1. OpenAI working. Except it needs “pandas”, sssssoooo …

  2. pip3 install pandas

  3. Working! But the commands keep returning ‘invalid json’ type errors, sssssssooooooo …

  4. Used an online json validator (mentioned above), saved the file via that validator.

  5. Profit!

3 Likes

Thanks for the updates. May they help someone in the future…

FWIW @robin.n.dean

I develop and test OpenAI API code daily on macOS using Ruby On Rails and the ruby-openai gem and it works great.

Its easier and more robust than PHP in my view.

Good on ya.

This, however, was day one for a web developer, not a terminal command soldier.

And if you don’t already know underwater basket weaving … well …

(you get the point)

1 Like

Yeah I know.

That is why I recommend you get started with Ruby v. PHP as a beginner. Ruby is a fully object-oriented scripting language and is easy (and fun) to develop with as well.

Also, the Rails console is invaluable for testing methods, ideas and debugging.

Anyway, glad you got PHP set up; enjoy the API!

2 Likes

Actually, I didn’t get the PHP thing to go.

I ended up going the Ruby route after all.

My world is all PHP > HTML5 > CSS > JS … with the occasional clunky attempt at curl & shell via PHP (ugh, I’m a cluts).

When presented with terminal + a variety of different languages I don’t speak at my command LOL … it gets rough.

But I got through it.

What I DID do with php was explode entire blog articles sentence by sentence into prompt:completion pairs. The system is currently processing my fine tuning through these means.

4000+ pairs. I’m excited to see how it turns out.

Yeah, I stopped processing text with PHP years ago (after countless years of PHP development) and only use Ruby “everywhere I can” this is easy to do with Ruby:

Your Task in PHP

explode entire blog articles sentence by sentence into prompt:completion pairs.

However, you code as you like :slight_smile:

Cheers and All the Best.

1 Like

Actually, I could use some help here.

I’m trying to generate blog posts in my voice, to be polished and edited after the fact.

So far, I’m $25 into completions that are complete gibberish.

So, I’m “learning” Python, which is excellent as it places the workload on my computer instead of the web server.

I still need to use PHP solutions to extract existing site data but I then write it to a file for local processing.

WHICH … by the way, I’m pretty bad at LOL. Not the code side. The logic/mindset of it.

Time will tell. Albeit with a price tag.

This was very helpful, thank you OP.

The error message I received was "OpenAI error:

missing `pandas` 

This feature requires additional dependencies:

$ pip install openai[datalib]"

But as you wrote, the solution is simply “pip3 install pandas”. Thank you.