Hello all,
So I need to train a model where I give an input in a natural human language and my model gives me the SCPI command.
I’m attaching a link which gives the idea about what SCPI command is Example of a command: page - 417 in the document: “https://download.tek.com/manual/5_6-Series-MSO54-MSO56-MSO58-MSO58L-MSO64-Programmer-Manual_EN-US_077130505.pdf”
My end goal is my prompt should be ‘Measure frequency on ref1’ and the output be ‘MEASUREMENT:ADDMEAS FREQUENCY’. This command is not directly available in the pdf, but there’s example of ‘MEASUREMENT:ADDMEAS RMS’ so hoping OpenAI will learn it
So, I parsed the manual (above link) , created a jsonl file, built a fine-tuning model and when I tested the model it gave irrelevant answers (Probably google searched it)
I was told to give multiple examples for a single command, but that becomes an issue as there 1300 commands and giving each command 5 examples, would make it 6500 examples.
So can you guys help me?
Thank you
Looks like GPT3 already understand SCPI so you probably don’t need finetuning. You just need fewshot training.
1 Like
Thank you, should I be using completions for this task and can you tell me how exactly can I train it ?
Also, the generated output (in the screenshot you’ve posted) is not actually the expected command (Expected command is MEASUREMENT:ADDMEAS FREQUENCY)
Another thing to add, SCPI commands varies from model of oscilloscope. My current aim is just to automate it for ‘https://download.tek.com/manual/5_6-Series-MSO54-MSO56-MSO58-MSO58L-MSO64-Programmer-Manual_EN-US_077130505.pdf’ this manual. So my expectation is to get the commands only from this pdf.
Thank you.
To be completely honest, and I apologize if this is too direct, if you don’t know how to do few-shot learning then you’re not ready to do finetuning. Spend more time reading the documentation and learning to use GPT3 first in order to get the most out of it. Finetuning is 100x more effort than prompt engineering.
3 Likes
Thank you. I’ll look into it.