Fine Tuning Question

I did finetuning and then sync results with Weights and Biases using below command.

$ openai wandb sync

Then I got summary message like below.

wandb: Run summary:
wandb:        classification/accuracy 0.95
wandb:           classification/auprc 0.99135
wandb:           classification/auroc 0.9911
wandb:            classification/f1.0 0.95385
wandb:       classification/precision 0.91176
wandb:          classification/recall 1.0
wandb:               elapsed_examples 961.0
wandb:                 elapsed_tokens 273001.0
wandb:               fine_tuned_model curie:ft-networks-co...
wandb:                         status succeeded
wandb:                  training_loss 0.15987
wandb:     training_sequence_accuracy 1.0
wandb:        training_token_accuracy 1.0
wandb:                validation_loss 0.3089
wandb:   validation_sequence_accuracy 1.0
wandb:      validation_token_accuracy 1.0

My question is, in here f1, precision, recall scores means validation score, not training score ? Is my understanding correct ?

And what is sequence_accuracy and token_accuracy. I would like to also know about these metrics.

Thank you,
Hiroshi

3 Likes

Thank you! @m-a.schenk I read the document and it was clear. I should have checked the documentation first. Sorry and thank you very much.

1 Like