Moderation endpoint not sufficient to avoid blocked training files

[jswny]

I have a dataset of user messages I am trying to fine-tune a model on. For each training sample, I pass the sample into the moderation API first and remove it from the dataset if it is marked as flagged. Then, I upload the remaining data samples and try to run a fine-tuning job.

However, despite the fact that the moderation endpoint says all my samples are okay, I still am blocked by the same error:

The job failed due to an invalid training file. This training file was blocked by our moderation system because it contains too many examples that violate OpenAI’s usage policies, or because it attempts to create model outputs that violate OpenAI’s usage policies.

How else can I possibly clean my dataset for fine-tuning if not with OpenAI’s own moderation endpoint?

[Teo2423]

I just had the same issue. Solved it by removing from the dataset the samples that had at least one category from response.results[0].categories as True. Pretty bad that flagged don’t have the same criteria as the finetuning validation tho. They should solve it.

[jswny]

I tried this and it still didn’t work. I excluded every sample that had even a single category flagged, it reduced my sample count to 6% of what it was before.

Still, my tiny training file is getting rejected because of the content filters. I don’t even know what use fine tuning will be if I can’t figure out what data is valid.

[d20]

Same experience here, very frustrating.

The aggressive moderation on training data is really not ideal, but could be made manageable with some tooling or visibility into exactly which examples in dataset are being blocked.

[lefteris.loukas]

Same here. I found 17 fine-tuning samples out of 900 to be “harmful” using the OpenAI Moderation API independently. So, I removed them.

But, still, after presenting the rest (non-harmful) samples, the fine-tuning UI still shows the same error:

The job failed due to an invalid training file. This training file was blocked by our moderation system because it contains too many examples that violate OpenAI’s usage policies, or because it attempts to create model outputs that violate OpenAI’s usage policies.

It seems that OpenAI has a different model behind the Moderation API and the fine-tuning moderation. I’ve messaged the support team but I haven’t got any response. It’s sad because this blocks our development of custom models.

There should be some more helpful indicator on the fine-tuning platform, something that shows which sample is harmful or which sample violates any OpenAI policies.

[viji1]

I am having the same issue. I found one objectionable row and removed it from my training dataset. I renamed and uploaded the training data file. However, I keep getting this error message and am unable to finetune. Did anyone find a work around?

[SahPet]

Same here - made a script to automatically exclude all jsonl elements that do not pass the Moderation API call, but Still fails with the very non-informative message “Too many files were skipped due to moderation”. Very frustrating.

import os
import requests
import time
from openai import OpenAI
import json

os.environ[“OPENAI_API_KEY”] = …

api_key = …

client = OpenAI(api_key=api_key)

def check_image_moderation(image_url):
# Moderate image using the OpenAI API
try:
response = client.moderations.create(
model=“omni-moderation-latest”,
input=image_url
)
moderation_result = response.results[0]
flagged_categories =

    # Collect categories that are flagged as True
    for category, is_flagged in vars(moderation_result.categories).items():
        if is_flagged:
            flagged_categories.append(category)

    # Print result in a simplified format
    if not flagged_categories:
        print(f"Moderation result for {image_url}: all false")
    else:
        print(f"Moderation result for {image_url}: flagged categories - {', '.join(flagged_categories)}")

    return moderation_result.flagged
except Exception as e:
    print(f"Error during moderation request for {image_url}: {e}")
    return True  # If there is an error, assume it's flagged

def filter_images(jsonl_path):
filtered_images =

with open(jsonl_path, 'r') as file:
    for line in file:
        try:
            data = json.loads(line)
            # Extract the image URL
            image_url = data["messages"][0]["content"][0]["image_url"]["url"]
            #print(f"Checking image URL: {image_url}")

            # Use the moderation API to check the image
            if not check_image_moderation(image_url):
                filtered_images.append(data)
        except KeyError as e:
            print(f"Skipping entry due to missing key: {e}")

return filtered_images

def save_filtered_jsonl(data, output_path):
with open(output_path, ‘w’) as file:
for entry in data:
file.write(json.dumps(entry) + ‘\n’)

def upload_file(file_path, api_key):
headers = {
“Authorization”: f"Bearer {api_key}"
}
filename = os.path.basename(file_path)
mime_type = “application/jsonl”

with open(file_path, 'rb') as file:
    files = {
        'file': (filename, file, mime_type),
        'purpose': (None, 'fine-tune')
    }
    response = requests.post("https://api.openai.com/v1/files", headers=headers, files=files)

response.raise_for_status()
return response.json()

def create_fine_tuning_job(training_file_id, validation_file_id, model):
response = client.fine_tuning.jobs.create(
training_file=training_file_id,
validation_file=validation_file_id,
model=model,
hyperparameters={
“n_epochs”: 2, # Adjust as needed
}
)
return response

def check_fine_tuning_status(job_id):
while True:
job = client.fine_tuning.jobs.retrieve(job_id)
print(f"Job status: {job.status}")
if job.status in [‘succeeded’, ‘failed’]:
break
time.sleep(60) # Check every minute

def main():
jsonl_train_path = “output_data_train.jsonl”
jsonl_val_path = “output_data_val.jsonl”
filtered_train_path = “filtered_output_data_train.jsonl”
filtered_val_path = “filtered_output_data_val.jsonl”
model = “gpt-4o-2024-08-06” # Your specific model name

# Filter training images
print("Filtering training images...")
filtered_train_data = filter_images(jsonl_train_path)
save_filtered_jsonl(filtered_train_data, filtered_train_path)
print(f"Saved filtered training JSONL to {filtered_train_path}")

# Filter validation images
print("Filtering validation images...")
filtered_val_data = filter_images(jsonl_val_path)
save_filtered_jsonl(filtered_val_data, filtered_val_path)
print(f"Saved filtered validation JSONL to {filtered_val_path}")



# Upload filtered training JSONL file
print("Uploading filtered training JSONL file...")
try:
    result_train = upload_file(filtered_train_path, api_key)
    training_file_id = result_train['id']
    print(f"Uploaded filtered training JSONL file. File ID: {training_file_id}")
except requests.exceptions.HTTPError as e:
    print(f"HTTP Error during training JSONL file upload: {e}")
    return

# Upload filtered validation JSONL file
print("Uploading filtered validation JSONL file...")
try:
    result_val = upload_file(filtered_val_path, api_key)
    validation_file_id = result_val['id']
    print(f"Uploaded filtered validation JSONL file. File ID: {validation_file_id}")
except requests.exceptions.HTTPError as e:
    print(f"HTTP Error during validation JSONL file upload: {e}")
    return

# Create fine-tuning job
print("Creating fine-tuning job...")
try:
    job = create_fine_tuning_job(training_file_id, validation_file_id, model)
    job_id = job.id  # Corrected to use dot notation instead of dictionary-style indexing
    print(f"Fine-tuning job created. Job ID: {job_id}")

    # Monitor fine-tuning job
    print("Monitoring fine-tuning job status...")
    check_fine_tuning_status(job_id)
except Exception as e:
    print(f"Error while creating fine-tuning job: {e}")

if name == “main”:
main()