BATCH Api errors max_tokens is too large

[_j]

This post has a quick script I wrote to count per-example tokens on a JSONL file, in the process validating the file format. It does not deal with functions.

Fine-Tuning of my personal blog

tiktoken, which I mentioned above, is a python token counting library that you can install simply with pip. pip install tiktoken Because I’m a fun guy, I just sat down and coded a per-line token counter for jsonl training files. import json import tiktoken class Tokenizer: def __init__(self, model_name): self.tokenizer = tiktoken.get_encoding(model_name) def count(self, text): encoded_text = self.tokenizer.encode(text) return len(encoded_text) def messag…

You can omit the max_tokens parameter from API calls, leaving the AI to potentially produce the maximum, or accept an input so large there is little context length remaining to form an answer.

A typical max_tokens setting for a response is 2000, returning an error if input + output reservation is over model capabilities, ensuring enough space is available to form a complete response.