File uploading issue with json

It seems it doesn’t upload json file anymore.
I’ve always used this python method:

  def file_upload(self,
                 file_name: str
                 ) -> object:
    """
    Upload a file to openai with given file_name

    Parameters
    ----------
    file : object
      DESCRIPTION.

    Returns
    -------
    object
      DESCRIPTION.

    """    
    file = None
    with open(file_name,'rb') as f:      
      try:
        file = openai.files.create(
          file=(os.path.basename(file_name),f),           
          purpose='assistants',
          )
      except Exception as e:
        self.logger.error(e)
    return file

But today it seems it accecpt only some json files.

ERROR 2024-01-02 10:38:42,182 Error code: 400 - {'error': {'message': "Invalid file format. Supported formats: ['c', 'cpp', 'csv', 'docx', 'html', 'java', 'json', 'md', 'pdf', 'php', 'pptx', 'py', 'rb', 'tex', 'txt', 'css', 'jpeg', 'jpg', 'js', 'gif', 'png', 'tar', 'ts', 'xlsx', 'xml', 'zip']", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Same problem experienced on playground.
image

Why?

p.s.
Happy New Year everyone!!

EDIT:
i think it depends on how is json file formatted.

With a json like this:

{
"test":"test"
}

it doesn’t work.

With json like this:

{
test:"test"
}

It works…
but both are valid json files :roll_eyes:

EDIT 2:
I think there’s something weird with JSON file:

This JSON file is uploaded fine even if is malformed (the comma at the end):

{
  "40268" : {
    "id" : 3
  },
  "40268" : {
    "id" : 4
  },
}

This JSON doesn’t work even if formally is correct!!!

{
  "40268" : {
    "id" : 3
  },
  "40268" : {
    "id" : 4
  }
}

I really don’t understand what could be the problem :roll_eyes:

I’ve put a few hours into this, making different uploaders, validators, and rewriters, and here’s my ultimate conclusion:

This is accepted as a .json:


{
  '40268' : {
    'id' : 3,
  },
  "40268" : {
    "id" : "test\nlines",
    "val" : +33,
    "3val" : .35,
  },
}

Why? A JSON5 library doing the upload validating.

Summary of Features

The following ECMAScript 5.1 features, which are not supported in JSON, have been extended to JSON5.

Objects

  • Object keys may be an ECMAScript 5.1 IdentifierName.
  • Objects may have a single trailing comma.

Arrays

  • Arrays may have a single trailing comma.

Strings

  • Strings may be single quoted.
  • Strings may span multiple lines by escaping new line characters.
  • Strings may include character escapes.

I’ve got a rewriter that will update the json if first refused, and it is taken in all cases above. But it is by regex. I’ll spend some more time later seeing if there’s a library feature that can do json->dict->json5 and force extra commas everywhere they can go.

The problem is that would get the file to code interpreter, but then you have a non “import json” file. Maybe just rename a txt?

1 Like

So it seems it’s JSON5, that’s ok. But anyway it doesn’t explain why “normal” json are refused. JSON5 is an extension of JSON, so it must be compliant with standard JSON file.
The strange thing is that if i upload a more structured json, with many other objects, it works :roll_eyes:
I really don’t understand.