GPT-4-Vision in Powershell - base64 works but URL does not

API
, ,
1

Has anyone managed to get gpt-4-vision-preview working in powershell?
Starting on the code here: OpenAI Platform
I’ve converted it to powershell like this:

$apiKey = "............."
$endpoint = "https://api.openai.com/v1/chat/completions"
$headers = @{
    "Content-Type" = "application/json"
    "Authorization" = "Bearer $apiKey"
}

$imageUrl = "https://www.google.com/images/branding/googlelogo/2x/googlelogo_light_color_272x92dp.png"

# Construct the body payload
$body = @{
    "messages" = @(
        @{
            "role" = "user"
            "content" = @(
                @{
                    "type" = "text"
                    "text" = "What’s in this image?"
                },
                @{
                    "type" = "image_url"
                    "image_url" = @{
                        "url" = $imageUrl
                    }
                }
            )
        }
    )
    "model" = "gpt-4-vision-preview"
    "max_tokens" = 3000
} | ConvertTo-Json

# Make the API request
$result = Invoke-RestMethod -Uri $endpoint -Headers $headers -Method Post -Body $body

# Access the response data
$result.choices[0].message.content

However the response seems to imply that it’s just processing the entire “type=image_url” content as text instead of as a formatted request, as the response I get is:

In .NET, a Hashtable is a collection of key-value pairs that are organized based on the hash code of the key [...]

I guess these might be early days and the feature’s not quite working yet, but I don’t see much discussion about it considering the millions of people who are assumedly jumping on board right now.

1 Like
2

The alternate option of converting a local image to base64 and sending it in the request works fine.

3

I think you should add “-Depth #DEPTHLEVEL#” to Convert-Json when using nested arrays.
I also would consider adding -Compress to the Convert-Json as well.

I’ve tried to test here, but my chatgpt-vision is not active.
Once it’s active I will return here with results.

4

yep… that was it.

Add “-Depth 10 -Compress” after your “ConvertTo-Json” and you’re good to go.

5

I appreciate your response but the issue was happening before I do any processing of it. After extracting the response’s message content, it was literally a paragraph explaining what a hashtable is.

6

Hi there, could you share the base64 version you had working?
I’m having some trouble with that, getting a response error around invalid JSON

7

Here’s a simplified version of what I ended up with

$imagePath = "C:\Users\<path to JPG or PNG>"

$imageBytes = [io.file]::ReadAllBytes($imagePath)
$base64Image = [convert]::ToBase64String($imageBytes)

$header = @{
    "Content-Type" = "application/json"
    "Authorization" = "Bearer $OpenApikey"
  }
  
$body = @{
    "model" = "gpt-4-vision-preview"
    "messages" = @(
        @{
            "role" = "user"
            "content" = @(
                @{
                    "type" = "text"
                    "text" = "$($myPrompt)"
                },
                @{
                    "type" = "image_url"
                    "image_url" = @{
                        "url" = "data:image/jpeg;base64,$base64Image"
                    }
                }
            )
        }
    )
    "max_tokens" = 4000
}
$bodyJson = $body | ConvertTo-Json -Depth 10

$response = Invoke-RestMethod -Uri "https://api.openai.com/v1/chat/completions" -Method POST -Headers $header -Body $bodyJson -ErrorAction Stop

$response.choices[0].message.content
1 Like
8

Thanks for getting back to me, I think I sorted it myself between when I asked and when you answered. For posterity though I came up with:


   $b64 = [convert]::ToBase64String((get-content -path $path -Encoding byte))


...

        $obody = @{
            model = "gpt-4-vision-preview"
            messages = @(
                @{
                    role = "user"
                    content = @(
                        @{
                            type = "text"
                            text = "summary"
                        },
                        @{
                            type = "image_url"
                            image_url = @{
                                url = "data:image/png;base64,$b64"
                            }
                        }
                    )
                }
            )
            max_tokens = 300
        } | ConvertTo-Json -Depth 10



        $url = "https://api.openai.com/v1/chat/completions"
        $response = Invoke-RestMethod -Method POST -Uri $url -Headers $oheaders -body $obody -ContentType application/json