Using image URL in images/edits request

API
41

Yeah, images can be tricky. Hoping you get it sorted.

42

me and @raymonddavey are almost positive there’s an issue on open AI’s end. either the endpoint is broken, or something vital is missing from the docs. we tinkered and tinkered last night, but nothing worked in the end. hopefully we hear something from the devs about it before too long

1 Like
43

You can specify the response_format as ‘b64_json’ to get the generated image returned as base64 data

  const response = await openai.createImage({
    prompt: prompt,
    n: 1,
    size: "256x256",
    response_format: "b64_json",
  });

  const base64 = response.data.data[0].b64_json;

This base64 data can then be used to create a buffer which can be passed into the API request

  const buffer = Buffer.from(base64, "base64");
  // Set a `name` that ends with .png so that the API knows it's a PNG image
  buffer.name = "image.png";

  const response = await openai.createImageVariation(buffer, 1, "256x256", "b64_json");

  base64Data = response.data.data[0].b64_json;

Full code to create a new image and a variation of the generated image:

  const response = await openai.createImage({
    prompt: "digital artwork of a robotic lion wearing a crown",
    n: 1,
    size: "256x256",
    response_format: "b64_json",
  });

const image = response.data.data[0].b64_json;

// This is the Buffer object that contains the generated image base64
const buffer = Buffer.from(image, "base64");
// Set a `name` that ends with .png so that the API knows it's a PNG image
buffer.name = "image.png";

const response = await openai.createImageVariation(buffer, 1, "256x256", "b64_json");

const variation = response.data.data[0].b64_json;

Capture
Capture888×435 62.4 KB

Hope this helps :slight_smile:

4 Likes
44

I wish more people here would reply like you @swiftrees with “working code” (and API responses and error messages) instead of opinions and theories without working code and detailed error messages from the API.

Well done!

Please stick around and keep posting working solutions in code!

:+1:

4 Likes
45

Thanks for sharing, I’ve tried using your strategy but I get an error requesting the png be rgba as opposed to rgb. I can’t quite figure out how to add the alpha channel.

edit: I see now openai.createImageEdit() requires the image to have transparency if a mask is not provided.

46

Based on all of these experiments, does anyone know if I can pass in a base64 url as an image parameter for edits or variations?
Unfortunately I’m limited to not being able to upload files directly in the fetch request, so I only can use either URLs or Base64 strings.

Any help would be appreciated.
Thank you!

47

ok so iv managed to get my one up and running by creating a mask exactly the dame size as th original image and with transparency in the mask in the area you want the most modification to take place

here is what’s in my payload

in the payload everything seems perfect, here.:

Content-Disposition: form-data; name=“prompt”
make the windows of the door in the image bright yellow

Content-Disposition: form-data; name=“n”
1

Content-Disposition: form-data; name=“size”
256x256

Content-Disposition: form-data; name=“image”; filename=“fauvism.png”
Content-Type: image/png
(i can only upload one image as a new user but this image is the same as the mask but complete without the transparency)

Content-Disposition: form-data; name=“mask”; filename=“mask.png”
Content-Type: image/png

48

This is a great thread, ty community!

Working with images is my next goal for my personal project. Recently I was able to integrate google image search to allow response to have embed images into the responses, and now I am looking to try also utilize google’s Vision api to feed back into the workflow.

I figure the actual coding for handling images is modular enough that my hope is I can find it not too difficult to use any number of APIs along side future modifications.

1 Like