How to use DALL-E 3 in the API.
Because new versions of the OpenAI Python library are being continuously released - and because API Reference and Cookbooks, and github are USELESS to describe what to do with the data return (or even show how to catch the API return) - I thought I’d demonstrate a basic application for you.
The script I’ve provided is linear, progressing through building the prompt and the parameters you’ll need to send to the images endpoint. We start with a basic request to DALL-E 2 with all the parameters you might employ, and then I add addition parameters needed to switch to DALL-E 3, and then tweak DALL-E 3 with its new options.
Price Warning: 4-12 cents per image
| Model | Quality | Resolution | Price |
|---|---|---|---|
| DALL·E 3 | Standard | 1024×1024 | $0.04 / image |
| Standard | 1024×1792, 1792×1024 | $0.08 / image | |
| DALL·E 3 | HD | 1024×1024 | $0.08 / image |
| HD | 1024×1792, 1792×1024 | $0.12 / image | |
| DALL·E 2 | 1024×1024 | $0.02 / image | |
| 512×512 | $0.018 / image | ||
| 256×256 | $0.016 / image |
Preliminary to just use:
Write your prompt, set your parameters
Within this script, I have the parameters outside the actual API call, in a dictionary. This allows easy amending. The keyword argument dictionary is then unpacked in to parameters.
With this method to “turn on” parts of the parameters, you can have dynamic code based on user input.
For the script as it is, remove the # of a comment to enable the additional DALL-E 3 parameters, or comment out the response_format line to switch from images included in API response to images that are a separate download.
Also, write something nice in the prompt strings for AI to generate.
(let a bot describe the code)
OpenAI DALL-E Image Generation Tutorial
This tutorial provides a step-by-step walkthrough of this script that uses OpenAI’s DALL-E image generation model to generate images based on a given prompt. The script showcases how to use the OpenAI Python library (version 1.2.3 or later) to make API calls, handle errors, process images with the Pillow library, and display the generated images using Tkinter.
1. Importing Required Libraries
The script begins by importing the necessary libraries. These libraries include openai for making API calls, datetime for handling timestamps, base64 and Pillow for image processing, requests for downloading images, and Tkinter for displaying the generated images.
2. Checking OpenAI Library Version
The script then checks the installed version of the OpenAI library. If the installed version is older than 1.2.3, the script raises a ValueError and instructs the user to upgrade the library.
3. Instantiating the OpenAI Client
The script then creates an instance of the OpenAI client. This is a new method from previous python libraries. This client instance will be used to make API calls. Note that the script uses the OPENAI_API_KEY environment variable to authenticate the client, which you can set in your operating system.
4. Setting Image Generation Parameters
Next, the script sets the parameters for the image generation request. These parameters include the model to use (dall-e-2), the number of images to generate (1), the size of the images (1024x1024), the prompt to guide the image generation, and a user ID for abuse monitoring.
5. Making the API Request
The script then makes the API request to generate the images. It uses a try/except block to handle potential errors that may occur during the request. If an error occurs, the script prints a relevant error message and re-raises the exception.
6. Processing the API Response
After successfully making the API request, the script processes the API response. It first extracts the timestamp from the response and formats it to create a filename prefix for the generated images. It then extracts the images from the response. If the images are returned as URLs, the script downloads the images; if the images are returned as base64-encoded JSON data, the script decodes the images.
6a. Saves the images to the script directory
This is just employing the runtime directory of the python script, creating timestamped images with a prefix.
7. Displaying the Generated Images
Finally, the script displays the generated images using Tkinter. It creates a new window for each image, resizes the image if necessary, and adds the image to a label in the window. (note that this display blocks further script execution until those thumbnail windows are closed)
Here’s the code.
I could have put it in many notebook-like parts so you can read it in the forum, but much better for you to just paste it into VSCode or IDLE and have a read there. Don’t run random code without understanding!
'''
DALL-E image generation example for openai>1.2.3, saves requested images as files
-- not a code utility, has no input or return
# example pydantic models returned by client.images.generate(**img_params):
## - when called with "response_format": "url":
images_response = ImagesResponse(created=1699713836, data=[Image(b64_json=None, revised_prompt=None, url='https://oaidalleapiprodscus.blob.core.windows.net/private/org-abcd/user-abcd/img-12345.png?st=2023-11-11T13%3A43%3A56Z&se=2023-11-11T15%3A43%3A56Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-11-10T21%3A41%3A11Z&ske=2023-11-11T21%3A41%3A11Z&sks=b&skv=2021-08-06&sig=%2BUjl3f6Vdz3u0oRSuERKPzPhFRf7qO8RjwSPGsrQ/d8%3D')])
requires:
pip install --upgrade openai
pip install pillow
'''
import os
from io import BytesIO
import openai # for handling error types
from datetime import datetime # for formatting date returned with images
import base64 # for decoding images if recieved in the reply
import requests # for downloading images from URLs
from PIL import Image # pillow, for processing image types
import tkinter as tk # for GUI thumbnails of what we got
from PIL import ImageTk # for GUI thumbnails of what we got
def old_package(version, minimum): # Block old openai python libraries before today's
version_parts = list(map(int, version.split(".")))
minimum_parts = list(map(int, minimum.split(".")))
return version_parts < minimum_parts
if old_package(openai.__version__, "1.2.3"):
raise ValueError(f"Error: OpenAI version {openai.__version__}"
" is less than the minimum version 1.2.3\n\n"
">>You should run 'pip install --upgrade openai')")
from openai import OpenAI
# client = OpenAI(api_key="sk-xxxxx") # don't do this, OK?
client = OpenAI() # will use environment variable "OPENAI_API_KEY"
prompt = (
"Subject: ballet dancers posing on a beam. " # use the space at end
"Style: romantic impressionist painting." # this is implicit line continuation
)
image_params = {
"model": "dall-e-2", # Defaults to dall-e-2
"n": 1, # Between 2 and 10 is only for DALL-E 2
"size": "1024x1024", # 256x256, 512x512 only for DALL-E 2 - not much cheaper
"prompt": prompt, # DALL-E 3: max 4000 characters, DALL-E 2: max 1000
"user": "myName", # pass a customer ID to OpenAI for abuse monitoring
}
## -- You can uncomment the lines below to include these non-default parameters --
image_params.update({"response_format": "b64_json"}) # defaults to "url" for separate download
## -- DALL-E 3 exclusive parameters --
#image_params.update({"model": "dall-e-3"}) # Upgrade the model name to dall-e-3
#image_params.update({"size": "1792x1024"}) # 1792x1024 or 1024x1792 available for DALL-E 3
#image_params.update({"quality": "hd"}) # quality at 2x the price, defaults to "standard"
#image_params.update({"style": "natural"}) # defaults to "vivid"
# ---- START
# here's the actual request to API and lots of error catching
try:
images_response = client.images.generate(**image_params)
except openai.APIConnectionError as e:
print("Server connection error: {e.__cause__}") # from httpx.
raise
except openai.RateLimitError as e:
print(f"OpenAI RATE LIMIT error {e.status_code}: (e.response)")
raise
except openai.APIStatusError as e:
print(f"OpenAI STATUS error {e.status_code}: (e.response)")
raise
except openai.BadRequestError as e:
print(f"OpenAI BAD REQUEST error {e.status_code}: (e.response)")
raise
except Exception as e:
print(f"An unexpected error occurred: {e}")
raise
# make a file name prefix from date-time of response
images_dt = datetime.utcfromtimestamp(images_response.created)
img_filename = images_dt.strftime('DALLE-%Y%m%d_%H%M%S') # like 'DALLE-20231111_144356'
# get the prompt used if rewritten by dall-e-3, null if unchanged by AI
revised_prompt = images_response.data[0].revised_prompt
# get out all the images in API return, whether url or base64
# note the use of pydantic "model.data" style reference and its model_dump() method
image_url_list = []
image_data_list = []
for image in images_response.data:
image_url_list.append(image.model_dump()["url"])
image_data_list.append(image.model_dump()["b64_json"])
# Initialize an empty list to store the Image objects
image_objects = []
# Check whether lists contain urls that must be downloaded or b64_json images
if image_url_list and all(image_url_list):
# Download images from the urls
for i, url in enumerate(image_url_list):
while True:
try:
print(f"getting URL: {url}")
response = requests.get(url)
response.raise_for_status() # Raises stored HTTPError, if one occurred.
except requests.HTTPError as e:
print(f"Failed to download image from {url}. Error: {e.response.status_code}")
retry = input("Retry? (y/n): ") # ask script user if image url is bad
if retry.lower() in ["n", "no"]: # could wait a bit if not ready
raise
else:
continue
break
image_objects.append(Image.open(BytesIO(response.content))) # Append the Image object to the list
image_objects[i].save(f"{img_filename}_{i}.png")
print(f"{img_filename}_{i}.png was saved")
elif image_data_list and all(image_data_list): # if there is b64 data
# Convert "b64_json" data to png file
for i, data in enumerate(image_data_list):
image_objects.append(Image.open(BytesIO(base64.b64decode(data)))) # Append the Image object to the list
image_objects[i].save(f"{img_filename}_{i}.png")
print(f"{img_filename}_{i}.png was saved")
else:
print("No image data was obtained. Maybe bad code?")
## -- extra fun: pop up some thumbnails in your GUI if you want to see what was saved
if image_objects:
# Create a new window for each image
for i, img in enumerate(image_objects):
# Resize image if necessary
if img.width > 512 or img.height > 512:
img.thumbnail((512, 512)) # Resize while keeping aspect ratio
# Create a new tkinter window
window = tk.Tk()
window.title(f"Image {i}")
# Convert PIL Image object to PhotoImage object
tk_image = ImageTk.PhotoImage(img)
# Create a label and add the image to it
label = tk.Label(window, image=tk_image)
label.pack()
# Run the tkinter main loop - this will block the script until images are closed
window.mainloop()
Also, I didn’t dare put it through a formatter or pep8 checker…at the very least due to my readable comments
Conclusion
By delving into my hack job of code, you should now have a good understanding of how to use the OpenAI Python library to generate images with DALL-E 2 and 3, process the generated images, handle potential errors, and display the generated images. You should now be able to modify this script to suit your own needs, or use it as a starting point to create your own program, functions, classes, GUIs, backends that use the new OpenAI Python library.
Example output using the same prompt:
Dall-E 2 - very “artistic” and can also make photorealistic people, but often with errors.
Dall-E 3 - a very “digital art” style is common, although it is versatile by prompting specifically for other styles. DALL-E 3 has an AI between you and the actual function that re-writes, translates, and sanitizes the input to remove mentions of people and 20th century artists.
