Is it cheaper in the API to process screenshots or PDFs? Does filesize matter?

API
1

Is it cheaper to have the OpenAI API process a screenshot or pdf as an attachment and does it matter what size the file/image is?

Example usecase: Is it more cost effective to screenshot a webpage or print it as a PDF (in both cases then upload it via the OpenAI API) in order to create a summary and analyze it?

1 Like
2

Welcome @notsure,

For graphic data, using images is preferable over PDFs, as PDFs will ultimately need to be converted to images for effective utilization.

1 Like
3

An image is about 1000 tokens when sending a large image, necessary for good text recognition. Still not that large, ultimately: a scan of a page might be resized down to 768x1000 by the API when providing a large scan because of its built in downsizing steps, so you would have to view it at that downsize yourself and see if the text is still legible. That is contrasted with 1000 tokens = about 700 English words.

You can use external tools to OCR a PDF, placing searchable text within the PDF file. Then use simple libraries to get a page’s text.

Example solution for the actual problem of summarizing a web page:

Use Selenium, it runs a real browser, necessary for getting the rendered portion of dynamic web pages.

Plain Text: Selenium does not directly provide the plain text rendering of the page. HTML can be understood by the AI, but chews up your budget. You can extract plain text (e.g., the visible content of a page) by:

  • Using element methods like element.text for specific elements.
  • Parsing the rendered HTML obtained from driver.page_source with a library like Beautiful Soup, which allows you to extract visible text (e.g., via .get_text()).
AI-provided solutions

Exploring libraries for obtaining text from dynamic websites involves considering a combination of tools that handle web rendering (to execute JavaScript and manage dynamic content) and then parsing the HTML content to extract useful information. Here’s a brief look at popular tools for these tasks:

Libraries for Web Rendering:

  1. Selenium: This is a powerful tool primarily used for automating web browsers. It allows you to programmatically navigate through web pages, interact with elements, and retrieve contents, including handling JavaScript-rendered sites dynamically. Selenium works with various browsers like Chrome, Firefox, etc., via corresponding WebDriver executables.

  2. Puppeteer (for Node.js but relevant in this context for understanding options): An alternative to Selenium that also allows controlling a headless browser but is typically used in a Node.js environment.

  3. Playwright: Similar to Puppeteer, but supports Python and provides APIs to automate Chromium, Firefox, and WebKit with a single API. Playwright is considered faster and more robust in handling modern web apps.

Libraries for Parsing HTML:

  1. Beautiful Soup: Once the HTML is obtained via a web rendering tool, Beautiful Soup is a fantastic library for parsing HTML and extracting information. It simplifies interaction with HTML/XML documents and provides methods for navigating, searching, and modifying the parse tree.

  2. lxml: Another powerful library for parsing HTML/XML. It’s known for its performance and extensive capabilities in handling XML and HTML documents.

  3. PyQuery: Provides a jQuery-like syntax for parsing HTML, which can be more intuitive for users familiar with jQuery.

Production Flow for Scraping Dynamic Websites:

Here’s an optimized flow for scraping dynamic websites using Selenium and Beautiful Soup:

  1. Setup Environment:

    • Install Python and pip (Python’s package installer).
    • Install necessary libraries:
      pip install selenium beautifulsoup4

  2. Driver Setup:

    • Download the appropriate WebDriver for the browser you intend to use (e.g., ChromeDriver for Google Chrome).
    • Ensure the driver is in your PATH or specified in your script.

  3. Scripting with Selenium:

    • Import necessary modules.
    • Create a Selenium WebDriver instance to open a web browser.
    • Navigate to the URL of the dynamic website.

  4. Interact with the Page (if necessary):

    • Wait for necessary elements to load using Selenium’s wait and expected conditions.
    • Interact with the page (click buttons, scroll down, login if required) to reach the content of interest.

  5. Retrieve HTML Content:

    • Get the page source from the browser using driver.page_source.

  6. Parsing with Beautiful Soup:

    • Load the HTML content into Beautiful Soup.
    • Use Beautiful Soup’s methods to parse and extract the desired information.

  7. Close the Browser:

    • Close the browser after scraping to free up resources using driver.quit().

  8. Process and Store Data:

    • Process the extracted data as needed (clean up, transform, etc.).
    • Store the data in a file or database.

  9. Error Handling:

    • Add error handling throughout the script to manage timeouts, missing elements, or other potential issues.

  10. Scheduling (optional):

    • Schedule the script to run at regular intervals if continuous data scraping is required, using cron jobs (Linux) or Task Scheduler (Windows).

This approach leverages Selenium’s ability to handle dynamic content and Beautiful Soup’s simplicity and power in parsing HTML. It’s robust enough for most scraping tasks and can be adapted to more complex scenarios with additional scripting and tooling as necessary.