File search storage usage bug

I think there must be a bug in your Usage portal related to file search storage. In the second usage chart below, it shows that we are using 133GB each day. In the first taken from the Spend categories, you’ll see that you’re charging about $0.52 for the same day. In reality, I can believe that we have about 5.2GB of storage – which would be consistent with the $0.52 per day. My guess is that there is just a bug in the usage chart.

Let’s take a look at your maximum day:

>>> usage_max=153_383_192_439
>>> print(usage_billed_gb:=(usage_max/2**30)-1)
141.84922968503088
>>> print(cost:=usage_billed_gb*0.10)
14.184922968503088

Let openai know to increase your daily bill to $14?


I can pile on. I’ve got vector stores in megabyte-hours:

Go to the depicted link:

https://platform.openai.com/usage/vector-stores


We can look another way: Use the organization admin API.

I shouldn’t get any “costs” line items for vector stores on the costs endpoint, as I’m well under 1GB.
However, If the vector store usage endpoint was supposed to return “current bytes” and not “billed bytes”, even bytes up-to-the-minute by parameter … that’s not getting done for me either.


API admin endpoint code for you to try, that will probably say the same “bytes” as the usage page. If so, you might need to ultimately iterate through the size of all vector stores in a vector store list - if the listing’s not broke also.

Report if you get nothing or problems with the printing in this code, as I have nothing to print. (Try “group_by”: [“project_id”] in get_recent_vector_usage()to break the printing too).

'''OpenAI admin framework – vector store usage and API costs printer
Environment:
  export OPENAI_ADMIN_KEY=...            # required “sk‑admin‑…” key
Dependencies:
  pip install httpx  (there if you have openai module)
'''
import json
import logging
import os
from datetime import UTC, datetime, timedelta
from typing import Any
import httpx

logger = logging.getLogger(__name__)

def admin_get(
    path: str,
    *,
    params: dict[str, Any] | None = None,
    timeout: float = 120.0,
) -> tuple[httpx.Response, dict[str, str]]:
    """GET wrapper for query-only OpenAI endpoints (no body/JSON)"""
    api_key = os.getenv("OPENAI_ADMIN_KEY")
    if not api_key:
        raise ValueError("Set OPENAI_ADMIN_KEY first (must be an *admin* key).")
    try:
        resp = httpx.get(
            f"https://api.openai.com/v1/{path}",
            headers={"Authorization": f"Bearer {api_key}"},
            params=params, timeout=timeout,
        )
        hdrs = dict(resp.headers)
        resp.raise_for_status()
        return resp, hdrs

    except httpx.HTTPStatusError as err:
        body_text = err.response.text.strip() or "<no body>"
        print(f"[Admin API] HTTP {err.response.status_code}: {body_text}", flush=True)
        logger.error("HTTP %s: %s", err.response.status_code, body_text)
        raise

    except httpx.RequestError as err:
        logger.error("Request error: %s", err)
        raise

def get_recent_costs(
    past_days: int = 14,
) -> dict[str, Any]:
    """
    Retrieve organisation‑level cost daily buckets up to today.

    Parameters
    ----------
    past_days
        Number of 24‑hour buckets (1 – 180).

    Returns
    -------
    Costs object with result buckets, amended with datetime. Example:
    {
        "object": "page",
        "data": [
            {
                "object": "bucket",
                "start_time": 1763337600,
                "end_time": 1763424000,
                "results": [
                    {
                        "object": "organization.costs.result",
                        "amount": {
                            "value": 0.06,  # can also be exponential, ie 6e-05
                            "currency": "usd"
                        },
                        "line_item": null,  # When group_by+=line_item
                        "project_id": null,  # When group_by+=project_id
                        "organization_id": "org-1234"
                    }
                ]
            }
        ],
        "start_datetime": "2025-11-17T00:00:00Z"  # added by function
        "has_more": false,
        "next_page": null
    }
    """
    past_days = max(1, min(past_days, 180))

    now = datetime.now(tz=UTC)
    start_dt = (now - timedelta(days=past_days)).replace(
        hour=0, minute=0, second=0, microsecond=0
    )

    params: dict[str, Any] = {}
    params.update(
        {
            "start_time": int(start_dt.timestamp()),  # send UNIX epoch time
            "end_time": int(now.timestamp()),  # also UNIX epoch time
            "limit": past_days,  # items before cursor pagination
            "bucket_width": "1d",  # only "1d"
            "group_by": ["line_item"],  # ["line_item" and/or "project_id"]
        }
    )

    resp, _ = admin_get("organization/costs", params=params)
    costs: dict[str, Any] = resp.json()

    # add human-readable start date to the response object
    for bucket in costs["data"]:
        start_dt = datetime.fromtimestamp(bucket["start_time"], tz=UTC)
        bucket["start_datetime"] = start_dt.isoformat().replace("+00:00", "Z")

    return costs

def printable_costs(costs: dict[str, Any]) -> str:
    """
    Format organisation cost buckets into a Markdown-compatible table.

    Parameters
    ----------
    costs
        The dict returned by ``get_recent_costs()``.

    Returns
    -------
    str
        A Markdown table showing daily costs per line item, auto-width.

    Example printable return produced
    -------
    | UTC Date             | Line Item                          | Amount (USD) |
    | -------------------- | ---------------------------------- | ------------ |
    | 2025-11-19T00:00:00Z | ft-gpt-4.1-nano-2025-04-14, input  |       0.0000 |
    |                      | ft-gpt-4.1-nano-2025-04-14, output |       0.0001 |
    |                      | ft-gpt-4o-2024-08-06, input        |       0.0001 |
    |                      | ft-gpt-4o-2024-08-06, output       |       0.0002 |
    """
    headers = ("UTC Date", "Line Item", "Amount (USD)")

    # Track rows as tuples of (date, line_item, amount_str)
    rows: list[tuple[str, str, str]] = []

    # Initialise column widths from header labels
    widths = [len(h) for h in headers]

    for bucket in costs.get("data", []):
        results = bucket.get("results") or []
        if not results:
            continue

        # Full UTC datetime string (already added in get_recent_costs)
        date_str = bucket.get("start_datetime", "")

        first_for_day = True
        for result in results:
            line_item = result.get("line_item", "")
            amount_value = result.get("amount", {}).get("value", 0.0)

            # Format amount to 4 decimal places
            amount_str = f"{float(amount_value):.4f}"

            # Only show the date once per day block
            date_cell = date_str if first_for_day else ""
            first_for_day = False

            row = (date_cell, line_item, amount_str)
            rows.append(row)

            # Update column widths based on this row
            widths[0] = max(widths[0], len(row[0]))
            widths[1] = max(widths[1], len(row[1]))
            widths[2] = max(widths[2], len(row[2]))

    # Build Markdown table lines
    lines: list[str] = []

    # Header row
    header_row = "| " + " | ".join(
        headers[i].ljust(widths[i]) for i in range(len(headers))
    ) + " |"
    lines.append(header_row)

    # Separator row, width-aligned with header/body
    separator_row = "| " + " | ".join(
        "-" * widths[i] for i in range(len(headers))
    ) + " |"
    lines.append(separator_row)

    # Data rows
    for date_cell, line_item, amount_str in rows:
        line = "| " + " | ".join(
            [
                date_cell.ljust(widths[0]),
                line_item.ljust(widths[1]),
                amount_str.rjust(widths[2]),
            ]
        ) + " |"
        lines.append(line)

    return "\n".join(lines)


def get_recent_vector_usage(
    past_days: int = 7,
) -> dict[str, Any]:
    """
    Retrieve organisation vector store daily bytes buckets up to today (now).
    Work in progress.

    Parameters
    ----------
    past_days
        Number of one-day buckets (1 – 31), default 7 like API.

    Returns
    -------
    object from API

    Example Return (if empty results buckets and limit=2 in "1d")
    -------
    {
      "object": "page",
      "has_more": true,
      "next_page": "page_AAAAAGkf3HXKKd36AAAAAGkeWgA=",
      "data": [
        {
          "object": "bucket",
          "start_time": 1763424000,
          "end_time": 1763510400,
          "start_time_iso": "2025-11-18T00:00:00+00:00",
          "end_time_iso": "2025-11-19T00:00:00+00:00",
          "results": [],
        },
        {
          "object": "bucket",
          "start_time": 1763510400,
          "end_time": 1763596800,
          "start_time_iso": "2025-11-19T00:00:00+00:00",
          "end_time_iso": "2025-11-20T00:00:00+00:00",
          "results": [],
        }
      ]
    }

    "results" contains bucket format items:
    {
        "object": "organization.usage.vector_stores.result",
        "usage_bytes": 1024,
        "project_id": "proj_abc"  # When group_by=project_id, otherwise null
    }
    """
    past_days = int(max(1, min(past_days, 31)))  # constrain input param, not raise

    now = datetime.now(tz=UTC)
    start_dt = (now - timedelta(days=past_days)).replace(
        hour=0, minute=0, second=0, microsecond=0
    )

    params: dict[str, Any] = {}
    params.update(
        {
            "start_time": int(start_dt.timestamp()),
            "end_time": int(now.timestamp()),  # also UNIX epoch time
            "limit": past_days,  # buckets before cursor pagination
            "bucket_width": "1d",  # 1m, 1h and 1d ... 1d is max 31 days page limit
            "group_by": [],  # optional ["project_id"], not demonstrated
            # "project_ids": [],  # empty array might return nothing?
        }
    )

    resp, _ = admin_get("organization/usage/vector_stores", params=params)
    usage: dict[str, Any] = resp.json()

    # Un-needed; this API now includes human-readable "start_time_iso"
    # Future: We could still proprietary "Zulu" the time to remove the +00:00 timezone
    # for bucket in usage["data"]:
    #    start_dt = datetime.fromtimestamp(bucket["start_time"], tz=UTC)
    #    bucket["start_datetime"] = start_dt.isoformat().replace("+00:00", "Z")

    return usage

def printable_vector_usage(usage: dict[str, Any]) -> str:
    """
    Format organisation cost buckets into a Markdown-compatible table.

    Parameters
    ----------
    costs
        The amended dict returned by ``get_recent_vector_usage()``.

    Returns
    -------
    str
        A Markdown table showing daily costs per line item. Human-interpretable.

    | UTC Date | Project ID | Usage (bytes) |
    | -------- | ---------- | ------------- |
    | ...
    """
    headers = ("UTC Date", "Project ID", "Usage (bytes)")

    # Track rows as tuples of (date, project_id, usage_str)
    rows: list[tuple[str, str, str]] = []

    # Initialise column widths from header labels
    widths = [len(h) for h in headers]

    for bucket in usage.get("data", []):
        results = bucket.get("results") or []
        if not results:
            continue

        # API returns e.g. "2025-11-19T00:00:00+00:00"
        # Make it a short, explicit UTC form by replacing the timezone with "Z"
        start_time_iso = bucket.get("start_time_iso", "") or ""
        if start_time_iso.endswith("+00:00"):
            date_str = start_time_iso.replace("+00:00", "Z")
        else:
            date_str = start_time_iso

        first_for_day = True
        for result in results:
            # When group_by=["project_id"] this is a real ID; otherwise null
            project_id = result.get("project_id")
            project_str = project_id if project_id is not None else ""

            usage_bytes = result.get("usage_bytes", 0)
            try:
                usage_int = int(usage_bytes)
            except (TypeError, ValueError):
                usage_int = 0
            usage_str = str(usage_int)

            # Only show the date once per day block
            date_cell = date_str if first_for_day else ""
            first_for_day = False

            row = (date_cell, project_str, usage_str)
            rows.append(row)

            # Update column widths based on this row
            widths[0] = max(widths[0], len(row[0]))
            widths[1] = max(widths[1], len(row[1]))
            widths[2] = max(widths[2], len(row[2]))

    # Build Markdown table lines
    lines: list[str] = []

    # Header row
    header_row = "| " + " | ".join(
        headers[i].ljust(widths[i]) for i in range(len(headers))
    ) + " |"
    lines.append(header_row)

    # Separator row, width-aligned with header/body
    separator_row = "| " + " | ".join(
        "-" * widths[i] for i in range(len(headers))
    ) + " |"
    lines.append(separator_row)

    # Data rows
    for date_cell, project_str, usage_str in rows:
        line = "| " + " | ".join(
            [
                date_cell.ljust(widths[0]),
                project_str.ljust(widths[1]),
                usage_str.rjust(widths[2]),
            ]
        ) + " |"
        lines.append(line)

    return "\n".join(lines)

apicosts = {}
apiusage = {}

def main() -> None:
    """Get and print some OpenAI admin endpoint data,
    leaving API responses in globals at at REPL console"""
    global apicosts, apiusage

    # Get OpenAI daily costs billed to an organization
    apicosts = get_recent_costs(past_days=14)
    #print(json.dumps(apicosts, indent=2))  # for debugging
    print(printable_costs(apicosts))

    # New - get the usage from vector store storage billed by day
    apiusage = get_recent_vector_usage(past_days=14)
    #print(json.dumps(apiusage, indent=2))   # for development
    print(printable_vector_usage(apiusage))  # this needs to be written

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    main()
1 Like

Thanks! I ran this code – actually a port into Typescript for my own convenience. It just calls the org/usage/vector_stores REST APIs. Anyway, the output is consistent with what I see in the UI – both prices and usage (bytes).

Then I noticed that if you take the usage numbers for a day in bytes, divide by 2^30 (GB), divide by 24 (hours/day), and multiply by 0.1 ($0.10 per GB/day), you’ll get exactly the corresponding price for that day in the table. So the usage numbers reported are the sum of the hourly usage numbers for the day. That’s not very intuitive, but at least I now understand what I’m looking at.

Thanks for your attention to this! I wasn’t aware of those admin APIs and am happy to know that they’re there when I need them again.

UTC Date Line Item Amount (USD)
2025-11-06T00:00:00Z gpt-5-mini-2025-08-07, input 0.0079
gpt-5-mini-2025-08-07, input 2.3430
gpt-5-mini-2025-08-07, cached input 0.0006
gpt-5-mini-2025-08-07, cached input 0.0475
gpt-5-mini-2025-08-07, output 0.0058
gpt-5-mini-2025-08-07, output 1.0373
file search storage 0.5780
file search tool calls 0.0025
file search tool calls 0.9125
2025-11-07T00:00:00Z gpt-5-mini-2025-08-07, input 0.0028
gpt-5-mini-2025-08-07, input 1.6016
gpt-5-mini-2025-08-07, cached input 0.0156
gpt-5-mini-2025-08-07, output 0.0018
gpt-5-mini-2025-08-07, output 0.8141
assistants api file search
file search storage 0.5839
file search tool calls 0.0050
file search tool calls 0.7350
2025-11-08T00:00:00Z gpt-5-mini-2025-08-07, input 0.0337
gpt-5-mini-2025-08-07, input 1.0512
gpt-5-mini-2025-08-07, cached input 0.0003
gpt-5-mini-2025-08-07, cached input 0.0078
gpt-5-mini-2025-08-07, output 0.0108
gpt-5-mini-2025-08-07, output 0.5870
file search storage 0.5952
file search tool calls 0.0275
file search tool calls 0.2450
2025-11-09T00:00:00Z gpt-5-mini-2025-08-07, input 0.9403
gpt-5-mini-2025-08-07, cached input 0.0056
gpt-5-mini-2025-08-07, output 0.4406
file search storage 0.5950
file search tool calls 0.2650
2025-11-10T00:00:00Z gpt-5-mini-2025-08-07, input 0.0045
gpt-5-mini-2025-08-07, input 2.2825
gpt-5-mini-2025-08-07, cached input 0.0003
gpt-5-mini-2025-08-07, cached input 0.0447
gpt-5-mini-2025-08-07, output 0.0058
gpt-5-mini-2025-08-07, output 1.1305
file search storage 0.5952
file search tool calls 0.0025
file search tool calls 1.1400
2025-11-11T00:00:00Z gpt-5-mini-2025-08-07, input 4.5193
gpt-5-mini-2025-08-07, input 1.8011
gpt-5-mini-2025-08-07, input 0.0042
gpt-5-mini-2025-08-07, cached input 0.0176
gpt-5-mini-2025-08-07, cached input 0.0340
gpt-5-mini-2025-08-07, output 4.6655
gpt-5-mini-2025-08-07, output 0.8306
gpt-5-mini-2025-08-07, output 0.0012
assistants api file search
file search storage 0.5909
file search tool calls 0.0100
file search tool calls 0.7900
2025-11-12T00:00:00Z gpt-5-mini-2025-08-07, input 6.3718
gpt-5-mini-2025-08-07, input 2.1092
gpt-5-mini-2025-08-07, input 0.0127
gpt-5-mini-2025-08-07, cached input 0.0207
gpt-5-mini-2025-08-07, cached input 0.0397
gpt-5-mini-2025-08-07, output 4.9934
gpt-5-mini-2025-08-07, output 0.9381
gpt-5-mini-2025-08-07, output 0.0113
file search storage 0.5589
file search tool calls 0.0025
file search tool calls 0.9675
2025-11-13T00:00:00Z gpt-5-mini-2025-08-07, input 4.6956
gpt-5-mini-2025-08-07, input 1.9069
gpt-5-mini-2025-08-07, cached input 0.0031
gpt-5-mini-2025-08-07, cached input 0.0165
gpt-5-mini-2025-08-07, output 3.8464
gpt-5-mini-2025-08-07, output 0.8329
assistants api file search
file search storage 0.5551
file search tool calls 0.0525
file search tool calls 0.7575
2025-11-14T00:00:00Z gpt-5.1-2025-11-13, input 0.2035
gpt-5.1-2025-11-13, cached input 0.0207
gpt-5.1-2025-11-13, output 0.0196
gpt-5-mini-2025-08-07, input 0.0073
gpt-5-mini-2025-08-07, input 2.2774
gpt-5-mini-2025-08-07, input 0.0045
gpt-5-mini-2025-08-07, cached input 0.0006
gpt-5-mini-2025-08-07, cached input 0.0479
gpt-5-mini-2025-08-07, cached input 0.0000
gpt-5-mini-2025-08-07, output 0.0053
gpt-5-mini-2025-08-07, output 0.8848
gpt-5-mini-2025-08-07, output 0.0047
file search storage 0.5577
file search tool calls 0.0075
file search tool calls 0.7850
file search tool calls 0.0050
2025-11-15T00:00:00Z gpt-5-mini-2025-08-07, input 0.0088
gpt-5-mini-2025-08-07, input 0.8831
gpt-5-mini-2025-08-07, cached input 0.0012
gpt-5-mini-2025-08-07, cached input 0.0082
gpt-5-mini-2025-08-07, output 0.0066
gpt-5-mini-2025-08-07, output 0.3635
file search storage 0.5535
file search tool calls 0.0050
file search tool calls 0.2650
2025-11-16T00:00:00Z gpt-5-mini-2025-08-07, input 0.7399
gpt-5-mini-2025-08-07, cached input 0.0053
gpt-5-mini-2025-08-07, output 0.3136
file search storage 0.5603
file search tool calls 0.1925
2025-11-17T00:00:00Z gpt-5-mini-2025-08-07, input 0.0112
gpt-5-mini-2025-08-07, input 2.3530
gpt-5-mini-2025-08-07, cached input 0.0002
gpt-5-mini-2025-08-07, cached input 0.0361
gpt-5-mini-2025-08-07, output 0.0042
gpt-5-mini-2025-08-07, output 0.9506
file search storage 0.5608
file search tool calls 0.0075
file search tool calls 0.9125
2025-11-18T00:00:00Z gpt-5-mini-2025-08-07, input 0.0086
gpt-5-mini-2025-08-07, input 3.4296
gpt-5-mini-2025-08-07, cached input 0.0002
gpt-5-mini-2025-08-07, cached input 0.0529
gpt-5-mini-2025-08-07, output 0.0030
gpt-5-mini-2025-08-07, output 1.3768
file search storage 0.5173
file search tool calls 0.0025
file search tool calls 1.2575
2025-11-19T00:00:00Z gpt-5-mini-2025-08-07, cached input 0.0003
gpt-5-mini-2025-08-07, cached input 0.0418
gpt-5-mini-2025-08-07, input 0.0028
gpt-5-mini-2025-08-07, input 2.8542
gpt-5-mini-2025-08-07, output 0.0029
gpt-5-mini-2025-08-07, output 1.0665
file search tool calls 1.0750
UTC Date Project ID Usage (bytes)
2025-11-06T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 148953696115
2025-11-07T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 151324695093
2025-11-08T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 153381050182
2025-11-09T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 153323885517
2025-11-10T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 153383192439
2025-11-11T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 153134816140
2025-11-12T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 144020423447
2025-11-13T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 143916750728
2025-11-14T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 143718985127
2025-11-15T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 142628522018
2025-11-16T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 144393040899
2025-11-17T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 144510066702
2025-11-18T00:00:00Z proj_C7puF5AXoXqVCPrPQYODPolv 133301886084
1 Like

Looks like the costs print parsing broke on line_item type “assistants api | file search” :beetle:

Need to replace any pipes (|) in the line item text to not break a markdown table…

UTC Date Line Item Amount (USD)
2025-10-23T00:00:00Z priority gpt-5-mini-2025-08-07, input
priority gpt-5-mini-2025-08-07, output
file search tool calls 0.0025
| UTC Date             | Line Item                                   | Amount (USD) |
| -------------------- | ------------------------------------------- | ------------ |
| 2025-10-23T00:00:00Z | priority | gpt-5-mini-2025-08-07, input     |       0.0000 |
|                      | priority | gpt-5-mini-2025-08-07, output    |       0.0000 |
|                      | file search tool calls                      |       0.0025 |