# Bug Report: GPT-5.1 Ignores Implicit Context in Strict Function Calling
## Quick Summary
GPT-5.1 with strict schemas (via `pydantic_function_tool`) does **not** extract implicit filter criteria from natural language queries, even with explicit instructions.
**Query:** `“Show me all colors for NorthPeak products”`
**Expected:** `filters={‘brand’:‘NorthPeak’}`
**Actual:** `filters={}` or `filters=null`
## Environment
- **Model:** `gpt-5.1` (version `gpt-5.1-2025-11-13`)
- **API:** Responses API (`client.responses.create()`)
- **SDK:** `openai==2.8.0`
- **Python:** 3.10+
## Reproduction (30 seconds)
### Step 1: Install SDK
```bash
pip install openai==2.8.0
```
### Step 2: Create test file `simple_repro.py`
```python
#!/usr/bin/env python3
import asyncio, os, json
from openai import AsyncOpenAI, pydantic_function_tool
from pydantic import BaseModel, Field
from typing import Dict, Any, Optional
class QueryParams(BaseModel):
entity: str
field: str
filters: Optional[Dict[str, Any]] = Field(
default_factory=dict,
description=“Extract filters. Example: ‘for NorthPeak’ → {‘brand’:‘NorthPeak’}”
)
async def main():
client = AsyncOpenAI(api_key=os.getenv(“OPENAI_API_KEY”))
tool = pydantic_function_tool(QueryParams, name=“query_data”,
description=“Extract filters from queries like ‘NorthPeak products’ → {‘brand’:‘NorthPeak’}”)
response = await client.responses.create(
model=“gpt-5.1”,
input=[{“role”: “user”, “content”: [{“type”: “input_text”,
“text”: “Show me all colors for NorthPeak products”}]}],
instructions=“Extract filters: ‘NorthPeak products’ → filters={‘brand’:‘NorthPeak’}”,
tools=[{“type”: “function”, “name”: tool[“function”][“name”],
“parameters”: tool[“function”][“parameters”],
“strict”: True, “description”: tool[“function”][“description”]}],
tool_choice=“auto”
)
for item in response.output:
if hasattr(item, ‘name’) and item.name == ‘query_data’:
args = json.loads(item.arguments)
print(f"Expected: filters={{‘brand’:‘NorthPeak’}}")
print(f"Actual: filters={args.get(‘filters’)}")
print(“
BUG” if not args.get(‘filters’) else “
OK”)
asyncio.run(main())
```
### Step 3: Run
```bash
export OPENAI_API_KEY=“sk-…”
python3 simple_repro.py
```
### Expected vs Actual
**Expected:**
```
Expected: filters={‘brand’:‘NorthPeak’}
Actual: filters={‘brand’:‘NorthPeak’}
OK
```
**Actual:**
```
Expected: filters={‘brand’:‘NorthPeak’}
Actual: filters=None # or {}
BUG
```
## Root Cause
The schema generated by `pydantic_function_tool()` includes:
```json
{
“filters”: {
“anyOf”: [
{“type”: “object”, “additionalProperties”: false},
{“type”: “null”}
]
},
“required”: [“filters”]
}
```
With `“additionalProperties”: false`, the model cannot add keys like `“brand”` without knowing they’re valid. It chooses the safest option: `{}` or `null`.
## Impact
- Breaks natural language → structured query conversion
- Requires users to explicitly structure their queries instead of using conversational language
- Makes GPT-5.1 less suitable for data query interfaces
## Attempted Workarounds
1. **Manual schema with `strict=False`** - SDK rejects it:
```
Expected Chat Completions function tool shape to be created using `openai.pydantic_function_tool()`
```
2. **Explicit examples in descriptions** - Model ignores them
3. **System-level instructions** - Model ignores them
4. **`json_schema_extra={“additionalProperties”: True}`** - Overridden by Pydantic
## Requested Features
1. **Allow `strict=False` in Responses API** - Give developers flexibility for natural language extraction
2. **Support `additionalProperties: true` for Dict fields** - Let models add arbitrary keys when appropriate
3. **Improve context extraction with strict schemas** - Train model to better extract implicit information
## Where to Submit
- **OpenAI Community Forum:** https://community.openai.com/
- **OpenAI Help Center:** https://help.openai.com/
- **OpenAI Developer Forum (Responses API):** Post in “Responses API” or “GPT-5” category
## Additional Test Cases
All fail to extract filters:
| Query | Expected filters | Actual |
|-------|------------------|--------|
| “colors for NorthPeak” | `{‘brand’:‘NorthPeak’}` | `{}` |
| “NorthPeak in Electronics” | `{‘brand’:‘NorthPeak’,‘category’:‘Electronics’}` | `{}` |
| “Electronics products” | `{‘category’:‘Electronics’}` | `{}` |
## Schema Example
Generated by `pydantic_function_tool()`:
```json
{
“description”: “Parameters for querying data with filters.”,
“properties”: {
“entity”: {
“description”: “Entity name (e.g., ‘product’)”,
“title”: “Entity”,
“type”: “string”
},
“field”: {
“description”: “Field to query (e.g., ‘color’)”,
“title”: “Field”,
“type”: “string”
},
“filters”: {
“anyOf”: [
{
“type”: “object”,
“additionalProperties”: false
},
{
“type”: “null”
}
],
“description”: “Extract filter criteria from user’s query. Examples: ‘for NorthPeak products’ → {‘brand’:‘NorthPeak’}”,
“title”: “Filters”
}
},
“required”: [
“entity”,
“field”,
“filters”
],
“title”: “QueryParams”,
“type”: “object”,
“additionalProperties”: false
}
```
Note the `“additionalProperties”: false` inside the `anyOf` - this prevents the model from adding filter keys.
-–
**Thank you for investigating this limitation!** This significantly impacts natural language query applications.
#!/usr/bin/env python3
“”"
Simple reproduction: GPT-5.1 ignores implicit context in function calling.
Expected: filters={‘brand’:‘NorthPeak’}
Actual: filters={}
Run: OPENAI_API_KEY=your-key python3 simple_repro.py
“”"
import asyncio
import os
from openai import AsyncOpenAI
from pydantic import BaseModel, Field
from openai import pydantic_function_tool
from typing import Dict, Any, Optional
class QueryParams(BaseModel):
entity: str = Field(description="Entity name")
field: str = Field(description="Field to query")
filters: Optional\[Dict\[str, Any\]\] = Field(
default_factory=dict,
description="Extract filters from query. Example: 'for NorthPeak' → {'brand':'NorthPeak'}"
)
async def main():
client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))
tool = pydantic_function_tool(
QueryParams,
name="query_data",
description="Query data. CRITICAL: Extract implicit filters (e.g., 'NorthPeak' → filters={'brand':'NorthPeak'})"
)
\# Convert to Responses API format
responses_tool = {
"type": "function",
"name": tool\["function"\]\["name"\],
"parameters": tool\["function"\]\["parameters"\],
"strict": tool\["function"\].get("strict", True),
"description": tool\["function"\]\["description"\]
}
response = await client.responses.create(
model="gpt-5.1",
input=\[{
"role": "user",
"content": \[{"type": "input_text", "text": "Show me all colors for NorthPeak products"}\]
}\],
instructions="Extract contextual filters from queries. 'NorthPeak products' → filters={'brand':'NorthPeak'}",
tools=\[responses_tool\],
tool_choice="auto"
)
\# Check result
for item in response.output:
if hasattr(item, 'name') and item.name == 'query_data':
import json
args = json.loads(item.arguments)
print(f"Expected: filters={{'brand': 'NorthPeak'}}")
print(f"Actual: filters={args.get('filters', {})}")
if not args.get('filters'):
print("\\n❌ BUG REPRODUCED: Model ignored 'NorthPeak' in query")
else:
print("\\n✅ Working as expected")
break
if _name_ == “_main_”:
asyncio.run(main())