Doing research how well LLM’s find sources. I want to compare gemini, grok and openai but can not find the info.
I think even free users get the gpt-5.2-chat-latest model, at least for a certain number of requests, after which they may switch to a “mini” model, but I’m not sure about the latter.
1 Like
You’re right. Gemini and Grok also give limited access to their latest models.
Do you think testing both the “mini” and the “latest” models would give insight into the free user experience?
These are my choices at the moment.
With ChatGPT, OpenAI runs a bunch of their own internal tools, and has a completely different set of their own features within ChatGPT. You cannot place real system messages on the API now either with GPT-5.2, or true top-level tools, a reservation OpenAI has kept for themselves, along with their own internal prompts run on API that are neither ChatGPT nor what you want.
Therefore, the goal to research “how well they find sources” cannot be met, as you are asking about a developed product, while on the API you develop your own AI products. You aren’t going to get these results, and cannot even simulate the message sequence even if no tools are called by AI and no internet results are injected.
Plus, then you have the unknown service level of “free”, where OpenAI has the ability to deliver whatever free is worthy of, given the current dynamic compute utilization by payers.
The closest would be to use the gpt-5.2-chat-latest model, which has no reasoning controls but is quite minimal reasoning, thinking about policy and refusing. Then place the same message sequence of “You are ChatGPT…” with the behavior tuneup language you can extract from ChatGPT after you have escalated your “user” privilege with prompting.
Better is gpt-5-chat-latest which actually acts like a non-reasoning model, but is not what is delivered today.
Simulate this context bloat?
Catalog of Available Tools
This document catalogs all tools available in this environment, organized by channel, tool recipient, and functional category. It is intended as a comprehensive reference, not a usage guide.
1. Core Conversation Channels
analysis
Purpose: Internal reasoning, planning, and tool orchestration. Not visible to the user.
- Decides when and how to call tools
- Performs private computation, validation, and planning
final
Purpose: User-facing response channel.
- Contains the final answer, explanations, or guidance
- No hidden reasoning or tool calls
commentary
Purpose: User-visible tool execution and UI-side effects.
- Canvas creation and updates
- User-visible Python execution
- Automations, widgets, image generation
2. Tool Catalog by Category
A. External Information Retrieval
web.run
Channel: analysis
Category: External info retrieval
Description:
- Performs live web search and page retrieval
- Required for time-sensitive, factual, or verifiable information
- Supports search queries and page opening
B. User-Connected / Uploaded Data Access
file_search.msearch
Channel: analysis
Category: User data connection
Description:
- Searches user-uploaded files and connected internal sources
- Supports multiple semantic queries
- Can be restricted by source and file type
file_search.mclick
Channel: analysis
Category: User data connection
Description:
- Opens and expands specific search results or documents
- Used after msearch to inspect content in detail
C. Coding, Computation, and Execution
python.exec
Channel: analysis
Category: Coding / computation (private)
Description:
- Executes Python code for internal reasoning
- No user-visible output
- Internet access disabled
python_user_visible.exec
Channel: commentary
Category: Coding / computation (user-visible)
Description:
- Executes Python code with visible output
- Can generate tables, plots, and downloadable files
- Used for data artifacts and demonstrations
container.exec
Channel: commentary
Category: Coding / container
Description:
- Executes shell commands inside a container
- Supports interactive sessions
container.feed_chars
Channel: commentary
Category: Coding / container
Description:
- Sends input to an interactive container session
container.open_image
Channel: commentary
Category: Coding / container
Description:
- Displays an image file from the container filesystem
container.download
Channel: commentary
Category: Coding / container
Description:
- Downloads external files into the container filesystem
D. Canvas / Document Management
canmore.create_textdoc
Channel: commentary
Category: Document & canvas management
Description:
- Creates a new canvas document
- Used for long-form writing, plans, or code files
canmore.update_textdoc
Channel: commentary
Category: Document & canvas management
Description:
- Applies regex-based updates to an existing canvas document
canmore.comment_textdoc
Channel: commentary
Category: Document & canvas feedback
Description:
- Adds inline comments and suggestions to a canvas document
E. UI Widgets and Utilities
genui.search
Channel: analysis
Category: UI utilities
Description:
- Discovers available interactive widgets (weather, currency, time, etc.)
genui.run
Channel: commentary
Category: UI utilities
Description:
- Renders a selected widget for the user
F. Image Generation
image_gen.text2im
Channel: commentary
Category: Media generation
Description:
- Generates images from text prompts
- Also used for image editing and style transfer
- Produces no textual response
G. Automations and Scheduling
automations.create
Channel: commentary
Category: Task automation
Description:
- Creates reminders, recurring tasks, or conditional checks
automations.update
Channel: commentary
Category: Task automation
Description:
- Modifies or enables/disables an existing automation
automations.list
Channel: commentary
Category: Task automation
Description:
- Lists all active automations
H. Memory, Context, and Personalization
personal_context.search
Channel: analysis
Category: Memory / context
Description:
- Retrieves stored personal context for the user
- Used when prior interactions matter
bio.update
Channel: commentary
Category: Memory
Description:
- Writes persistent user memory (if enabled)
I. User Settings
user_settings.get_user_settings
Channel: commentary
Category: Preferences & personalization
Description:
- Retrieves current UI and personality settings
user_settings.set_setting
Channel: commentary
Category: Preferences & personalization
Description:
- Updates appearance, accent color, or personality
J. API Tool Discovery
api_tool.list_resources
Channel: commentary
Category: Tool discovery
Description:
- Lists available API-like resources and tools
api_tool.call_tool
Channel: commentary
Category: Tool invocation
Description:
- Invokes a discovered API tool or executable resource
K. System / Guardian Constraints
Safety & Policy Layer (implicit)
Channel: Internal
Category: Guardian / safety
Description:
- Enforces safety, policy, and capability constraints
- Not directly callable
3. Summary
This catalog enumerates all available tools by channel, recipient, and functional category, covering:
- External and internal information retrieval
- User data access
- Code execution and containers
- Canvas-based document workflows
- UI widgets and media generation
- Automations and scheduling
- Memory, context, and personalization
- Tool discovery and invocation
It is designed as a stable reference for understanding the full operational surface of this environment.