# Custom GPT → Plugins: 100-scenario baseline **URL:** \ **Category:** Use cases and examples\ **Tags:** custom-gpt\ **Created:** [September 19, 2026, 6:01pm UTC](https://community.openai.com/t/custom-gpt-plugins-100-scenario-baseline/1399153 "2026-09-19T18:01:47Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![GregoryMayor](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/gregorymayor/32/768773_2.png) [@GregoryMayor](https://community.openai.com/u/GregoryMayor)\ **Post date:** [September 19, 2026, 6:01pm UTC](https://community.openai.com/t/custom-gpt-plugins-100-scenario-baseline/1399153/1 "2026-09-19T18:01:47Z")
# Custom GPT → Plugins: 100-scenario baseline OpenAI’s planned transition from Custom GPTs to Plugins raises an important question for anyone who has spent significant time refining a GPT: **How do you know whether the migrated version still behaves like the system you spent months — or years — building?** OpenAI’s official migration guidance recommends preparing existing workflows and testing the replacement Plugin before switching: [OpenAI — Custom GPT retirement and migration FAQ](https://help.openai.com/en/articles/20001519-custom-gpt-retirement-and-migration-faq) Rather than migrate first and evaluate afterwards, I decided to establish a measurable baseline for my existing Custom GPT, **GM INFO - IA Technologies ©** , before moving it to the new Plugin architecture. My objective was not to prove that the current GPT is “perfect”. It was to create a sufficiently demanding reference point that I can reuse after migration to identify regressions, behavioral differences, improvements, or unexpected changes. ## The approach I created a second, more advanced non-regression assessment based on **100 scenarios** , designed to evaluate the current Custom GPT before migration. The complete process ultimately produced a **100-page report**. The first part of the work focused on designing and structuring the test protocol and questionnaire. ChatGPT Work was used for the multi-step workflow and document production, with Codex involved in the technical generation of the Word documents. The 100 responses were then evaluated in a **separate conversation** , deliberately separating the evaluation stage from the original response-generation process as an initial measure to reduce evaluation contamination. The first evaluation produced an unexpected result: > **300/300** A perfect score looked suspicious enough to justify another review rather than simply accepting it. So I requested a second examination of the complete evaluation. That review identified one constraint that had not been fully respected in **Q063**. The corrected baseline became: > **299/300 — 99.67%** > > - 99 responses: 3/3 > - 1 response: 2/3 > - 0 critical failures For me, the interesting part is not really the 299/300 score. It is the fact that the evaluation process itself demonstrated why an apparently perfect AI-generated evaluation should sometimes be challenged and independently reviewed. The second pass actually found something the first evaluation had missed. ## Why establish the baseline before migration? A migrated Plugin may appear to work correctly while still introducing subtle behavioral changes. Without a pre-migration reference, it becomes difficult to distinguish between: - behavior that was already present in the Custom GPT; - a genuine regression introduced during migration; - an improvement; - differences in instruction interpretation; - differences caused by tools, retrieval, Plugin selection or execution; - normal variability between model runs. My current approach is therefore straightforward: **Phase 1 — Custom GPT:** establish and document the baseline. **Phase 2 — Plugin:** migrate the system. **Phase 3 — Non-regression:** run the same benchmark again, keeping the original scenarios and scoring criteria as stable as possible. **Phase 4 — Delta analysis:** investigate every meaningful difference between the two implementations instead of looking only at the final aggregate score. That final comparison is ultimately more important to me than the current **299/300** result. ## Important limitation This benchmark should not be interpreted as an external certification. It evaluates the written behavior tested by this particular protocol. It does **not** prove that the future Plugin migration will be regression-free, and it does not yet measure every aspect of real-world Plugin execution. The Plugin version will introduce additional dimensions that deserve their own testing, including tool invocation, capability selection, access permissions, failure handling, reference-material behavior and potentially different execution paths. That is precisely why I wanted a documented baseline before changing the architecture. ## Full context and original test publication I documented the complete pre-migration experiment in this LinkedIn post: [Original LinkedIn post — Advanced pre-migration test for GM INFO - IA Technologies ©](https://www.linkedin.com/posts/gregory-mayor-b21151156_intelligenceartificielle-chatgpt-chatgptwork-activity-7505895445412442112-kkfI) The LinkedIn publication also provides access to the complete **100-page evaluation report** for anyone interested in examining the test methodology and results in greater detail. ## A question for the Developer Community I would be very interested in feedback from developers who are preparing similar migrations. **How would you design a rigorous pre/post-migration regression test for a complex Custom GPT moving to Plugins?** In particular, I am interested in three questions: 1. **Would you keep the complete pre-migration benchmark unchanged for the first Plugin comparison, or evolve part of the test suite to account for Plugin-specific behavior?** 2. **How would you further reduce evaluator bias or correlated errors when AI is involved both in producing and evaluating test results?** 3. **Which Plugin-specific dimensions would you consider mandatory before concluding that a migration has successfully preserved the original workflow?** My intention is to repeat the evaluation after migration and compare the results **scenario by scenario** , including failures, improvements and behavioral differences — not simply publish another overall score. If this methodology is useful to others preparing the same transition, I will also share the **post-migration results and delta analysis** in this thread.