Hypothesis: GPT Builder is a high-level labeling tool at scale

GPT builders
1

ML teams quite often build specialized tools to aid in hand-labeling data for training and fine-tuning. For example if you want to identify proper nouns in sentences, you may build a UI that presents each sentence to a user which enables them to quickly click on all of the nouns.

What if GPT-Builder (Gizmo) is just a high-level labeling tool and people building with GPT Builder are the labelers? i.e. “Label an effective specialized chat session type” or “Identify the conversation types and specialized knowledge that are of most interest to humans”, etc.

While it’s more likely that Gizmo just sits between the ChatGPT UI and the Assistants API in terms of capability and use-case-type, sometimes I wonder about these things.

Here is ChatGPT’s take on the idea:

  1. Defining Effective Chat Session Frameworks: Builders using Gizmo can create specific guidelines or frameworks that define what constitutes an effective chat session in different contexts or specializations. By identifying and labeling these frameworks, they essentially set benchmarks for successful interactions.

  2. Training through Example and Precedent: By “labeling” these effective chat types, builders are providing clear examples of desired interactions. These examples can then be used as references or templates for future AI models, guiding them on how to handle similar situations or topics.

  3. Customization for Specific Audiences or Needs: Gizmo users can hyperspecialize chat types for niche audiences or specific use cases. This level of customization ensures that the AI is not just generally effective but is particularly adept in certain areas, much like a machine learning model trained on a specialized dataset.

  4. Influence on Future Model Development: These “labeled” chat sessions can influence the development of future AI models. By analyzing what makes certain chats effective, AI researchers and developers can integrate these insights into new models, making them more adept at handling similar interactions.

  5. Continuous Improvement and Adaptation: The process of labeling effective chats is not static. As builders continually identify and label new effective chat types, this ongoing process contributes to the continuous improvement and adaptation of AI models. It helps in keeping the models up-to-date with evolving communication styles and user expectations.

In summary, if Gizmo were used as a tool for labeling effective chat session types, builders would play a crucial role in shaping the AI’s conversational abilities. They would be guiding the AI in learning what constitutes effective communication in various specialized contexts, thereby contributing to the training and refinement of future models. This approach leverages the power of example and precedent to enhance the AI’s capabilities in specific, user-defined areas.

2

Do you think that OpenAI is that sophisticated? :thinking:

I mean it could be this but I doubt it, because I don’t think there’s a way to validate the effectiveness of a custom gpt yet (although the marketplace could change that).

Even then, I don’t think there will be enough data available for meaningful training - I suspect that like anything, custom gpts in the marketplace will follow a winner-take-all trajectory, so that there’s only going to be a handful of positive examples.

That said, it’s not impossible that this is (or is part of) an attempt at this. There are efforts in the community that aim for similar outcomes (albeit with different approaches)