Useful feature: Allow sending "name" for unseen ChatML assistant prompt & in fine tuning

The “name” parameter of the chat completions API might not have high awareness or quality documentation (and it is not an option in Assistants), but it is something that begs for enhancement.

What is “name”? One can send a name along with a user role message, a name like “Bob” or “example” (with the limitation that it cannot have spaces). The AI will recognize this as a user’s name from some aspect of pretraining, being able to answer the user by name.

One also can pass assistant messages with a name.

You can see that a name might be useful for fine tuning, also: instead of giving distinct system prompts within a training set to send the AI in different directions to perform different roles, one could train an AI by only its name to select tasks or change behavior.

For example:

messages=[
    {
    "role": "system", "name": "programming",
    "content": "You are a helpful AI assistant."
    },
    {
    "role": "user", "name": "example"
    "content": "Who is Zendaya?"
    },
    {
    "role": "assistant", "name": "brief"
    "content": "Zendaya is an American actress and singer."
    },
    {
    "role": "user", "name": "example"
    "content": "Who is Dua Lipa?"
    },
    {
    "role": "assistant", "name": "extensive"
    "content": """
Dua Lipa is an acclaimed British singer, songwriter, and former model, who has made significant contributions to the contemporary music scene with her distinctive blend of pop, disco, and electronic music. Born on August 22, 1995, in London, England, to Albanian parents from Kosovo, she grew up in a creative and musical environment that greatly influenced her career path.

### Early Life and Career Beginnings

Dua Lipa's interest in music was evident from a young age. She attended the Sylvia Young Theatre School before moving to Kosovo with her family in 2008. Despite the relocation, Lipa was determined to pursue music. She moved back to London at the age of 15 to follow her dreams, where she worked in various jobs while posting covers of her favorite songs on YouTube.

### Breakthrough and Musical Style

Her distinctive voice and unique style soon caught the attention of the music industry, leading to her signing a deal with Warner Music Group in 2015. Dua Lipa's self-titled debut album was released in 2017, featuring hits like "New Rules" and "IDGAF". The album was a critical and commercial success, establishing her as a rising star in the music world.

Dua Lipa's music is known for its empowering themes, often revolving around love, self-respect, and female empowerment. Her sound is a blend of various genres, with influences from pop, disco, and electronic music, combined with her deep, sultry vocals, creating a modern yet nostalgic feel.

### Achievements and Recognition

Dua Lipa has received numerous accolades for her work, including multiple Grammy Awards, BRIT Awards, and MTV Europe Music Awards. Her second album, "Future Nostalgia" (2020), received widespread acclaim for its cohesive sound and innovative approach to pop music. It featured hits like "Don't Start Now" and "Levitating", further cementing her status as a global pop sensation.

### Impact and Influence

Beyond her music, Dua Lipa is known for her distinctive fashion sense and has become a style icon for many. She is also an active advocate for various social causes, including women's rights and refugee support, using her platform to raise awareness and inspire change.

### Conclusion

Dua Lipa's journey from posting covers on YouTube to becoming a global pop star is a testament to her talent, determination, and innovative approach to music. With her distinctive style, powerful vocals, and catchy tunes, she has not only dominated the charts but also influenced the musical landscape, making her one of the most influential figures in contemporary pop music.
""".strip(),

You can see that by a name of “brief” or “extensive”, we could train the AI on how to respond (current AI models don’t take much to multi-shot-as-training, though, still spitting out their over-trained ChatGPT form).

However, there is one missing component: You cannot give the final assistant prompt (that the AI is supposed to complete upon) a name. You thus cannot use name to direct the assistant to be an identity or follow the name, despite such training.

(Background: the messages you send to the API, which are encoded in the ChatML container format, have the beginnings of an additional assistant message appended, which triggers the AI to write as itself.)

Proposal: allow specification of name for final assistant prompt.

That would be a relatively easy add to chat completions, not as part of messages, but another API parameter. Used in conjunction with fine-tuning or chat history, this gives enhanced possibilities for real-world applications.

(…and then add another parameter to turn off the closing of “assistant” messages and this injection, so the model can actually complete again.)


Background:

The chat message format places messages into AI context that start with [token]assistant[token], where token is a special token that cannot be encoded within messages. Adding a name simply extends this to [token]assistant:yourname[token]

2 Likes

It’s been like that for going on a year or more now, problem has always been the undocumented nature of names outside of those predefined, but I can see a lot of use-cases where this would be extremely useful, meeting notes and agents spring to mind.

2 Likes