Feature Request: A Native Music Module for ChatGPT

ChatGPT Feature requests
, ,
1

Feature Request: A Native Music Module for ChatGPT

I recently ran a small experiment trying to “teach” ChatGPT Voice to sing.

The experiment did not assume that ChatGPT can currently sing in a controlled musical sense. It clearly cannot: there is no dedicated pitch-controlled singing system behind Voice. The point was instead to explore how far the existing system can be pushed, and what the remaining gap actually looks like.

Day 1 — Voice, rhythm and pitch experiments → Watch on X

Day 2 — Audio imitation + score-reading test → Watch on X

In the audio test, ChatGPT Voice could occasionally approximate singing, but it did not reliably follow the actual pitch and timing of the source and tended to anticipate lyrics it already knew.

In the score-reading test, after instructions about notes, intervals, syllables and duration, the result became more song-like — but the melody was still largely improvised rather than faithfully derived from the notation.

This is exactly what makes the experiment interesting: it gives a rough measure of what is already possible and, more importantly, of what is still missing.

What seems to be missing is a native musical layer connecting audio perception, symbolic music representation and vocal generation.

Such a module could allow ChatGPT to:

  • detect pitch, rhythm, tempo, chords and key from audio;

  • convert between audio and symbolic formats such as MIDI / MusicXML / notation;

  • read and reproduce a score with controlled pitch, timing and duration;

  • compose, harmonize, edit and compare musical material;

  • analyze themes, motifs, harmony and rhythm while reasoning jointly over audio + notation + text.

Some of these building blocks already exist. For example, Spotify Basic Pitch can extract pitch information and generate MIDI from musical audio.

The pieces already exist. What is missing is their integration into ChatGPT.

Such a capability could be useful far beyond singing: for music education, transcription, composition, score analysis and interactive practice.

Would OpenAI consider adding native music capabilities like these to ChatGPT in the future?

2

Now this is a really nice feature request!

Note: The following was created by me then expanded and polished using ChatGPT, with a final review by me before posting.

Gregorian chant might be an interesting area to include when considering a native music module.

It is quite different from much of the music that would probably be used initially for testing:

  • Gregorian chant is fundamentally vocal and monophonic—a single melodic line—and is traditionally sung without instrumental accompaniment.

  • Traditional chant notation normally uses a four-line staff and neumes, rather than modern five-line staff notation.

  • More importantly, its rhythm generally is not organized into regular measures governed by a time signature. The interpretation depends much more on the text, phrasing, syllables, neumatic groupings, pauses, and musical articulation than on fitting notes into a fixed beat grid.

  • A single syllable can also contain a long melisma, with many notes sung on that one syllable.

That seems particularly relevant to the proposal here. A music system that performs well only when given something resembling MIDI—fixed pitches at fixed start times and durations—would handle one important model of music, but not all musical notation or performance traditions.

Gregorian chant could therefore provide some useful additional tests:

  1. Text ↔ notation ↔ audio alignment. The relationship between the words, syllables, neumes, and sung pitches is fundamental to chant.

  2. Pitch without a strong metric framework. The system would have to reproduce a melody without depending upon a conventional beat grid.

  3. Prosody and phrasing. The natural accent and flow of the text contribute to how the chant is performed, making this partly a language-understanding problem as well as a music problem.

  4. Melismas. Mapping one syllable to many notes would test whether the system really understands the score rather than simply aligning one note with one syllable.

  5. Alternative notation systems. Supporting neumes and square notation would test whether the proposed musical layer can represent music more generally instead of being tightly coupled to modern five-line notation.

  6. A simpler starting point in some respects. Because chant is normally a single unaccompanied melodic line, there are no chords, orchestration, or multiple simultaneous voices to solve first. That could make it useful for isolating pitch, phrasing, notation, and vocal-generation problems.

There is even an existing computer-readable notation worth exploring: GABC, used by the open-source Gregorio project to represent and engrave Gregorian chant. A GABC score ties the text directly to the corresponding neumes and pitches.

So rather than testing only something like:

modern score → MIDI-like representation → singing

it might be useful to test:

Latin text + neumes → symbolic chant representation → correctly phrased singing

Gregorian chant looks deceptively simple because it is usually only one voice, but that simplicity removes a lot of unrelated musical complexity and exposes some interesting problems involving pitch, notation, language, prosody, phrasing, and timing. That could make it a useful test case for exactly the sort of native musical layer being proposed here.

You might also find this of interest:

3

Thanks for sharing the experiment and the music use cases. We'll pass along the request for a native module connecting audio, notation and controlled musical output. We don't have a timeline to share.