Sample:
https://vimeo.com/manage/videos/572882138
Joey from Open AI support suggested that I post some ideas here. This an introductory note for somehow reach the tech community for developers for Open A, this is not a creative discussion.
Basic ideas is to comix Free Form Natural Language with mathematical standard rigorous formal languages e.g. surface and volumes specifications.
In the video above we are able to simulate the Physics a Soft Cloth in Graph-form, and shortly we do the simulation entirely in a Message-passing Neural Net. But that Neural Net is not and cannot be a part of Open AI!
Therefore we are in need of Hybrid Multi-task Multi-pass semantic generative Free Form Neural Nets.
For the Free Form English we are exploring Open AI and we can handle the rest by our own learned nets.
Q: How can we build such a Hybrid System without requiring any alterations to Open AI?
Example: Apropos of the above simulation, we like to allow phrases:
“Zigzag Twill, light green Silk cloth with Annulus cutting and with radiuses 0.5 to 1.0 centred at the edge of the table …”
Zigzag Twill is texture (uv-image), light green is the colour (RGB or Hue), Silk is the substance (from Substance 3D?).
Yet the Annulus is a geometric term, radiuses 0.5 to 1.0 are to be mapped into the Annulus formal geometrical language (easy to do for us) and and centred at could be an Open AI semantic or we have to develop that to set the centre at some xyz.
I find no tech docs to even beginning to feast my mind on how to do such integration, and it could be that I am did not look in the right places.
Please make astute technical comments by those with under-the-hood knowledge of Open AI.
Much in your debt,
Dara O Shayda
Wexford, Ireland