MS Guidance - real-time interactive prompts. Anyone else exploring this?

Prompting
1

Anyone else working with MS Guidance?
Guidance

Works w both the OpenAI and Azure apis for GPT 3.5-turbo and GPT-4

Guidance enables you to control modern language models more effectively and efficiently than traditional prompting or chaining. Guidance programs allow you to interleave generation, prompting, and logical control into a single continuous flow matching how the language model actually processes the text. Simple output structures like Chain of Thought and its many variants (e.g. ART, Auto-CoT, etc.) have been shown to improve LLM performance. The advent of more powerful LLMs like GPT-4 allows for even richer structure, and guidance makes that structure easier and cheaper.

Features:

  • Simple, intuitive syntax, based on Handlebars templating.
  • Rich output structure with multiple generations, selections, conditionals, tool use, etc.
  • Playground-like streaming in Jupyter/VSCode Notebooks.
  • Smart seed-based generation caching.
  • Support for role-based chat models (e.g. ChatGPT).
  • Easy integration with Hugging Face models, including guidance acceleration for speedups over standard prompting, token healing to optimize prompt boundaries, and regex pattern guides to enforce formats.
2 Likes
2

I keep meaning to spend some time looking into this but haven’t gotten around to it yet. How has your experience with it been?

3

I wish I could say more.
I got discouraged that the openai API is limited (you can only put interesting stuff in the assistant role). Not really sure how much that matters in practice, but then I switched to OS LLMs, only to discover Guidance loads a transformers model internally in order to get inbetween the tokenizer and the model. That means you have to wait for model to load on every test. Useless

I then got some draft code from the devs to work with a remote OS server, but decided after a couple of days that was too far off my main goal, and abandoned it.

So the reality is I haven’t actually used it to do anything yet other than run one or two simple tests.

That might change in the next day or so.

1 Like
4

I think this should be highlighted a bit more:

I had a look through the guidance GitHub and didn’t notice that.
I did notice that there’s a fairly large library of prompt’s that may be useful as an inspiration for making your own “guidance” system :laughing: