Policy Synth "Engineer", an automatic programming agent for Typescript

ChatGPT Use cases and examples
,
1

I’m happy to share an early version of our our open source Policy Synth “Engineer” project, a multi-scale AI agent solution for automatic Typescript programming.

Policy Synth Engineer

Features

  • Automatically programming for; new features, fixing bugs and refactoring Typescript projects from a simple specification

  • Automated web research (with many layers of quality increasing pairwise voting) for: documentation, code examples and error solution search.

  • Built to auto-program Policy Synth itself, Your Priorities, All Our Ideas and our other open source Typescript civic-tech projects

Why Typescript only (for now)?

  • Typescript is a unique fit to LLMs and automatic programming

  • Typescript is Javascript with types on top and Javascript is by far the most common programming language on the web and in any training dataset

  • Typescript has past few years itself also been the fasted growing programming language

  • Typescript has “types” providing a uniquely useful context for the LLMs that is missing in Javascript and Python

  • Typescript classes can be compiled with the Typescript compilers, in a review and code planning loop to fix auto-generated code, automatically.

Working already with GPT-4 (but will get better with “gpt2”):

By using pairwise voting extensively, automated web research for documentations, examples, and now about 25 different agent types, I’ve got it working with GPT-4! But it needs to go through many reviews and compiling loops, and makes many mistakes. We expect Engineer to work better & faster with “gpt2” (or whatever it will be).

Multi scale agent flows

image
image1920×1034 78.7 KB

image
image2574×1386 224 KB

image
image2574×1386 193 KB

image
image2574×1386 259 KB

2

Just to add that Engineer is now working with gpt-4o at 2x speed compared to gpt-4, and with what seems, better coding results! :slight_smile: I’ve push the model upgrade to github.

3

First GitHub Issue completed by Policy Synth “Engineer” in about 7 minutes, fully automatically: Create LLM Abstractions for OpenAI, Claude Opus and Google Gemini with a common base class · Issue #15 · CitizensFoundation/policy-synth · GitHub

Slightly ironically, this ticket is about adding a base model class and support for Claude and Gemini to Policy Synth Agents, which of course we want to have but maybe less need just today as gpt-4o is so far ahead! :slight_smile: Let’s see what Google brings to the table tomorrow.