A(G)I and you: how much of your own work can you automate?

Community
,
1

AI tools and models like GPT-5.5 are becoming more capable across coding, writing, research, operations, design, analysis, and many other workflows.

I’m curious how far people here are getting in practice: How much of your own work can you automate today?

Are you mostly automating tasks where you already have professional judgment, or using AI for support tasks outside your core expertise?

Where does the remaining work show up? Is it review, correction, integration, accountability, domain knowledge, taste, or edge cases?

Is the hardest part still at the end, or is it spread throughout your workflow?

As these tools mature, cost could become part of the discussion too. If today’s usage benefits from subsidized inference costs, which workflows would you still consider worth paying for at closer to real cost?

Looking forward to what you want to share!

2

Heya VB, hope you’re well.

I’ve actually stumbled on a task I cannot automate at all. I’m at hundreds of hours of conversation across multiple models—ChatGPT/Codex, 5.4, 5.5, ClaudeSonnet/Opus 4.6, 4.7. We’re writing a system ontology; and what we’re doing amounts to one very , very long philosophical conversation about JSON schema. I’m running out of context every day of the last two months… and I thought this would take a week tops.

A few weeks (or months now) ago, I tried firing up some subagents to try hammering out the full ontology in a single step. We went through about five iterations from various [very detailed] prompts before it became clear that we needed to go one-by-one for a given “domain” of the overall ontology. We ended up with “a good starting place.” That’s it.

Now, we’re at hours and hours of discussion for each DOMAIN schema. It’s actually very exciting to have a topic that is so challenging for frontier models and little old me.

I have as much professional judgement on this matter as anyone—it’s really more the attention span for the work that would be lacking.

We have been producing (and working extensively on) a single skill / subagent pair, and paying minute attention to every phrase we create. At this point, we have over 50 Diff Coins—our term for a single JSONL that’s intended to train a model. So, we’re not formally fine tuning yet, and we have a dataset that COULD be used to fine tune, but I just have the model look through the extensive research before we begin work.

We are definitely becoming more efficient, but wow, I’m shocked. This isn’t “waste” conversation either, many of our steps might alter a definition slightly or need a fidelity sweep through the entire ontology, (like 5000+ lines of JSON Schema, and 7 documents with 7 more to go.)

Automation and Augmentation

This task is definitely “ai augmentation.”

Given the experience with it, my first real try at fully annotating a dataset for fine tuning, for example—I am going to be very wary of any automation outside of my expertise.

That said, i also think one of the main barriers between Automation and Augmentation is the lack of precisely this sort of Ontology.

I’m working under the hypothesis that a completed ontology (a json structure that shows how everything conceptually fits in a system) will greatly enhance efficiency and all other attempts at full Automation.

The actual results of an AI building with this ontology “in mind” have been very promising.

But if my experience is typical of people designing ontologies, then the barriers to entry with regard to thought, time and compute are considerable.

The Hard Part

Clearly this is the hard part. lol I’m still shocked by “how hard,” even as I become increasingly “pleasantly surprised” by how previous moves are starting to compound into “model experience.”

But, the “extra” work, geeze. I’m working in a limited range of documents, and it’s STILL hours reading, and re-reading stuff a model wrote, or I wrote, or was added or appended to some concept—and rewriting language so it matches exactly what I have “in mind”, which is absolutely necessary for avoiding down-stream hallucinations which will immediately propagate.

What about annotating the data set? Geeze. What about automating the dataset in the first place?

But all of this would be called "review, correction, integration, accountability, domain knowledge, outside of domain knowledge, taste, edge cases, research for both familiar and unfamilliar topics, integrating new things (like the Agent SKILL protocol, MCP, or A2A just this morning), designing skills and subagents organically while we work (which has been one of the greatest assets so far), annotating data.

Cost Subsidization

Why, thank you for asking! I actually think this type of work would be very worth subsidizing because it’s “compute and attention” heavy, but we’re ultimately trying to make downstream operations more efficient.

So, in theory, I would be more willing to pay full price for compute with a [philosophically sound] ontology helping the Models understand “where things belong”. To be perfectly frank, a single turn with Codex GPT 5.5 Extra High (thinking) burnt my 5 hour context limit all at once. :scream:

Though, you know, as I think about it, what about subsidizing “non-wasteful workflows.” Like, are you making full use of your conversations, outputs, harvesting mistakes as a dataset for small model training (as you go): Give that dev a subsidy!

(I think the term “tokenmaxxing” is embarrassing. If you’re making efficient use of compute, and running out of tokens, that’s one thing. But if you’re just burning compute to pad a statistic…)

Practically

My goal is to work with businesses to develop their own ai-first system. I can say, without a doubt, that the hardest part will be getting the experts to sit down and annotate their own datasets.

I just throw up my hands and tell them the accuracy WILL ONLY come from them doing THEIR homework (no way am I annotating a set whose topic I don’t understand. Nope. Not after all this. I’ll build it, I’ll make it easy to annotate, but annotate myself? Nope.)… but I’m betting that if they have a complete ontology to start with, “full auto” will be achievable far sooner.

3

Hi @thinktank,

Yes, it has been a while since the last AI Pulse. Hope you’re doing well!

I agree and would use the figure of speech: the devil is in the details. I notice those details most in areas where I either have the most expertise or the highest expectations.

For example, I can automate recipe selection for using up leftover Easter eggs and happily accept a mediocre semifreddo recipe. But if I want to make Keller’s “Coffee and Donuts,” then it is on me to ask the right questions and understand what makes that recipe special. And realistically, my kids may still be perfectly happy with the mid version.

That is just a more tasteful example of the same problem: getting the basics right is still a hard obstacle to overcome, even with AI augmentation.

On the other side, we also need to lift the results beyond the point where the outcome is merely acceptable. Those are the two main challenges I mean when I talk about getting “past the 80%.”
And how much of this can be automated?

4

Not a direct answer, but something related to think:

TL;DR: AI will only perform well when you know what you are asking it for, otherwise not even AGI can solve a problem without proper context. Same goes for humans when hiring someone to do a job.

5

I have been testing this pretty heavily over the past year.

My answer is that a lot more can be automated than I originally thought, but only once the work is moved out of single chat sessions and into a proper operating workflow.

For me, the big shift was not simply asking a model to do a task. It was building a system around the model that handles state, project history, files, managed runs, review points, exports, and repeatable execution.

At that point, the human role changes. I am still responsible for taste, judgment, direction, review, correction, and deciding what is good enough. But the amount of raw production work the system can carry is much larger than I expected.

The remaining work shows up mostly in supervision, architecture, verification, and deciding what should happen next.

So I do not think the useful question is only “how much can AI automate?” I think the better question is “what kind of operating environment does the model need before automation becomes reliable?”

That is where I think the real field is forming.

6

This is a valid observation. Without a proper framework and pipeline, the vision capabilities of GPT-5.5 do not always outperform 4o. But with a few more years of experience and the generally improved capabilities, I can usually just put together the requirements quickly and get results that work well enough.

And yes, the original question is definitely aimed at developers using AI to automate away parts of their own work.

7

Yes, that makes sense.

I think we may be talking about slightly different levels of the problem.

A framework and pipeline definitely help, but what I mean by operating environment is more than prompt structure or choosing the right model.

I mean the runtime around the model: persistent state, managed runs, files, artifacts, review points, write back, recovery, and enough visibility that the work can continue reliably instead of depending on one chat context.

Model choice still matters, but the biggest unlock for me has been treating the model as one part of a larger stateful system.

8

Looking at the interesting replies from both @thinktank and @Pimpcat, I think I may need to rephrase the question.
This is not about a specific process, like “How I automated invoice processing across teams.” It is more about a broader work philosophy:

“My work is done when I am no longer needed.”

How much of that statement is becoming true for developers working in the AI space?

I am interested in this because the claim that coders may no longer be needed in the near future is still hanging over us.

9

Agree with @Pimpcat

I built an otherwise deterministic workflow system on top of Sidekiq and it directs when to apply a work step, and most of those leverage LLMs.