A few weeks ago I posted about using encoded human expertise to process an astronomical image. It got a good reaction, but the more interesting part happened after I hit publish, when I kept pulling on the thread and landed somewhere I didn't expect.
Here's the short version of where I ended up: the opportunity to blow the lid off quality and throughput on real work right now is enormous, but only if we resist the most obvious way of using AI to get there. Let me explain.
The first attempt, and what it got wrong
In the original post, I had an LLM write fresh, reusable Python code to process a Messier object image, the kind of deep-sky target amateur astrophotographers spend hours capturing and processing. The code worked. It was a genuinely useful demonstration of encoding expertise into a repeatable workflow, and I still think that's the right direction for a lot of tasks.
But something about it nagged at me. The LLM was, in effect, reinventing image processing from scratch. Stretching, background extraction, color calibration, noise reduction: all of this is decades-old, hard-won knowledge encoded painstakingly into existing software. Writing a fresh script to approximate that work, no matter how capable the model, means throwing away everything the software already does well and betting that the LLM can catch up in real time.
That's a strange trade to make. It's a bit like asking a brilliant new hire to write their own compiler instead of using gcc, on the theory that they're smart enough to figure it out. Sure, maybe they can. But why would you want them to?
Worse, the quality of output was a bit like a slot-machine. Sometimes good, sometimes awful and yet the LLM confidently claiming victory.
A better question
There's already public, open-source software that astroimaging experts have spent years refining. Siril is one example: free, powerful, and full of tools that took real domain expertise to build. Like most specialized software, it also has a real learning curve. I spent half a day tinkering with Siril to get a good result on a hyperbolic stretch technique that a YouTuber had posted a detailed video on. Half a day, for one image, on a technique someone else had already worked out and generously documented.
So I asked a different question. Instead of having an LLM write a script from scratch and iterate on it, what if it could just control and direct Siril itself?
I fed the YouTube transcript into an LLM agent and let it drive Siril directly, running on my low-powered laptop. The results genuinely surprised me. What the AI agent produced in about 20 minutes was better than what took me many hours of manual tinkering using the exact same software. Mind, kind of, blown.
The workflow that emerged
What came out of this experiment is a workflow I think generalizes well beyond astroimaging:
-
Encode human directives. Using MCP for X, I captured the hyperbolic stretch parameters I wanted, the target look for the final image, and the quality checks the output needed to pass. This is the expertise layer: not code, but a structured decision tree that represents what a knowledgeable human would actually be checking for.
-
Hand off execution to an agent that controls existing software. Rather than writing new code to replicate what Siril already does, the agent drives Siril directly, the same way a human would, just faster and without needing a half-day learning curve first.
-
Validate automatically against the decision tree. The agent checks its own output against the MCP for X quality criteria before calling the job done. This is the step that turns "an LLM did something" into "an LLM did something I can trust," because the check is grounded in an explicit, human-authored standard rather than the model's own judgment of what looks good.
Encode, execute, validate. That loop is the part I think matters most.
Why this generalizes
I've spent close to two decades in healthcare analytics and actuarial work, and the parallel here is hard to miss. Actuarial workflows, IBNR reserve estimation, risk adjustment, reserve triangulation, are full of exactly this pattern: specialized software and methodologies, built by domain experts over years, that an LLM could either try to reinvent from a blank page, or learn to direct with the right guardrails in place.
The instinct to have an LLM "just write the code" is understandable. It feels more flexible, more impressive, more like the AI is doing the real work. But in a lot of deep, niche-expertise domains, the real work has already been done. The software exists. The methodology exists. What's been missing is a fast, reliable way to direct that existing work using the specific judgment and standards of someone who actually knows the domain.
That's a different kind of AI project than "build a smarter script." It's closer to: capture what an expert actually checks for, let an agent operate the expert-grade tools that already exist, and confirm the output meets the bar before it ships. It's less flashy than watching an LLM generate code from scratch. I think it's also a lot more useful, because it doesn't ask the model to be an expert in astrophotography, or actuarial science, or whatever the domain happens to be. It asks the model to be an excellent operator of expertise that a human already encoded.
Where this leaves us
None of this requires the biggest, most expensive model, or some exotic new architecture. It requires being disciplined about not asking AI to reinvent the wheel just because it can. The software, the methods, the hard-won domain knowledge: all of it already exists, in most fields worth automating. The opportunity isn't replacing it. It's building the layer that lets an agent operate it with real judgment, checked against a standard a human actually set.
I'm excited about what this looks like for data analytics, for actuarial work, and yes, for astroimaging, three fields that have almost nothing in common except that all of them are full of expert software waiting for a better way to be directed. If the workflow above holds up as I apply it more broadly, I think it's a genuine glimpse at what a lot of expert work is going to look like fairly soon: not AI replacing the tools, but AI finally being trusted to use them well.