Over the past several weeks, I’ve been spending a lot of time thinking about systems.
Some of that thinking has taken the form of writing.
If you’ve come across any of my recent posts, they might seem like they cover very different topics:
- cataloging rocks in a backyard
- building AI systems using MCP
- working with documents, images, and real-world data
At first glance, they don’t appear to have much in common.
But they’re all exploring the same underlying idea.
The Common Thread
Across all of these posts, the focus has been on a specific kind of problem:
How do we turn messy, real-world inputs into structured, usable systems?
That problem shows up in many different forms.
Sometimes the input is physical:
- objects
- artifacts
- environments
Sometimes it’s digital:
- documents
- images
- logs
Sometimes it’s dynamic:
- motion
- behavior
- sensor data
But the challenge is the same.
The input is unstructured.
The system needs structure.
The Backyard Quarry
One way I explored this idea was through a small project I called the Backyard Quarry.
It started with a simple observation:
There are a lot of rocks in the yard.
From there, the problem evolved into something more interesting:
- how to represent physical objects as data
- how to capture images and measurements
- how to build pipelines around that data
- how to search and organize it
- how to think about digital twins
What began as a small experiment became a way to explore system design in a constrained, tangible setting.
MCP and AI Systems
In parallel, I’ve been writing about building AI systems using MCP.
On the surface, this looks very different.
Instead of rocks, the inputs are:
- documents
- APIs
- models
- agent workflows
But the structure is familiar.
- inputs are ingested
- processed
- transformed
- routed
- used by applications
The system still needs to handle:
- variability
- scale
- imperfect data
- orchestration
Different inputs.
Same patterns.
From Objects to Systems
One of the more useful realizations in working through these ideas is this:
The problem is rarely about the individual object. It’s about the system that handles many objects over time.
Whether the object is:
- a rock
- a document
- a sensor reading
The questions become:
- how is it represented?
- how does it enter the system?
- how is it transformed?
- how is it stored?
- how is it retrieved?
These are system-level questions.
A Shared Architecture
Across these different domains, a common architecture begins to emerge.
The labels change depending on the domain.
But the structure remains consistent.
Why This Matters
Understanding this pattern makes it easier to approach new problems.
Instead of starting from scratch each time, you can ask:
- Where does the data come from?
- How does it enter the system?
- What transformations are required?
- How will it be used?
This reduces complexity.
It also makes systems more predictable.
What I’m Interested In
Going forward, I’m particularly interested in systems that sit at the boundary between:
- the physical world and digital systems
- unstructured inputs and structured data
- human workflows and automated processes
That includes areas like:
- digital archiving
- photogrammetry and 3D capture
- AI-assisted analysis
- systems that track objects or behavior over time
These problems are messy.
Which is part of what makes them interesting.
A Continuing Exploration
The posts I’ve been writing are not meant to be definitive.
They’re part of an ongoing exploration.
A way to think through problems in public.
And occasionally, a way to use a slightly unusual example — like a pile of rocks — to make broader ideas easier to see.
If You’re Interested
If any of this resonates, you might find these useful:
The Backyard Quarry Series
A systems-focused look at modeling and working with physical objects starting with Turning Rocks Into Data.
MCP and AI Systems
A technical exploration of building agent-based systems and data pipelines. I'd suggest starting with The End of Glue Code: Why MCP is the USB-C Moment for AI Systems.
More to come.
And if nothing else, it turns out that even a backyard can be a good place to think about system design.
Top comments (10)
starting with physical examples before jumping to the AI system is the move. most "how I use MCP" posts lead with architecture diagrams and lose me in the first paragraph.
I'm pleased that the approach resonated. It’s easy to get lost in the abstraction of architecture diagrams, but seeing the real-world data first usually makes the technical implementation feel less like a theoretical exercise and more like a solved problem.
yeah exactly - I've noticed teams spend way more time debating diagrams than they would if they'd just played with the actual tool for 10 minutes first. the physical example shortcuts that whole 'but what does it actually DO' loop
Exactly. When you’re dealing with messy, real-world data like 19th-century ledgers or physical sensors, the 'idealized' diagram usually falls apart immediately. Starting with the physical object forces the system to be grounded in reality from day one.
the 19th-century ledgers point lands - edge cases aren't exceptions, they're the actual shape of the domain. idealized models just delay discovering that
The architecture pattern holds well across domains, but it gets harder to close when the transformation step is an AI agent rather than a deterministic function. A rock-to-data pipeline is reproducible; an agent call isn't - same input, different model state, different output. That means "how is it transformed?" becomes a first-class question you can't answer from the output alone, because the transformation carries provenance (model version, prompt, context window state) that needs to be captured alongside the result. In agentic MCP pipelines especially, the transformation step is where auditability tends to break first.
You’ve identified the exact 'black box' problem in digital forensics. If the transformation isn't deterministic, then the prompt and the model state effectively become part of the raw data. In my view, the 'Metadata' of the transformation is just as important as the record itself—without capturing that context window state, the audit trail ends at the agent call.
There's something quietly useful about the realization that the problem is rarely the object—it's the system that handles many objects over time. That reframe changes where you spend your energy. Instead of solving for the rock, or the document, or the sensor reading, you start solving for the pipeline shape that can accommodate all of them without needing to be rewritten each time.
What I find myself thinking about is how much of system design is really just pattern recognition across domains that don't look related until you squint. The fact that a backyard rock cataloging project and an MCP-based AI pipeline surfaced the same structure suggests the pattern is genuine, not something you imposed retroactively. It's like discovering the same bone structure in wildly different animals—once you see it, you can't unsee it.
The messy boundary between physical and digital is where things get especially interesting to me. Structured data is comfortable. Unstructured inputs are where most of the real world lives, and most of our tooling still assumes someone else already did the hard part of making it machine-readable. The systems that sit on that boundary and do the translation work feel like they're going to define a lot of what gets built in the next few years.
I'm curious—when you were working through the Backyard Quarry, did any of the patterns surprise you by showing up in places you didn't expect? Or was it more of a confirmation of things you'd already felt intuitively but hadn't mapped explicitly?
You’ve hit on exactly what made this project so rewarding. It wasn't so much a 'surprise' as it was a profound confirmation of intuition.
When you've spent years building enterprise systems, you develop a gut feeling for where the 'stress fractures' will appear. Seeing the same 'bone structure' emerge in a backyard rock catalog as in a high-scale MCP pipeline was a validation that these patterns aren't just artifacts of our tools—they are fundamental laws of data integrity and scale. The most satisfying part was watching the 'messy' physical reality of a rock finally click into a structured digital twin without the pipeline needing to change its shape.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.