I, like many others, tried to take a crack at building agents at work for problems requiring deep domain knowledge. This has been fun, and there have been more failures learning moments than pure successes. I wanted to write down one of the main design choices that keeps coming up while building these kinds of systems.
The cold start problem
When I first started working on internal AI agents, I made a mistake that I think is pretty common. I started by comparing harness SDKs like LangChain, CrewAI, the OpenAI agents SDK. After all... to build an agent you first have to build a harness, right? These SDKs provide a core agent loop and focus their innovation on how you customize and mold that loop to your own problems. You bring your own tools, even basic ones like read(file), and in return you get a system that is entirely your own.
This route may be the right choice for you and the problems you're trying to solve!
What I found when building with these SDKs is that I was spending an awful lot of time writing boilerplate code. I was dealing with details that are crucial for the agent's basic functionality but do nothing to drive the project forward like loading it with appropriate domain knowledge.
For most innovation-style side projects you're taking on at work, and for those projects where you are building with individuals who are not themselves deeply experienced in applied AI, structured outputs, tool calling, etc. I'm of the opinion that you should not roll your own harness.
I think this may be counter-intuitive for most folks. When you get pitched an internal data investigation bot your head starts thinking about writing custom query or search_confluence tools, etc. You should instead just give your agent a sandboxed bash shell and access to a psql or snow cli. Maybe write a custom confluence cli tool for it to use (codex was able to one-shot mine in 5 mins).
You need the Django of AI
You should borrow one of the extremely well-built pieces of software like the excellent pi - the coding agent underneath OpenClaw. I also use the pi coding agent daily. The built-in tools - read, write, edit, bash - are more than enough to solve most problems. If you need extra functionality, you can either write a CLI utility that can be called from bash, or easily plug in a custom tool or extension.
Rolling your own harness is similar to building a SaaS with something ultra-lightweight like Flask or FastAPI. It can be done, it has been done before, and there are many cases where that is the best decision to make. However, for most greenfield SaaS projects, you can get away with using something more opinionated like Django. You're locked into the conventions of the project, but those conventions are perfectly fine for most people. If you're building an agent that needs to read, write, and edit files, and run bash commands to solve problems, then pick an off-the-shelf coding harness to skip to the good stuff. Bypass the implementation of basic file and execution tools, as well as the other boilerplate code that isn't unique to the problem you're solving. Why would you roll your own users, groups, auth, admin panel, etc when Django provides an incredibly robust solution for all of this right out of the box? Time is money. Why roll your own suite of custom build and migration scripts when manage.py will do? Don't waste your time reinventing the wheel. You can always swap out harnesses with something custom down the line if you think you need it. But it's likely that You Aren't Gonna Need It.
Budgeting your innovation tokens
There was an article I read a long time ago featuring a concept I keep coming back to at work called "innovation tokens" (Choose Boring Technology).
If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that’s existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you’re in trouble.
This dates the article a bit to claim NodeJS would spend an innovation token, and it's also highly team-dependent, but grant me your suspension of disbelief for just a moment and let's continue. If you choose to write an AI agent, you've spent one (or two) of your innovation tokens right off the bat. You choose to build a custom pipeline for ingesting context into a vector database for semantic search? Arguably one or two tokens right there. You have to roll your own harness from the ground up, including every tool and every connection to your internal domain knowledge? You're cooked. Rules are meant to be broken, so I think it's fair to say we have to choose to go into some sort of innovation token debt if we want to build a system with applied AI right out of the gate. I'll stop my Dan McKinley impersonation before we beat this metaphor to death. There are tons of other great posts and thinkers in this space. Shout out to grugbrain.dev for another one of my favorites.
The boring tech for AI agents
The point is, you can build a kick-ass, fully integrated agent without touching an agent loop or writing custom tools. I did it with Postgres, Django, React, and ironically a NodeJS service that handled agent runs and streaming websockets back and forth between the user and the agent.
Author's Note: John is anti AI slop. Ouachita Labs blog posts are always written with care by a human brain, proofread with Claude Code.