I tried to vibe code Hex

I spent 5 hours building something my company sells, an AI analytics agent. The first hour was magic; the last 4 turned me into a product manager. Building to learn is valuable, necessary in the AI era, but building to ship is a different beast entirely. The cost and effort required to build something have never been lower. But why are we being pulled to build these things?

My experience gave me a strong sense of ownership over what I created. It's intoxicating, and what I love about data work. Building that dashboard, or shipping that insight, this felt a lot like that.

Until it wasn't.

The moment you bring more people into it, complexity explodes. Not because the questions are harder, but the scale creates new challenges that aren’t solved by better data models. They are scary words like security, collaboration, and privacy… way outside my skill set. This is the trap: Claude made me feel like I could do those things well.

More people, more problems

Agents require the same level of context, no matter the tooling. Fresh out of the box, they don’t understand your business, data, metrics, goals, anything. So my time spent creating markdown files for my fake dataset was net-neutral. What I spent much more time on was the agent’s harness.

I had to tell it what it even was, the tasks it should perform, how it should output the information, how it should store past conversations and insights, the list got long really fast. Then I realized that I hadn’t done any analytics in about 3 hours, just skills tuning and creation. It was a great learning experience, but now I have to maintain this.

The oh shit moment came when I wanted to connect this to Slack so I could eventually build a Slackbot-style version. I imagined my Head of Data earnestly asked me to build this for our company. What would that take? Once I started thinking about adding more people to this, it got even more complex.

I wasn't just building for myself anymore. I had to think about data privacy, permissions, shareability, refresh logic, answer-quality review, and model-upgrade fragility — and none of it touched a single chart.

Every new user wasn’t just another question. It was another system I had to design.

Scale means making decisions about things far beyond analytics or “getting the right answer” from an agent. The true power of data isn’t just that an analyst can get faster insights; it’s the self-service nirvana we’ve all been chasing.

When you go down this road, you realize the token & subscription savings are offset by the cost of an FTE (or 2 or 3) who maintains this piece of infrastructure. Because that’s what you’re building if you intend to scale this outside of yourself.

From analyst to product manager

Some people are building to prove they’re still needed. Analysts are the most visible targets in the AI-replacement conversation, and watching an agent write perfect SQL is unnerving.

You are valuable to organizations if you understand the business, can curate context for the agents, and tell compelling stories that drive action with data. None of those things requires you to build the entire system. It does require you to learn what these tools can do and which parts of your job you should automate. Building the whole thing yourself means you’re rebranding as a product manager, not a better analyst.

We should all stop and ask ourselves why we went down this road in the first place. Was it to learn? Or was it to deploy something in production for others to use?

If it’s the first one, build on! Because learning how these tools are set up, what makes them work well, and where you find value makes you a sharper buyer and evaluator of others’ technologies. The same learned experiences we had grappling with which data model approach to go with. You can see a solution, open the hood, and understand where this will/won’t work for your organization.

If you’re doing it to deploy, ask yourself: What net benefit are you providing outside of “I built it my way”? I’m not saying don’t build anything. But focus on building the things that will support your role and the broader business. There are far more interesting questions around context windows, data modeling, semantic models, and agent evals that require your attention.

Build to learn, not to deploy

If you’re thinking about trying this, I say do it. Feel what it feels like and where it gets brittle. Take that experience into your search and find a tool that connects to your stack and delivers the customization you want, with the collaboration and shareability you know you’ll need.

Evaluating which AI analytics tools are real vs. just crisp demos will remain a challenge. Learning through building makes you a better critic of this technology. Build to understand it, and what deploying it actually means.