AI: because your team has better things to do than writing boilerplate SQL

AI is everywhere right now — in headlines, on conference stages, and in the "have-you-seen-this?" links your colleagues are Slacking you at 11 p.m. The hype is deafening, and if you're leading a data team, it can feel like everyone expects you to have an "AI strategy" yesterday.

Long before the Notebook Agent went live, our data team had been road-testing it (and plenty of other AI tools) in real workflows. We found that while people are still central, AI has a huge potential to drive impact faster.

Our experiments have taught us that unlocking real value from these tools doesn’t come from bolting AI onto everything. It comes from prioritizing high-impact use cases and testing solutions.

And the starting point is deceptively simple — AI has to make the team faster before it can make the team smarter. Here’s how we start.

Step 1: Build for speed by automating the low-hanging fruit

Every data leader knows the problem: too much of your team’s time gets eaten by dull, but necessary tasks. These are tasks that are repetitive, fiddly, or just plain boring — writing boilerplate SQL, cleaning up messy CSVs, reformatting documentation. They need to be done, but aren't the interesting, high-impact data work that got us excited about our jobs in the first place.

AI-powered tools can help us move faster through these tasks, which shouldn’t be overlooked. Every hour saved is an hour we can put toward higher-value work.

But let’s not pretend it’s a silver bullet for every task you give it. If you've tried using AI in a complex codebase, you've definitely had at least one "this-would-have-been-faster-by-hand" moments. The key is learning where it helps, and where it doesn’t.

Our principles for using AI to speed up data work

1. Test liberally in your workflows. Spot the repetitive tasks that eat up your team’s time — writing docs, cleaning data, early analysis drafts — and ask, could AI do this faster? Run small experiments to find out.

2. Time-box your experiments. We set a limit on how long we'll spend trying an AI approach. If it's going badly, we bail early.

3. Keep humans in the loop. We joke that we've "hired a junior analyst in the form of an LLM" — one who still needs reviews and a lot of guidance. If AI gets us from zero to a C+, closing the gap to an A is still much faster than starting from scratch.

4. Make sure your work still influences an outcome. Just because we can do something faster doesn't mean we should do more of it. Data work is valuable when it influences decisions and drives business outcomes.

5. Keep going. Some of these experiments will be massive failures, but that doesn’t mean you should stop using this technology. After a handful of tests, you’ll start to see patterns in which tools work best, what tasks AI excels at, and where you might want to hold off for a few more months.

What tasks AI has accelerated and what tasks it’s not ready for (yet)

Through our experiments, we've found some clear patterns so far for the work of our internal data team.

AI has been genuinely helpful for:

• Drafting dbt documentation

• Creating first drafts of semantic layers in MetricFlow

• Making simple changes in dbt models, like pulling new fields from Salesforce

• Jump-starting any kind of writing — RFCs, high-impact communications, documentation of tribal knowledge, or anything else

• Asking what triggers a specific product analytics event in the codebase

• Bootstrapping new analyses and apps

Where AI just isn't up to the task yet:

• Making changes to complex data models through multiple parts of the DAG

• Identifying the source of most non-trivial bugs

• Polished final outputs for a lot of the tasks above

The only way to learn is to test things. You’ll learn what works faster and have more impact by jumping in than debating the perfect strategy. Start small, prove value, and keep going.

Step 2: Unlocking new insights that actually matter

Once AI is helping the team move faster, there’s more time for generating new insights. This is where AI feels like magic — chewing through unstructured data, surfacing patterns you’d never spot manually, and opening doors to work that used to be out of reach.

AI is great for upskilling, letting data folk work more quickly and confidently at the edge of their skill sets. Projects that might have taken research, lots of trial and error, or partnering with a more experienced teammate are suddenly accessible.

But uncovering interesting findings isn’t the same as driving impact. Without discernment, you’ll end up with “trivia insights” — cool findings that never influence a single decision.

Here’s how we evaluate where AI-driven insights add value. Perhaps unsurprisingly, it’s nearly the same way we determine whether any analysis will meaningfully impact the business.

How to make sure AI analysis is resulting in meaningful change

• Ask yourself what this impacts. Before using AI to do an analysis, we first ask: What would we do differently if we knew this? If the answer is “not much,” we skip it.

• Tie the finding back to a decision-maker. If no one will act on the results, it’s trivia, not insight.

• Prototype cheaply. Don’t build for production until you’ve validated value. Even if it’s useful, ask if it’s worth making recurring — a one-off analysis might be enough.

• Keep domain experts in the loop. AI might cluster churn reasons, but a human still has to interpret and suggest next steps.

What AI has enabled and what it’s not ready for (yet)

Here are some examples of impactful insights we’ve unlocked at Hex with the Notebook Agent.

Analyzing customers' project metadata to understand how they're using Hex. This helps our Customer team successfully support our customers’ use cases and proactively improve their experiences.

Letting our Head of Data (that’s me) build forecast models despite very rusty memories of forecasting methods. Those models allow Account Executives to go into renewal talks with more confidence about expected growth so they can land on terms that work for both the customer and Hex.

Digging into token usage from the Notebook Agent (meta!) so we can understand customer and user-level usage and make better decisions about how to position the agent.

Creating and quickly iterating on a regression analysis to understand what account features are most highly correlated with booking a meeting with our sales team.

When it works, the leverage we get from a data team member plus AI is powerful.

Advanced tasks where it hasn’t worked (yet)

Here are just a few tasks where AI hasn’t met our bar for impact so far:

• Automating insights or changes from monthly reporting.

  We’ve tried having LLM-generated weekly reports that explain the week’s performance and what changed since the prior week. So far, we just can’t get the tone right — LLMs have tended somewhere between over-eager and alarmist, and their suggestions often aren’t actionable. We hope improving our context and prompting will get us there soon, though.

• Pixel-perfect visualization.

  You can prompt and prompt, and the agent will usually get us 80% there, but sometimes, it’s just faster to just tweak visualization settings manually to get the exact look and feel we’re going for to visualize those insights. Fortunately, it’s easy to go back and forth from prompts to UX.

  .

Honestly, a lot of the examples I originally intended to include here have already gotten much better! The underlying models and Hex’s Notebook Agent are evolving so fast that what was very frustrating yesterday might be entirely possible tomorrow.

Conclusion: AI shifts data work from chores to impact

The job of data leaders isn’t to chase every AI shiny object. It’s to cut through the hype and prove that data work is making your organization more successful. Speed matters because without it, the business moves on without you. Depth matters because dashboards alone aren’t enough. AI, when applied thoughtfully, helps the data team deliver more impactful insights at the speed of the business.