Exploratory Product Analysis for Product & Engineering Teams
Though titles and team structure vary, there’s a general formula for how analytics changes businesses: the data team collaborates, investigates, and shares their insights. But with new agile tools like Hex in the market, we’re starting to see functional teams like engineering and product glean their own insights, built on top of the hard work of a data team. This guide will talk through how you can empower your Product and Engineering teams to ask their own questions of your data, find useful answers quickly, and ultimately build better products.
Why is this so hard? Data context and inflexible tools
To understand how to build more self-sufficient analytical Product & Engineering teams, it’s worth exploring why so little self-directed insights are discovered by Product and Engineering teams today. Among our customers, we’ve seen two major setbacks these teams encounter when they try to do analytics themselves: difficulty understanding data and its context, and the rigidity of product analytics tools.
Finding the right data with the right context
For companies running on a reasonably modern data stack, there’s a loose standard for data collection: product interaction events are instrumented, collected, and stored in a data warehouse. This is great, because anyone with a decently technical bent can query that data. But it’s also not so great, because it leads to more questions than answers:
Which table(s) does the data I need sit in?
What does this column mean? And why is it missing so much data?
What filters should I apply to get a meaningful result?
Are we even tracking the right events to answer my question?
This isn’t a new problem, and it’s why there’s so much excitement around the emerging idea of a semantic layer. Without some semblance of that semantic layer or rich context on data, asking Product and Engineering teams to query data directly is like asking someone to swap the engine in a car without any underlying knowledge about how the engine works.
Answering questions and finding insights starts with having the right data organized in the right way. And without a data team who can make that a reality and the right tools to surface that information, Product and Engineering teams are often stuck without knowing where to look. Or even worse, they query the wrong data, get the wrong results, and wrong decisions get made.
The rigidity of product analytics tools
The good news is that directly querying the warehouse is rarely the only option that Product and Engineering teams have – specialized product analytics tools provide simpler UIs for pulling and visualizing the data you need. But these tools are designed for highly specific use cases, and can be frustrating to use:
They’re mostly limited to event data – it’s difficult to access data from other tools (like Stripe or Hubspot) or aggregated data points.
New data needs to be manually added in – each data point needs to be instrumented by an engineer.
Licensing makes it difficult to share results broadly – which means that teams often resort to screenshots, which are their own can of negative worms.
The same is true of BI tools, where adding in new data models can be a multi-week affair. These tools were designed for regular reporting dashboard building, not an open ended exploratory use case.
What your Product and Engineering team can do with the right tools
We’ve seen organizations like Modern Treasury, Notion, Algolia, and StubHub successfully use Hex to create a flexible, agile environment for their Product & Engineering teams to run their own investigations. They’ve generally relied on a strong semantic layer for shared definitions and data context, with some “guided applications” in Hex as starting points for queries. Here are some of the most common use cases:
1. Tracking feature adoption and performance
The first thing you want to know after a launch is whether and how people are using the product or feature. Instead of waiting for new data models to be added to a BI or product analytics tool, you can analyze feature effectiveness in real time in Hex by querying the warehouse directly. Because Hex is built for flexibility, the nature of these analyses can change as you get new information about a feature (a new filter, an updated definition of usage) – you’re not locked into the initial data model.
2. Exploratory, ad-hoc investigations
More often than not, critical insights come from genuine curiosity or a hunch. Enabling your teams to follow their intuition and investigate what they’re seeing will lead to better products in the long run – but your data environment needs to be flexible enough to support that. In Hex, instead of waiting for the right data, Product and Engineering teams can jump in and query the data they’re looking to understand. They can also easily transition between SQL, Python, charts, and no-code environments so they’re always getting the best tool for the job.
3. Overall product reporting
Creating reports and dashboards to monitor your product usage, WAU, MAU, feature adoption, churn, and other similar metrics is easy in Hex. But unlike traditional BI tools, your functional teams can explore from there and dive into further analysis. We’ve seen Hex customers set up reporting infrastructure for Product and Engineering teams with powerful drill downs as starting points for diving deeper.
4. Contextual data for product research and rollouts
To understand what to prioritize on the roadmap and the nuance of how to handle the details, looking at user interaction patterns with your existing product surface is the best place to start. Hex’s ability to pull in data from all different sources, from product data sets in your warehouse to demographic data from Salesforce and Clearbit, makes it easy for Product and Engineering teams to get the right data to understand their users. For big launches, your data team will likely get involved; but for smaller ones, PMs and engineers can dive into these results themselves.
What does a flexible, agile environment for Product & Engineering teams look like?
For Product and Engineering teams to be able to meaningfully explore and derive insights from their data, they need an environment that solves the aforementioned tooling issues and gives them the ability to rapidly, iteratively ask their own questions of data. We’ve built Hex on top of years of expertise talking to customers and understanding how they can empower their organizations to be self-sufficient when it comes to data.
1. Warehouse-native, all the data
Today, we’re seeing teams increasingly use the data warehouse as a sort of nervous system for their data – storing much more than just the basics for product analytics. A flexible data environment needs to be built on top of that warehouse so teams have access to all of the data they’d need to glean insights.
2. A semantic layer and schema definition
To get value out of your data, your Product and Engineering teams need to understand the ins and outs of what each data point means and when to use it. Teams are adopting different degrees of a semantic layer that sits on top of raw data and defines common metrics like users, revenue, and the like. With shared definitions of business concepts, Product and Engineering teams can query in confidence that a “user” means the same thing to everyone. As dbt’s semantic layer becomes more popular, a native integration with it is a helpful addition.
A well built out semantic layer is ideal, but not always practical for every team and every stage. In the meanwhile, Product and Engineering teams need the ability to quickly see and understand data in the warehouse. The ability to simply view schemas in your warehouse, identify which columns are in each and what data types they contain is table stakes. And to when it’s not clear where to start, Hex Magic uses AI to help build your queries for you.
3. Polyglot query capabilities
Product and Engineering teams have all different kinds of backgrounds and language preferences, but most product analytics tools force you into one query method. Hex allows you to query and work with your data in whatever way you know how – SQL, Python, charts, and no-code transformations.
4. Collaboration and sharing are native
The way your teams build products is default collaborative; the way your teams look at product data should be too. Collaboration is built right into Hex, from the ability to work together on a shared canvas, to easily sharing your work, down to a particular cell or visualization. No screenshots necessary here.
Data teams aren’t going anywhere; if anything, the more advanced tooling gets the more important they are to a company’s heartbeat. But empowering Product and Engineering to explore their own data and answer their own questions – without needing to rely entirely on the already capacity constrained data teams – can help companies make better decisions faster.