tl;dr: A lot has changed in our data stacks, but it's not clear they're having more impact. At Hex we're working on this, by helping teams explore, collaborate, and share – creating knowledge that informs better decisions.
Let's start with why, 10 years into the “data era”, data teams aren’t as impactful as we hoped they'd be.
Where we’re coming from
In 2012, HBR called Data Science “the sexiest job of the 21st century.” But to anyone in the industry today, working with data can be disappointing and sad. Things have progressed a lot – we have more organizational respect, modern ELT, and a decade of compounding education. But if you were to ask data teams if they’re having a significantly higher impact than when we started this whole thing, your answers would be ambivalent at best.
For a long time the story was that we couldn't do “real” analysis work because we spend all of our time plumbing and cleaning. That was true for a while! But now we have a modern data stack™ with pipelines and warehouses and observability, generating refined data at our fingertips.
And yet, it still seems like data isn’t doing what we had hoped for. If you’re an analyst or data scientist, you spend your day digging through data trying to find insights. When you do, however, the most you can hope for is that someone can find the screenshot of a chart inside of a slide deck you sent in an email when they want to make a decision. For every “data team is a trusted partner” story, there are one hundred ignored recommendations, forgotten discoveries, and misinterpreted dashboards.
This is not the data-driven future we were promised.
Going from data to knowledge
At Hex, we believe the disconnect is because we have been looking at the role of analysts and data scientists all wrong. The job of the data team isn’t just churning out analysis – it’s about creating knowledge.
Knowledge is the sum of facts – everything that we learn from data, day in and day out. Every query, notebook, presentation, and "quick Slack with some numbers" generates facts, but the open secret is that organizations know a lot less about their world (customers, business, etc.) than they realize; there's a serious shortage of knowledge accumulating from these facts.
Knowledge needs to be shared, discovered, and referenced. It should build on itself; every analysis or insight should just be one page in a bigger book. This is ostensibly why we have all of this [** waves arm toward a data stack ** ]. We want to turn a mass of data into a base of knowledge, so we can make better decisions.
But our current tools aren't built for this. Our work doesn't compound – we're learning things and immediately forgetting them. 10 years into this modern data thing, we're looking at results_notebook_v3_final.pdf and wondering where the hell did this come from?
We toil away in siloes, our workflows divvied up between tools based on how "technical" we are. If I'm writing in Python, I'm using completely different tools than an analyst in SQL, or a business user pointing-and-clicking, and our insights wind up living separately.
This lack of knowledge-building is especially painful as organizations change. New people join. Others leave. As new questions arise, how do you know if they’re new at all? Wait... someone did an analysis about this once... do you have that PDF somewhere?
Hex: a platform for data and knowledge
This crisis in knowledge is as much a culture problem as it is a software one. But our tools shape the way we work, and we're in need of better ones. A solution focused on this would empower users to do a few things:
Allow anyone to ask and answer questions of data
Share their work in ways that lead to actual decision making by stakeholders
Organize and compound the facts they generate each day
This is what we're building at Hex.
Hex is a workspace for collaborative analytics and data science. It's all about empowering individuals to learn things, and organizations to know things, so they can make better decisions.
As I write this, there are thousands of data scientists, analysts, analytics engineers, and data-curious people of all backgrounds using Hex at places like Notion, Fivetran, Mentimeter, Workrise, and Loom to analyze data, and create knowledge.
We built Hex to be both more powerful and more accessible than legacy tools: it’s a "low-floor, high-ceiling" platform that opens the door to novices while not constraining experts. Hex lets everyone ask and answer questions, whether they're using no-code, SQL, Python, or R.
Great analysis (and knowledge) is a collaborative endeavor, so Hex also makes it easy for teams to work together, including real-time multiplayer, comments, and a sane versioning model.
Finally, projects in Hex are incredibly easy to share as interactive data apps that anyone can use. You can drag and drop to create a UI, and publish with 1 click. Apps are organized in the Knowledge Library, which makes it easy to discover and manage outputs.
Combined, all of this means that more people can contribute and work together on turning data into knowledge, and that the end results are shareable, discoverable, and useful to the whole organization.
We're just getting started. Got some data that you'd like to turn into knowledge? Click the button below to get started for free. Interested in helping realize this dream? We'd love to have you on the team.