Death to Dashboards, Really?

Over the past couple of years there have been a number of articles about the death of dashboards.

See, for instance, Taylor Brownlow’s piece on Towards Data Science titled Dashboards are Dead, and the pithily named deathofdashboards.com.

The argument goes something like this:

• Dashboards are the default option for analytics, and everyone gets a dashboard. Soon, there are too many dashboards in the enterprise. You find yourself with a serious report sprawl problem. It all sucks. The answer is to invert the report sprawl problem and to get rid of dashboards entirely.
• With dashboards, you get ‘death by 1,000 filters’.  Brownlow writes: “it was clear that the dashboards were not answering everyone’s questions, which was either a failure of the dashboard design step, or a failure in other tools to provide the answers people needed.” Therefore, dashboards need to go.
• People don’t trust dashboards. Brownlow again: “People started disparaging dashboards as ‘wrong’, and blatantly ignoring them. Many saw them as a threat to their jobs and if they saw numbers they didn’t expect, chalked it up to ‘bad data’.” Because you have a data quality problem, dashboards need to go.

Brownlow argues the solution is to use notebooks — that is, the style of code notebooks that Project Jupyter made popular. She argues that notebooks solve many of the problems with dashboards — for instance, people would be more willing to trust the process (because they can read the code that generated the notebooks, alongside the author’s commentary); that they would have the power and flexibility to answer any question (so long as they know the language in which the notebook is written); and they have a way to collaborate on, present and share these documents with a wider audience.

On the other hand, deathofdashboards.com argues that the solution is to use “a system that tracks all of your metrics and uncovers insights that even humans cannot find. And once it finds that, it tells you the entire story behind it.”

In other words, if you use these tools, these articles seem to say, you will solve all your problems.

The only issue with this argument is that it takes a whole bunch of different problems and mushes them up together.

People, Process, Tools

Adoption of effective business analytics is a people, process, and tools (PPT) problem. Companies that blame tools alone don’t get very far.

To put this another way:

• If your organization isn’t particularly data oriented, no amount of sophisticated tooling will save you.
• If obsession over data quality isn’t a cultural norm, or if existing data processes do not take data quality seriously, than no amount of dashboarding or notebooking would save you.
• If your data team exclusively uses dashboards for exploration, then you have bigger problems than using dashboards or notebooks alone. You have a data team maturity problem, not a tooling one.

What makes things doubly tricky is that analytics is often regarded as a cost function within the organization. You shouldn’t try to solve PPT problems with tooling alone; doing so runs the risk of overrunning your department’s budget — and with it, the probability of growing the team's influence within the org.

People and Process First

So what is the more nuanced view of things? The more nuanced view is that it is more productive to rule out people and process problems first, before going after tooling.

If you have too many dashboards, you need to figure out a way to manage report sprawl. This is a process problem and may be managed if you dedicate some time and effort to document who uses what dashboards, and when.

Admittedly, some tools are better at this than others (and, speaking as a vendor ourselves, we have strong opinions about how to do just this), but in principle it is possible to solve this problem using whatever means is available to you. At Holistics, we worked quite closely with a data team lead who used a wiki to keep track of important dashboards. His team was disciplined about tracking which dashboards were important, and for whom. Was it ideal? No. But it got the job done for months after implementation.

If you have a trust issue with your data, switching to notebooks isn’t going to do much. Data quality is a culture problem, and must be dealt with at the organizational level first. Ralph Kimball has a pretty good whitepaper on this (read our summary here); Uber has shown that it is possible to create a high-trust-in-data environment using dashboards alone. The point isn’t to pick dashboards or notebooks or self-serve data marts. The point is to fix your data quality problem by fixing the organizational norms around quality.

If your team isn’t logging into a dashboard to look at their metrics, then it is worth asking if the dashboards you have created for them are that useful in the first place. Putting some process around new dashboard creation (and using an alternative process for ad-hoc requests), would do wonders for this problem.

Finally, and most importantly, if your data team uses dashboards for exploration, then they are grossly misunderstanding the purposes of a dashboard.

Dashboards are good for:

• Non-technical users. These are usually busy business people who have zero patience for technically sophisticated solutions, and simply want timely information for their decisions.
• Presenting the most important leading and lagging indicators for the business.

Dashboards are bad for:

• Ad-hoc analysis.
• Exploratory data spelunking, like when you want to tease out non-obvious insights within your business.

You won't take a spoon to a knife fight. Similarly, you shouldn’t reach for a dashboard as the answer to every analytics problem. Sometimes a data export is what is needed; other times, you can get by with a decentralized data mart. (Although that comes with its own set of problems).

At Holistics, we use something called datasets, which is a collection of data models that may be used for ad-hoc analysis. But using our product is besides the point; there are equivalent tools on the market, and you should figure out what those are and use them instead.

Wither the Notebook?

The point of this article isn’t to argue that dashboards are good and notebooks are bad.

The world is rarely that simple. There may yet be a place for notebooks in the enterprise, the same way that there has long been a place for dashboards.

My point here is to warn against conflating a people or process problem with a tools problem. All the notebooking or dashboarding in the world isn’t going to save you if you operate in a low data-trust, low data maturity environment. It isn't going to save you if your data team is treated dismissively by the rest of the org. And it isn't going to save you if your company doesn’t take data analytics seriously.

Fix those first. Tools are always easier to fix in comparison. And don’t throw the baby out with the bathwater when you can’t get dashboarding to work.