The Silos of Data Organization: Data Analysts vs Business Users

As a Data Analyst, you usually work with two types of stakeholders: the clients (business users) who ask for data, and the support (data engineers) who provide you with the data pipeline/raw data needed to do your work.

In theory, everyone is supposed to row in the same direction, working together nicely to make the business better.

In practice, it’s very common that you find yourself in a situation where the stakeholders are frustrated with each other. Things are moving slowly (or not at all). You can't stand the business people (they ask all the wrong questions)! All the cool Statistics stuff you learnt are not of use, you just spend most of your time second-guessing what they're trying to get at! Oh and, the data engineers are not responding to your requests! ...

What happens there?

You're facing what we call silos of data organizations: each of the data stakeholders only thinks about their own goals and working styles, without considering how to work well with each other.

In this week's (and next week's) post, I'd like to explore these relationships and what we can do about them.

The silos between Data Analysts and Business Users

As a Data Analyst, ask yourself: “When was the last time you get a data request from Business Users where they tell you what data they want with all the context, the reasons and complete analytical reasoning around why they want that data?”.

Yeah. If you're like most data analysts, the answer is: "Hmm, depends. But probably a long time ago".

How many times have you gotten back to them with some report, only to be responded with “Hmm doesn’t look like what I’m looking for? Can you try to change…”

This goes on for a few cycles, then you lose your patience. You have no motivation to produce high-quality results. You are annoyed and stressed by the constant requirement changes. You feel even worse when you are criticized for getting numbers wrong (sometimes). You told yourself: “I thought I’m supposed to use my brain to work on important analyses, turn out I’m just an English-to-SQL translator”.

(An English-to-SQL translator, just in case it's not already clear enough, is someone who take a data request communicated in verbal language, translate that into the corresponding SQL query, execute it, get the results, and email that back to the requester. Well, how do you like this as a career?)

Why is this happening? This happens because the business users do not know what is necessary for the analyst to understand them. When they aren’t sure of what they want, they put out dozens of requirement changes, leaving the analyst confused and frustrated.

Are the business users to blame? Not really. As said, they just don’t know better! It is (ironically) the data team’s job (i.e. your job) to train and educate the Business Teams to have better data-driven thinking.

Of course, you will sometimes find certain people who are naturally data-driven in their thinking. These people are absolute pleasures to work with. They ask the right question, lay out their line of thinking clearly, and naturally provoke your data-geekiness.

Now. Let’s think from the business users’ perspective. Are they feeling frustrated too? Yes. They feel powerless because they have to go through Data Team for most of their data questions.

An analogy would be: When I was a student living in a rented apartment with the landlord, every time I went out and return home late after 11 pm, I’d have to call and ask the landlord to open the gate. This … worked, but errr, now I think twice (or thrice) when I want to hang out late. The fear of bothering someone!

That’s exactly what the business users feel. So after a few episodes of the above (plus the slow response time and mismatch in expectation of the report’s results), they stop relying on using data to inform their business decisions altogether.

So. What can we do to solve this situation?

A couple of (non-exhaustive) things that come to my mind:

(People) First, a mindset fix — Data Department should run themselves like a Product Organization, not an Outsourcing Team. They should be proactive about thinking how to add business values, instead of being reactive about it. They should engage with their business counterparts as equal thinking partners, rather than merely English-to-SQL translators.

I would also go out on a limb and suggest: The Data Department should aspire themselves to be the internal McKinsey consultants of the organization, bringing an analytical set of eyes to the business problems. This is, of course, extremely difficult to achieve. Where can you find good data-driven business thinkers with good data technical skills? But one should always dream.

(Process) Second, the Data Department should have put in place proper programs to systematically raise the data literacy levels of the entire org. I mentioned this last week on building a goldilocks-rule program so that the data-novice business users can “teach themselves data-driven thinking” without them even being aware of it.

(Tools) Third, for trivial data extraction, Data Department should put in place proper “self-serve” infrastructure and tooling, so that business teams don’t have to bother them with these constant ad-hoc questions.

Before I end this, for the Product-minded reader out there, I’ll leave you with this user-story style JTBD:

“Given I am a business user who is still novice about data-driven thinking.
When I try to make use of data to aid with my business decisions
There should be
a) A built-in training program for me to gradually learn to ask the right data questions.
b) An easy software interface for me to access the answer to the questions above.
c) A supportive data colleague to guide and consult me along the journey”

That’s it for this week. Next week, I’ll discuss the remaining silo: the relationships between Data Analysts and Data Engineers.

Do you agree or disagree with the above? How is your situation like? Let us know!