Guidebook / Chapter 5. Conclusion

The End

So what have we shown you?

We’ve shown you that all analytical systems must do three basic things. Those three things give us a useful framework for talking about and thinking about building data infrastructure. The three things that you must do are, again:

  1. You must collect, consolidate and store data in a central data warehouse.
  2. You must process data: that is, transform, clean up, aggregate and model the data that has been pushed to a central data warehouse.
  3. And you must present data: visualize, extract, or push data to different services or users that need them.

Within these three steps, we’ve taken a look at each with a fair amount of nuance:

  • We’ve examined the rise of the modern data warehouse as being the center of most contemporary analytics stacks. We’ve also explored how this shift happened.
  • We’ve shown you how ELT provides powerful benefits that stretch beyond minor operational improvements.
  • We’ve shown you the outline of a contemporary dimensional data modeling approach, shaped by all that is possible with modern technology.
  • We’ve demonstrated how to do that using a new breed of tools we’ve named “data modeling layer tools”, and walked you through a set of concepts that are common to those tools.
  • Finally, we’ve sketched for you the shape of the entire BI tool landscape.

The Future

What do we think the future holds for data analytics?

In Chapter 4, we explained that much of the business intelligence world is confusing because competing approaches from different paradigms tend to stick around. A newcomer to the field would be overwhelmed as to which approach to take; more experienced practitioners may themselves be taken aback by the sheer amount of tools, ideas, and approaches that seem to emerge every couple of years.

We don’t think this is going to change. Data analytics and business intelligence continue to play a major role in most modern businesses, and vendors are highly motivated to get new products, new ideas and new approaches off the ground. This isn’t a bad thing! It’s just the way it is. Our hope is that this book gives you a solid enough foundation to understand new changes in the industry as they emerge over the next decade.

In the short term, we think that many of the trends we’ve pointed out throughout this book will continue to proliferate. The industry’s current shift to standardize around SQL will only strengthen; new BI tools will continue to adapt to the power of the modern data warehouse. We (rather biasedly, it must be said) believe that the approach we have described in this book is The Way Of The Future, and that it would eventually seep into large enterprises — that is, whenever it is that large enterprises finally move their operations to the cloud. This may take decades, and may also never happen — because this is a bit of wishful thinking on our part. But we believe it to be true.

Now: how does this affect your career?

If you are a data analyst, one implication is that you should have passing familiarity with all the approaches from all three paradigms in the business intelligence world. This doesn’t mean that you must master them — but it does mean that you should be aware of the alternative approaches that exist. This is because — as we’ve pointed out — old approaches in business intelligence stick around for a long time.

Concretely, what it looks like is the following: if you are working as a data analyst in a startup today, you may find yourself operating in a ‘first wave’ BI environment if you decide to move to a larger, older company tomorrow. Conversely, if you are a data analyst in an old-school data department, you may be taken by surprise if you leave and find yourself in an ELT-first paradigm at a younger company.

As a business intelligence tool vendor ourselves, we find ourselves chomping at the bit to change the industry. But if you are a data practitioner and not a vendor, it is probably a good idea to be pragmatic. Maintain awareness of all the approaches that exist in our industry. Stay abreast of new developments. That way, you won’t be terribly surprised when you see a data team that does things very differently from the way you did things in the past.

We hope you enjoyed this book, and that you’ve learned a great deal from it. If you thought this was useful, we would be very grateful if you shared this book with the people who need it — new data professionals, startup founders looking to set up a data analytics capability for the first time, product people who just want the bare minimum to hit the ground running.

We’d also love to hear from you if you have feedback or thoughts on the book, you can:

Godspeed, and good luck.