This is the first article in a series in which we will talk about how Data Warehouse Automation can greatly improve Decision Support Systems (DSS) and Business Inteligence (BI) development.

The purpose of the series is to provide some answers to some of the questions below:

  1. why should you even care about Automating Business Intelligence (BI)?
  2. what would be a good place to start with a Data Warehouse Automation (DWA) exercise?
    When would be the right time to do so?
  3. methods, tips, tricks + some solid tools to help work with both people and technology variables:
    the importance of embracing and managing change
  4. Data Warehouse Automation as a foundation for building a strong stakeholder/user community
  5. Some general consideration when building, growing and maintaining a Decision Support Eco-System
    Howto integrate BigData or other valuable data sources

This (first) article will focus on describing context (and potential use) of automation.

Thank you, Simon Sinek.

Credit where credit is due.

Inspiration for this part comes from Simon Sinek's 'Start with the Why pitch'. His 'How great leaders inspire action' TED talk manages to capture the essence of working with purpose and vision in business. In his inspiring pitch, Mr. Sinek focusses on the reason why we are doing things, rather than on how need to get things done.

Would you agree that most of the work we do in BI is framed round how to get things done, rather than asking ourselves why we want to achieve this?

When you are getting involved in the intricate world of Business Intelligence, you may find that tooling and deployment method can sometimes take over from business vision. I sometimes catch myself forgetting why I am getting involved in doing what I do. Business Intelligence should be the materialisation of business vision. The purpose of Business Intelligence is about instilling self-confidence and self-awareness into people, in order to help coprporate decision making. In order to have any kind of impact, this effort should be true to business vision.

Questions, questions

Business Intelligence is about asking simple questions (*):

  1. Are we doing things right?
  2. Are we doing the right things?

from which all other Business Questions can be derived (**)

Business Intelligence can deliver great value, simply because it allows people to get reliable feedback on complex action patterns. These actions can happen on an individual level, or even cross departments, roles and teams. When properly used, BI can help build confidence in teams, confirm business opportunities. Decision Support can even allow for early warning systems when things (like the economy) start behaving in unexpected ways. BI can be -by far- the best way to help anticipate and to some extent safeguard against undesireable business outcomes.

Data Warehouse Automation serves as an acceletor for this process. (***)

Start with the 'Why'

The problem with simple questions is that you do not necessarily get simple answers.

Some questions and answers are predictable - which is why standardiazed BI solutions (Industry Specific Business Intelligence solutions, Industry specific sets of KPI's or ERP-standardized systems) can mean such a big leap forward, a real kick-start.
But they often get us stuck right after that.

As we progress, our questions will become more profound, so inheritably more valuable, and - unfortunately - more complex to answer (*****).

The most valuable questions will inevitably pop up as we move along. Their answers will be way outside the realm of our initial system.That is why it is of utmost importance that a Decision Support Endeavour is able to deliver standard stuff fast, and non-standard stuff even faster.

This is where Data Warehouse Automation comes into play.

Data Warehouse Automation is a concept which will enable you to accelerate both standard and non-standard business questions, yielding in faster implementations of standard solutions, but also capable of answering questions-as-they-come-along faster - cutting time to market by as much as 80% or more, but I digress.

This could be a great advantage. It does come with inherent risk.

That's why you need to reconsider the 'Why?', the business vision, every step of the way.

About Agile

In Agile methodology, a best practice for developing and deploying BI, we cut Business Intelligence vision into smaller pieces, called user stories. User stories serve a double purpose: they need to make sense to End Users on the Business side, for the develoment team, they need to contain concrete, physical elements to work with.

This is what a typical user story could look like:

As the marketing manager, I would like to know how many customers have reacted to a campaign and if the resources, discounts, and response capture has yielded a good return.

It fulfills the requirements above. It is easy to understand for the Business users. As within any story, some protagonists (Marketing Management and Customers) have been identified. Measureable elements like discounts, responses and return have been named. In the Agile approach, it will be possible to further eliminate vagueness and ambiguity, through intensive dialogue.

  • What is a campaign all about?
  • What are it's composing elements (dimensions)?
  • What types of customer reactions are of interest?
  • What do you mean exactly by resources?
  • How should we visualize this?
  • And so on ...

By the end of the exercise, the team will have a concensus on what to build. In true form to the Agile approach, results will be available fast, provided of course the user story gets chosen and planned for development. This can happen within just a few days or weeks.

There is one element missing.

We do not know whether the result of the exercise will be sufficient for the end-users. When we work out this particular user story - when we build the damn thing, will the Business users still think it to be valuable? What will they then do with it.

In other words: what will be the value of handling this particular user story? Starting with the 'Why' can be of tremendous help. It is a matter of mindset rather than method. Meaning this should be helpful regardless of method, or wether we are talking BI or other Business Projects. What is it that we are aiming to achieve? As difficult as it may sometimes be for the technical teams to explain techical issues, likewise, most business people find it hard to agree on purpose of achievements.

Let's try rephrasing the question above by using the following leading words:

In order to ...

Three little words can make all the difference.

Three little words which allow end-users to put purpose in words. What is it that they are trying to make happen? How will they act?

Let's give the above example a go:

In order to ... determine how to spend future marketing resources, focus and campaign efficiency, I need to be able to measure customer campaign responses. I also want to investigate relationship between discounts and purchase value and measure the return on campaigns and resources.

In order to ... is a good way to start descibing your next user story.

It will serve to determine value. In turn, this will serve to prioritize and make choices.

Data Warehouse Automation?

In the ensuing articles we will investigate ways you can use Data Warehouse Automation techniques to leverage usage and help all people on the team. We will discuss some techniques on how to widen your audience and even take into account new approaches concerning 'BigData' and 'Internet of Things' or Machine Generated Datasets.

For now, I will be happy to just sum up reasons why our customers use tha DWA approach, in no particular order:

  • It enables them to quickly test and try out some tracks, choose the most valuable ones (automated testing and prototyping)
  • It eleminates the need to worry about things like data source changes (automated data lineage and change data)
  • It enables (automates) transitions from old to new platforms (as supported by the platform)
  • It allows to cater and serve a multitude of users on a multitude of platforms (automate multiple data output formats)
  • It help govern data discovery and harness findings from departemental teams (streamline test/development and production)
  • It automatically generates documentation and code as we go along (so you do not have to create it by hand)

Read on to the next article of the series...

or drop me a line

Dirk Remacle



(*) see other Blog entry on the topic of simple questions and complex answers

(**) (yes, consider this a challenge)

(***) And we, at TimeXtender, have software that can help accomplish this. (****)

(****) Or what did you expect?

(*****) If it was easy, everybody'd be doing it and we would not be having this conversation.