With the significant investment that organisations have made or continue to make into their Data Warehouse, it’s no wonder that Data Warehouse Automation was such a hot topic at the TDWI World Conference in Chicago in May this year.

Why are we talking about Data Warehouse Automation?

Simply put, the demand around BI predominantly being driven by business users looking for capabilities they can interact with (such as dashboards, visualisation and analytics) and additional sources of data means your data warehouse department needs to consider automation of the “smart” data warehouse features that take too much time, are difficult to achieve and often become obsolete as skills move on.

Some of the ongoing challenges facing environments that have not adopted Data Warehouse Automation technology could be:

  • Agility and slowness in adopting to change within the business
  • Developer lock-in as the system was custom developed
  • Optimisation of the environment
  • Upgrades or maintenance results in additional workload
  • Spiraling costs as maintenance costs increase
  • Governance is consistently unhappy as there is limited documentation

In essence, a Data Warehouse Automation technology (like TimeXtender) takes care of the normally costly and difficult data warehousing processes that are critical to your strategy. In addition, it allows your team to focus on the more impactful and quite frankly, more satisfying processes involved in providing a BI environment. I recently helped the BI department of a well-known brand in South Africa, to put a business case together on why they should move to Data Warehouse Automation and the results were incredibly favourable, providing:

Savings
  • Reduce dependency on senior BI technical skills
  • Reduce development time by an estimated 80%
  • Reduce BI development costs over 24 months – by approximately R4.275m
  • Reduce ongoing BI Maintenance costs – by approximately R150 000 per month
Improvements
  • Increase business impact by becoming more relevant
  • ​Increase adoption and confidence in BI by delivering quicker
  • Increase value in the information you provide for strategic decision making
Strategic Importance of a Data Warehouse Automation platform
  • Improvement in data compression and reduction in storage needs by 40%- 90%.
  • Automated generation of documentation of all integration, development and BI, along with version control – that can be used internally or externally for Governance and Compliance.
  • Reduce the risk related to your BI environments design by ensuring all development aligns to global Best-Practice Kimball Methodology
  • Reduce the risk, related to losing or gaining specific skills
  • Remove tedious and error-prone scenarios of developing multiple data sources
  • Rapid prototype or proofing of customer data for integration

“People Adding Business Value” versus “People Maintaining Environment”

A typical BI project will involve four iterations or releases to business. After each release, enhancements or changes will need to be accommodated in a subsequent release, and the prior release will require ongoing maintenance by an individual.

 

Custom build vs. TimeXtender DWH

 

In a Custom Build Data Warehouse

  • Initial release will take an estimated six months, and businesses involvement will be high, but typically only when you get closer to the six month period will the business start seeing results
  • Inevitably there will be changes, because priorities have changed and another release will be required
  • Subsequent releases will typically take three months each
  • Each release will provide additional value to the business, but with each release, a certain number of resources become focused on building new business features or value while other resources become more focused on maintaining the current release

Result:

  • 18 months to 24 months to achieve all four releases
  • One person per release required to maintain each release – resulting in adding people to your team to ensure you keep delivering new business value
  • Costs increase over time and then maintain (but don’t drop) when you stop releasing
  • Total people cost for four releases = R6 075 000
  • Ongoing maintenance costs = R225 000 per month
Custom Build Data Warehouse Build
Release 1
at 6-9 mths
Release 2
at 12 mnths
Release 3
at 15 mnths
Release 4
at 18 mnths
Thereafter
Build Build Build Build Maintain
Build Build Build Maintain Maintain
Build Build Maintain Maintain Maintain
Build Maintain Maintain Maintain Maintain
    Add 1 Build Build  
      Add 1 Build  
People Cost per Release (monthly cost)
B = R300K
M=0
B = R225K
M=R75K
B = R300K
M=R150K
B = R300K
M=R225K
B = 0
M=R225K

 

In a TimeXtender Automated Data Warehouse Approach

  • Initial release will take an average of four to six weeks, and businesses will start seeing results within days to weeks, as the team deliveries iteratively, thus ensuring greater relevance to business
  • Each release thereafter will typically take six weeks each (sometimes less)
  • Each release will provide additional value to the business, but because the maintenance of the system is automated with TimeXtender, only one resource is required to maintain the system through all releases, whilst the other resources can focus completely on adding new business value

Result:

  • Six months to achieve all four releases
  • Only one person require to maintain all releases
  • Total people cost for four releases = R1 800 000 (saving of R4.275m over 18 months)
  • Ongoing maintenance costs = R75 000 per month (saving of R150 000 per month)
TimeXtender Automated Data Warehouse
Release 1
at 6 weeks
Release 2
at 12 weeks
Release 3
at 18 weeks
Release 4
at 24 weeks
Thereafter
Build Build Build Build N/A
Build Build Build Build N/A
Build Build Build Build N/A
Build Maintain Maintain Maintain Maintain
People Cost per Release (monthly cost)
B = R300K
M=0
B = R225K
M=R75K
B = R225K
M=R75K
B = R225K
M=R75K
B = 0
M=R75K

 

Assumptions in the above calculations:

  • People cost of R75K per month per BI resource (average between salaried staff of R50K and external consultant of R100K per month)
  • Custom Bi Development cycles are based on a typical Microsoft BI Deployment approach of 6 months for release 1, and thereafter 3 months per release.
  • Excludes cost of Business Users and their involvement.
  • Additional savings could be achieved in TimeXtender Build, as typically you can use a lower costing resource to build your environment.
  • Additional savings could be achieved in TimeXtender Maintenance, as typically you can use a lower costing resource to maintain and extend your environment.