Insights

View full profileHerman Ohlhoff

Enterprise DW Lead & Head of Consulting (SA)
Bluefin Solutions

The real problem with trying to be agile and responsive in SAP BW

Let's be honest, when someone says "rapid delivery of BI solutions", SAP BW is probably not the first thing that pops into your mind.  In contrast, tools like QlikView have made surprising inroads within traditionally SAP-only organisations purely on the basis of the perceived speed and agility of delivery.  So what is the problem?  What is it about SAP BW that's putting on the brakes?

I was recently involved in a proof of concept exercise with a client, where SAP BusinessObjects Universes were created over SAP BW as well as over another database platform. Their feedback was that they loved using the tool over the other database platform, but were not very impressed with SAP BW as a source.  When drilling into this a little deeper I discovered that their main complaint against SAP BW was that it took too long to get things done or to change things - it was perceived as an unresponsive tool.

This was not the first time I'd come across a client complaining about SAP BW being inflexible or cumbersome to work with.  In fact, I'd guess it's probably a fairly common perception.  However, it did make me think about the merits of this argument and whether fundamentally SAP BW technology hindered rapid development of solutions.  

Apples with apples

The first step in trying to clarify this issue for myself was to ensure I was comparing apples with apples.  Imagine two developers - an SAP BW developer and  a Sybase IQ (or any other DW platform) developer - working to the exact same requirement in their respective systems having to create a number of database objects.  Would the pure technical development time in SAP BW be any longer than in another system? Having little experience of building data warehouse solutions outside SAP BW, I found this a little difficult to answer emphatically.  However, based on the fact that I've seen proof of concept/demo systems developed in SAP BW in less than a day, I decided this couldn't be the root problem of BW's non-agile reputation.

Ok, so you can't just dig up the roads

No, you can't just remove an attribute from an InfoObject. You can't just delete the master data in a table if it's used in an InfoProvider somewhere.  There are plenty of blockers to getting some things done quickly (or at all) within SAP BW.  In a sense, SAP BW is too good for its own good from this perspective.  The internal checks and balances that are in place will sometimes prevent you from getting something done, but it's usually because the thing that you are trying to do could well destroy the integrity of (at least some part of) your data warehouse. And that level of control is what you'd expect from an enterprise quality data warehousing application; it is the approach that has given SAP products the reputation for stability that they have.

Moving data around

Another aspect to consider is the speed at which data can be moved into the system or within the system.  The tricky bit here is that there are very few processes in BW that simply load data into a table and do nothing else.  Whether it's generating SID's when loading a cube or having to activate data in a DSO,  the application performs all sorts of other activities around loading processes.  This can often create the impression that loading data takes a long time and is slow in BW.  However,  if you were to load data into a write-optimised DSO without generating SID's I'd be surprised if it's any slower than an alternative platform.

System & landscape governance

This, I believe, is the crux of the matter.  Most SAP BW installations I've encountered have been deployed as strategic enterprise data warehouse solutions.  With this comes the need to ensure availability and stability of the solution.  This usually means at least a 3-tier landscape combined with some pretty stringent approval processes for moving changes through the landscape.  Often it also means pre-scheduled periodic slots for moving changes into the production environment.

In contrast, the platforms being compared to SAP BW at our clients are often deployed as non-mission-critical and tactical reporting solutions.  They seldom have 3-tier landscapes and are subject to much less governance and control.  In fact, it wouldn't be that unusual for developers to be allowed to make changes straight in the production environment.  This clearly creates a situation much more conducive to a quick turnaround of changes.  However, I would argue that if the same landscape and level of governance were applied to both systems they would be perceived as equally unresponsive.

The underlying technology therefore is not the real problem.  The real problem is that the need for stability and control is at war with the need for agility and speed.

Rethinking our SAP BW landscape

My view is that the key to achieving more agility is to reconsider our approach to our landscapes.  Clearly we need to balance the need for stability with the need for responsiveness and our traditional models are not working so far.  The diagram below outlines a possible approach to achieving this.

This approach suggests introducing a layer within the production environment where changes can be made rapidly without introducing the risk to the integrity of the larger data warehouse.  Think of it as having a production sandpit area.  This will of course require breaking quite a few rules as it will involve:

• Allowing developers to make changes in production.
• Allowing duplication of data.  Instead of making changes to the stable core, solutions are replicated in the flexible area and modified there.
• Don't reuse master data.  Create new infoobjects instead and load the necessary data from the stable core. Be ready to throw them away again if no longer needed.  This gets around the need to consider knock-on effects on the stable core.
• Don't bother with trying to follow naming standards or conventions - just get it built as quickly as you can.

Sound dangerous? That's because it is - as an approach to the entire environment. But that's not what I'm proposing. These rules will only be applied to the flexible layer - not to the stable core system.  And the important element here is that everything in this layer is temporary and eventually goes one of two ways: it either gets discarded completely if the requirement is no longer valid or it gets migrated to the stable core.  As part of the migration process all the usual rules will be applied again.

There are some very obvious downsides to this approach e.g. the need for duplicate effort when migrating the solution from the flexible layer to the stable core.  However, the benefits of being able to respond much faster to new requirements and produce 'production prototypes' must be weighed against these disadvantages. 

It's not for everyone (in fact, as far as I'm aware it's not for anyone at this point), but rather than blame the underlying technology for a lack of agility, isn't it time we consider that our conventions might be to blame?  It might just be time to try something unconventional.