Head of Business Analytics & Technology
Bluefin Solutions
Updated: The SAP HANA FAQ - answering key SAP In-Memory questions
22 Nov 2011
Business Intelligence (BI), Business Objects, HANA, In-Memory, SAP NetWeaver Platform
It's now six months since SAP HANA 1.0 SP02 went into General Availability (GA) and two weeks since SAP HANA 1.0 SP03 went into ramp-up, SAP's mechanism to release new products into the market. This FAQ was horribly out of date and now seemed like an excellent time to revisit what's changed in the last six months and make it accurate once more. I hope you enjoy it. It's not an official SAP FAQ, but it is information pulled together from industry experts from in and outside of Bluefin Solutions who have worked with and know the product intimately.
1. SAP HANA overview
1.1 What are the product names?
HANA is an overarching product name that forms several components. When we talk about the in-memory database (previously IMDB or NewDB), we now call it SAP HANA Database. When the database is installed onto an appliance, we call the appliance a SAP HANA Appliance. The toolset used to administer and model SAP HANA is called SAP HANA Studio. Applications built on HANA will be marked "powered by SAP HANA".
This means that the new version of SAP's data warehouse product based on SAP HANA Database 1.0 SP03, is called NetWeaver BW 7.3, powered by SAP HANA.
1.2 What is SAP HANA Appliance 1.0 SP02?
SAP HANA 1.0 is an analytics appliance that consists of certified hardware, an In Memory DataBase (IMDB) an Analytics Engine and some tooling for getting data in and out of HANA. You build the logic and structures yourself, and use a tool e.g. SAP BusinessObjects, to visualise or analyse data.
1.3 What are the limitations of SAP HANA 1.0 SP02?
HANA 1.0 as an appliance is effectively a database and analytics appliance. You can load data in one of three different ways, from just about any location - in real time, or in batch, from SAP and non-SAP data-sources. Currently on SAP BusinessObjects BI4, Explorer and Analysis for Excel are supported as reporting tools, but other tools are expected to be supported in the future. As much as anything, SAP are limiting support for integrated scenarios so they can get the stable product stable.
As an aside, we are already working on projects that include HANA 1.0 SP02 integrated with SAP Event Insight and the Sybase Unwired Platform. HANA is well on its way to becoming a general purpose database and there are ODBC drivers that should be supported generically, in time.
1.4 What is SAP HANA 1.5, 1.2 or 1.0 SP03?
On 11th November 2011, SAP HANA 1.0 SP03 was released into Ramp-Up, along with the application layer for SAP's Data Warehouse product, called NetWeaver BW 7.3, powered by SAP HANA. This will allow any SAP NetWeaver BW 7.3 Data Warehouse to be migrated into a HANA appliance. HANA 1.0 SP03 specifically also accelerates BW calculations and planning, which means you get even more performance gains.
Bluefin Solutions' initial tests with this product suggest that it will be revolutionary to the 14,000 SAP NetWeaver BW customers, removing issues with information availability and performance. SAP NetWeaver BW 7.3, powered by HANA, is expected to be available to the general market in Q2 2012.
1.5 What's the difference between SAP HANA and IMDB?
HANA is the name for the current BI appliance (HANA 1.0) and the BW Data Warehouse appliance (HANA 1.0 SP03). Both of these use the SAP IMDB Database Technology (SAP HANA Database) as their underlying RDBMS. Expect SAP to start to differentiate this more clearly as they start to position the technology for use cases other than Analytics.
1.6 If I can run NetWeaver BW on IMDB/HANA, why can't I run the Business Suite/ERP 6.0?
Simply because it's not mature enough yet to support business critical applications. From a technology perspective, it is already possible to run the Business Suite on IMDB and SAP has trialled moving some large databases into HANA already.
Expect the first non-analytics product running on the SAP HANA database platform to be the small-business oriented SAP Business One - a version of SAP's ERP platform designed for businesses with 1-100 employees.
1.7 What is SAP HANA great at?
The best thing that HANA brings to the table is the ability to aggregate large data volumes in near real-time - and to have the data updated in near real-time. SAP's demos show hundreds of billions of records of data being aggregated in a matter of seconds. SAP has built a set of Analytics Apps on top of HANA and this are set to be great point use cases to get customers up and running quickly.
The really great SAP HANA apps that have been created mix three big performance improvements. First, the performance of in-memory analytics, second, an inefficient design and third, a change in process that allows further improvements. This is what SAP's CTO Vishal Sikka affectionally call the "100,000x club".
In addition, SAP NetWeaver BW 7.3, powered by SAP HANA looks like it will be a no-brainer for the majority of SAP's 14,000 BW customers. The improvements in performance and flexibility it allows resolve many of the classic data warehouse problems that have plagued the market for 20 years.
1.8 What is SAP HANA bad at?
SAP HANA improves the biggest bottleneck that exists in standard database platforms - the spinning disks. In-memory technology is typically 100-1000x faster than disk for this reason. However in-memory technology does not always calculate any faster than existing technologies, especially since some analytics engines out there have a lead on maturity.
So, there are scenarios where I/O was not the bottleneck and calculations were, that SAP HANA may not provide benefits. This is especially the case when sending out large datasets to front end tools like SAP BusinessObjects BI4 which then have further calculations to perform.
SAP is doing great things in this in the next version of SAP HANA and BusinessObjects BI, where the calculation layer of BusinessObjects will actually reside in HANA.
1.9 What does SAP HANA cost?
SAP HANA is priced by the 64GB unit right now, and there is some discounting based on volume. As usual with SAP licenses, it's best to contact your account exec directly and talk to them. The minimum purchase amount is currently 64GB, and the smallest appliance is 128GB, which is upgradeable to 256GB. This means if you buy 64GB today, you can easily incrementally expand up to 256GB.
1.10 Why is SAP HANA so fast?
Regular RDBMS technologies put the information on spinning plates of iron (hard disks) from which the information is retrieved. HANA stores information in electronic memory, which is some 50x faster (depending on how you calculate). HANA stores a copy on magnetic disk, in case of power failure or the like. In addition, most SAP systems have the database on one system and a calculation engine on another, and they pass information between them. With HANA, this all happens within the same machine.
1.11 Does HANA/IMDB replace Oracle?
It's the elephant in the room, but once the Business Suite runs on IMDB, Oracle won't be needed any more by SAP customers who purchase HANA. This doesn't affect anything in the short term because many of those people buying HANA today will still need an Oracle ERP system.
However if you run an Oracle or DB2 data mart that performs poorly, you could replace this outright with SAP HANA and that would allow you to actually eliminate some licenses today.
1.12 What is this about 10:1 compression with SAP HANA compared to Oracle?
A typical uncompressed Oracle or Microsoft SQL Server database, when put into HANA, will be 10x smaller than before and this is due to the way that HANA stores information in a compressed format. Note that most databases are now compressed and these numbers may not fit your scenario, and to add to this you need 2x the RAM as your database, plus room for growth. HANA sizing is still a dark art.
Our initial sizing suggests that for NetWeaver BW, the key database tables will compress between 5x and 10x.
1.13 You mean I have to buy an SAP HANA only 2.5x smaller than my big Oracle RDBMS? What about archiving and data ageing?
Yes, in some instances you may have to buy a HANA appliance that is only 2.5x smaller than it would be under Oracle. And data ageing isn't part of the 1.0 release, but SAP is certainly working on it pretty hard. Let's hope they release something faster than you need to buy a bigger HANA appliance!
1.14 What's the wider market opportunity for IMDB?
This is the interesting thing - no one knows yet, and few analysts seem to have cottoned on that the wider market opportunity might be huge. Think not just SAP applications but any third party that requires ultra-high speed. Think not just an appliance but a development platform. Time will tell.
2. SAP HANA database hardware
2.1 What hardware is supported right now?
Talk to your hardware vendor - all of the major vendors e.g. HP, IBM, Dell, Cisco, Fujitsu and Hitachi have certified HANA offerings now. Technically HANA will run on any Intel x64 based system from your laptop through to the big 40-core, 2TB RAM servers. It is however only supported on a small number of big rack-mount servers like the Dell R910 and HP DL980.
There is now a supported hardware list on SAP's website - http://service.sap.com/pam (login required). Appliances vary from 128GB to the IBM scale-out appliance which is 16x512GB nodes (8TB).
2.2 Why doesn't SAP HANA run on blades?
The first HANA blades have now been certified by Hitachi as of November 2011 and are the first built with network storage. Other vendors haven't got blade chassis that can power the huge computing required of SAP HANA with the top-end Intel E7 CPUs and 1TB RAM.
http://www.hds.com/assets/pdf/hitachi-solution-profile-converged-platform-for-sap-hana.pdf
Expect the other vendors to follow suit when they can!
2.3 Does SAP make its own IMDB/HANA hardware?
Yes, but only in the labs so far. There are no public plans to compete against IBM/HP/Dell in this space, but it may make sense for SAP to enter the appliance market, especially in the context of Data Centres and even more so in the context of the SAP Business byDesign cloud offering, which will run on HANA.
2.4 How big does SAP HANA scale?
Theoretically at least - very well. The biggest single-server HANA hardware will run most mid-size workloads - 2TB of in-memory storage is equivalent to 5-20TB of Oracle storage. The way that HANA works means that it is possible to chain multiple systems together - meaning that scalability has thus-far been determined by the size of customers' wallets. Do note that whilst SAP talk up "Big Data" quite a lot, HANA currently only scales to the small-end of Big Data, which refers to the kind of huge datasets that FaceBook or Google have to store - not Terabytes, but rather Petabytes.
2.5 What storage subsystem does SAP HANA use?
This varies from vendor to vendor but most use direct attached storage (DAS). Both regular magnetic disks and SSD storage can be used for the backup of the database (HANA runs in memory remember, so disk storage is just for backup, and later, for data ageing) but so far, all the certified hardware configurations use magnetic disks. Note that you require 2x storage that you have RAM, which is 2x the database size - i.e. storage size = 4x database size. In most cases there is additional ultra-high speed SSD storage for log files.
The Hitachi solution is the first SAP HANA certified solution that uses network attached storage (NAS), which is expected in time to be the dominant storage type for HANA. However IBM at least are eschewing network storage in favour of local storage, plus replication using their proprietary GPFS technology. IBM claim that - for now at least - this is the most reliable and cost-effective mechanism for SAP HANA.
3. Technical FAQ
3.1 What source databases does HANA support in real-time?
There are two mechanisms that HANA supports for near-real-time data loads. First is the Sybase Replication Server (SRS), which works with SAP or non-SAP source systems running on Microsoft, IBM or Oracle databases. This was expected to be the most common mechanism for SAP data sources but there remain some license challenges around replicating data out of Microsoft and Oracle databases, depending on how you license the database layer of SAP. If you buy your database license direct from the vendor then you are fine, but if you buy it through SAP then you may have a restricted license that does not allow for usage of SRS.
For those scenarios, SAP have a second choice of replication mechanism called System Landscape Transformation (SLT). SLT is also near-real-time and works from a trigger from the SAP Business Suite products. This is both database-independent and pretty neat, because it allows for application-layer transformations and therefore greater flexibility than the SRS model. Note that SLT only works with SAP source systems.
3.2 What source databases does HANA support for batch loads?
If you use SAP BusinessObjects Data Services 4.0 for bulk loads then pretty much anything. BO-DS is a very flexible Extract, Transform & Load tool that supports many databases. Data Services was previously called Data Integrator, and was previously called Acta, prior to being acquired by Business Objects.
You can reasonably load into HANA using Data Services every 10 minutes and Data Services allows for excellent flexibility because you can take care of complex business transformations including e.g. address verification outside of HANA, which may allow simplified modelling within HANA.
3.3 What additional limitations does Sybase Replication Server present?
SRS has additional restrictions which are worth bearing on mind. It can only replicate Unicode data and does not support IBM DB2 compressed tables at this time.
4. Follow-ons, corrections & credits
This is a work in progress and your help correcting me, clarifying some things I may have not explained so well or even just asking a question that I haven't covered would be really useful for the wider market. Let me know and I'll expand this as the months go on!
I'd like to particularly thank Vitaliy Sygyzmundovych (@Sygyzmundovych), Jon Reed (@jonerp) and Vijay Vijaysankar (@vijayasankarv) who have given invaluable input into this. But also to Ethan Jewett, Karin Tillotson, Mike Bestvina, Danny Rhode and others.
Comments
Antonio Caldas 16 Feb 2012
Hi Josh.
Thanks for this complete information on HANA. It is really a great article.
Regards
Antonio Caldas
Vitaliy @Sygyzmundovych 10 Feb 2012
> I heard that Hana does not use indexes? Is this true? I thought that in-memory systems would use the new T-Tree index.
Mike, you raised an interesting question indeed. Firstly, there is a split between column-based and row-based data stores. Afaik, T-Tree is used by TimesTen (Oracle), which was row-based IMDB. I frankly do not know what SAP is using for indexes on row-based tables in HANA db (HDB). Secondly, column-based tables in HDB are legacy of TREX, so their structure is inverted index itself. But because HDB is SQL database it has table indexes concept, which are still used, e.g. in case of secondary indexes on BW objects' tables. And again - interesting question what structure is used in that case: B-Tree, T-Tree or something else.
Vitaliy @Sygyzmundovych 08 Feb 2012
Maloof, you need to start from the differentiator that HANA has been created with the thought of supporting traditional SAP products. E.g. it has the same "Client" concept as it is available in NetWeaver ABAP. I do not think you can find Coherence or Gigaspace been certified for use with SAP. So, in a nutshell: say "HANA" think "SAP".
Maloof 08 Feb 2012
how does HANA compare to other In-Memory data management solutions such as Coherence and Gigaspace. Is this a replacement or an enhancement
Rama Shankar - Atlanta, GA 07 Feb 2012
John, this is a great book. It is a must read for Data Architects who are moving towards modelling in-memory computing solutions. Thanks for sharing this chapter...from the book.
John Appleby 07 Feb 2012
HANA 1.0 SP03 (previously known as HANA 1.5) is currently in RampUp. Solution Validation (general release) is expected in H1 of this year. Hopefully soon! It's pretty fantastic stuff and surprisingly mature.
You are right that HANA 1.0 SP03 is required if you would like to use HANA as a database for a NetWeaver BW ABAP environment.
@Mike
HANA doesn't really have a concept of indexes per se, because every element is stored in a row- or column-store depending on the table definition. Because it is stored in-memory, there is no need for aggregates or table indexes. If you're interested in the details, here's a great link:
http://www12.sap.com/platform/pdf/In-Memory%20Data%20Management.pdf
mike hill 07 Feb 2012
I heard that Hana does not use indexes? Is this true? I thought that in-memory systems would use the new T-Tree index.
Is HANA 1.5 available for General release ? 06 Feb 2012
Wanted to know if HANA 1.5 is available for general release. As per discussions at the Demo booths at SAP Teched 2011, I understand that Data modeling in HANA using ABAP is available only from HANA 1.5 onwards.
Hence, wanted to know, if HANA 1.5 is released
Vyankatesh Datar 31 Jan 2012
Dear John Appleby,
Truelly nice briefing about HANA. I would appreciate if you add some value about difference between Modeling in BW7 and BW 7.3.
Rama Shankar - Atlanta, GA 15 Dec 2011
John:
Great job on the updated HANA FAQ article.
Thanks,
Rama Shankar ( SAP BI Solutions Consultant)
Vitaliy @VitalBI 09 Dec 2011
Glenn, SAP sales is always the best source of all licensing information, but if I got it right, than you need to pay extra license only when using new Planning Kit and only when using it eith BPC for NetWeaver. -Vitaliy
Glenn Cheung 09 Dec 2011
@john: good stuff!
Heard at Teched that in-memory IP-planning functions require additional licensing... :-(
Do you know if RDSes like COPA accelerator come at a cost or can be used/modified for free?
Cheers,
Glenn
John Appleby 26 Nov 2011
Hiya,
As Vitaliy says, you can easily upgrade from HANA SP02 to SP03, it is just a newer version of the DB appliance software.
And from there you could easily install a central instance of SAP BW, using the existing HANA database and a new schema: HANA DB supports multiple schemas.
There is the sizing issue to consider, as Vitaliy comments. There is also a support issue to consider, as SAP do not yet support multiple schemas with BW. I am sure they will in future.
Regards,
John
Vitaliy @VitalBI 26 Nov 2011
Yenanda, did you mean upgrading the hardware or upgrading the software? There are couple of answers:
1. First thing if you want to use "BW powered by HANA" is either to become current Ramp-up customer, or wait until it is in General Availability
2. Yes, you can upgrade HANA 1.0 SPS2 to SPS3 running on exactly the same box.
3. You need to upgrade "the box", i.e. the hardware only if your memory requirements are going above currently sized. E.g. if your SPS2 is using box with 1TB RAM, but use case for SPS3 requires the sizing above that - you will need to pgrade the hardware.
-Vitaliy
Yenanda 26 Nov 2011
Hello John Appleby,
Thanks for a very good article. One more FAQ that is being asked is if have SAP HANA appliance (HANA SP2), can I upgrade same box to SP3 and ultimately use it for "SAP NW 7.3 Powered by HANA"?
Mike Bestvina @techdisruptive 24 Oct 2011
@John - some update on this too. I just read that HANA as a DB for BW will basically speed up everything so this is another selling point as ETL will be much quicker. I forget the term but you will also be able to "contract" InfoCubes so the star schema is less expanded. Also DSO activation can be sped up tremendously.
PSReddy 24 Oct 2011
Good coverage. How HANA will differentiate with Exalogic (of Oracle)?
Jayanta 24 Oct 2011
SAP Installations(esp SMB) typically have 30-40% capacity peak.
Why not use the 70%-60% idle time to build caches in database or in filesystems.
Then we may get sizzling MIS & Portals in R3!
Without any investment except for effort to
design and build caches and consume them
even in peak time.
No HANA, no EP, no BW - only R3.
Jayanta of Kolkata
Baris 04 Aug 2011
Why do you think that HANA is not a datawarehouse?
Ayoub 23 Jun 2011
Thanks for this great Article.
I am excited about the new SAP technology and hope it will spread all the world "in real time" :)
Bala Prabahar 23 Jun 2011
Hi John,
Thanks for your quick response. Yes, I saw 3.1; however 3.1 points to licensing issue whereas SAP note appears(my interpretation) to indicate a technical issue.
Now I'm clear. If it is a licensing issue, not a big deal; it can (if SAP/Oracle wants to) hopefully be taken care of.
Thanks again,
Bala
John Appleby 23 Jun 2011
Hi Bala, I cover that in point 3.1 - perhaps not clearly enough. The problem with SRS & Oracle is a license issue. Business Suite customers who buy their Oracle license through SAP are generally prohibited from extracting data out of Oracle by their license terms.
I heard that SAP were working on this with Oracle and Microsoft and an announcement is expected to be made. Let's keep our fingers crossed!
Bala Prabahar 23 Jun 2011
John,
Good efforts. Question on Oracle:
Note 1513496 states:
"Sybase Replication Server based replication of data from Oracle DBs into SAP Hana 1.0 is not allowed".
Does this mean Oracle is not supported by HANA 1.0? In simple terms, does this mean HANA 1.0 is not useful to SAP customers who use Oracle db?
Thanks,
Bala
Vitaliy @Sygyzmundovych 22 Jun 2011
John, I think the proper question instead of "Why doesn't HANA run on blades?" would be "Why HANA is not offered on blades?" because I believe it is really a matter of products offer and then the question why. As I look for "why" - I think SAP started with the premise that 90% of datasets for HANA use cases will fit into 1TB RAM. Therefore "distribution" (multi-node processing) functionality of SAP HANA database has been originally planned for what was known as "HANA 1.5" to support BW-on-HANA. For single node systems rack servers are just cheaper comparing to single blade with all needed supporting components.
Vitaliy @Sygyzmundovych 22 Jun 2011
Danny: for the moment we just know that "data aging" functionality is in plans for SAP HANA database (I stopped using IMDB, as it seems SAP Marketing is drifting from this name to "SAP HANA database" - probably to once again justify their existence). Accordingly to plans this data aging should allow you to define not only what tables to store permanently in RAM, and which not, but even to slice data within one table and to store let's say only 2 recent years of data in RAM permanently, while rest is loaded from drives on need basis.
And btw, there are different h/w configurations offered by vendors. There are some indeed with SSDs only, but most have both SSDs for log and HDDs for the rest.
John Appleby 22 Jun 2011
@Danny
Sure, SSD, but it's the same thing really. You could use regular disks if you had enough of them, SSDs just make it cheaper.
Data ageing is expected for HANA but I'm not aware whether it will hit in the 1.0 SP03 timeline. In the meantime, you will need a shedload of HANA if you have a big DW. There aren't that many BW implementations bigger than 5TB anyhow, so most should fit on a single box.
@mike
Agreed, will update with some info on BWA. I don't believe that BWA 7.2 supports planning accelerations though.
Danny Rohde 22 Jun 2011
That's a good start, definitely more than SAP delivers righ now ;)
Please note that HANA will not use spinning drives, but only SSD, this is necessary to maintain half decent performance for DB logs as well as in the case of disaster recovery.
What is your opinion regarding HANA under BW? I would expect that it needs to be a mix of in memory and disk based. Otherwise these large DWH will not fit onto HANA or you may end up with two DWH, one fast and one for historic data.
Mike Bestvina @techdisruptive 22 Jun 2011
Good article as always John. Just wanted to highlight something often overlooked for deciding between BW 7.3 and HANA.
"HANA 1.0 SP03 specifically also accelerates BW calculations and planning, which means you get even more performance gains."
Pushing calculations (and other runtimes traditionally unavailable in BWA, such as F4 member help) will be available with BW 7.3 and BWA. So if customers have BWA and are looking to improve OLAP performance, the cost-benefit ratio of going to BW 7.3 might outweigh going directly to HANA.