Using SAP Solution Manager for test management: how to support your Test Manager

12 January 2015

Lindsay (Stanger) Kelly

Lindsay (Stanger) Kelly

Enterprise Mobility Consultant

Let’s take another step towards a seamless project cycle, by helping the Test Manager in the final stages. I believe the role of Test Manager is one of the most valuable roles within a project team, and without it there is too much additional work loaded onto the Project Manager, who, by the time we are in the testing phase, is likely already dealing with pre-go-live and business readiness preparation for the impending release of the solution on the end users.

It is therefore essential that the Test Manager is supported as much as possible to ensure a smooth final project phase, and what better way to do this than provide the Test Manager with the right tools for the job.

Is Solution Manager the final piece of the project puzzle?

When you’re already running SAP it makes sense to invest in a test management tool which is designed to integrate directly with all your other SAP environments, making SAP Solution Manager the obvious choice.

As with so many good tools, there is an initial time and monetary impact of deciding to get SAP Solution Manager, but in the long term, once you’re up and running a successful implementation, the amount of time you will save, and as a consequence money, during the testing phases of the project will rapidly make the move worthwhile.

In my experience, the test phases of a project are often the ‘least respected’: usually by this stage the end users are keen to get their hands on the solution, the developers are eager to move on the next fun project, and the project managers are running out of time/budget. This is natural as the last phase of a project, testing is usually squeezed to allow for more time earlier in the project for, and it is easy to then rush through the testing, without giving proper consideration as to the impact. Solution Manager formalises this process, and makes it much easier to keep track of progress and completeness of testing, both past and present, and enables quick communication between testers and developers.

I recently took on the role of Test Manager (quite a change from my usual role in development!), managing the testing phases of two projects simultaneously. The client decided to use Solution Manager to aid with the testing phases, based on past experience where lack of, or the wrong tool had caused delays and inefficiencies. This being one of my first hands-on experiences with the tools available, I was a little sceptical to begin with, but since getting my hands dirty with it, I can now only sing the praises.

I already have a Test Manager, so why do I need Solution Manager?

There are three key benefits to using Solution Manager for your test management:

  1. Reusability/repeatability: In some respects SAP Solution Manager can be considered a document repository; for every project you will need SIT, UAT and Regression test scripts, all of which will need storing somewhere. By loading them into Solution Manager, you make it really easy to find reusable scripts when you are on the next project. Often, regression tests will follow old UAT scripts, and this saves significant time and effort to reuse rather than recreate
  2. Speedy communication and resolution: While testing, bugs will inevitably be uncovered. The progress of raising a bug and getting it fixed can be slow and painful, but SAP Solution Manager takes away that pain. When a tester raises an issue (or Test Message) they can assign it directly to the developer who needs to fix it (or to the development lead who can delegate). This then sends an automated email to the developer telling them that they have a new issue to resolve. Once they complete a fix, they can reassign the defect back to the tester for retest, and another automated email is sent. And all of this without the need for a middle-man
  3. Monitoring who’s doing what: Solution Manager allows you to plan out Test Plans and Packages script by script, and assign each one to an individual. These can then be put in a sequence, if required. When the Test Plan is released (i.e. you want testing to begin) the testers all receive an automated email telling them where to find their worklist. It is then simple to see, at a glance, who has complete, started and/or failed what, by looking at the Test Evaluation statistics. The main benefit of this is that the Test Manager need not constantly interrupt testers to get status update, but instead can see at a glance their progress based on the stats.

Hopefully you are starting, now, to see how Solution Manager is a brilliant tool to support the role of the Test Manager. Who needs to buy into the idea of using Solution Manager in order for me to gain the maximum benefit?

This is the first question you should ask yourself, and the answer is fairly straightforward: you will need almost everyone within your organisation who might be involved in testing, at some stage or another, to buy into the idea. And failing this you will need a few champions from each of the following:

  • The business. Those who will be doing UAT and Regression Testing
  • Developers. Those who will be performing SIT, and resolving defects
  • Project Managers. Those who will need to allow, within their project timelines, time for training, and who are willing to enforce the use of Solution Manager
  • The Test Manager. Who will be using, and setting up the tool.

People can be resistant to change, but if you can outline the benefits, it’s clear for anyone to see how implementing and using SAP Solution Manager for test management will benefit them in turn.

Business buy-in

Understandably, getting business buy-in is probably the hardest. Depending on the size of your organisation it will be challenging to get all potential future tester to buy into the idea of using a new tool, and not forgetting that people are often reluctant to accept change, even if it is for the best. It isn’t going to happen overnight, or even during your first project. I believe that the point at which the business are truly bought in is when they have all had 1) Sufficient training in the tool and 2) A chance to use it in practice, during the test phase of a project.

Developer / defect resolver buy-in

Getting developer buy-in and training is also key. One of the main benefits of SAP Solution Manager is the way that defects can be raised and assigned to developers, only requiring input from the Test Manager when the delegation of a test defect hasn’t already been made. Getting this right does not happen by chance. Developers need to be aware of their work-lists of defects, as well as the timeframes in which these need resolution. Often developers are on multiple projects, but this is where a development lead/manager can take ownership of the defect lists for the team, and prioritise accordingly.

How can I ensure SAP Solution Manager is used effectively?

Having a main Solution Manager sponsor who is willing to enforce its use can make all the difference, especially when it comes to getting all of the prerequisite steps set-up. Their involvement will decrease overtime, as the solution starts to be maintained by the organisation as a whole.

Each project is different but I would recommend the following:

  • Identify testers for the project (SIT, UAT and regression) as early as possible. By doing so, you will give yourself and them as much time to prepare as possible
  • Enable testers and developers with training in SAP Solution Manager across all the areas they will need. This is crucial, as it is easy to accidentally miss small, but key, aspects - for example what to do after a defect has been raised.

What training is required?

Business training

The training for business training falls into three main categories:

1. Test phase set-up

  • How and where to write and upload test scripts to Business Process Steps
  • How to create Test Plans & Packages for an existing Project, containing their scripts
  • How to assign scripts to specific users, and how to create sequences within a test package.

2. Executing tests

  • How to access their test worklist
  • How to run a test
  • How to update the Test Note (documentation of test)
  • How to save a version of the Test Note, after a failure, and create a new one for the re-test
  • How to change the test status.

3. Managing defects

  • How to raise a defect and assign it to the correct person
  • How to deal with a defect which has been re-assigned to them for re-testing, after it has been resolved, and how to change the status/reassign it to someone else/add comments

Developer training

Developers will need training in everything that the business are training in (for SIT), but also the following:

  • How to access their defect list
  • How to add comments to defects
  • How to change the status of defects
  • How to reassign defects, either back to the person who raised it, for re-testing, or to another developer
  • How to see defects assigned to other developers

Prerequisite set-up required

There are a number of pre-requisite steps which need completing before SAP Solution Manager can be used in the testing phases, and this applies to all projects. Much of this can be, and is often, reused between projects, so it is essential that enough time and consideration is put into each of these points.

  • Creating Projects in SOLAR_PROJECT_ADMIN
  • Linking Solution Manager, and then individual projects, to the relevant landscape, and systems
  • Creating the Business Scenario list in SOLAR01
  • Creating the Business Processes and their hierarchy
  • Creating the Business Process Steps, and linking them to the relevant transactions
  • Writing and uploading reusable (where possible) test scripts - should be done by a combination of the Testers and the Test Manager
  • Ensuring scripts are kept up to date for future projects to use - should be owned by the Solution Manager sponsor

If done correctly, these steps will ensure reusability across other projects.

Crucially, regression test scripts can often be UAT scripts from previous projects, so when new functionality is built, ensure the UAT scripts are written in such a way that they could be used along the line during the regression phase of a later project - saving time, effort, and annoyance down the line.

Communicating with you testers: Emails

There is a fine balance to be had with emails in general: if you send too many, people stop reading them, if you send too few, people don’t have all the information they need when they need it.

With Solution Manager the Test Manager benefits from the automated emails which are sent based on workflows. This can prove very valuable in keeping people informed as to what they need to do next and why. This means that Test Managers won’t need to send many emails themselves, as Solution Manager will do it automatically for them.

The different emails you can set-up, which remove the need for management intervention

The reason the automated emails are so useful, is that they avoid the need for the Test Manager to be running around, and passing messages. Here are a few examples of where they come if very handy:

  • “Test Plan Ready for Testing”: An email is sent to each tester telling them that they can begin testing, and what test packages are assigned to them. It provides a link to the users’ worklist, where they can see and work through all of their tests
  • “Next Tester in the Sequence”: When a test is completed, if it is part of a sequence, the next tester is emailed, to tell them it is their turn to start testing
  • “Defect Assigned to You”: When a defect is raised and assigned to a developer, the developer receives an email which links them to the defect details. Then when they assign it back to the tester to be re-tested, the tester receives an email telling them that the test defect is ready for a re-test.

As you can see from this list, a lot of what would usually be done by the Test Manager, is done automatically by the emails! But once again, this is only useful if, a) the recipient actually reads and understand the email, and b) the recipient knows what to do with the message they have been given, and has had the appropriate Solution Manager training

If you still aren’t convinced…

Hopefully you have read this post with an open mind, and are now aware of the benefits that Solution Manager could bring you, and your organisation, if used to aid in your Test Management. If you’re still unsure, I think the key points to take away are these:

  • Save yourself time (and therefore money) down the line, but building a reusable store of test scripts, which can be easily applied to new projects
  • Help your Test Manager to do his job as efficiently and effectively as possible by automating much of their role as possible, and therefore freeing them up for other work
  • Keep the testing phase of projects running rapidly and smoothly, by making communication between the developers, Test Manager and testers seamless

 

View comments

Comments

Blog post currently doesn't have any comments.

About the author

Lindsay (Stanger) Kelly

Enterprise Mobility Consultant

Working with new and emerging technologies in enterprise mobility is where Lindsay’s focus lies. From designing and developing responsive user interfaces with SAPUI5, through to creating the communication layer with SAP Gateway, Lindsay’s client experiences have provided her with the tools to build end-to-end solutions, with user experience as the key driver.  Recently she has been working with others at Bluefin to build a Fiori resourcing app to enable our consultants to plan and manage project resources.

Lindsay’s expertise in implementation and customisation within the Fiori application landscape was greatly valued whilst working as Gateway Development Lead on one of the UK’s first SAP Fiori implementations.  

Progressing from Fiori to UI development, using the SAPUI5 libraries, Lindsay has found a natural joy and passion: building device responsive user interfaces to meet users’ requirements and provide a quality user experience by following the Fiori design principles.
 
When planning out new mobile roadmaps, Lindsay uses her technical expertise, in combination with her desire to dig deep into clients’ systems and processes. This ensures the best strategic choices are made, whilst keeping in view the importance of user-centric design.

Lindsay’s passionate about guaranteeing clients’ self-sufficiency and delivers the SAP Gateway part of a three day Fiori training course. This leaves attendees with a greater understanding of, and better ability to support, maintain, and build from scratch their own OData service.

Organisation is Lindsay’s middle name (actually it’s Florence, but who’s checking!). As a fierce note-taker and documentation-writer nothing is left to chance. 

Outside of work you'll find Lindsay sneaking off on Snowboarding adventures, singing with Rock Choir and volunteering with the Newman Holiday Trust.

Bluefin and SAP S/4HANA - welcome to the one horse race