Just where you need automation, you don’t find it.

I’m indebted to a new Facebook friend who emailed me the following statistic: in the test market 74% still don’t use an automated solution, and 53% are using general-purpose business software like Word and Excel.

In the SAP environment, the figures could be higher, with as many as 95% manual testing. Strange don’t you think, when there are dozens of small test tools out there, and the big monsters too? (What do you use for example? – I’d like to hear.) Anyway, maybe this polarity in test tools is the problem. Who wants lots of separate cobbled together tools for different applications, or big expensive software that’s really complicated to use?

There has to be a happy medium for SAP in particular. You work in a customized environment, with numerous patches and support packs, major new versions and your own changes. So why can’t test tools meet those requirements – i.e. be affordable, easy to implement, and quick to use and maintain? My old company, Borland, believe the power of test tools should be hidden under the hood, leaving you to get the job done easily. Some testing newcomers do have easy to use solutions but they only cover one area or can’t be applied to a broad range of applications.

Take web based applications, for example, where different browser and operating system combinations need a broad testing capability to ensure a good user experience. Or even more challenging – in the SAP world – mobile applications which create a whole new level of testing challenges. There’s just no way manual testing is going to give any kind of assurance with so many devices and interfaces out there to test.

I’ve been trying out all the latest test software, but I’m no SAP expert, so anyone out there who’s working hands-on in SAP, I’d welcome feedback on where testing’s going in your environment. Let me know what’s needed.

This entry was posted in Insights. Bookmark the permalink.

3 Responses to Just where you need automation, you don’t find it.

  1. Steve Dykstra says:

    Welcome back. Glad to see you are writing about the trials and tribulations of not only software developement but packaged software. I have been working with companies using SAP for quite some time now and while software development certainly requires good software quality practices, implementing a very flexible one size fits all packaged application has major risks that need to be mitigated.

    One risk area that is the same or similar to developing software is that you can still easily create systems that are not a good fit for the business. So all the upfront work to gather good requirements and make sure your tests are validating them is just as important in the packaged world.

    One way packaged apps are uniique is that you need to configure a lot of complex business processes so they fit together seamlessly. A configuration mistake here or there will create a process that cannot flow from end to end. So while you might have less crashes or errors to deal with you still have to validate things work from end to end and just make sense.

    Anyway glad you are having a look at this and writing about it. Sure seems like people could use some help in this area.

  2. Michael Thuma says:

    SAP follows a different philosophy.
    1) You test the impact of customization, impact of master data changes to the transactional data.
    2) The impact of changes to the business logic.

    Of course 1) + 2) combined appears too – Roll-out projects especially.

    You have 2 major ‘use cases’
    a) Rollout projects 1) +2) – heavy use of regression.
    b) ‘Continuous improvement’ – something is put productive and the whole userbase is testing in production. If an error can be found -> The requirement changed. Tamagotchi and a whole ECO system taking care (system owner, application owner,…).

    Why Excel – Technically the SAP SAPGUI testing part is an integrated Batch-Input more or less. SAP development is not engineering it’s still customization.

    SOX – Strict separation of technical challenges from the business knowledge. Is a user a tester, a SAP user is not part of the IT organization. The majority is still IT knows logic, but in these cases the developers can simply separate the business logic from the VIEW.

    Dependent on the technology I would hook into the device emulator and catch the ‘function call /message transfer’. Honestly I have no insight on SAP mobile technology. CRM will be very likely different than traditional Netweaver ERP.

    In 2006 I started to experiment with a middleware from the Delphi world (Remobjects). I fact these days somehow implementing a little test server and replicating the security information could have been helpful. My thought was another one – use the BAPIs/RFCS call them via a service or better trace the session make the trace available. (no longer state of the art I think). Of course this can be done more intelligent but SAP and mobile in 2006 was – illusion of replicating SAP GUIs via a middleware.

    A business function the does the job is usually an asynchronous background process executing a function call. In Oracle wording – a function call storing the screen values in ref cursors and executing a PL_SQL_JOB. There is absolutely no magic. SapScript once provided a library that allow to retrieve a GUI screens meta-data and honestly by making use of RFC and DEBUGGING in the GUI (ABAP_DEBUG=1, USE_SAP_GUI=1) you can do a lot. Test the business logic from the moment ‘message’ received to checking what happens when the business logic is activated.

    I have no idea what frontend technology SAP uses at the moment. ITSMobile (is still alive maybe), the Sybase approach (messaging combined with apps – I think focusing on standard shipped), from the last strategic announcement (Titanium) sound realistic, but I think could be replaced soon. Maybe it’s for the NextGen ABAP stack only. We will see.

  3. Michael, I couldn’t agree more.

    SAP isn’t like any other environment – it involves many different approaches to testing. You can bet plenty of people are still using Excel lists and manual testing, and while that will never disappear completely, I’ve been hearing on the grapevine that many SAP clients want to test more holistically.

    It’s no wonder – manual testing actually costs big bucks that you won’t get back – SAP reckon that three quarters of an update/change budget is eaten up by testing. We should be automating these processes to the max and staying in tune with SAP´s ALM strategy, including SAP Solution Manager and Test Management for managing the testing process throughout the SAP deployment lifecycle.

    This also includes load and performance testing which is key for SAP deployments, particularly business critical situations where new business processes and models have to be deployed for new customers – I’m thinking about SAP’s new ARIBA platform, here.

    So why not rely on non-script based, business-requirements and process-driven testing of SAP ERP and NetWeaver applications, SAP Enterprise Portals, mobile Apps and SAP native GUI? Let’s work towards a test automation ‘factory’ which automatically detects and tests changes of business blueprints, transactions or the whole SAP solution directory.

    Basically, we need to push the boundaries of SAP testing. And you know where to come for that capability.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s