Jon Reed: Welcome to this podcast interview with our special guest, Sue Martin, the Global Certification Portfolio Manager at SAP. We're here with Sue today to get the latest and greatest information about SAP certification and ask her questions we've always wanted to ask about SAP's view on the best use of SAP certification for SAP professionals.

This podcast is hosted by SearchSAP.com and was made possible by a joint collaboration between SearchSAP; my site, JonERP.com; and B2B Workforce, an SAP partner.

Sue, thanks a lot for joining our call today. Why don't you tell us how you got involved in the SAP certifications world and what your current role is.

So I did an initial project and an analysis which sort of included stakeholder analysis, a market research study comparison of other IT certification programs, but also included a couple of surveys where we went out and asked certified individuals what they would like to see from the program and then put that together in a recommendation plan to SAP, who, thankfully, thought it was a really good idea. I have now been with SAP as the Global Certification Portfolio Manager really to implement that plan, that transformation strategy, for one and a half years now.

Reed: There's a lot of confusion but also interest in SAP's new three-tiered certification program. Tell us more about why this program was rolled out, what the levels are and what distinguishes each level.

Martin: What we initially did was go out to our certified population. As you probably know, Jon, SAP global certification has been around since the mid-90s and we had about 120,035 consultants out in the market around the world. When I came into the project to evaluate global certification, we really weren't quite sure if we were still adhering to their needs - are we still listening to them, making sure that we've got a valuable certification program that really boosts their competitive advantage, etc. - so that was one of the first things we did.

We worked together with a research organization, ITSMA, who did a survey of all of the people who'd certified with SAP in the two years previous, and this was initially in 2006. We went out to them and asked them what we can do better, how SAP certification has really enhanced their career so far, how could it enhance their career more positively, what sort of benefits packages would they be expecting to see - they've got their certified status, so what would they expect from SAP in terms of benefits associated with the certified status - and the multiple tiers really came out of this wish list.

Right at the top was that people felt that SAP offered certification right at the beginning of their career to get them up to speed and to talk about the different elements of our software solutions, but they didn't feel that it was something that really enhanced their career at a later date. They came back to us and said that they all talk about lifelong learning opportunities, etc., but our certification program doesn't actually reflect that.

They want something they can go to their customers with, that they can win contracts with; something that reflects their experience, how much they've actually been out there, they've been in projects. It doesn't just tell their customers they know a piece of SAP software from back to front. That's where the multiple tiers came from.

Reed: The three tiers we have are the Certified Associate, the Certified Professional and the Certified Master. Is there one that is the equivalent of the classic certification, if you will, and then the other levels build on that?

Martin: The Associate level is pretty well equivalent to our traditional, new-hire certification, so that's really targeted at individuals who are in the first year of their career. It's still primarily looking at a little bit more knowledge recall rather than experiential: What have I learned in the classroom? Am I fit to go out there and really play a role in a project? But it's primarily in a mentored situation.

The Professional level that you referred to is the second level that builds on that, so we're going much more towards individuals who've been in their careers for quite a few years, five years or more, who have quite extensive project experience. It's not about just knowing the software; it's about how they can actually apply those skills and the experience they've got in the market and in real life projects. It's much broader: it's about integration skills, it's about developing feasible solutions, it's about interpreting customer requirements.

Then we have the third level, which is really the crème de la crème, the Master Certification, which is very much focused towards the experts in their field. Using the term "experts," I don't want to imply that they're so specialized into an area that it sort of narrows as it gets further up the pyramid. They are specialists in their area, but part of that specialization skill is really the broader integration skills: How can I sit down with my client and talk to him or her about the vision for the next ten years? And then putting together solutions for that - so it's really the crème de la crème.

Reed: What is the availability right now of the level two and level three certifications? Is it possible at this point to go out and get a Certified Master certification, or are these levels getting rolled out more gradually in certain areas?

Martin: Our main focus at the moment is rolling out the Professional level. All of the Associate certifications are obviously coming out of the door as we update; so, the normal cycle of things. Since TechEd last year, that was really the big push for the major technology and development professional exams: getting the Java and ABAP professional exams out the door or the ABM professional exams and also CRM and a couple of other areas that are really very strategic for us.

So most of the professional exams are out there. We have the rest of the professional exams for FI, CO, etc., which will all be coming out in the third quarter, and we're currently using the experience and feedback that we've got coming back from the Professional exams to design the Master exams that will be coming out in the fourth quarter.

Reed: On my skills blog on SAP/BX and elsewhere, what we get a lot are mostly excitement about these new levels and the prospect of actually considering the hands-on component, but also some curiosity about how that would be assessed. I just want to get some clarity: Do both the Professional and Master levels include the assessment of a hands-on component, and if so, how is that going to be assessed?

Martin: There were a couple of things that came out in the survey. One was, as you say, a lot of interest in terms of hands-on components. I'd like to take a little step back here. From a psychometric validity perspective, what we chose to do, first of all, was to take multiple choice questions for the Professional level and gather some historic data. I don't want to drown us in scientific blabber, but it's very difficult to build a performance-based, hands-on exam if you don't collect historical data about how questions perform, etc. You have to have some level of historical data there to really be able to build an effective performance-based exam; if it's really hands on, you say, "How do I actually click this button?" etc.

That is a concept that we're definitely looking at, and we've spoken to a couple of providers who have very good reputations in that field and that is definitely something that we're looking at for next year, to be able to do hands-on exams. Particularly for the development areas, I think it's very relevant. I think for our application certifications, hands-on is not something that really adds value at the Professional level.

Have you looked at some of our Professional sample questions that are on the new SAP.com certification site, Jon? They're higher level questions, so it's no longer about whether it's the green button or the red button, it's more about if there was a particular scenario, how would you react, which would be the solution that you would choose for your customer. We've worked together with partners and consultants, with real-life experience to put those questions together, much more than taking it out of the training material and putting your questions down. When you look at sample questions, it's easier to understand how we actually get those higher level questions out there.

Reed: Sounds like those sample questions might challenge me a little bit, which would probably be a good thing.

Martin: Well, that's the feedback that we got. When we originally put our CRM Professional exams out there, we asked people who took them to give us written feedback in terms of how they felt the exam level was. We have a couple of people who said, "That was very challenging, but I haven't been out in the field for two years, so it's understandable."

The level of feedback that we got was excellent, and I think we're well on track. We're evaluating on an ongoing basis how we're actually doing, so we're looking at all of the item behavior and test behavior; on front, we have a couple of people who are monitoring that on a daily basis to make sure we are still on track. The feedback we've gotten so far has been very positive.