5 KPIs to Evaluate Cloud and Managed Hosting Providers – Part 1

It is part of my lot in life to evaluate the service offerings of many cloud providers within the context of identifying viable alternatives for my customers who are either considering a migration to cloud services or who are seeking to outsource the day-to-day operation of their data centers.  Over time I have developed  KPIs that are critical in terms of selecting a viable Cloud or Managed Hosting provider.  While this analysis may not follow the Gartner Magic Quadrant, it is a checklist that I have found accurately predicts success – or trouble – that may exist in moving forward with a particular provider.

KPI 1 – The service catalog is too narrowly defined.  I can’t adequately explain to you the dismay I feel when I see a service catalog that is only focused on support for Red Hat Linux and Windows Server 2008 as a supported operating system. From a large Enterprise customer perspective, what in the world are they supposed to do with their applications that are deployed on HPUX, SUSE Linux, Oracle Enterprise Linux, Solaris, zOS or AS/400?  I have found that many Managed Hosting providers attempt to use contractors or partners to bifurcate the management of operating systems and platforms outside of the defined service catalog.   My advice? Caveat emptor – Let the Buyer Beware.  I have seen situations where a provider used a very well-defined interface agreement and service level agreement with the partner from a lifecycle management perspective that created a very transparent support process for the customer.  I have also seen several train wrecks develop from this approach.

KPI 2 – The vendor doesn’t have a defined cross-tower on-boarding team.   For most large Enterprise customers, the concept of “self-service” for Infrastructure as a Service (IaaS) or Storage as a Service (StaaS) solutions is a non-starter.  These customers are looking to either reduce headcount within the IT organization from a support perspective or are looking to use the Cloud or Managed Hosting provider as a strategic outsourcer to support the reallocation of IT budget to new services or applications.  My experiences with customers suggest that when they sign up for “self-service,” very few actually move applications to an IaaS VM or move data to the storage service.

One of the key differentiators in the Cloud and Hosting marketplace is the ability for a provider to bundle migration services in support of their service portfolio. These services facilitate transformation to the Cloud or Managed Hosting environment.  However, the ability to migrate is just one nuance of the analysis.  The second issue is to determine whether or not the on-boarding team has the ability to provide a single point of contact to the customer.  Reviewing the various operational silos that a Cloud or Managed Hosting service touches, you have to ask yourself one key question: How is the provider driving governance and the service assurance process? Unless the process is simple enough to clearly understand and consistent irrespective of the incident in question, you will not have a seamless experience with this provider once you migrate to a production status.

KPI 3 – Will the Provider step up and accept the corner cases?  This is what separates the men from the boys.  It is one thing to agree to manage an x86 Linux platform running VMWare with Microsoft Exchange, but it is quite a different thing to step up and manage two zSystem G10s running WebSphere Application Server and DB2 configured as GDPS nodes utilizing XRC data replication globally.  Do you support End of Life platforms and operating systems? Certain software developers are contractually bound to continue support for them.  In my experience, the willingness of a hosting provider to take on the corner cases is the key performance indicator that often drives the selection decision.

Enterprises can’t afford to migrate all of their applications off of non-standard platforms. Certain platforms (such as the Oracle Exadata, IBM Power 7 and IBM zSeries) can’t be replaced, especially when they are driving high-volume transaction processing, Big Data processing, or support for intensive processing tasks such as genome sequencing in the life sciences industry.

At the end of the day, I advise my customers that using multiple providers in this space is not a viable option, primarily due to governance and life cycle management issues.  From my perspective, a Cloud and Managed Hosting provider who refuses to step up strikes out.

What are your thoughts on these KPIs? Be sure to watch for my follow-up post on KPIs 4 and 5 on engineering gravity and CCMPs.
The Networking Exchange Blog Team About NEB Team