SLAs and Cloud Testing

A test team’s job is to report test results, not set or guarantee that you will meet the SLAs.

In the rush to cloud services, with everything-as-a-service, you will hear people talking about SLAs. What is this about and what does it have to do with testing?

A Service Level Agreement, or SLA, is a contract a service provider promises for a defined level of service, such as response time, throughput or capacity.

When a customer signs up for service, the provider promises, in contract, certain levels of service. The most important aspect is usually availability. 

Availability is the ability to access the system. Everyone wants their service available all the time. This is an impossibility for both good and bad reasons. Good reasons – downtime, patches, new build migration and system upgrades. Bad reasons – system crashes, security problems – denial of service, network/infrastructure problems.

Downtime happens and SLAs are meant to provide a promise from the provider, of how available the system will be.

This is an important part of marketing, sales and contracts for any cloud service provider, from HP and Amazon to consumer products in the cloud like Netflix and Foursquare.

Gartner analyst Lyida Leong blogged that Amazon Web Services, which Gartner named a market-leader in infrastructure-as-a-service cloud computing, has the “dubious status of ‘worst SLA (service level agreement) of any major cloud provider.” She also wrote, “HP’s newly available public cloud service could be even worse.”

What are reasonable SLAs for availability? What is common? The answer differs based on the service. For example, many people use “4 9s” which represents 99.99% uptime:

Think about this: for four nines availability allows 1 minute downtime per week. Wow. How safe do you think it is for a company to guarantee this? In one full year, that means down time of less than an hour.

So what does this mean for testing? Testing SLAs is all about system performance testing; load testing, stress testing. It is measurement of the various attributes of the product; capacity, response time, against agreed upon standards. What you have to remember is a test team’s job is to report test results, not set or guarantee you will meet the SLAs.

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

Throw away clunky hyper-visors, and stop thinking about computer hardware and software license during your development projects. The first thing you think about when you hear “The Cloud” may not be development and testing. The Cloudy market is filled with SaaS applications, hosting, and cloud-based file systems. All are very useful, and offer a clear ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
Over the years we’ve provided an extensive number of articles, videos, and infographics that provide a wealth of knowledge about Continuous Delivery.
By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments. Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...
As a software development company, what is your goal? What is the one thing you feel you need to do to ensure you have a job at the beginning of each wonderful work week? The answer is actually quite simple; You need to deliver a quality product. Like how I used the word simple? Although the answer I ...
Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
Eric Minick brilliantly demonstrates how automation works in Continuous Delivery using this real-world “case study”. As a product manager at IBM working on our UrbanCode products, I think a lot about Continuous Delivery, how to do it and what it does for teams. Continuous Delivery techniques enable teams to deliver software faster and with less ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
LogiGear Magazine June Issue 2018: TESTING in DEVOPS

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe