Book Review: Testing Cloud Services: How to Test SaaS, PaaS and IaaS

The book is an incredibly effective and valuable guide that details the risks that arise when deploying cloud solutions. More importantly, it provides details on how to test cloud services, to ensure that the proposed cloud service will work as described.

It is a great start to the topic. The 6 chapters detail a paradigm that cloud architects, managers and designers can use to ensure the success of their proposed cloud deployments.

The first two chapters are a very brief introduction to cloud computing. In chapter 3, the authors detail the role of the test manager. They write that the book is meant to give substance to the broadening role of the test manager within cloud computing. They encourage firms to make sure the test manager is involved in all stages of cloud computing; from selection to implementation. In fact, they write that it is only a matter of time until this service will be available in the cloud, in the form of TaaS – Testing as a Service. 

Besides the great content, the book is valuable since it has many checklists and questions to ask. One of the reasons cloud hype is so overly pervasive, is that the customers believe what the marketing people say, without asking enough questions. It would have been an added benefit if these questions and checklists would be made available in softcopy to the reader.

In chapter 4, the book details performance risks. As to performance, an important aspect of selecting the correct cloud provider is scalability of the service. This then requires a cloud specific test to determine if the scaling capacity (also known as elasticity) of the provider will work efficiently and effectively in practice.

An extremely important point the authors make is that when choosing a cloud service, many firms don’t immediately think of having a test environment, because the supplier will themselves test the service. The absence of a test environment is a serious risk.

About 2/3 of the book is in chapter 5 – Test Measures. The chapter mostly details the test measures for SaaS, but also does address IaaS and PaaS testing. The chapter spends a lot of time on the importance of performance testing.

An important point detailed in the chapter is that of testing elasticity and manual scalability. This is an important topic since testing elasticity is a new aspect of performances testing. The objectives of elasticity tests are to determine if the performance of the service meets the requirements across the load spectrum and if the capacity is able to effective scale. The chapter details various load tests to perform.

In the section on guarantees and SLAs, the authors make numerous excellent points, especially in reference to cloud providers that may guarantee very high availabilities, but often hide behind contract language. They provide a number of good points to consider in regards to continuity guarantees, including determining what is meant exactly by up- and down-time; for example, is regular maintenance considered downtime or not.

Another key topic detailed is testing migration. The authors write that when an organization is going to use a service for an existing business process, a migration process is necessary. This includes the processes of going into the cloud, and backing the service out of the cloud.

With all of the good aspects to this book, a significant deficiency in it is that it lacks any mention of specific software testing tools to use. Many times the authors write that “there are many tools, both open source and commercial, that can” but fail to name a single tool. The reader is left gasping at a straw knowing of the need to perform tests, but clueless as to what the best tools to use are. Given the authors expertise in the topic, that lacking is significant.

The only other lacking in the book is in section 5.3 on testing security, the authors fail to mention any of the valuable resources on the topic from the Cloud Security Alliance. Specifically the Cloud Controls Matrix (CCM) and Consensus Assessments Initiative (CAI) questionnaire.

With that, Testing Cloud Services: How to Test SaaS, PaaS IaaS should be on the required reading list of everyone tasked with cloud computing. This is the first book to deal with the critical aspect of testing as it related to cloud computing. The ease of moving to the cloud obscures the hard reality of making a cloud solution work. This book details the hard, cold realities of turning the potential of cloud computing, in the reality of a working solution.

Had the designers of the Obamacare website taken into consideration the key elements of this book, it is certain that the debacle that ensued would have been minimize and the administration would not have had to send out a cry for help. The Obamacare website will turn into the poster child of how to not to create a cloud solution. Had they read Testing Cloud Services: How to Test SaaS, PaaS IaaS, things would have been vastly different.

Ben Rothke
Ben Rothke, CISSP, CISM, CISA is an information security manager with Wyndham Worldwide Corp., the world’s largest hospitality firm. Rothke has over 15 years of industry experience in information systems security and privacy. His areas of expertise are in risk management and mitigation, security and privacy regulatory issues, design & implementation of systems security, encryption, cryptography and security policy development, with a specialization in the financial services and aviation sectors. He is the author of “Computer Security – 20 Things Every Employee Should Know” (McGraw-Hill). He is also a frequent speaker at industry conferences, such as RSA and MISTI, and holds numerous industry certifications.

The Related Post

Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...
It’s no secret that the cloud is growing at an exponential rate. By 2016, two-thirds of the world’s server workloads will exist in the cloud. But according to Cisco’s 2012 Cloud Index, less than half of server workloads currently run in the cloud. Closing the gap between current capabilities and future requirements is a mission-critical ...
Making the leap to CT is easier than you think— follow this guide to transform your testing process No pain, no gain! Achieving Continuous Testing shouldn’t take a “Hans and Franz” attitude. It should be painless, more like a natural progression from implementing certain practices over time.
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...
How to Adopt the “Third Way” in the Dojo’s Method to Master CD In The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations the authors describe “The Three Ways” – the underlying principles forming the basis for all DevOps practices. 
Throw away clunky hyper-visors, and stop thinking about computer hardware and software license during your development projects. The first thing you think about when you hear “The Cloud” may not be development and testing. The Cloudy market is filled with SaaS applications, hosting, and cloud-based file systems. All are very useful, and offer a clear ...
Special considerations that should be applied to an application running in the cloud. Over the last weeks, I have found myself in several rather intense discussions about “cloud testing”: what it is, what it isn’t, and what it means for testing and QA professionals. The major source of confusion in these discussions usually revolves around ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions. ...
How to ensure a successful test-driven environment In order to ensure a higher quality product is released in the end, many teams have turned to test-driven development. Under this scenario, quality assurance metrics professionals first create various QA tests, and then software engineers code based on these tests, typically while using a robust enterprise test management ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe