Make Design a Priority for Cloud Testing

Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be easily ramped up by access to additional computing resources. How this happens, and the configuration of those resources, can vary. Designing tests that can simulate these real-world scenarios and that are maintainable takes a significant amount of up-front planning.

When Cisco’s Telepresence division migrated the initial PC based applications to the cloud one of the primary goals was to automate as many of the test routines as possible, and to create the automation in a way that would allow it to scale as the application itself grew and changed. Rather than try to build the automation around a particular automation software, the focus was on test design first and tool second. In the end this proved to be the right approach.

Cloud Testing Considerations

Cloud deployments require testing the application, platform and infrastructure. Depending on whether it’s a private or public cloud there may be access issues that need to be addressed, and these will need to be accommodated in the test design. Below are several of the primary considerations the test teams for this project had to address.

Application changes and on premise interfaces

  • Would moving the application to the cloud require changes to the application to fit the environment available on the cloud platform?
  • How should the test cases be designed and test automation implemented to provide the required test coverage for these changes?

Data migration

  • Would new tests be needed for the storage services available on the cloud platform?
  • Data moved from managed servers to cloud storage may have to be encrypted for security and compliance needs. How would the additional security requirements be tested?

Performance

  • Was a plan needed to load and stress test the application on the cloud platform to ensure that system response is as per SLAs?

Scalability

  • Would the test scenarios be robust enough to simulate on-demand needs of the application?

Priorities for testing

Testing the applications being migrated to the cloud required a careful assessment from the perspective of what changes were required for cloud deployment. This assessment helped the test team understand application areas that needed specific focus during testing on the cloud platform. In addition, knowledge of the cloud platform capabilities for scalability, availability and disaster recovery had to be disseminated to the teams to assure successful test planning.

The testing plan was based on a model that included testing across the application, platform and infrastructure.

The Process

The project consisted of separate teams for portal testing and testing the supported devices. The Portal team was responsible for testing web applications that included a user portal and an administration portal that controlled all account information and updates, subscriptions information and modifications. Proper setup of the test environment required the team to become familiar with the target cloud platform.

Both LogiGear and Cisco teams collaborated to develop the ideal test plan for the existing web applications to be deployed in the cloud. Test engineers designed and developed python-based automated test cases utilizing the Action-Based-Testing (ABT) method. The big advantage of using ABT for this project was the ability to create well-defined test cases that were able to run independently of each other. This approach localized the impact of functional changes under test allowing the automation to be easily maintained. Maintainability of automated tests is essential to any automation project, but critical for testing of cloud applications where the application owner may not be involved in changes to the infrastructure the application runs on. It is precisely these types of projects where module-based test design proves its value.

This project required additional resources for computations and data storage to provide a true representation of the production data and expected load. Using virtual machines (VMs) proved to be the most effective way to accomplish this, plus VMs made it possible to scale quickly. After the initial testing, the VMs were saved as images in cloud storage, and all the cloud resources were released to keep the overall testing costs low. After the initial tests the VMs were restored as needed to recreate the test environment for subsequent releases.

Key learning

Test design planning for the cloud migration project was ultimately the primary component in achieving success for this project. Automation was the only way to handle the number of tests, and the amount of data to be tested, and in turn, the module-based approach made it possible to maintain tests easily even in a very fluid development environment.

When moving to the cloud there are two important test considerations. The first is the need to test all aspects of the cloud environment, starting with the cloud client through the application (SaaS), platform (PaaS) and Infrastructure (IaaS) layers. The second consideration is the capability of the team to plan and design the tests. Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Joe Luthy
Joe has over 15 years product marketing experience in technology and Internet services for major companies including AT&T and Verisign.TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.

The Related Post

This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
    Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions ...
…On what you need to know before making the transition to EaaS 1. What are the main differences between cloud-based environments and cloud infrastructure? An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work. For example, a simple 3-tier application, with a web front-end component, a business logic ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
Sauce Lab’s Perspective on Integration with Continuous Testing The term ‘DevOps’ implies that implementing an effective development and production workflow is as easy as developing the collaborative interaction between developers and IT operations. On the surface, DevOps may appear as a simple and straightforward idea but, as you will find out, there is more than meets ...
How Halliburton leveraged outsourcing to achieve their goals. Organizations are focusing on speed, both in Continuous Integration and rapid deployment as a competitive advantage. Many software development organizations can significantly shorten development cycles by implementing one or a combination of Agile practices, continuous integration & deployment methods, and feature branches. While these frameworks and techniques ...
Special considerations that should be applied to an application running in the cloud. Over the last weeks, I have found myself in several rather intense discussions about “cloud testing”: what it is, what it isn’t, and what it means for testing and QA professionals. The major source of confusion in these discussions usually revolves around ...
  LogiGear_Magazine_June 2016_Testing in the New World of DevOps  
Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing. The Fundamentals ...
Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...
LogiGear Magazine June Testing in Continuous Delivery Issue 2017

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe