Make Design a Priority for Cloud Testing

Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Moving existing applications to a cloud environment adds new dimensions to testing. One of the primary reasons for moving to the cloud is scalability. Capacity to handle traffic and data transfer can be easily ramped up by access to additional computing resources. How this happens, and the configuration of those resources, can vary. Designing tests that can simulate these real-world scenarios and that are maintainable takes a significant amount of up-front planning.

When Cisco’s Telepresence division migrated the initial PC based applications to the cloud one of the primary goals was to automate as many of the test routines as possible, and to create the automation in a way that would allow it to scale as the application itself grew and changed. Rather than try to build the automation around a particular automation software, the focus was on test design first and tool second. In the end this proved to be the right approach.

Cloud Testing Considerations

Cloud deployments require testing the application, platform and infrastructure. Depending on whether it’s a private or public cloud there may be access issues that need to be addressed, and these will need to be accommodated in the test design. Below are several of the primary considerations the test teams for this project had to address.

Application changes and on premise interfaces

  • Would moving the application to the cloud require changes to the application to fit the environment available on the cloud platform?
  • How should the test cases be designed and test automation implemented to provide the required test coverage for these changes?

Data migration

  • Would new tests be needed for the storage services available on the cloud platform?
  • Data moved from managed servers to cloud storage may have to be encrypted for security and compliance needs. How would the additional security requirements be tested?

Performance

  • Was a plan needed to load and stress test the application on the cloud platform to ensure that system response is as per SLAs?

Scalability

  • Would the test scenarios be robust enough to simulate on-demand needs of the application?

Priorities for testing

Testing the applications being migrated to the cloud required a careful assessment from the perspective of what changes were required for cloud deployment. This assessment helped the test team understand application areas that needed specific focus during testing on the cloud platform. In addition, knowledge of the cloud platform capabilities for scalability, availability and disaster recovery had to be disseminated to the teams to assure successful test planning.

The testing plan was based on a model that included testing across the application, platform and infrastructure.

The Process

The project consisted of separate teams for portal testing and testing the supported devices. The Portal team was responsible for testing web applications that included a user portal and an administration portal that controlled all account information and updates, subscriptions information and modifications. Proper setup of the test environment required the team to become familiar with the target cloud platform.

Both LogiGear and Cisco teams collaborated to develop the ideal test plan for the existing web applications to be deployed in the cloud. Test engineers designed and developed python-based automated test cases utilizing the Action-Based-Testing (ABT) method. The big advantage of using ABT for this project was the ability to create well-defined test cases that were able to run independently of each other. This approach localized the impact of functional changes under test allowing the automation to be easily maintained. Maintainability of automated tests is essential to any automation project, but critical for testing of cloud applications where the application owner may not be involved in changes to the infrastructure the application runs on. It is precisely these types of projects where module-based test design proves its value.

This project required additional resources for computations and data storage to provide a true representation of the production data and expected load. Using virtual machines (VMs) proved to be the most effective way to accomplish this, plus VMs made it possible to scale quickly. After the initial testing, the VMs were saved as images in cloud storage, and all the cloud resources were released to keep the overall testing costs low. After the initial tests the VMs were restored as needed to recreate the test environment for subsequent releases.

Key learning

Test design planning for the cloud migration project was ultimately the primary component in achieving success for this project. Automation was the only way to handle the number of tests, and the amount of data to be tested, and in turn, the module-based approach made it possible to maintain tests easily even in a very fluid development environment.

When moving to the cloud there are two important test considerations. The first is the need to test all aspects of the cloud environment, starting with the cloud client through the application (SaaS), platform (PaaS) and Infrastructure (IaaS) layers. The second consideration is the capability of the team to plan and design the tests. Having the right skills and experience, even if you have to go outside, is essential for designing tests for large-scale cloud deployments.

Joe Luthy
Joe has over 15 years product marketing experience in technology and Internet services for major companies including AT&T and Verisign.

TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.

The Related Post

LogiGear University announces the launch of a new, free video series on Testing in DevOps and Continuous Testing which is available today.
Fitting QA into a modern DevOps group In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities. However, for folks outside the engineering team – say in Operations ...
DevOps for Test Teams By Michael Hackett Now that Dev Teams have had a little time to settle into the Agile, the new wave of process optimization has arrived. DevOps. DevOps has been described as Agile on Steroids. DevOps has also been described as Agile for Operations/IT. I like both of those descriptions as well ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
If you haven’t already caught a glimpse, we announced in January the Testing in DevOps and Continuous Testing videos series, available on YouTube!
Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...
Do you want to speed up your automated tests by a factor of 10 and deploy your application continuously? In this article we share how the JIRA development team at Atlassian has accomplished this using Stages in Bamboo. Stages have allowed the JIRA Development team to take a week’s worth of testing and condense it ...
    Eric Minick is internationally recognized as a leading authority on continuous delivery and DevOps. Eric joined IBM four years ago with the acquisition of UrbanCode where he had worked as a developer, technical seller, and evangelist for a decade. Today, he has responsibility for leading the product management team overseeing continuous delivery solutions ...
By Jez Humble and David Farley Continuous Delivery from Jez Humble and David Farley is an important contribution to the field of software development. It takes continuous integration to the logical conclusion and covers how to set up a continuous integration system, delving into everything from check-in to delivery to production. It doesn’t state you ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
It is a fundamental role for testing teams to align their test design, test automation, and test case development with DevOps–not only to verify that code changes work but that the changes do not break the product. A key differentiator of DevOps is testing maturity. An organization can automate their integration, testing, delivery, and monitor, ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions. ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe