An Interview with Skytap’s Sumit Mehrotra

…On what you need to know before making the transition to EaaS

LogiGear_Skytap's Sumit

1. What are the main differences between cloud-based environments and cloud infrastructure?

An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work.

For example, a simple 3-tier application, with a web front-end component, a business logic component and a database component requires, at a minimum, the following infrastructure components: a few VMs, a few storage disks, and a network. All of these components are required, and they need to work together as an environment for the application to be functional. Note, that as the complexity of the application stack grows, the definition of an environment grows bigger than just infrastructure components. It can include VPNs, public IP addresses, an object store, and other service end-points.

From an application development and testing perspective, the application environment is the starting point of productive work. However, we see teams spending a large amount of time just getting the infrastructure components to a working state and hence losing out on a good portion of their productive work. Working with environments as a unit of work can enable these teams to be more productive and hence service customer and business requirements at a faster pace with higher quality.

2. What is “environment or infrastructure as code?” How can we use this idea in testing?

As the name suggests, infrastructure-as-Code is practice of representing infrastructure as code blocks so that they can be executed whenever needed to build an application stack (environment) in a repeatable and consistent manner. Depending on the actual tool used, it can allow you a variety of things including:

  • Deploy basic infrastructure elements, VMs, disks, networks, etc.
  • Install and configure operating systems
  • Deploy and configure application components
  • Populate application data

With a repeatable and consistent process, bugs can be reproduced, fixed, and validated efficiently. Developers can recreate the same environment that a tester is using to find bugs by using the same Infrastructure-as-Code scripts as the tester is using. This practice goes a long way in solving the problem of ‘it-works-on-my-machine’ syndrome.

There are various paradigms available to create consistent environments in a repeatable manner. Two such popular paradigms are “patterns” and “clones.” With patterns, you can create the environment from scratch each time—allowing you to progress the same exact code at each point of your software delivery pipeline. But it takes time for the environment to be provisioned and can be error-prone especially, for very complex environments. With clones, you can create another copy of the environment from a pre-existing copy. The process can be very fast, even for complex environments. However, moving the same clone through your SDLC can be a heavyweight process.

There is no right or wrong paradigm here. Successful teams have chosen a combination of paradigms and have made trade-offs between time and tooling consistency, to achieve the velocity they need in their software delivery lifecycle.

3. How do companies use cloud-based environments to facilitate testing in DevOps?

Testing especially in the DevOps world has the following key requirements:

  • Instant self-service access to application stacks for testing
  • Support the complexity and scale of the application stack needed at various stages of testing
  • Consistent and reproducible ways to create test environments
  • Ability to collaborate and share environments with developers and testers as part of a feedback loop. Constant feedback is a key aspect of Agile development
  • Ability to ensure that the right resources are available at any time and are being utilized judiciously

All of these requirements are hard to meet without a cloud-based platform. So, cloud-based environments are essential to facilitate testing workflows.

4We know that getting the right environments for testing can be difficult, and sometimes it’s out of a Test team’s control. How are innovative organizations solving this challenge?


Organizations are increasingly turning to the cloud for on-demand and self-service access test environments, but not every cloud platform is suited for the needs of the applications in an organization’s portfolio. Innovative organizations evaluate the environment needs of their applications, and some of the criteria that we have seen customers use are as follows:

  • Configurability – Can the platform deliver test environments that capture the complexity of my application at each stage of testing?
  • Consistency – Can the platform deliver test environments that are in the exact state that we need them to be, whenever I want?
  • Collaboration – Can the platform make it easier for my team (Devs + QA + Ops) to work together more productively?
  • Control – Does the platform have controls to ensure the right people have the appropriate resources to do their jobs?

5. How long does it take to build these environments? Some Test teams do not have dedicated IT members—does the Test team often do this?

The time to get a fully configured environment to test can vary in range from a few minutes to a few weeks. The availability of cloud platforms, and especially IaaS has reduced the far-end of the range to a few days primarily by taking the acquisition of hardware out of the equation. This is the lowest layer of the environment and typically, the one that used to take the longest.

Reducing the time to build these environments to minutes requires more than just infrastructure. It requires a platform that can support the deployment of fully configured environments. Paradigms like patterns and clones, discussed earlier, are essential to achieve that speed of deployment.

Additionally, these paradigms help with reuse of expertise within the organization as well. Environments capture the expertise needed to deploy various aspects of the application, be it infrastructure (IT team), presentation layer (UI team), business logic (app server team), database layer (DBAs). All the tester has to do is click a button or call an API to deploy the environment. The individual components are automatically deployed based on the definition of each component captured in the environment definition.

6.  How does this impact running an automated test suite?

Automated testing at each stage can be made more efficient and predictable by using consistent, fully configured environments that can be deployed using API calls. Continuous Integration and Continuous Delivery can actually be realized if the environments of desired size and configuration can be deployed within seconds and minutes versus hours and days.

7.  Getting the right data to test with, or setting up the right data is also a problem area for many teams. What strategies or tools help with this?

A few strategies can help with test data management:

  • Populating data in base environments which can then be cloned into new copies of the environment when needed. Once testing is done, the environment along and data can be thrown away. It is cheap to recreate a new environment with the desired data set.
  • Creating a database as part of environment creation. In this model, the data is created by using some automation code. This slows the set-up of the environments but ensures that the data creation is based on the latest desired state of the test data set.
  • Using data virtualization and data-masking tools to continuously keep a close-to-production copy of test data for testing. This is especially important in the later stages of testing like pre-prod certification testing.

8.  Are cloud-based environments also a popular choice for those doing continuous integration and or continuous testing?

Absolutely. The key requirement for CI/CD—speed of execution and feedback—cannot be achieved until test environments can be deployed on-demand, in an automated fashion, in a consistent state, and with desired complexity.

9.  We know containers are a stripped down wrapper around software components or services. How does Skytap use containers?

Skytap views containers as another component of the application stack just like all other components: infrastructure, middleware and applications. As customers modernize parts of their application that are a good fit for container technology, they are able to use them in Skytap environments, along with the other parts of their application that are not containerized. This is a very common scenario for traditional enterprise applications that tend to incorporate new technologies and are modernized over time.

With containers in Skytap, not only are customers able to use the container toolchain with container hosts in Skytap, but they are also able to still achieve the same efficiencies in their SDLC that they have traditionally done with environments. Additionally, Skytap customers have full control over their container hosts. This is not available in many container services, but is often desired by our enterprise customers. Customers can run containers on host VMs in their Skytap environments. They can also use the Skytap driver for Docker Machine to manage containers running on hosts in Skytap. If you are interested in learning more about Skytap’s support for containers, please contact

10.  How do you see the use of containers impacting testing?

Container technology can affect testing in many ways. Some of these are:

  • Deploying test environments: Application test environments that are entirely container-based or those that use containers for some application components require new ways for deploying the test environments while still meeting the requirements of consistency.
  • Testing both code and deployment: Containers encapsulate both the code changes and the deployment process by design. Containers deploy fully configured application components across the testing pipeline. Testing that traditionally only used to focus on code changes and deployment changes in isolation will need to accommodate both.
  • Integration with tools across the testing toolchain: Consideration must be given to the choice of build, testing, and automation tools that can work with traditional application components side-by-side with container-based components.
Sumit Mehrotra

Sumit Mehrotra is Sr. Director of Product Management at Skytap, a role in which he is responsible for product strategy and roadmaps. Prior to Skytap, Sumit worked at Microsoft in different roles and has shipped a number of products, including Windows Azure and Windows operating system. Sumit holds an MBA from University of Chicago Booth School of Business and a Masters in Computer Science from Boston University.

Sumit Mehrotra
Sumit Mehrotra is Sr. Director of Product Management at Skytap, a role in which he is responsible for product strategy and roadmaps. Prior to Skytap, Sumit worked at Microsoft in different roles and has shipped a number of products, including Windows Azure and Windows operating system. Sumit holds an MBA from University of Chicago Booth School of Business and a Masters in Computer Science from Boston University.

The Related Post

It’s no secret that the cloud is growing at an exponential rate. By 2016, two-thirds of the world’s server workloads will exist in the cloud. But according to Cisco’s 2012 Cloud Index, less than half of server workloads currently run in the cloud. Closing the gap between current capabilities and future requirements is a mission-critical ...
  LogiGear_Magazine_June 2016_Testing in the New World of DevOps  
From adopting the culture, to implementing Continuous Delivery With the relative newness of DevOps, there are not yet a ton of DevOps books. That’s why we’ve assembled a list of the 7 best DevOps books based on four criteria: the number of ratings from Amazon, the average Amazon rating, number of ratings from GoodReads and the ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
LogiGear Magazine June Issue 2018: TESTING in DEVOPS
How Halliburton leveraged outsourcing to achieve their goals. Organizations are focusing on speed, both in Continuous Integration and rapid deployment as a competitive advantage. Many software development organizations can significantly shorten development cycles by implementing one or a combination of Agile practices, continuous integration & deployment methods, and feature branches. While these frameworks and techniques ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions.
LogiGear Magazine June Testing in Continuous Delivery Issue 2017
Eric Minick brilliantly demonstrates how automation works in Continuous Delivery using this real-world “case study”. As a product manager at IBM working on our UrbanCode products, I think a lot about Continuous Delivery, how to do it and what it does for teams. Continuous Delivery techniques enable teams to deliver software faster and with less ...
Do you want to speed up your automated tests by a factor of 10 and deploy your application continuously? In this article we share how the JIRA development team at Atlassian has accomplished this using Stages in Bamboo. Stages have allowed the JIRA Development team to take a week’s worth of testing and condense it ...
LogiGear Magazine June Issue 2020: Transform Your SDLC With Continuous Testing
Cloud computing has been the buzzword in the world of Information Technology for quite some time and it is likely to retain that status in the coming years. Cloud computing has been helping business enterprises deliver services faster and cheaper compared to all other existing delivery models. Small and medium business enterprises have changed their ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news