Continuous Iteration in Automation

I’ve been teaching a lot lately, was in India for one week, and I’m off to Seattle in two weeks to teach on performance topics. I thoroughly enjoy teaching, it allows me to stay sharp with current trends, and provides a nice break from the “implementation focus” that I generally have day to day.

One topic however that has been gnawing at me lately is the notion of “continuous iteration” in automation. With virtualization, my feeling is that you should be running the automation you create all the time. I’ve coined a new term lately, “Test Everything all the Time”.

I find that many automation teams shelve the automation until runs are needed. I’m finding this to be a real bone of contention when it comes to automation credibility. The automation should always be in a state of readiness, in that I can run all my tests all the time if I want to. We used to be plagued by machine constraints and “unstable” platforms, but I’ve found that these issues are becoming increasingly less, but we seem to be still holding on to old automation ideas.

I’ve setup our labs here with machines that are constantly running our clients’ automation, while the testers are always adding more tests to that bank of tests. Automation should free us up to do this, but I still see too many people attending automation tests, which to me is like watching paint dry.

We have a “readiness” dashboard on all of tests all the time, so if I choose to run a set of tests immediately, we can dispatch those tests to a bank of machines and run them. It makes my job as a quality director easy, in that I never ever ask if the “automation” is ready, I have that information at my fingertips, for all types of tests. I’ve attached a simple dashboard report that I’ve used since I started working on automation projects in the early 90’s.

Remember, your automation needs to be credible and usable, not one or the other.

7-Sep-09
Test Cases Created and in Development25
Test Case Create Goal for Week40
Goal Delta-15
Test Cases to be Certified150
Test Cases Modified* (timing only)5
TOTAL TEST CASES IN PRODUCTION1253
Passive (Functional) Test Cases631
Negative Test Cases320
Boundary Test Cases40
Work Flow Test Cases78
User Story Based Test Cases24
Miscellaneous Test Cases160
Test Cases Ready for Automation1065
Test Cases Ready for Automation %85%
Test Cases NOT Ready for Automation %15%
Test Cases NOT Ready for Automation188
Test Case Error/Wrong5
Application Error23
Missing Application Functionality59
Interface Changes (GUI)2
Timing Problems70
Techincal Limitations6
Requests to Change Test(change control)23
Sum Check188
TOTAL MACHINE TIME REQUIRED TO RUN23:45:00
TOTAL MACHINES AVAILABLE TO RUN TESTS7
TOTAL SINGLE MACHINE RUN TIME3:23:34

 

LogiGear Corporation

LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers public and corporate software-testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast and cost-effective results. Since 1994, LogiGear has worked with hundreds of companies from the Fortune 500 to early-stage startups, creating unique solutions to exactly meet their needs. With facilities in the US and Vietnam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs. For more information, contact Joe Hughes + 01 650.572.1400

LogiGear Corporation
LogiGear Corporation provides global solutions for software testing, and offers public and corporate software testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast, cost-effective results. Since 1994, LogiGear has worked with Fortune 500 companies to early-stage start-ups in, creating unique solutions to meet their clients’ needs. With facilities in the US and Viet Nam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs.

The Related Post

Mobile usage today is not just a trend but it is an essential shift in how people communicate with each other, interact with the world, and do business. According to a ComScore, in 2014 the number of mobile users surpassed the number of computer users and is showing strong growth over time, towards some point in ...
LogiGear Magazine – April 2014 – Test Tool and Automation
LogiGear Magazine January Trends Issue 2017
I got some comments on my post “Test Everything all the Time” — most notably people commenting that it’s impossible to test “everything”. I can’t agree more. The intention of the post was to make the point that we need to be able to test “everything we can” all the time. That is, you should ...
Mobile testers need to take a different approach when it comes to Test Automation.
Two dominant manual testing approaches to the software testing game are scripted and exploratory testing. In the test automation space, we have other approaches. I look at three main contexts for test automation: 1. Code context – e.g. unit testing. 2. System context – e.g. protocol or message level testing. 3. Social context – e.g. ...
The Cloud demands that we be as nimble as possible, delivering features and fixes in almost real-time fashion. Both customer and provider rely on software development that can maintain quality while being light on its feet and constantly moving. In addition, Cloud-oriented systems tend to be highly complex and dynamic in structure — more than ...
LogiGear Magazine – March 2011 – The Agile Test Automation Issue
Automated Testing is a huge part of DevOps, but without human-performed quality assurance testing, you’re increasing the risk of  lower-quality software making it into production.  Automated Testing is an essential DevOps practice to increase organizations’ release cadence and code quality. But there are definitely limits to only using Automated Testing. Without human quality assurance (QA) ...
This book isn’t for everyone, but everyone can get some value out of it. What I mean by that rather confusing statement is that folks working in Agile environments will likely want to throw the book across the room while folks in more bureaucratic environments like CMMI or other waterfall environments will likely get a ...
June Issue 2019: Testing the Software Car
The huge range of mobile devices used to browse the web now means testing a mobile website before delivery is critical.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe