February 2007 - Tests Built-to-Last: Software Tests as an Asset
By Hung Nguyen, CEO, President and Founder, LogiGear CorporationIntroduction Traditionally, software testing is not treated the same as code development. In many organizations, software testing is looked upon as a costly and time consuming task that stands in between development and product shipment. In such organizations, tests are developed, the software product is tested, and the product is subsequently shipped. When the next release is available, the cycle is repeated - new tests are created and run, and the new product is shipped. Even though many advancements, such as extreme programming, test-driven, and Agile development have been adopted to speed up and enhance the quality of software development, testing is still the same expensive and time consuming process it has always been. The problem is not inherently in software testing. The problem is in the approach to software testing. Just as development advancements required philosophical shifts, changes in methodology and changes in the supporting tool sets, software testing requires similar changes in philosophy, methodology, and supporting tools. The philosophical shift needed for software testing is to view developed software tests as an asset; an asset that has its own intrinsic value and adds to the company's balance sheet, as do software applications. These software tests codify intellectual property, are reusable, are easily maintainable, and provide process documentation and visibility. To truly be an asset, software tests must be highly automated, and reusable in the long-term across various versions or revisions of the systems under test. Only by undertaking a shift in vision will an organization be poised to truly optimize software testing and realize significant cost and time savings, while at the same time improving software quality. The remainder of this article will discuss the "Software Tests as an Asset" philosophy, introducing Action Based TestingT (ABT) as the methodology that implements that philosophy, and automation-enabling technologies that support the methodology. "Software Tests as an Asset" Philosophy Software testing, and the software tests that are created, need to be looked upon as an easily maintained, renewable and reusable resource that inherently captures what an organization knows about how the software under test is supposed to work, and provides visibility into the quality of the software under test. Some of the most fundamental precepts of the "Software Tests as an Asset" philosophy are:
Clearly this is fundamentally different from the more "traditional" approaches to software testing. Only by adopting such a dramatic shift in philosophy will an organization be able to significantly improve the speed and reliability of testing, while reducing costs and improving software quality. By adopting such a philosophy, an organization will have software tests that are built to last. The tests will be reusable as is, or with small amounts of maintenance, to test future generations of the software under test.
ABT - the Methodology to Support the Philosophy Action Based Testing (ABT) is the methodology that implements the "Software Tests as an Asset" philosophy. In ABT, tests are written in English, by non-technical test engineers who use a combination of keywords and data (parameters) to build tests. Tests cases are written in test lines using action keywords (such as "login") and data or parameters (such as "login name" and "login password"). Multiple test lines are aggregated into test cases, and multiple test cases are aggregated into test modules. What emerges, by design, is an inherently self-documenting, hierarchical test plan for the software under test that can be easily understood by non-technical individuals. Highly technical automation engineers worry about the small amount of work that it takes to code the underlying actions that actually automate the test lines created above, actions such as "FieldInput", or "ButtonClick". For example: To implement a login test, the test engineer would create a test line that provided the action keyword "Login" and pass the parameters "login name" and "login password". To automate this would require only two actions:
Automation engineers worry about defining the interface specifications in an Interface Definition Worksheet, and script the actions using a harness or scripting language. It is easy to see both how simple actions can get reused over and over again in numerous ways and in different combinations, and how actions can be easily aggregated to create more complex test cases. In Action Based Testing, test engineers focus on test design, and automation engineers focus on automation. TestArchitect as the Toolset that Supports the Methodology LogiGear's TestArchitectT is a test automation toolset that implements ABT's hierarchical methodology. Using TestArchitect, automation engineers write test cases in English using action keywords and parameters in a Test Module spreadsheet, and define data formats and conditions using the Interface Definition spreadsheet. The level of technical skill to do this is no more than is needed to use Microsoft® Excel. Automation engineers, who focus on the automation of test cases, use Action Definition spreadsheets that define the actions in English, and Interface Definition spreadsheets that define the interfaces with which the software under test will interact. Finally, the automation engineer works with a harness, or the scripting language that is actually used to automate the individual actions, or simply links to built-in actions included with the TestArchitect toolset. The Benefits of "Software Tests as an Asset" Philosophy Organizations that adopt the "Software Tests as an Asset" philosophy, and supporting methodologies and toolsets, will accrue many benefits, including:
All of this will lead to increasing the speed and coverage of testing, reducing the costs of testing, and improving the quality of the software under test. Adding a Global Resource Strategy to Further Reduce Costs The hierarchical nature of the "Software Tests as an Asset" philosophy and the ABT methodology actually make it easier to adopt global resource strategies to further reduce testing costs. Using this philosophy and methodology, it is easier to implement a distributed resource architecture where onshore testing engineers write test cases and offshore automation engineers script actions and run the test cases. Doing so can further drive down the costs of software testing. Conclusion Economies of scale in software testing, as many organizations have already done in development, require a philosophical change where the organization comes to look upon software tests as an asset. Then tests will be built to last. Adopting ABT as a supporting methodology with TestArchitect as the implementation tool, allows an organization to test more, test faster, and test at a lower cost, while improving software quality. Implementing a global resource strategy along with these other changes will allow an organization to gain maximum cost benefits from their software testing efforts. |
