LogiGear Magazine

The Changing Landscape of Software Testing

By Hans Buwalda, CTO, LogiGear Test automation is significant and growing, and yet I read many forum comments and blog posts about test automation not delivering on expectations. It’s true that test More »

Leveraging Global Talent for Effective Test Agility

By Todd Little & Suzanne Elliott, Halliburton Joe Hughes, LogiGear Corporation Florin Simion, Simco International Ltd. Over the years many agile proponents have come out strongly against offshoring some of the development More »

Testing Netflix on Android

By Amol Kher, Wello Inc. When Netflix decided to enter the Android ecosystem, we faced a daunting set of challenges: 1. We wanted to release rapidly (every 6-8 weeks). 2. There were More »

Test Automation is Not Automatic

By Randall Rice, Rice Consulting Recently while teaching a workshop on Testing Dirty Systems, I uttered this “Randyism” off the top of my head, “Test automation is not automatic.” I realized immediately that More »

Misconceptions About Test Automation

April 29, 2013 No comments

By Hans Buwalda, CTO, LogiGear

Test automation is significant and growing, and yet I read many forum comments and blog posts about test automation not delivering on expectations. It’s true that test automation can improve reliability while minimizing variability in the results, speed up the process, increase test coverage, and ultimately provide greater confidence in the quality of the software being tested, but in (too) many cases the benefits never fully materialize.

A significant part of the problem results from misconceptions about software test automation. Many view the automation of tests as a low tech activity that the testers can take care of on top of their test design efforts. Unfortunately, many test tools on the market encourage this vision by making automation “friendly” with nice looking features and support for end users to do their own automation. However, automation is in essence software development—you try to program a computer to do something that you do not want to do yourself anymore. As with any software, automated tests tend to be complex and they can break when something unanticipated happens.

LogiGear Magazine – April 2013 – Test Automation

April 23, 2013 No comments

LogiGear Magazine – April 2013 – Test Automation

Test Automation Games

April 23, 2013 No comments

By Jonathan Kohl

Two dominant manual testing approaches to the software testing game are scripted and exploratory testing. In the test automation space, we have other approaches. I look at three main contexts for test automation:

1. Code context – e.g. unit testing.

2. System context – e.g. protocol or message level testing.

3. Social context – e.g. GUI testing.

In each context, the automation approach, tools and styles differ. (Note: I first introduced this idea publicly in my keynote “Test Automation: Why Context Matters” at the Alberta Workshop on Software Testing, May 2005)

Leveraging Global Talent for Effective Test Agility

April 23, 2013 No comments

By Todd Little & Suzanne Elliott, Halliburton

Joe Hughes, LogiGear Corporation

Florin Simion, Simco International Ltd.

Over the years many agile proponents have come out strongly against offshoring some of the development team, and in particular against having a remote testing team. We made use of not one, but two separate outsourcing providers located in two distant locations. While we had many challenges, what we found was that by starting with an overall testing strategy and an understanding of the strengths and constraints, we were able to achieve outstanding results. In particular, we were able to reduce defects found in customer beta testing by 84% and known customer issues at deployment by 97%.

Landmark Graphics is a wholly owned subsidiary of Halliburton and is the premier provider of software and technology services for the upstream oil and gas industry. Its software solutions help geoscientists and engineers make highly complex technical and business decisions. The specific product line involved in this case study is DecisionSpace Nexus[1][2], a next generation reservoir simulation software suite which utilizes a finite difference mathematical model to allow companies to accurately model hydrocarbon assets, enabling rapid decisions on high-dollar development scenarios.

Testing Netflix on Android

April 23, 2013 No comments

By Amol Kher, Wello Inc.

When Netflix decided to enter the Android ecosystem, we faced a daunting set of challenges:

1. We wanted to release rapidly (every 6-8 weeks).

2. There were hundreds of Android devices of different

shapes, versions, capacities, and specifications which need to playback audio and video.

3. We wanted to keep the team small and happy.

Of course, the seasoned tester in you has to admit that these are the sort of problems you like to wake up to every day and solve. Doing it with a group of fellow software engineers who are passionate about quality is what made overcoming those challenges even more fun.

Test Automation is Not Automatic

April 23, 2013 No comments

By Randall Rice, Rice Consulting

Recently while teaching a workshop on Testing Dirty Systems, I uttered this “Randyism” off the top of my head, “Test automation is not automatic.” I realized immediately that I had just concisely stated the problem in making test automation a reality in many organizations.

Most testers know that test automation is not automatic. (Wouldn’t it be great?) However, management many times does not know or accept that reality.

There are some test tools, such as unit test tools, that are practically automatically applied. My remarks in this article are aimed at the capture/playback and scripting tools for test automation.

Automation Frameworks and How to Build a Simple One

April 23, 2013 No comments

By Karthik KK

An automation framework is a way to organize your code in meaningful manner so that any person who is working with you can understand what each file contains.

Automation frameworks differ based on how you organize your code – it can be organized based on your data, so that any person who wants to use or edit data files such as an excel sheet can do so easily. These types of frameworks are known as data-driven frameworks.

Keyword-driven frameworks are those which can be written with keyword functions such as: Login, ClickButton, SearchList etc. These enable automation engineers to work within the framework easily, without ambiguity in function or code.

The combination of the above is called a Hybrid framework. There are some other frameworks which are named according to their usage such as Modular frameworks, structural frameworks etc.

In The News – April 2013

April 23, 2013 No comments

Bugs in Tax Software Stop People From Filing

Tax filing in the US was more difficult this year due to a glitch in some tax software programs.

“We’ve seen that on a number of returns. Actually, several hundred now,” said Income Tax Administrator for the City of Grand Rapids John Schaut. “Our counsel to tax payers who are using income tax preparation software to prepare their city returns is that they be patient, and check their software support site for updates.”

Letter From the Editor – April 2013

April 23, 2013 No comments

Automation is a mantra in testing. Anyone associated with software development wants more test automation, but it’s often misunderstood. People who do test automation know how difficult it can be. But some people do not understand that automation is code, and that it needs to have architecture and design just like production code. They do not understand that it needs code review, and especially ongoing maintenance, which can be more expensive than maintenance of the production code. These misunderstandings about automation in addition to misunderstandings about testing in general, make test automation a political problem as much as it is a software problem.

Book Review: Experiences of Test Automation

April 22, 2013 No comments

By Tad Anderson

Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including both successes and failures.

Many books simply provide you the success path. This book also provides you with the steps you could possibly be taking that could lead to failure, helping you to change your path before failing.

The book starts with a nice overview of the case studies and an introduction to the key issues addressed by the case studies. Besides each case study being summarized, it also includes introducing the topics and pointing out the chapter they can be found in. They are broken down into management and technical issues.