Regression testing is selective retesting of a system or component to verify that modifications have not
caused unintended effects in previously working modules of software/application. This type of testing is
typically associated either as a ‘challenge’ or ‘unimportant’ by developers; but a good tester always enjoys
breaking the software to scour out each and every fault from it. Having said that even regression testing
can become a challenge for the testers as well. Here are some of the reasons:-
1. The Number of test cases in regression suite increases with each new feature.
2. Sometimes, the execution of entire regression test suite becomes difficult due to time and budget constraints.
3. Minimizing test suite while achieving maximum test coverage is not a cake walk.
4. Determination of frequency of Regression Tests after every modification or every build update or after a
bunch of bug fixes is always a challenge.
The graph below depicts the same
Hence selecting test cases for regression testing is quite a tricky task. Today, I am sharing some tips that
I found to be most effective while selecting test cases for regression testing
Below are tips:-
1. Include the test cases which have frequent defects:
Some areas in product are so error prone that they usually fail with a small change in code. We can keep track
of failing test cases due to such areas throughout the product cycle and cover them in regression test suite.
2. Include the test cases which verify core features of the product:
Prior to designing of the test cases figure out all core features of application. Ensure that, test case cover
all functionality mentioned in requirement document. One can make use of traceability matrix to make sure that
no requirement is left untested.
Example:- Client would never expect a Home Page/Login Page/Key functionalities of his application getting failed.
3. Include the test cases for Functionalities which have undergone more and recent changes:-
SRS keeps on getting updated. Sometimes, the updates are not in full change to the previous SRS. But to some extent
(may be 15-30 %) changes may happen for every version upgrade. We testers have to agree that, it’s difficult to
keep writing (modifying) test cases, as the SRS keeps on getting updated often and this leads to some end moment
internal defects and changes into the code which in turn might break some already existing functionalities,
So it’s a “Must” (a must with capital M ) to always include such test cases which has recent changes.
4. Include all the Integration test Cases:
Even if Integration testing is a separate part of software testing cycle, its test cases should be included
in regression test suite.
A last moment fix, in already tested application can break the integrity between different modules. For
example, data might get lost across an interface, messages might not get passed properly or interfaces might not be
implemented as specified.
5. Include all Complex Test Cases:-
Some functionality of the system may only be accomplished by following some complex sequence of GUI events.
For example, to open a file a user may have to click on the File Menu and then select the Open operation,
and then use a dialog box to specify the file name, and then focus the application on the newly opened
window. Obviously, increasing the number of possible operations increases the sequencing problem exponentially.
This can become a serious issue even if one of the steps is not working—A whole functionality comes to HALT,
and End-User feels like a crap . That’s why all such complex test case should be included in regression test suite.
6. Prioritize the test cases for regression testing:
Prioritize the test cases depending on business impact, critical & frequently used functionalities. It is always
helpful if some analysis is done to find out what test cases are relevant and what are not. It is a good approach
to plan and act for regression testing from the beginning of project before the test cycles. One of the ideas is
to classify the test cases into various Priorities based on importance and customer usage. Here it is suggested
the test cases be classified into three categories:-
| Priority-0 |
Priority-1 |
Priority-2 |
|
Sanity test cases which checks
basic functionality(as per the
SRS of product) and are run
for pre-system acceptance and
when product goes through
major change. These test cases
deliver a
very high project value to
both engineers and to
customers. |
This includes the test cases
which tests essential
functionalities that deliver high
project value to both engineers
and customers. |
These are executed as a part
of ST cycle and selected
for regression testing on
need basis.
These test cases deliver
moderate
project value. |
Selection of test cases based on priority will greatly reduce efforts spent on regression testing.
7. Categorize the selected test cases: –
Regression testing becomes very difficult when the application scope is very huge and there are continuous increments
or patches to the system. In such cases selective tests needs to be executed in order to save testing cost and time.
Categorizing test cases makes this work easier. We can categorize them as
1) Reusable Test Cases:
It includes test cases which can be repetitively used in succeeding regression cycles. This can be automated so that
set of test cases can be easily executed on new build.
2) Obsolete Test Cases:
These test cases are defect specific and can’t be used in succeeding cycles. Smart way is to use them is when relative
defect occurs.
8. Choose the test cases on “Case to Case” basis:
There could be several right approaches to regression testing which needs to be decided on “case to case” basis:
•Case 1: If the criticality and impact of the bug fixes are LOW, then it is enough that a test engineer selects
few test cases from TCDB (Test case DB) and executes them. These test cases can fall under any Priority (0, 1
or 2).
•Case 2: If the criticality and the impact of the bug fixes are Medium, then we need to execute all Priority-0
and Priority-1 test cases. If bug fixes need additional test cases from Priority-2, then those test cases can
also be selected and used for regression testing. Selecting Priority-2 test cases in this case is desirable but
not a must.
•Case 3: If the criticality and impact of the bug fixes are High, then we need to execute all Priority-0, Priority-1
and carefully selected Priority-2 test cases.
•Case 4: One can also go through the complete log of changes that happened (can be obtained from Configuration management
engineer) because of bug fixes and select the test cases to conduct regression testing. This is an elaborate
process but can give very good results. I hope this will help you to find out what test cases should be included
in your regression test suite.
9. Change the regression test cases whenever required.
RESET of test case is not expected to be done often. Resetting of the test cases needs to be done with following
considerations;
a. When there is a major change in the product
b. When there is a change in the build procedure which affects the product
c. Large release cycle, where some test cases were not executed for a long time
d. You are in the final regression test cycle with a few selected test cases
e. When there is a situation where the expected results of the test cases could be quite different from previous
cycles.
On similar lines why to execute test cases manually if we can automate them and reuse it?
In RT Task, including an automation suite is always painless and feasible as it covers a wide area of testing scenario.
Infact Regression test suite should be 99.99% automated (It may require executing the RT task after every new build is
fired).Ahaa!! It sounds good, Running a test suite in night hours and getting test report in morning, what a start to
your day, when you realize your half of the work is already done.
