Software Testing
This article provides basic and advanced concepts of software testing.
Software Testing: Verification and Validation; Error, Fault, Bug and Failure; Unit and Integration Testing; White-box and Black-box Testing; Basis Path Testing, Control Structure Testing, Deriving Test Cases, Alpha and Beta Testing; Regression Testing, Performance Testing, Stress Testing.
Software Testing is the process of executing a program with the aim of finding errors.
- To make our software perform well it should be error-free
- If testing is done successfully it will remove all the errors from the software.
Principle of Testing
- All the test should meet the customer requirements
- To make our software working, testing should be performed by a third party
- Exhaustive testing is not possible. As we need the optimal amount of testing based on the risk assessment of the application.
- All the test to be conducted should be planned before implementing it
- It follows the Pareto Rule (80/20) which states that 80% of errors come from 20% of program components
- Start testing with small parts and extend it to large parts
Software Verification
Verification is the process of confirming if the software is meeting the business requirements and is developed adhering to the proper specifications and methodologies.
- Verification ensures the product being developed is according to design specifications
- Verification answers the question – “Are we developing this product by firmly following all design specifications?” – Are we building the product right?
- Verification concentrates on the design and system specifications.
Software Validation
Validation is process of examining whether or not the software satisfies the user requirements. It is carried out at the end of the SDLC. If the software matches requirements for which it was made, it is validated.
- Validation ensures the product under development is as per the user requirements.
- Validation answers the question – “Are we developing the product which attempts all that user needs from this software?” – Are we building the right product?
- Validation emphasizes on user requirements.
Difference between verification & Validation
| Verification | Validation |
| Static Testing | Dynamic Testing |
| It includes checking documents, design, codes and programs | It includes testing & validating the actual product |
| It does not include the execution of the code | It includes the execution of the code |
| Methods used in verification 1. Reviews, 2. Walkthrough, 3. Inspections, 4. document checking | Methods used in validation 1. Black Box Testing 2. White Box Testing 3. Non-Functional Testing |
| It can find the bugs in the early stage of the development | It can only find the bugs that could not found by the verification process |
| Specifications | User Requirements |
| Are we building the product right? | Are we building right product? |
Error/Mistake
These are actual coding mistakes made by developers. In addition there is a difference in output of software and desired output, is considered as an error. Error is a human action that produces an incorrect results.
Defect/Fault
A Defect is a deviation from the Requirements. A software defect is a condition in a software product which does not meet a software requirement ( as stated in the requirement specifications) or end-user expectations. In other words, a defect is an error in coding or logic that causes a program to malfunction or to produce incorrect/unexpected result. This could be hardware, software, network, performance, format or functionality.
Bug
A Bug is the result of a coding Error in the program which causes the program to behave in an unintended or unanticipated manner. It is an evidence of fault in the program. Bugs arise from mistakes and errors, made by people, in either a program’s source code or its design.
Failure
It is a deviation of the software from its intended purpose. It is the inability of a system or a component to perform its required functions within specified performance requirements. Failure occurs when fault executes.
Static Testing
Performed to check defects in software without actually executing the code
- early stage
- source of errors
- experienced person walkthrough
- peer review / checking doc of each other
- verification of document
Dynamic Testing
There are two types in Dynamic Testing
- Functional Testing
- Non-Functional Testing
Functional Testing
Functional Testing is a type of software testing that validates the software system against the functional requirements/specification
The purpose of Functional tests is to test each function of the software application, by providing appropriate input, verifying the output against the Functional Requirements
- Identify test input (test data)
- Compute the expected outcomes with the selected test input values
- Execute test cases
- Comparison of actual and computed expected result
There are Two types in Functional Testing
- White Box Testing
- Black Box Testing
Comparison of white-box testing and black-box testing
The following are some key distinctions between white box testing and black box testing:
| White Box Testing | Black Box Testing |
| Focus on Internal design of the code | Ignores Internal design of the code & it focus only on output |
| Also called Structural or Glass box Testing | Also called Functional Testing |
| For Verification | For Validation |
| The developers can perform white box testing. | The test engineers perform the black box testing. |
| It is necessary to comprehend programming languages in order to do WBT. | It is not necessary to comprehend programming languages in order to do BBT. |
| Examine the source code and the code’s logic. | Based on the required definition, we will check if the application is functional in this. |
Unit Testing
While coding, the programmer performs some tests on that unit of program to know if it is error free. Unit Testing is performed under white-box testing approach. Unit Testing helps developers decide that individual units of the program are working as per requirement and are error free.
- White Box/Glass-Box
- Transparent
- Code is visible
- Unit testing done by developer
Integration Testing
Even if the units of software are working fine individually, there is a need to find out if the units if integrated together would also work without errors.
For example, argument passing and data updating etc.,
Software Engineering defines variety of strategies to execute Integration testing viz.
- Big Bang Approach
- Incremental Approach – Which is further divided into the following
a) Top Down Approach
b) Bottom up approach
c) Sandwich Approach – Combination of Top down and Bottom up
Big Bang Testing
- It is an integration testing approach in which all the components or modules are integrated together at once and then tested as a unit.
- This combined set of components is considered as an entity while testing.
- If all of the components in the unit are not completed, the integration process will not execute
Advantages
- Convenient for Small Systems
Disadvantages
- Fault Localization is difficult
- Since the Integration testing can commence only after “all” the modules are designed, the testing team will have less time for execution in the testing phase.
- Since all modules are tested at once, high-risk critical modules are not isolated and tested on priority.
Incremental Testing
In the incremental testing approach, testing is done by integrating two or more modules that are logically related to each other and then tested for proper functioning of the applications. Then the other related modules are integrated incrementally and the process continues until all the logically related modules are integrated and tested successfully.
Incremental approach in turn is carried out by two different methods
- Bottom up Integration Testing
- Top Down Integration Testing
Bottom Up Integration Testing
Bottom-up Integration Testing is a strategy in which the lower level modules are tested first. These tested modules are then further used to facilitate the testing of higher level modules. The process continues until all modules at top level are tested. Once the lower level modules are tested and integrated, then the next level of modules are formed.
Advantages
- Fault Localization is easier
- No time is wasted waiting for all modules to be developed unlike Big-bang approach
Disadvantages
- Critical Modules (at the top level of software architecture) which control the flow of application are tested last and may be prone to defects
Top-down Integration Testing
Top down integration testing is a method in which integration testing takes place from top to bottom following the control flow of software systems. The higher level of modules are tested first and then lower level modules are tested and integrated in order to check the software functionality.
Stubs are used for testing if some modules are not ready.
Advantages
- Fault Localization is easier
- Possibility to obtain an early prototype
- Critical modules are tested on priority; major design flaws could be found and fixed first
Disadvantages
- Needs many Stubs
- Modules at a lower level are tested inadequately
Stubs and Drivers are the dummy programs in Integration testing used to facilitate the software testing activity.
- These programs act as a substitutes for the missing models in the testing
- They do not implement the entire programming logic of the software module but they simulate data communication with the calling module while testing.
Stub is called by the Modules under Test. (used in top-down)
Driver – Calls the Module to be tested( used in bottom-up)
Sandwich Testing
Sandwich Testing is a strategy in which top level modules are tested with lower level modules at the same time lower modules are integrated with top modules and tested as a system.
- It is a combination of Top-down and Bottom-up approaches therefore it is called Hybrid Integration Testing
- It makes use of both stubs as well as drivers.
System Testing
The software is compiled as product and then it is tested as a whole. This can be accomplished using one or more of the following Tests
Functionality Testing – This test all functionalities of the software against the requirements
Performance Testing – This test proves how efficient the software is. It tests the effectiveness and average time taken by the software to do desired task. Performance testing is done by means of load testing and stress testing where the software is put under high user and data load under various environment conditions.
Security & Portability – These test are done when the software is meant to work on various platforms and accessed by number of persons
Acceptance Testing
When the software is ready to hand over to the customer it has to go through last phase of testing where it is tested for user-interaction and response. This is important because even f the software matches all user requirements and if user does not like the way it appears or works, it may be rejected.
Alpha Testing – (Developer team – work env) The team of developer themselves performed alpha testing by using the system as if it is being used in work environment. They try to find out how user would react to some action in software and how the system should respond to inputs.
First phase of Testing
Beta Testing – After the software is tested internally, it is handed over to the users to use it under their production environment only for testing purpose. This is not as yet the delivered product. Developers expect that users at this stage will bring minute problems, which were skipped to attend.
- By end user
- aim is to cover unexpected errors
- cost, shipping, duration of Test is done
- Beta Testing is 2nd phase of Testing
- Beta Testing known as Field Testing
- Beta Testing is pre-released testing
- Aim is to place the software in the user hand & check for all levels
Levels of Testing
- Unit Testing – Done by Developers
- Integration Testing – Done by Testers
- System Testing – Done by Testers
- Acceptance Testing – Done by End Users
| Functional Testing | Non-Functional Testing |
| Functional Testing is performed using the functional specification provided by the client and verifies the system against the function requirements | Non-Functional Testing checks the Performance, reliability, scalability and other non-functional aspects of the software system |
| Functional Testing is executed first | Non-Functional Testing should be performed after functional testing |
| Business requirements are the inputs to functional testing | Performance parameters like speed, scalability are inputs to non-functional testing |
| Functional Testing describes what the product does | Non-Functional testing describes how good the product works |
| Easy to do Manual Testing | Tough to do Manual Testing |
What is Basis Path Testing?
Basis Path Testing in software engineering is a White Box Testing method in which test cases are defined based on flows or logical paths that can be taken through the program. The objective of basis path testing is to define the number of independent paths, so the number of test cases needed can be defined explicitly to maximize test coverage.
In software engineering, Basis path testing involves execution of all possible blocks in a program and achieves maximum path coverage with the least number of test cases. It is a hybrid method of branch testing and path testing methods.
Advantages of Basic Path Testing
- It helps to reduce the redundant tests
- It focuses attention on program logic
- It helps facilitates analytical versus arbitrary case design
- Test cases which exercise basis set will execute every statement in a program at least once
What is Control structure testing?
Control structure testing is used to increase the coverage area by testing various control structures present in the program. The different types of testing performed under control structure testing are as follows-
- Condition Testing
- Data Flow Testing
- Loop Testing
Test Case
Deriving Test Cases
The test case is defined as a group of conditions under which a tester determines whether a software application is working as per the customer’s requirements or not. Test case designing includes preconditions, case name, input conditions, and expected result. A test case is a first level action and derived from test scenarios.
What is Regression Testing?
Regression testing is a black box testing techniques. Regression Testing is done after modification of system. It is used to authenticate a code change in the software does not impact the existing functionality of the product. Regression testing is making sure that the product works fine with new functionality, bug
fixes, or any change in the existing feature.
What is Stress Testing?
Stress Testing is testing used to check the accessibility and robustness of software beyond usual functional limits. It mainly considers for critical software but it can also be used for all types of software applications.
- It is also known as Endurance Testing, fatigue testing or Torture Testing.
- The stress testing includes the testing beyond standard operational size, repeatedly to a breaking point, to get the outputs.
- It highlights the error handling and robustness under a heavy load instead of correct behaviour under regular conditions.
- To analyses how the system works under extreme conditions, we perform stress testing outside the normal load.
- The primary purpose of executing the stress testing is to confirm that the software does not crash in lacking computational resources like disk space, memory, and network request.
- We can use stress testing to discover hardware issues, data corruption issues.
What is performance testing?
It is the most important part of non-functional testing. Checking the behaviour of an application by applying some load is known as performance testing. Generally, this testing defines how quickly the server responds to the user’s request.
Types of Performance Testing
Following are the types of performance testing:
- Load testing
- Stress testing
- Scalability testing
- Stability testing