Software life cycle testing means that testing occurs in parallel with the development cycle and is a continuous process. The software testing process should start early in the application life cycle, not just in the traditional validation testing phase after the coding phase has been completed. Testing should be integrated into application development. In order to do so, there needs to be a commitment on the part of the development organization and close communication with the quality assurance function.

A test plan is started during the requirements phase. It is an organization of testing work. It is a document describing the approach to be taken for the intended testing activities and includes the items to be tested, the types of tests to be performed, test schedules, human resources, reporting procedures, evaluation criteria, etc.

During logical, physical, and program unit design, the test plan is refined with more details. Test cases are also created. A test case is a specific set of test data and test scripts. A test script guides the tester through a test and ensures consistency among separate executions of the test. A test also includes the expected results to verify whether the test met the objective correctly. During the coding phase, test scripts and test data are generated. During application testing, the test scripts are executed and the results are analyzed.

The application development cycle proceeds from user requirements and design until the code is completed. During test design and development, the acceptance test criteria was established in a test plan. As more details are refined, the system, integration, and unit testing requirements are established. There may or may not be a separate test plan for each test type, or one plan may be used.

During test execution, the process is reversed and starts with unit testing. Integration tests are performed which combine individual, unit tested pieces of code. Once this is completed, the system is tested from a total system point of view. This is known as system testing. System testing is a multifaceted test to evaluate the functionality, performance, and usability of the system. The final test is the acceptance test, which is a user-run test that verifies the ability of the system to meet the original user objectives and requirements. In some cases the system test serves as the acceptance test.

If you will recall, the PDCA approach, i.e., plan, do, check, and act, is a control mechanism used to control, supervise, govern, regulate, or restrain a system. The approach first defines the objectives of a process, develops and carries out the plan to meet those objectives, and checks to determine if the anticipated results are achieved. If they are not achieved, the plan is modified to fulfill the objectives. The PDCA quality cycle can be applied to software testing.

The Plan step of the continuous improvement process, when applied to software testing, starts with a definition of the test objectives, e.g., what is to be accomplished as a result of testing. Testing criteria do more than simply ensure that the software performs according to specifications. Objectives ensure that all responsible individuals contribute to the definition of the test criteria to maximize quality.

A major deliverable of this step is a software test plan. A test plan is the basis for accomplishing testing. The test plan should be considered an ongoing document. As the system changes, so does the plan. The test plan also becomes part of the system maintenance documentation after the application is delivered to the user. The outline of a good test plan includes an introduction, the overall plan, and testing requirements. As more detail is available, the business functions, test logs, problem and summary reports, test software, hardware, data, personnel requirements, test schedule, test entry criteria, and exit criteria are added.

The Do step of the continuous improvement process when applied to software testing describes how to design and execute the tests included in the test plan. The test design includes test cases, test procedures and scripts, expected results, function/test case matrix, test logs, etc. The more definitive a test plan is, the easier the test design will be. If the system changes between development of the test plan and when the tests are to be executed, the test plan should be updated accordingly, i.e., whenever the system changes, the test plan should change.

The test team is responsible for the execution of the tests and must ensure that the test is executed according to the plan. Elements of the Do step include selecting test tools, defining the resource requirements, defining the test setup conditions and environment, test requirements, and the actual testing of the application.

The Check step of the continuous improvement process when applied to software testing includes the evaluation of how the testing process is progressing. The credo for statisticians, “In God we trust. All others must use data,” is crucial to the Deming method. It is important to base decisions as much as possible on accurate and timely data. Testing metrics such as the number and types of defects, the workload effort, and the schedule status are key.

It is also important to create test reports. Testing began with setting objectives, identifying functions, selecting tests to validate the test functions, creating test conditions, and executing the tests. To construct test reports, the test team must formally record the results and relate them to the test plan and system objectives. In this sense, the test report reverses all the previous testing tasks.

Summary and interim test reports should be written at the end of testing and at key testing checkpoints. The process used for report writing is the same whether it is an interim or a summary report, and, like other tasks in testing, report writing is also subject to quality control, i.e., it should be reviewed. A test report should at least include a record of defects discovered, data reduction techniques, root cause analysis, the development of findings, and recommendations to management to improve the testing process.

The Act step of the continuous improvement process when applied to software testing includes devising measures for appropriate actions relating to work that was not performed according to the plan or results that were not anticipated in the plan. This analysis is fed back to the plan. Examples include updating the test suites, test cases, test scripts, reevaluating the people, process, and technology dimensions of testing.


Also See:

• Spiral Testing Approach