IEEE Std 829 Software Test Plan (IEEE Standard for Software and System Test Documentation)

This article will examine the Test Plan in detail. We shall refer mainly to the IEEE Std 829-2008 (IEEE Standard for Software and System Test Documentation) in this article.

1. Test Plan definition

The IEEE Std 829 defines a Test Plan as, "(A) A document describing the scope, approach, resources, and schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning. (B) A document that describes the technical and management approach to be followed for testing a system or component. Typical contents identify the items to be tested, tasks to be performed, responsibilities, schedules, and required resources for the testing activity. (adopted from IEEE Std 610.12-1990 [B2])"

2. Why is a Test Plan required ?

The Test Plan is the fundamental document for testing. The purpose of the Master Test Plan, as stated by the IEEE Std 829 is to provide an overall test planning and test management document for multiple levels of test (either within one project or across multiple projects).

The Test Plan is a medium of communication, used by the testing team to communicate to the other project participants about the team's intent, expectations and understanding of the test activity to be performed. It is important that all the stakeholders of the project review and sign-off on the test plan. The Test Plan helps the testing team to set the right expectations about the intended test activities. It makes for sound business sense to have the review and sign-off happen before starting the test campaign. This will help in avoiding any mis-understanding around the testing effort as well as protect the testing team from potential complaints about the testing performed later in the project.

The Test Plan is a product of the test planning process. The Test Plan may be referred to both as a map and a blueprint for testing. It is a comprehensive document that offers clarity to all the project's stakeholders about testing. It addresses the how, what, when, who, where and why of testing and helps the testing group to focus.

3. What constitutes a Test Plan ?

The IEEE Std 829 provides an outline of the Master Test Plan (MTP). MTP involves selecting the constituent parts of the project's test effort, setting the objectives for each part, setting the division of labor in terms of time and resources, and the interrelationships between the parts, identifying the risks, assumptions, and standards of workmanship to be considered and accounted for by the parts, defining the test effort's controls and confirming the applicable objectives set by quality assurance planning. It also identifies the number of levels of test, the overall tasks to be performed and the documentation requirements.

Master Test Plan outline

1. Introduction

This section identifies the document and describes the entire test effort, including the test organization, test schedule and the system characteristics (such as complexity, risk, safety level, security level, desired performance, reliability, and/or cost) selected as important to stakeholders, and arranged into discrete levels of performance or compliance, to help define the level of quality control to be applied in developing and/or delivering the software. A summary of required resources, responsibilities, tools and techniques may also be included here.

Identifier: This uniquely identifies the version of the document by including information such as the date of issue, author(s), approver signatures, status (e.g. draft, reviewed, corrected or final), reviewers and managers. This information may be placed either at the beginning or end of the document.

Scope: This describes the purpose, goals and scope of the test effort. Identify the project(s) for which this plan is written and the specific processes and products covered by the test effort. Describe the inclusions, exclusions, assumptions and limitations. Test tasks will reflect the overall test approach and the development methodology. For example, if the development is based on the waterfall methodology, then each level of testing will be executed once. However, if the development is based on an iterative methodology, then there will be multiple iterations of each level of testing. The test approach identifies what will be tested and in what order for the gamut of testing levels such as component, component integration, system and acceptance. The test approach identifies the rationale for testing or not testing as also the selected order of testing. The test approach may also identify the type of testing performed at the different levels listed earlier.

References: This lists all applicable reference documents, both internal and external.

System overview and key features: This describes the purpose and key features of the system or software product under test or reference where the information can be found.

Test overview: This describes the test organization, test schedule, system characteristics (such as complexity, risk, safety level, security level, desired performance, reliability, and/or cost) selected as important to stakeholders, test resources, responsibilities, tools, techniques and methods necessary to perform testing.

While describing the test organization, an organization chart may be included to clarify the reporting structure. Include information on the authority for resolving issues raised by testing, and the authority for approving test products and processes.

The test schedule describes the test activities within the project lifecycle and milestones. Summarize the overall schedule of testing tasks, identifying where the task results feed back to the development and related process such as quality assurance and configuration management. Also, describe the task iteration policy for re-execution of test tasks and any dependencies.

The test resources requirement should be summarized to include, staffing, facilities, tools and any special procedural requirements such as security, access rights, etc.

Include information on the responsibilities for testing tasks. Responsibilities may be primary (task owner / leader) or secondary (providing support) in relation to specific test related responsibilities.

Describe the hardware, software, test tools, techniques, methods and test environment to be used in testing. Include information pertaining to acquisition, training and support for each tool, technology and method. Include the metrics to be used by the testing effort.

2. Details

This section describes the test processes, test documentation and test reporting requirements.

Test processes and definition of test levels: Describe the test activities and tasks for all development lifecycle processes. List the number and sequence of levels of test. Levels of test may include component, integration, system, acceptance, security, usability, performance, stress, interoperability, regression, etc. Not all projects will have all the levels of test. Some projects may have fewer levels of test and could combine multiple levels. The test processes may either be described here or reference to already defined standards may be provided.

In addition, the IEEE Std 829 recommends that for each test activity, the following topics be addressed.
  • Test tasks: Identify the test tasks
  • Methods: Describe the methods and procedures for each test task, including tools. Also, define the criteria for evaluating the test task results
  • Inputs: Identify the required inputs for the test task. Specify the source of each input. Inputs may be derived from preceding tasks or activities
  • Outputs: Identify the required outputs from the test task
  • Schedule: Describe the schedule for the test tasks. Establish specific milestones for initiating and completing each task, for obtaining input and for delivery of output
  • Resources: Identify the resources for the performance of the test tasks. Example of resources include people, tools, equipment, facilities, budgets, etc.
  • Risks and Assumptions: Identify any risks and assumptions associated with the test tasks. Include recommendations to eliminate, reduce or mitigate risks identified
  • Roles and responsibilities: Identify for each task, who has the primary and secondary responsibilities for task execution and the nature of the roles they will play
Test documentation requirements: Here, define the purpose, format and content of all other testing documents that are to be used (in addition to those that are defined in the "Test reporting requirements" section)

Test administration requirements: These are needed to administer tests during execution and involve describing the following.
  • Anomaly resolution and reporting process: Describe the method of reporting and resolving anomalies. This would include information about the anomaly criticality levels, authority and time line for resolution.
  • Task iteration policy: Describe the criteria for repeating testing tasks when its input is changed or task procedure is changed. Example, re-executing tests after anomalies have been fixed.
  • Deviation policy: Describe the procedures and criteria for deviation from the MTP and test documentation. The information for deviations includes task identification, rationale and effect on product quality. Also, identify the authorities responsible for approving deviations.
  • Control procedures: Identify control procedures for test activities. These procedures describe how the system, software products and test results will be configured, protected and stored. They may also describe quality assurance, configuration management, data management, compliance with existing security provisions and how test results are to be protected from unauthorized alterations.
  • Standards, practices and conventions: Identify the standards, practices and conventions that govern the performance of testing tasks.
Test reporting requirements: This specifies the purpose, content, format, recipients and timing of all test reports. Test reporting includes test logs, anomaly reports, level interim test status reports, level test reports, master test report and any optional reports defined during preparation of the test plan.

3. General

This section includes the glossary of terms, acronyms, description of the frequency and process by which the master test plan is changed and base-lined and may also contain a change page mentioning the history of changes (date, reason for change and who initiated the change).

A result of the test planning process must be a clear and agreed upon definition of the product's quality and reliability goals in absolute terms. There must be no room for subjective interpretation. Everyone on the project team must know what the testing team intends to do and the quality goals. Test planning is not about filling in a template or writing a document. Test planning is an important activity that must involve testers and representatives from all functions part of the project team. Getting everyone on the same page and on agreement regarding what is to be tested, why it is to be tested and how it is to be tested is key to testing success.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.