Testing School-How to Build the Plan

How to Build the Plan

1. Analyze the product.

• What to Analyze
• Users (who they are and what they do)
• Operations (what it’s used for)
• Product Structure (code, files, etc.)
• Product Functions (what it does)
• Product Data (input, output, states, etc.)
• Platforms (external hardware and software)
• Ways to Analyze
• Perform product/prototype walkthrough.
• Review product and project documentation.
• Interview designers and users.
• Compare w/similar products.
• Possible Work Products
• Product coverage outline
• Annotated specifications
• Product Issue list
• Status Check
• Do designers approve of the product coverage outline?
• Do designers think you understand the product?
• Can you visualize the product and predict behavior?
• Are you able to produce test data (input and results)?
• Can you configure and operate the product?
• Do you understand how the product will be used?
• Are you aware of gaps or inconsistencies in the design?
• Do you have remaining questions regarding the product?

2. Analyze product risk.

• What to Analyze
• Threats
• Product vulnerabilities
• Failure modes
• Victim impact
• Ways to Analyze
• Review requirements and specifications.
• Review problem occurrences.
• Interview designers and users.
• Review product against risk heuristics and quality criteria categories.
• Identify general fault/failure patterns.
• Possible Work Products
• Component risk matrices
• Failure mode outline
• Status Check
• Do the designers and users concur with the risk analysis?
• Will you be able to detect all significant kinds of problems, should they occur during testing?
• Do you know where to focus testing effort for maximum effectiveness?
• Can the designers do anything to make important problems easier to detect, or less likely to occur?
• How will you discover if your risk analysis is accurate?

3. Design test strategies.

• General Strategies
• Domain testing (including boundaries)
• User testing
• Stress testing
• Regression testing
• Sequence testing
• State testing
• Specification-based testing
• Structural testing (e.g. unit testing)
• 
  
• Ways to Plan
• Match strategies to risks and product areas.
• Visualize specific and practical strategies.
• Look for automation opportunities.
• Prototype test probes and harnesses.
• Don’t over plan. Let testers use their brains.
• Possible Work Products
• Itemized statement of each test strategy chosen and how it will be applied.
• Risk/task matrix.
• List of issues or challenges inherent in the chosen strategies.
• Advisory of poorly covered parts of the product.
• Test cases (if required)
• Status Check
• Do designers concur with the test strategy?
• Has the strategy made use of every available resource and helper?
• Is the test strategy too generic could it just as easily apply to any product?
• Will the strategy reveal all important problems?

4. Plan logistics.

• Logistical Areas
• Test effort estimation and scheduling
• Testability engineering
• Test team staffing (right skills)
• Tester training and supervision
• Tester task assignments
• Product information gathering and management
• Project meetings, communication, and coordination
• Relations with all other project functions, including development
• Test platform acquisition and configuration
• Possible Work Products
• Issues list
• Project risk analysis
• Responsibility matrix
• Test schedule
• Agreements and protocols
• Test tools and automation
• Stubbing and simulation needs
• Test suite management and maintenance
• Build and transmittal protocol
• Test cycle administration
• Problem reporting system and protocol
• Test status reporting protocol
• Code freeze and incremental testing
• Pressure management in end game
• Sign-off protocol
• Evaluation of test effectiveness
• Status Check
• Do the logistics of the project support the test strategy?
• Are there any problems that block testing?
• Are the logistics and strategy adaptable in the face of foreseeable problems?
• Can you start testing now and sort out the rest of the issues later?

5. Share the plan.

• Ways to Share
• Engage designers and stakeholders in the test planning process.
• Actively solicit opinions about the test plan.
• Do everything possible to help the developers succeed.
• Help the developers understand how what they do impacts testing.
• Talk to technical writers and technical support people about sharing quality information.
• Get designers and developers to review and approve all reference materials.
• Record and reinforce agreements.
• Get people to review the plan in pieces.
• Improve review ability by minimizing unnecessary text in test plan documents.
• Goals
• Common understanding of the test process.
• Common commitment to the test process.
• Reasonable participation in the test process.
• Management has reasonable expectations about the test process.
• Status Check
• Is the project team paying attention to the test plan?
• Does the project team, especially first line management, understand the role of the test team?
• Does the project team feel that the test team has the best interests of the project at heart?
• Is there an adversarial or constructive relationship between the test team and the rest of the project?
• Does any member of the project team feel that the testers are “off on a tangent” rather than focused on important testing tasks?