Focus: session 3 of module 5; test progress monitoring and control.
Key Learnings:
Techniques and metrics used for monitoring test preparation and execution.
Test reporting using test matrix.
Main Topics: Monitoring progress of test activities, reporting test status, controlling testing activities when issues arise.
Monitoring Test Activities
Importance of Tracking: Essential to track execution and progress in all phases of the software testing life cycle.
Test Execution: Collecting execution data such as tests passed, defects raised/resolved; reporting these to stakeholders.
Test Monitoring: A test management task tracking the test’s progress against planned outcomes.
Reports: Generated to compare actual results with planned results, finding deviations.
Metrics for Test Progress Monitoring
Common metrics:
Number of test cases prepared/run.
Number of test cases passed/failed.
Number of defects (open, closed, in-progress).
Metrics Collection: Gathering data during test execution phase, combining them for stakeholder reports.
Techniques Variation: Depends on stakeholder preferences, project goals, regulatory requirements.
Purposes of Test Monitoring
Feedback: Provides feedback to test team and managers on progress.
Visibility: Offers project teams visibility on test results.
Status Measurement: Confirms test execution against exit criteria defined in test plan.
Data for Future Estimations: Gathers data for more accurate future test estimations.
Common Metrics Examples
Total test cases: prepared, run, passed, failed.
Total number of incidents raised, fixed.
Planned vs actual effort for executing test cases.
Test environment completion percentage.
Extent of test coverage (requirements, risks, code, etc.).
Status of various testing activities (analysis, design, implementation).
Testers' confidence levels on product quality.
Reporting Test Status
Effective Communication: Communicating test monitoring findings to stakeholders.
Understanding Results: Helps stakeholders understand test cycle results and exit criteria fulfillment.
Release Decisions: Based on test status reports, stakeholders make decisions about product release.
IEEE 829 Standard Test Summary Report Template
Includes: Test report identifier, summary of testing, project/product overview, variances, assessment, summary of results, evaluation, summary of activities, approval.
Adaptation: Organizations may modify this template to suit their needs; can vary from detailed to one-page summary reports.
E.g., simple summary includes total/passed/failed test cases, number of defects, exit criteria status.
Risk-Based Testing Example
Importance: Focuses on high-risk areas and ensures appropriate test coverage.
Template Example: Functionalities categorized by risk, planned vs actual test cases, percentage of test cases to be executed, outstanding defects.
Test Control
Purpose: Bringing project under control if deviations occur.
Corrective Actions: Taken to get the project back on track in case of issues.
Examples:
Rescheduling performance tests based on user activity (e.g., from weekdays to weekends).
Defects verification by developers before retesting to save time/effort.
Re-prioritizing test execution based on module readiness.
Adding more resources to meet tight deadlines.
Summary
Discussed test progress monitoring and control activities.
Covered monitoring, reporting, and controlling testing activities.
Introduced IEEE 829 standard template for reporting test status.
Control activities help manage deviations effectively.
Conclusion
Essential for successful test management and delivery.
Ensures visibility, accurate estimation, and informed decision-making.