Work Order: Test Enhancement Audit

This work order provides a structured process for auditing and improving the test coverage for a specific area of functionality within a Pigweed module. It focuses on a deep analysis of the existing code and tests to create a targeted plan for enhancement.

Scaffolding Documents

<module>/test_audit_design.md: A document to analyze the current test coverage and design the new tests.
<module>/tasks.md: A task list to track the implementation of the new tests.

`test_audit_design.md` Skeleton

# Test Enhancement Audit: [Module Name] - [Feature Area]

## 1. Background

- **1.1. Motivation:** Why is this audit being performed? (e.g., general
  hardening, recent bug, proactive quality improvement). Link to a bug report if
  applicable (e.g., `Bug: b/123456789`).
- **1.2. Scope:** A brief description of the feature area being audited.

## 2. Scoping & Sizing

- **2.1. Estimated Size:** A rough estimate of the effort required (e.g., S, M,
  L, XL, or person-days).
- **2.2. Key Risks:** What are the biggest risks or unknowns in this audit?

## 3. Current Functionality Deep Dive

_This section is critical for building a shared understanding of the existing
system. It must be completed before analyzing the tests._

- **3.1. System Overview:** A detailed, narrative description of how the
  relevant parts of the system work today. This should be based on a thorough
  reading of the code and not on assumptions.
- **3.2. Key Code Components:**
  - A list of the key classes, functions, and data structures.
  - For each component, a brief description of its role and key parameters.
- **3.3. Files Read:** A list of all files that were read to build this
  understanding.

## 4. Test Coverage Analysis

_With a deep understanding of the code, now analyze the existing tests._

- **4.1. Existing Tests:**
  - A list of the test files and specific test cases that cover this
    functionality.
- **4.2. Coverage Gaps:**
  - A narrative description of the current test coverage, based on a thorough
    review of the code and tests.
  - **Testing Levels:** An analysis of unit, integration, and language-specific
    (e.g., C++ vs. Python) test coverage.
  - What scenarios are well-tested?
  - What are the gaps? (e.g., edge cases, error conditions)

## 5. Proposed New Tests

- **5.1. Test Plan:** A high-level plan for the new tests to be added to fill
  the identified gaps.
- **5.2. Detailed Test Cases:** A list of the new test cases to be written,
  including:
  - A description of the scenario being tested.
  - The expected outcome.

## 6. Blind Alleys

- A section to document any implementation attempts that were abandoned. For
  each blind alley, describe the approach, why it failed, and what was learned.

## 7. Testing Strategy

- How will the new tests be implemented? (e.g., new test file, adding to
  existing tests)

`tasks.md` Skeleton

# Tasks: Test Enhancement Audit - [Module Name] - [Feature Area]

**Work order:** test_enhancement_audit: [Module Name] - [Feature Area]

- [ ] **Setup:** Scaffolding documents created.
- [ ] **Planning:**
  - [ ] `test_audit_design.md` complete.
    - [ ] Scoping & Sizing complete.
    - [ ] Current Functionality Deep Dive complete.
    - [ ] Test Coverage Analysis complete.
    - [ ] Proposed New Tests complete.
  - [ ] `tasks.md` complete.
- [ ] **Critic:**
  - [ ] Stage 1: Initial Review complete.
  - [ ] Stage 2: Deep Expert Review complete.
- [ ] **Implementation:**
  - [ ] Test Case 1...
  - [ ] Test Case 2...
- [ ] **Verification:**
  - [ ] Code review complete.
  - [ ] All tests passing.
- [ ] **Work order review (Optional):**
  - [ ] Stage 1: Initial Feedback complete.
  - [ ] Stage 2: Deep Analysis complete.
- [ ] **Integration & Cleanup:**
  - [ ] Final documentation review complete.
  - [ ] Scaffolding files deleted.
  - [ ] Final commit ready.

Phases

Setup Phase

Determine the target module (e.g., pw_string) and the specific feature area to audit.
Create test_audit_design.md and tasks.md in the module directory using the skeletons above.

Planning Phase

This phase is iterative. The “Deep Dive” and “Test Coverage Analysis” sections of the design document should be developed in tandem, as understanding of the existing system informs the test analysis, and the test analysis may prompt further investigation into the code.

Crucially, complete the “Current Functionality Deep Dive” section of test_audit_design.md before proceeding. This requires a thorough review of the implementation.
With a deep understanding of the code, analyze the existing tests and identify gaps.
Iterate on the “Proposed New Tests” section until the user is satisfied with the plan.
Create a detailed task list in tasks.md for implementing the new tests.
Flush the context to the scaffolding files.

Critic Phase

Stage 1: Initial Review: Adopt the persona of a skeptical, expert principal software engineer. Thoroughly review the test_audit_design.md and all referenced code. Poke holes in the plan. Look for flawed assumptions, missing edge cases, and potential negative interactions with other parts of the codebase.
Stage 2: Deep Expert Review: Based on the findings from the initial review, conduct a second, more intensive critique. Challenge every assumption.
Document all findings and discuss them with the user before proceeding.
Flush the context to the scaffolding files.

Implementation Phase

Identify the smallest verifiable test case to add from the list.
As you implement, your understanding of the code and tests will evolve. Update the “Deep Dive” and “Test Coverage Analysis” sections of test_audit_design.md to reflect new insights.
Implement the test case, ensuring it passes and that existing tests continue to pass.
Mark the task as complete in tasks.md.

Verification Phase

Review the test_audit_design.md and the implemented tests to ensure they are aligned.
Ensure that all new and existing tests pass.
Confirm with the user that the test coverage is now sufficient for the audited area.

Integration & Cleanup Phase

Ensure any relevant external documentation (e.g., module READMEs) is updated to reflect the improved testing.
Delete the scaffolding files (test_audit_design.md, tasks.md).
Prepare the final commit.