Review Tests

Review Tests#

Systematic review of test code for gaps, weak assertions, cheating or circular tests, and end-to-end data independence.

Description#

The review-tests skill audits test files for common quality problems: gaps in coverage, weak or meaningless assertions, circular tests that reuse production logic to compute expected values, and end-to-end tests that lack independent comparison data. It produces a prioritised, actionable list of findings without modifying any files.

Triggers#

The skill activates when you mention:

  • Reviewing tests for gaps or cheating

  • Auditing test quality

  • Checking test independence

Explicit invocation:

/wf:review-tests

Workflow#

  1. Confirm scope

    • Use paths provided by the user, or search for test files in a given root directory.

    • If scope is missing or ambiguous, ask for explicit paths.

    • Does not require a Git repository.

  2. Gather test files

    • Search within the provided scope for common test patterns: tests/, test_*.py, *_test.*, *.spec.*, __tests__, spec/.

  3. Identify code under test

    • For each test file, list the production modules, functions, and helpers it exercises.

    • Build a map of test to production code.

  4. Cheating / circular test checks

    • Flag tests that compute expected outputs using the same production logic under test.

    • Flag tests that import production helpers to build expected values unless the helper does not overlap with the behaviour under test and is independently tested elsewhere.

  5. Coverage gaps and weak assertions

    • Check boundary values, malformed inputs, and error paths.

    • Flag assertions that only verify “no error” without checking meaningful outcomes.

    • Flag over-mocking that bypasses core logic.

    • Note nondeterminism (time, randomness, filesystem, network) without controls.

  6. End-to-end tests

    • Verify E2E tests use independent comparison data.

    • Flag data produced by production code without justification.

    • Suggest concrete missing E2E scenarios for critical workflows.

  7. Output

    • Prioritised, actionable list of findings.

    • Each finding includes Severity, File:Line, Problem, Suggestion.

    • No files are modified.

Output Format#

Findings are presented directly in the conversation as a prioritised list:

## Test Review Findings

| # | Severity | File:Line | Problem | Suggestion |
|---|----------|-----------|---------|------------|
| 1 | High | tests/test_parser.py:42 | Expected value computed by `parse()` itself (circular) | Use a hand-crafted expected dict |
| 2 | Medium | tests/test_utils.py:18 | Only asserts no exception raised | Assert return value matches expected output |
| 3 | Low | tests/test_cli.py:91 | No test for missing input file | Add parametrised error-path case |

Example Usage#

Review tests in a specific directory:

> /wf:review-tests audit tests/ for gaps and cheating

Review specific test files:

> /wf:review-tests check tests/test_parser.py and tests/test_utils.py

Review tests for a non-Git project:

> /wf:review-tests review ~/scripts/tests/

Cheating Detection#

The skill specifically targets a common anti-pattern: tests that use the same code path to generate both the actual and expected values. For example:

# BAD: circular test - uses production function for expected value
def test_transform():
    result = transform(input_data)
    expected = transform(input_data)  # same function!
    assert result == expected

# GOOD: independent expected value
def test_transform():
    result = transform(input_data)
    expected = {"key": "manually computed value"}
    assert result == expected

The skill also checks that any production helpers used in test setup are independently tested elsewhere in the test suite.

Limits#

  • Read-only: The skill does not modify any files.

  • Scope required: The user must provide file or directory paths.

  • No Git requirement: Works on any directory, not just repositories.

Contents