Testing
The testing infrastructure for autoswe is built on pytest and is designed to validate both the operational integrity of the deployment pipeline and the data consistency of the generated site. The configuration ensures that internal scripts are importable during test execution, while the test suite itself is divided into integration tests for the acceptance wrapper and validation tests for the accuracy check module. These tests utilize synthetic environments, including temporary Git repositories and stubbed sandbox scripts, to verify correct behavior under happy paths, failure conditions, and edge cases without requiring external dependencies.
Pytest Configuration
Section titled “Pytest Configuration”The repository uses a conftest.py file to manage the Python path for tests. This configuration ensures that the journey_server script, located in the scripts directory, is accessible to test files via sys.path. This allows tests to directly import and interact with the server logic.
The configuration resolves the repository root relative to the conftest.py file and inserts the scripts path at the beginning of sys.path if it is not already present. This setup is critical for tests that need to import modules from the scripts directory, such as the accuracy_check module.
Acceptance Wrapper Integration Tests
Section titled “Acceptance Wrapper Integration Tests”Happy Path and Sandbox Validation
Section titled “Happy Path and Sandbox Validation”
Precondition Checks and Idempotency
Section titled “Precondition Checks and Idempotency”
Accuracy Check Validation Tests
Section titled “Accuracy Check Validation Tests”The tests/test_accuracy_check.py file contains tests for the accuracy_check module, which recomputes site headline numbers from raw sources to flag mismatches. These tests use synthetic site data to validate the integrity of iteration counts, slug uniqueness, and epoch prefixing.
Source-Independent Checks
Section titled “Source-Independent Checks”The tests exercise checks that do not depend on external sources, such as verifying that the number of accepted iterations matches the count of iterations with an ACCEPT outcome. test_consistent_iters_pass provides valid site data and asserts that all checks pass, including totals.itersAccepted, totals.itersAttempted, iter slugs unique, and iter slugs epoch-prefixed.
Error Detection
Section titled “Error Detection”test_wrong_accepted_count_flagged tests the detection of mismatched counts by providing site data where the accepted count does not match the actual number of accepted iterations in the iters list. The test asserts that the totals.itersAccepted check fails.
test_duplicate_or_unprefixed_slugs_flagged tests the validation of iteration slugs. It provides data with duplicate slugs and slugs that are not epoch-prefixed. The test asserts that the iter slugs unique and iter slugs epoch-prefixed checks fail, ensuring that the accuracy check correctly identifies these structural issues in the site data.