Skip to content

pytest-park

Build Documentation PyPI - Package Version PyPI - Python Version Docs with MkDocs uv linting: ruff ty prek security: bandit Semantic Versions Copier License

Organise and analyse your pytest benchmarks

Features

  • Load pytest-benchmark JSON artifact folders and normalize runs, groups, marks, params, and custom grouping metadata.
  • Compare reference runs against candidate runs over time with per-case and per-group delta summaries.
  • Build custom grouping views with precedence across custom groups, benchmark groups, marks, and params.
  • Associate optional profiler artifacts with benchmark runs for code-level analysis context.
  • Serve an interactive local NiceGUI dashboard for exploratory benchmark comparison.

Installation

With pip: 

python -m pip install pytest-park

With uv: 

uv add --group test pytest-park

How to use it

# Print version
pytest-park version

# Start interactive mode (no arguments)
pytest-park

# Inspect a benchmark folder
pytest-park load ./benchmarks

# Analyze grouping distribution
pytest-park analyze ./benchmarks --group-by group --group-by param:device

# Compare a candidate run against a named reference tag/run id
pytest-park compare ./benchmarks --reference reference --candidate candidate-v2 --group-by custom:scenario

# Compare latest run against second-latest run when --reference/--candidate are omitted
# By default, this will compare and print details for all methods.
pytest-park compare ./benchmarks

# Compare a specific method only
pytest-park compare ./benchmarks --method test_func1

# Normalize method names by removing configured postfixes
pytest-park analyze ./benchmarks --original-postfix _orig --reference-postfix _ref

# Launch interactive dashboard
pytest-park serve ./benchmarks --reference reference --original-postfix _orig --reference-postfix _ref --host 127.0.0.1 --port 8080

Benchmark folder expectations

  • Input artifacts are pytest-benchmark JSON files (--benchmark-save output) stored anywhere under a folder.
  • Reference selection uses explicit run id or tag metadata (metadata.run_id, metadata.tag, or fallback identifiers).
  • Default comparison baseline is latest vs second-latest run when --reference and --candidate are omitted.
  • Grouping defaults to: custom groups > benchmark group > marks > params.
  • Custom grouping tokens include custom:<key>, group, marks, params, and param:<name>.
  • Method normalization supports optional --original-postfix and --reference-postfix to align benchmark names across implementations.

pytest-benchmark group stats override

If your benchmark method names encode postfixes and parameter segments, you can override pytest_benchmark_group_stats using the helper from this package:

# tests/conftest.py
from pytest_park.pytest_benchmark import default_pytest_benchmark_group_stats


def pytest_benchmark_group_stats(config, benchmarks, group_by):
    return default_pytest_benchmark_group_stats(
        config,
        benchmarks,
        group_by,
        original_postfix="_orig",
        reference_postfix="_ref",
        group_values_by_postfix={
            "_orig": "original",
            "_ref": "reference",
            "none": "unlabeled",
        },
    )

This stores parsed parts in extra_info["pytest_park_name_parts"] with base_name, parameters, and postfix.

If you use postfixes in benchmark names, expose matching pytest-benchmark options in the same conftest.py:

def pytest_addoption(parser):
    parser.addoption("--benchmark-original-postfix", action="store", default="")
    parser.addoption("--benchmark-reference-postfix", action="store", default="")

Docs

uv run mkdocs build -f ./mkdocs.yml -d ./_build/

Update template

copier update --trust -A --vcs-ref=HEAD

Credits

This project was generated with 🚀 python project template.