calibrax.ci¤
CI pipeline tools for regression gating and automated bisection.
CI Guard¤
CIGuard compares the latest run against the stored baseline and returns a
GuardResult indicating pass/fail. Use with the CLI calibrax check command
for CI exit-code signaling.
calibrax.ci.guard
¤
CI regression gate for automated performance checks.
Compares the latest (or specified) run against the stored baseline and flags regressions that exceed a configured threshold.
GuardResult(*, passed, regressions, threshold, baseline_id, current_id)
dataclass
¤
Result of a CI regression check.
Attributes:
| Name | Type | Description |
|---|---|---|
passed |
bool
|
True if no regressions were detected. |
regressions |
tuple[Regression, ...]
|
Detected regressions (empty tuple if passed). |
threshold |
float
|
Regression threshold used for the check. |
baseline_id |
str | None
|
ID of the baseline run used for comparison. |
current_id |
str
|
ID of the run being checked. |
to_dict()
¤
Serialize to a JSON-compatible dictionary.
CIGuard(store, threshold=0.05)
¤
Regression gate that compares runs against a stored baseline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
store
|
Store
|
Storage backend containing runs and baselines. |
required |
threshold
|
float
|
Relative change threshold (e.g. 0.05 = 5%). |
0.05
|
Initialize the CI guard.
check(run_id=None)
¤
Check for regressions against the baseline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
run_id
|
str | None
|
Run to check. Defaults to latest run. |
None
|
Returns:
| Type | Description |
|---|---|
GuardResult
|
GuardResult indicating pass/fail with regression details. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If no baseline is set or run not found. |
Bisection Engine¤
BisectionEngine binary-searches git history to find the commit that introduced
a regression. Restores the original HEAD after completion.
calibrax.ci.bisection
¤
Git bisection engine for performance regression root-cause analysis.
Binary-searches through git history to find the commit that introduced a performance regression, using user-provided benchmark and regression detection functions.
BisectionResult(*, culprit_commit, total_steps, tested_commits, is_regression_found)
dataclass
¤
Result of a git bisection for performance regression.
Attributes:
| Name | Type | Description |
|---|---|---|
culprit_commit |
str | None
|
Hash of the commit that introduced the regression, or None if no regression was found. |
total_steps |
int
|
Number of bisection steps performed. |
tested_commits |
tuple[str, ...]
|
All commit hashes that were tested. |
is_regression_found |
bool
|
Whether a regression-causing commit was identified. |
to_dict()
¤
Serialize to a JSON-compatible dictionary.
BisectionEngine(repo_path, benchmark_fn, regression_fn)
¤
Binary search through git history to find regression-causing commit.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
repo_path
|
Path | str
|
Path to the git repository. |
required |
benchmark_fn
|
Callable[[str], Run]
|
Function that checks out a commit and runs benchmarks. Receives a commit hash string, returns a Run. |
required |
regression_fn
|
Callable[[Run], bool]
|
Function that checks if a Run exhibits regression. Returns True if the run shows regression. |
required |
Initialize the bisection engine.
bisect(good_commit, bad_commit)
¤
Binary search between a known good and bad commit.
Restores the original HEAD after bisection completes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
good_commit
|
str
|
Commit hash known to be regression-free. |
required |
bad_commit
|
str
|
Commit hash known to have the regression. |
required |
Returns:
| Type | Description |
|---|---|
BisectionResult
|
BisectionResult with the culprit commit (if found). |