calibrax.storage¤
JSON-per-run file store for persisting benchmark runs. Provides saving, loading, querying, baseline management, trend extraction, external data ingestion, and ASV-compatible export.
calibrax.storage.store
¤
JSON-per-run file backend with baseline management.
Directory layout:
benchmark-data/
+-- runs/
| +-- abc123.json
| +-- ...
+-- baselines/
| +-- main.json
+-- config.json
Store(path)
¤
JSON-per-run file backend with baseline management.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path | str
|
Root directory for storing runs, baselines, and config. |
required |
Initialize the store and create directory structure.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path | str
|
Root directory for storing runs, baselines, and config. |
required |
save(run)
¤
Save a run as JSON.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
run
|
Run
|
The run to persist. |
required |
Returns:
| Type | Description |
|---|---|
Path
|
Path to the saved JSON file. |
load(run_id)
¤
Load a run by ID.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
run_id
|
str
|
Unique identifier of the run. |
required |
Returns:
| Type | Description |
|---|---|
Run
|
The deserialized Run. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the run does not exist. |
list_runs(branch=None)
¤
List all runs, optionally filtered by branch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
branch
|
str | None
|
If set, return only runs on this branch. |
None
|
Returns:
| Type | Description |
|---|---|
list[Run]
|
Runs sorted by timestamp descending (newest first). |
latest()
¤
Load the most recent run.
Returns:
| Type | Description |
|---|---|
Run
|
The most recent run by timestamp. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the store is empty. |
query(**tags)
¤
Find runs where any point matches all given tag filters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**tags
|
str
|
Key-value pairs that must all match on at least one point. |
{}
|
Returns:
| Type | Description |
|---|---|
list[Run]
|
List of matching runs. |
set_baseline(run_id)
¤
Copy a run to baselines/main.json.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
run_id
|
str
|
ID of the run to set as baseline. |
required |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If the run does not exist. |
get_baseline()
¤
Load the current baseline, or None if not set.
Returns:
| Type | Description |
|---|---|
Run | None
|
The baseline Run, or None if no baseline has been set. |
ingest(path, format='auto')
¤
Import results from an external JSON file and save to store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path
|
Path to the external JSON file. |
required |
format
|
str
|
Import format (currently only "auto" / JSON supported). |
'auto'
|
Returns:
| Type | Description |
|---|---|
Run
|
The imported Run. |
export_asv(output_dir)
¤
Export store data in ASV-compatible JSON format.
Creates benchmarks.json (benchmark definitions) and per-commit
result files in results/{machine}/{commit}.json.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output_dir
|
Path | str
|
Directory to write ASV-formatted output. |
required |
Returns:
| Type | Description |
|---|---|
Path
|
Path to the output directory. |
extract_trend(metric, point_name, tags, *, n_runs=None)
¤
Extract time-series trend for a metric across stored runs.
Scans all runs for points matching (point_name, tags), extracts the named metric, and returns a TrendSeries ordered oldest-first.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
metric
|
str
|
Metric name to track. |
required |
point_name
|
str
|
Point name to match. |
required |
tags
|
dict[str, str]
|
Tags that must all match on the point. |
required |
n_runs
|
int | None
|
If set, return only the N most recent data points. |
None
|
Returns:
| Type | Description |
|---|---|
TrendSeries
|
TrendSeries with one TrendPoint per matching run. |