calibrax.metrics.functional.regression¤
Regression metrics for comparing predicted and target values. Includes standard error metrics (MSE, MAE, RMSE), percentage-based metrics (MAPE, SMAPE), robust losses (Huber, log-cosh), and goodness-of-fit measures (R-squared, explained variance).
Regression metrics for benchmark result comparison.
Pure functions for computing standard regression metrics between predictions and targets. All functions accept JAX arrays and return scalar values.
Includes 13 metrics: MSE, MAE, RMSE, R-squared, MAPE, relative error, explained variance, max error, Huber loss, quantile loss, log-cosh loss, SMAPE, and CRPS.
mse(predictions: Any, targets: Any) -> Any
¤
Mean squared error.
Computes the average of squared differences between predictions and
targets: mean((predictions - targets)^2).
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric -- violates the triangle inequality.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Mean squared error as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
mae(predictions: Any, targets: Any) -> Any
¤
Mean absolute error.
Computes the average of absolute differences between predictions and
targets: mean(|predictions - targets|).
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). True metric -- satisfies identity, symmetry, and triangle inequality.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Mean absolute error as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
rmse(predictions: Any, targets: Any) -> Any
¤
Root mean squared error.
Computes sqrt(mean((predictions - targets)^2)).
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). True metric -- satisfies identity, symmetry, and triangle inequality.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Root mean squared error as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
r_squared(predictions: Any, targets: Any) -> Any
¤
Coefficient of determination (R-squared).
Computes 1 - SS_res / SS_tot where SS_res is the residual sum of
squares and SS_tot is the total sum of squares.
Note
Direction: HIGHER (1.0 = perfect fit, 0.0 = mean predictor). Range: (-inf, 1]. Not a true metric -- not a distance function.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
R-squared value as a scalar value. Values near 1.0 indicate good fit. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
mape(predictions: Any, targets: Any) -> Any
¤
Mean absolute percentage error.
Computes mean(|targets - predictions| / |targets|).
Note
Direction: LOWER (0.0 = perfect, 1.0 = 100% error). Range: [0, inf). Not a true metric -- not symmetric in general and division by target magnitude breaks metric space axioms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
MAPE as a scalar value (0.0 = perfect, 1.0 = 100% error). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
relative_error(predictions: Any, targets: Any) -> Any
¤
Mean relative error (L2 norm ratio).
Computes ||predictions - targets||_2 / ||targets||_2.
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric -- normalization by target norm breaks the triangle inequality.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Relative error as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
explained_variance(predictions: Any, targets: Any) -> Any
¤
Explained variance score.
Computes 1 - Var(targets - predictions) / Var(targets). Similar to
R-squared but uses variance of residuals instead of sum of squares.
Note
Direction: HIGHER (1.0 = perfect). Range: (-inf, 1]. Not a true metric -- not a distance function. Unlike R-squared, explained variance is invariant to constant bias in predictions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Explained variance as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
max_error(predictions: Any, targets: Any) -> Any
¤
Maximum absolute error.
Computes max(|prediction_i - target_i|) -- the worst-case error.
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric on distributions (operates on individual errors, not on distribution distance).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Maximum absolute error as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
huber_loss(predictions: Any, targets: Any, *, delta: float = 1.0) -> Any
¤
Huber loss (robust regression loss).
Quadratic for small errors (|e| <= delta), linear for large errors.
L = 0.5 * e^2 if |e| <= delta, else delta * (|e| - 0.5 * delta).
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric. Degrades to 0.5 * MSE for small errors, to delta * MAE for large errors. More robust to outliers than MSE.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
delta
|
float
|
Threshold where loss transitions from quadratic to linear. |
1.0
|
Returns:
| Type | Description |
|---|---|
Any
|
Mean Huber loss as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
quantile_loss(predictions: Any, targets: Any, *, quantile: float = 0.5) -> Any
¤
Quantile (pinball) loss for quantile regression.
Asymmetric loss: L = q * max(0, t - p) + (1 - q) * max(0, p - t).
At quantile=0.5, equivalent to 0.5 * MAE.
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric -- asymmetric by design.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
quantile
|
float
|
Target quantile in (0, 1). 0.5 = median. |
0.5
|
Returns:
| Type | Description |
|---|---|
Any
|
Mean quantile loss as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
log_cosh_loss(predictions: Any, targets: Any) -> Any
¤
Log-cosh loss.
Computes mean(log(cosh(predictions - targets))). Smooth approximation
to MAE: approximately quadratic for small errors, linear for large errors.
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Not a true metric. Twice differentiable everywhere (unlike MAE), making it suitable for second-order optimization methods.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Mean log-cosh loss as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
smape(predictions: Any, targets: Any) -> Any
¤
Symmetric mean absolute percentage error.
Computes mean(|p - t| / ((|p| + |t|) / 2)). Unlike MAPE, SMAPE is
symmetric: SMAPE(p, t) = SMAPE(t, p).
Note
Direction: LOWER (0.0 = perfect). Range: [0, 2]. Not a true metric -- normalization breaks metric space axioms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Predicted values. |
required |
targets
|
Any
|
Ground truth values. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
SMAPE as a scalar value. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If shapes do not match. |
crps(predictions: Any, targets: Any) -> Any
¤
Continuous ranked probability score for ensemble forecasts.
Computes the empirical ensemble CRPS:
mean(|X - y|) - 0.5 * mean(|X_i - X_j|) averaged over samples.
Note
Direction: LOWER (0.0 = perfect). Range: [0, inf). Proper scoring rule for probabilistic forecasts.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predictions
|
Any
|
Forecast ensemble with shape |
required |
targets
|
Any
|
Observed targets with shape |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Mean empirical CRPS as a scalar JAX array. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If inputs do not have compatible ensemble forecast shapes. |