# Machine Learning & Telemetry Metrics Reference

> Formulations, interpretations, and mathematical definitions for classification, regression, time-series, and WebGPU telemetry metrics.

## Classification Metrics

### 1. 1 - AUC Error (ROC Loss)

$$\text{Error} = 1 - \text{AUC}$$

Represents the probability that a randomly chosen positive instance receives a lower predicted probability score than a randomly chosen negative instance. Optimal: $0.0$, Random guessing: $0.50$.


### 2. Cross-Entropy Log-Loss

$$\mathcal{L}_{\log} = -\frac{1}{N} \sum_{i=1}^N \sum_{c=1}^C y_{i,c} \ln(p_{i,c})$$

Penalizes overconfident incorrect predictions. Evaluated for multiclass classification.


### 3. F1-Score (Macro & Weighted)

$$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$

Harmonic mean of precision and recall. Weighted F1 accounts for class imbalance.


## Regression Metrics

### 1. Root Mean Squared Error (RMSE)

$$\text{RMSE} = \sqrt{\frac{1}{N} \sum_{i=1}^N (y_i - \hat{y}_i)^2}$$

Measures average prediction error magnitude in the original units of the target variable.


### 2. Coefficient of Determination ($R^2$)

$$R^2 = 1 - \frac{\sum_{i=1}^N (y_i - \hat{y}_i)^2}{\sum_{i=1}^N (y_i - \bar{y})^2}$$

Proportion of variance explained by the model relative to predicting the mean.


## Time-Series Forecasting Metrics

### 1. Scaled Quantile Loss (SQL)

Evaluates probabilistic forecast calibration across 9 quantile levels ($\alpha \in \{0.1, 0.2, \dots, 0.9\}$):

$$\text{QL}_\alpha(y, \hat{q}_\alpha) = 2 \left( (y - \hat{q}_\alpha)(\alpha - \mathbb{I}_{y < \hat{q}_\alpha}) \right)$$

Normalized by in-sample seasonal naive baseline error to make metrics comparable across heterogeneous series.


### 2. Mean Absolute Scaled Error (MASE)

$$\text{MASE} = \frac{\frac{1}{H} \sum_{t=1}^H |y_t - \hat{y}_t|}{\frac{1}{T-m} \sum_{t=m+1}^T |y_t - y_{t-m}|}$$

Scale-invariant point forecast metric where values $< 1.0$ indicate performance superior to a one-step seasonal naive forecast.


## WebGPU Execution Telemetry

- **Preprocess Latency (ms):** Vectorized encoding, quantile transform, and tokenization via WebAssembly.

- **Kernel Execution (ms):** Pure GPU execution time across forward-pass compute pipelines.

- **Memory Overhead (MB):** Peak GPU buffer allocation during attention computation.