Model Monitoring
Your ML platform computes accuracy, drift and bias. Wytness accepts those figures through its API per registered agent, judges each one against the threshold rules you set, raises a breach through your existing connectors, and keeps the signed record and trend as evidence. Wytness never computes drift itself.
What to send
One request carries up to 500 metric points for one agent. Every point names the metric, its value, when it was observed and, optionally, the evaluation window, a segment label and the tool that computed it. Re-sending the same metric at the same observation time overwrites the point instead of duplicating it, so retries are safe.
| Field | Required | Meaning |
|---|---|---|
| metric | yes | A documented name (table below) or custom:<name> |
| value | yes | A finite number |
| observed_at | yes | ISO-8601 timestamp with a timezone offset |
| window | no | { start, end } — the period the figure was computed over |
| segment | no | A slice label such as a region or cohort |
| source | no | dataiku · azure_ml · mlflow · custom (default custom) |
| meta | no | Any JSON object you want kept with the point |
Metric names
| Family | Names |
|---|---|
| Quality | accuracy · precision · recall · f1 · auc · rmse · mae · log_loss |
| Drift | drift_psi · drift_ks · drift_js |
| Fairness | bias_disparity · bias_equal_opportunity |
| Data | data_quality |
| Anything else | custom:<name> — letters, digits, dot, dash or underscore |
Send metrics with curl
curl -X POST https://api.wytness.ai/v1/registry/credit-scorer/metrics \
-H "X-API-Key: $WYTNESS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"metrics": [
{
"metric": "accuracy",
"value": 0.93,
"observed_at": "2026-11-17T09:00:00+00:00",
"window": { "start": "2026-11-16T09:00:00+00:00", "end": "2026-11-17T09:00:00+00:00" },
"source": "dataiku"
},
{ "metric": "drift_psi", "value": 0.07, "observed_at": "2026-11-17T09:00:00+00:00", "source": "dataiku" }
]
}'
# → {"status":"ok","metrics_recorded":2,"alerts_raised":0}Send metrics from Python
Any job that can make an HTTPS request can report — a Dataiku scenario step, an Azure ML pipeline step, an MLflow callback or a plain cron job. No SDK is required.
import os
from datetime import datetime, timezone
import requests
API = "https://api.wytness.ai"
AGENT_ID = "credit-scorer"
now = datetime.now(timezone.utc).isoformat()
payload = {
"metrics": [
{"metric": "accuracy", "value": 0.93, "observed_at": now, "source": "azure_ml"},
{"metric": "drift_psi", "value": 0.07, "observed_at": now, "source": "azure_ml"},
{"metric": "bias_disparity", "value": 0.03, "observed_at": now, "segment": "EU", "source": "azure_ml"},
]
}
response = requests.post(
f"{API}/v1/registry/{AGENT_ID}/metrics",
headers={"X-API-Key": os.environ["WYTNESS_API_KEY"]},
json=payload,
timeout=30,
)
response.raise_for_status()
print(response.json()) # {"status": "ok", "metrics_recorded": 3, "alerts_raised": 0}Thresholds and alerts
An owner or admin sets the rules on the agent page under Model monitoring, one metric at a time: alert when the value is below, at or below, above, or at or above a figure, with the severity the alert should carry. On every intake Wytness judges the newest value against the rules. A breach raises an anomaly alert of kind model_metric_threshold that names the agent, the metric, the value, the threshold and the severity, with a link to the agent page — so Microsoft Sentinel, Splunk, ServiceNow, Slack and webhook connectors deliver it exactly as they deliver every other alert. One alert per rule per 24 hours while the breach holds; every change to the rules is written to the audit log.
What lands on the Evidence Pack
Every Evidence Pack carries a Model monitoring section: the latest value of each metric in the period, the threshold rules in force, and the breaches the period raised. The agent page shows the same figures with their trend.
Questions about model monitoring? Ask us.
Contact us