模型预测准确性下降的可视化监控方案

模型预测准确性下降的可视化监控方案

核心监控指标配置

在模型运行时监控中，准确性下降通常表现为以下关键指标：

1. 准确率(Accuracy)变化趋势

# 每小时计算准确率并记录到Prometheus
from sklearn.metrics import accuracy_score
import pandas as pd

def monitor_accuracy(model, X_test, y_test):
    predictions = model.predict(X_test)
    accuracy = accuracy_score(y_test, predictions)
    # 推送指标到Prometheus
    prometheus_client.Gauge('model_accuracy', 'Current model accuracy').set(accuracy)
    return accuracy

2. 精确率(Precision)和召回率(Recall)波动

from sklearn.metrics import precision_recall_fscore_support

def monitor_precision_recall(model, X_test, y_test):
    predictions = model.predict(X_test)
    precision, recall, f1, support = precision_recall_fscore_support(y_test, predictions)
    
    # 记录各指标到监控系统
    prometheus_client.Gauge('model_precision').set(precision[1])  # 假设二分类
    prometheus_client.Gauge('model_recall').set(recall[1])

可视化监控面板配置

Grafana仪表板设置步骤：

1. 创建数据源连接Prometheus
2. 添加以下查询语句：
   • model_accuracy
   • model_precision
   • model_recall
3. 设置时间窗口为最近7天
4. 配置阈值告警：当准确率连续3个周期下降超过5%时触发

告警规则配置方案

# alerting.yml 配置文件
rules:
  - alert: ModelAccuracyDegrade
    expr: (model_accuracy < 0.8) or (increase(model_accuracy[1h]) < -0.05)
    for: 30m
    labels:
      severity: critical
    annotations:
      summary: "模型准确率下降超过阈值"
      description: "当前准确率 {{ $value }}，低于设定阈值0.8"

复现步骤：

1. 部署Prometheus + Grafana监控环境
2. 集成上述代码到模型推理服务
3. 设置告警规则并测试告警触发
4. 通过Grafana查看指标趋势图