docs(11-01): complete plan 11-01 - 回测指标扩展
新增 NDCG@5、MRR、命中率分布等排名质量评估指标
This commit is contained in:
@@ -0,0 +1,111 @@
|
||||
---
|
||||
phase: 11-predictv3
|
||||
plan: 01
|
||||
subsystem: backend
|
||||
tags: [predictv3, backtest, metrics, ndcg, mrr]
|
||||
requires:
|
||||
- PRED-02
|
||||
- PRED-05
|
||||
provides:
|
||||
- NDCG@5 计算方法
|
||||
- MRR 计算方法
|
||||
- 命中分布计算方法
|
||||
- 扩展的回测返回结构
|
||||
affects:
|
||||
- _runBacktestV3 返回值
|
||||
- 前端回测结果展示
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: [TDD-style validation, edge case handling]
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- application/admin/model/History.php
|
||||
decisions:
|
||||
- 使用 50 期作为最小数据量阈值
|
||||
- hit_distribution 使用 rank_N 键名格式便于前端解析
|
||||
- 空预测或无效数据返回默认值而非抛出异常
|
||||
metrics:
|
||||
duration: ~5min
|
||||
tasks_completed: 3
|
||||
files_modified: 1
|
||||
completed_at: "2026-05-01T07:15:00Z"
|
||||
---
|
||||
|
||||
# Phase 11 - Plan 01: 回测指标扩展 Summary
|
||||
|
||||
## One-liner
|
||||
|
||||
扩展 `_runBacktestV3` 方法,新增 NDCG@5、MRR、命中率分布等排名质量评估指标,提升算法评估能力。
|
||||
|
||||
## Changes Made
|
||||
|
||||
### Task 1: 实现 NDCG@5 计算方法
|
||||
|
||||
新增 `_calculateNDCG` 方法:
|
||||
- 实现归一化折损累积增益计算
|
||||
- 公式:DCG = Σ(1/log2(rank+1)),IDCG = Σ(1/log2(i+1))
|
||||
- 添加空预测保护和数据完整性检查
|
||||
- 返回 0-1 范围标准化值
|
||||
|
||||
### Task 2: 实现 MRR 和命中分布计算方法
|
||||
|
||||
新增 `_calculateMRR` 方法:
|
||||
- 实现平均倒数排名计算
|
||||
- 公式:MRR = Σ(1/rank_i) / N
|
||||
- 添加空预测保护
|
||||
|
||||
新增 `_calculateHitDistribution` 方法:
|
||||
- 统计各排名位置(1-5)的命中次数
|
||||
- 返回格式:{rank_1: n, rank_2: n, ..., rank_5: n}
|
||||
- 用于前端柱状图可视化
|
||||
|
||||
### Task 3: 扩展 _runBacktestV3 返回结果
|
||||
|
||||
修改返回结构,新增字段:
|
||||
- `ndcg_5`: NDCG@5 值
|
||||
- `mrr`: 平均倒数排名
|
||||
- `hit_distribution`: 各排名命中分布
|
||||
- `precision_5`: Top-5 精确率
|
||||
- `data_warning`: 数据量不足警告
|
||||
- `data_sufficient`: 数据量是否充足标志
|
||||
|
||||
添加数据量检查:不足 50 期时返回默认值和警告提示。
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| application/admin/model/History.php | +134 行(3个新方法 + 返回结构扩展)|
|
||||
|
||||
## Acceptance Criteria Verification
|
||||
|
||||
| Criteria | Status |
|
||||
|----------|--------|
|
||||
| `_calculateNDCG` 方法存在 | PASSED |
|
||||
| `_calculateMRR` 方法存在 | PASSED |
|
||||
| `_calculateHitDistribution` 方法存在 | PASSED |
|
||||
| 所有方法包含空预测保护 | PASSED |
|
||||
| 所有方法包含函数级注释 | PASSED |
|
||||
| hit_distribution 结构为 rank_N 格式 | PASSED |
|
||||
| _runBacktestV3 返回结构包含新字段 | PASSED |
|
||||
| 数据量检查(50期阈值)| PASSED |
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None.
|
||||
|
||||
## Self-Check
|
||||
|
||||
- [x] 所有新增方法已实现
|
||||
- [x] 所有验收标准已通过
|
||||
- [x] 提交已完成 (483a616)
|
||||
- [x] 代码包含函数级注释
|
||||
Reference in New Issue
Block a user