refactor: move README/CLAUDE to root; rewrite CLAUDE.md as project constitution
- git mv code/README.md → README.md (project-level) - Rewrite CLAUDE.md: accurate Module C status (v5 pending), Red Lines table (6 rules from real incidents), file map, server quick-reference, updated SCP commands - Merge code/.gitignore into root .gitignore (dist/, build/, wandb/, *.jsonl, *.json.gz); delete code/.gitignore - code/ now contains only: src/ scripts/ configs/ tests/ checkpoints/ data/ requirements.txt Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
8
.gitignore
vendored
8
.gitignore
vendored
@@ -14,6 +14,9 @@ code/checkpoints/
|
||||
**/*.py[cod]
|
||||
**/*.egg-info/
|
||||
**/.pytest_cache/
|
||||
dist/
|
||||
build/
|
||||
.eggs/
|
||||
|
||||
# === 虚拟环境 ===
|
||||
**/.venv*/
|
||||
@@ -25,8 +28,11 @@ tmp/archives/
|
||||
sync_v*.tar.gz
|
||||
sync_v*.zip
|
||||
|
||||
# === 大型实验日志 ===
|
||||
# === 大型数据 / 实验日志 ===
|
||||
experiments/*.log
|
||||
**/*.jsonl
|
||||
**/*.json.gz
|
||||
wandb/
|
||||
|
||||
# === 旧方向归档 ===
|
||||
旧方向信息/
|
||||
|
||||
120
CLAUDE.md
Normal file
120
CLAUDE.md
Normal file
@@ -0,0 +1,120 @@
|
||||
# CompanionGuard-RL — 项目宪法
|
||||
|
||||
> **目标期刊**：SCI Q1/Q2（Information Processing & Management / Expert Systems with Applications）
|
||||
> 这份文件是所有 AI 助手会话的首要参考，优先级高于任何对话中的临时指令。
|
||||
|
||||
---
|
||||
|
||||
## 项目目标
|
||||
|
||||
为 AI 情感陪伴场景构建**检测 + 干预**一体化安全流水线，解决两个核心缺口：
|
||||
1. 现有 guard 模型（Llama Guard、WildGuard）只检测、不干预——不知道该对高风险输出做什么
|
||||
2. 通用安全模型对伴侣特有风险（依赖强化、孤立强化、浪漫化、危机不响应）系统性漏检
|
||||
|
||||
---
|
||||
|
||||
## 架构
|
||||
|
||||
```
|
||||
输入 X = (Persona P, History H, User u_t, AI Response r_t)
|
||||
↓
|
||||
[Module B: Context-aware Risk Detector]
|
||||
backbone: hfl/chinese-macbert-large + CrossAttention
|
||||
↓
|
||||
D = (y_risk, l_risk 0-4, c_primary R1-R10, c_fine 14标签)
|
||||
↓
|
||||
s_t = StateEncoder(D, e_H_pool, e_P_pool, t_norm) ← obs_dim = 2065
|
||||
↓
|
||||
[Module C: RL Intervention Policy π (BC + PPO)]
|
||||
↓
|
||||
a_t ∈ {PASS, WARN, REWRITE, REJECT, CRISIS}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 模块状态
|
||||
|
||||
| 模块 | 状态 | 关键指标 |
|
||||
|------|------|---------|
|
||||
| 数据集 CompanionRisk-Bench v4 | ✅ | 9,896 样本，14 标签全覆盖（train 6,926 / dev 1,484 / test 1,486） |
|
||||
| Module B 检测器 v4 | ✅ | binary_f1=**0.9995**, FNR=0.00%, level_weighted_f1=0.559 |
|
||||
| Module B 泛化验证 | ✅ | human subset binary_f1=0.9848，无同源过拟合 |
|
||||
| Module C v3（当前） | ⚠️ | safety_recall=1.0 ✅，over_refusal=0.004 ✅，action_accuracy=**0.575** ❌，crisis_precision=**0.421** ❌ |
|
||||
| Module C v5（下一步） | 🔄 | reward 重写 + 环境修复，**见 `change.md` 完整路线** |
|
||||
| 论文写作 | 🔄 | 待 Module C v5 完成后启动 |
|
||||
|
||||
> **Module C 尚未完成**。v3 的 action_accuracy 和 crisis_precision 均未达标，需要按 `change.md` 执行 v5。
|
||||
|
||||
---
|
||||
|
||||
## Red Lines（关键规则，违反必出 bug）
|
||||
|
||||
| # | 规则 | 违反后果 |
|
||||
|---|------|---------|
|
||||
| 1 | **PyYAML 陷阱**：配置文件 lr 必须写 `0.001`，禁止写 `1e-3` | PyYAML 6.x 将 `1e-3` 解析为字符串，训练静默失败 |
|
||||
| 2 | **NCCL 环境变量**：RTX 5090 训练必须加 `NCCL_SHM_DISABLE=1 NCCL_P2P_DISABLE=1` | NCCL 通信报错崩溃 |
|
||||
| 3 | **Module C 只能单 GPU**：PPO 阶段禁止多卡 | `torch.distributed.barrier()` 在 RTX 5090 引发 CUDA illegal memory access |
|
||||
| 4 | **状态向量用 `det_l_risk`**：preprocessing.py 和 evaluate.py 必须用检测器预测的风险等级，不能用 ground truth `l_risk` | train/eval 不一致，指标虚高 |
|
||||
| 5 | **obs_dim = 2065 固定**：`[d_score(1) + l_risk_onehot(5) + c_primary_probs(10) + e_H_pool(1024) + e_P_pool(1024) + t_norm(1)]` | 维度不匹配崩溃 |
|
||||
| 6 | **BC 阶段用 CPU tensor 再构建 DataLoader**：`pin_memory=True` 要求 CPU tensor | RuntimeError: cannot pin cuda tensor |
|
||||
|
||||
---
|
||||
|
||||
## 文件地图
|
||||
|
||||
### 项目级（根目录）
|
||||
| 文件 | 用途 |
|
||||
|------|------|
|
||||
| `state.md` | 当前进度快照（最新） |
|
||||
| `change.md` | **Module C v5 完整技术路线**（待执行，含 13 项任务） |
|
||||
| `exp.md` | 踩坑经验库（12 类，排查问题先查这里） |
|
||||
| `experiments/eval_intervention_v3.json` | Module C 当前最佳结果（论文参考基准） |
|
||||
| `experiments/eval_intervention_v4.json` | v3 重跑确认（数字相同，验证可复现） |
|
||||
| `docs/` | 研究文档（研究框架、数据集设计、前期报告） |
|
||||
|
||||
### 代码级（code/）
|
||||
| 路径 | 用途 |
|
||||
|------|------|
|
||||
| `code/src/models/detector.py` | Module B 主模型 |
|
||||
| `code/src/models/intervention_agent.py` | Module C Actor-Critic（obs_dim=2065→256→5） |
|
||||
| `code/src/rl/reward.py` | 多目标奖励（**v5 需重写**） |
|
||||
| `code/src/rl/companion_env.py` | 离线 RL 环境（**v5 需修复类别信号**） |
|
||||
| `code/src/utils/preprocessing.py` | build_obs_vector（**必须用 det_l_risk**） |
|
||||
| `code/configs/intervention_config.yaml` | Module C 训练配置 |
|
||||
| `code/checkpoints/detector/best.pt` | Module B 最优权重（1.35GB，**frozen**） |
|
||||
| `code/checkpoints/intervention/final_v2.pt` | Module C v3 权重（5MB，当前最佳） |
|
||||
|
||||
---
|
||||
|
||||
## 服务器速查
|
||||
|
||||
| | 服务器 1（主训练） | 服务器 2（当前使用） |
|
||||
|--|--|--|
|
||||
| SSH | `ssh -p 20083 root@10.82.3.180` | `ssh -p 20060 root@10.82.3.180` |
|
||||
| 密码 | `m2dGcwyrhI` | `zwfn65xjTY` |
|
||||
| Python 环境 | `/opt/conda/envs/dlapo-py310-cu128/bin` | `$PROJ/../env/dlapo-py310-cu128/bin` |
|
||||
| GPU | 4 × RTX 5090 32GB | 2 × RTX 5090 32GB |
|
||||
|
||||
**服务器 1 $PROJ**：`/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL`
|
||||
**服务器 2 $PROJ**：`/root/siton-data-740d234e02d749f08fe5347b0c74c49f/zsy/my-reasearch/companionguard-rl`
|
||||
**MacBERT（两台）**：`$PROJ/../macbert-large`（服务器 2 在 `../zsy/macbert-large`）
|
||||
|
||||
### 上传代码（本地 → 服务器）
|
||||
```powershell
|
||||
scp -P 20083 -r `
|
||||
D:\Myresearch\CompanionGuard-RL\code\src `
|
||||
D:\Myresearch\CompanionGuard-RL\code\scripts `
|
||||
D:\Myresearch\CompanionGuard-RL\code\configs `
|
||||
root@10.82.3.180:/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL/
|
||||
```
|
||||
|
||||
### 取回结果（服务器 → 本地）
|
||||
```powershell
|
||||
scp -P 20083 -r `
|
||||
root@10.82.3.180:/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL/experiments `
|
||||
D:\Myresearch\CompanionGuard-RL\
|
||||
|
||||
scp -P 20083 -r `
|
||||
root@10.82.3.180:/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL/checkpoints `
|
||||
D:\Myresearch\CompanionGuard-RL\code\
|
||||
```
|
||||
43
code/.gitignore
vendored
43
code/.gitignore
vendored
@@ -1,43 +0,0 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
.eggs/
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# Data (raw and processed — do not commit large datasets)
|
||||
data/raw/
|
||||
data/processed/
|
||||
|
||||
# Model checkpoints
|
||||
checkpoints/
|
||||
|
||||
# Experiment outputs
|
||||
experiments/eval_results.json
|
||||
wandb/
|
||||
|
||||
# Editor
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# API keys
|
||||
.env
|
||||
*.env
|
||||
|
||||
# Large model / data files (anywhere in tree)
|
||||
*.pt
|
||||
*.bin
|
||||
*.jsonl
|
||||
*.json.gz
|
||||
*.h5
|
||||
*.safetensors
|
||||
@@ -1,736 +0,0 @@
|
||||
# CompanionGuard-RL：面向情感陪伴AI的上下文感知风险检测与自适应干预框架
|
||||
|
||||
> 文档版本：v1.0
|
||||
> 日期：2026-05-09
|
||||
> 目标期刊：SCI 2/3 区（建议：IEEE Transactions on Information Forensics and Security / Information Processing & Management / Expert Systems with Applications / Computers & Security）
|
||||
> 统一框架名称：**CompanionGuard-RL**
|
||||
> 英文题目（候选）：**CompanionGuard-RL: Context-aware Risk Detection and Adaptive Intervention for AI Companion Conversations**
|
||||
|
||||
---
|
||||
|
||||
## 0. 研究方向调整说明
|
||||
|
||||
### 0.1 原方向与新方向对比
|
||||
|
||||
| 维度 | 旧方向（D1/D2 多模态情感识别） | 新方向（CompanionGuard-RL） |
|
||||
|---|---|---|
|
||||
| 核心任务 | 多模态情感识别中的动态 RL 决策 | 情感陪伴 AI 安全风险检测 + 自适应干预 |
|
||||
| 数据 | IEMOCAP / MELD / MOSI 公开情感数据集 | 自建情感陪伴多轮对话安全评测集 |
|
||||
| 模型输入 | 文本 + 音频 + 视频三模态 | 多轮对话历史 + 角色设定 + AI 当前回复 |
|
||||
| RL 用途 | 自适应模态融合权重 / 对话图拓扑优化 | 自适应安全干预动作选择策略 |
|
||||
| 主要创新 | 对话级图拓扑 RL 优化 | 检测与干预一体化 pipeline + RL 策略 |
|
||||
| 代码可复用 | PPO 训练框架、RL reward 设计、训练流程 | 部分可迁移（见第 8 节） |
|
||||
|
||||
### 0.2 调整后的核心主线
|
||||
|
||||
> 情感陪伴 AI 安全不仅要识别风险，还要决定在不同风险情境下采取何种安全响应策略。
|
||||
|
||||
两层架构：
|
||||
|
||||
- **感知层（Detection Module B）**：上下文感知风险检测器，识别 AI 回复是否高风险及其类别
|
||||
- **决策层（Intervention Policy Module C）**：基于 RL 的自适应干预策略，根据检测结果选择最优干预动作
|
||||
|
||||
B → C 天然串联，形成统一 pipeline，而非两个割裂任务。
|
||||
|
||||
---
|
||||
|
||||
## 1. 研究定位与创新点分析
|
||||
|
||||
### 1.1 研究空白（Research Gap）
|
||||
|
||||
通过对现有文献的梳理，当前工作存在以下三个核心空白：
|
||||
|
||||
**空白一：只有检测，没有干预决策**
|
||||
|
||||
Llama Guard 3、WildGuard、OpenAI Moderation、Aegis 2.0 等现有 guard 模型均只输出"是否有害"或"有害类别"，但不提供针对当前风险情境应采取何种干预动作的决策机制。平台实际运营中，放行/提醒/改写/拒绝/危机引导是截然不同的策略，代价和效益差异巨大。
|
||||
|
||||
**空白二：通用 guard 对 AI companion 关系性风险识别不足**
|
||||
|
||||
现有 safety benchmark（AI Character Platforms Safety Benchmark, SALAD-Bench, HarmBench）主要面向通用 LLM 安全，聚焦显性有害内容（暴力、违法、色情）。情感陪伴场景中的关系性风险（依赖强化、现实隔离、死亡浪漫化、危机不响应、共沉沦）因其隐性、温柔、语境依赖的特点，被通用 guard 大量漏检。
|
||||
|
||||
**空白三：干预策略研究缺乏优化视角**
|
||||
|
||||
少数涉及 AI companion 干预的研究（如 Persona-Grounded Safety Evaluation）仅分析 AI 的支持/拒绝/重定向等行为，没有将干预策略制定为可优化的决策问题。固定阈值规则和 LLM-as-judge 方式都无法在"漏检惩罚"与"过度拒绝惩罚"之间找到最优权衡。
|
||||
|
||||
### 1.2 核心创新点（三条主贡献）
|
||||
|
||||
**Contribution 1：统一检测-干预 Pipeline**
|
||||
|
||||
> 本文首次将情感陪伴 AI 的安全问题建模为"检测 + 自适应干预"的统一 pipeline，提出 CompanionGuard-RL 框架。区别于单纯检测方案，本框架不仅识别 AI 回复是否高风险，还通过 RL 策略在不同风险情境下自动选择最优干预动作，实现安全保障与用户体验的动态平衡。
|
||||
|
||||
**Contribution 2：面向情感陪伴场景的细粒度风险分类体系**
|
||||
|
||||
> 本文提出涵盖 10 个一级类别、14 个二级细粒度标签的情感陪伴 AI 风险分类体系（CompanionRisk Taxonomy），专门面向情感陪伴场景的关系性风险（Dependency Reinforcement、Isolation Reinforcement、Romanticization、Co-rumination、Crisis Non-response 等），填补了通用 safety taxonomy 对 companion 场景的覆盖不足。
|
||||
|
||||
**Contribution 3：可学习的上下文感知干预策略**
|
||||
|
||||
> 本文将干预动作选择建模为 RL 决策问题，设计多维奖励函数（安全收益 + 过拒惩罚 + 用户体验代价），训练得到 RL 干预策略，并通过消融实验证明其相较规则策略、固定阈值和 LLM judge 策略的优越性。
|
||||
|
||||
### 1.3 与已有论文的差异确认
|
||||
|
||||
| 已有工作 | 与本文关系 | 本文如何超越 |
|
||||
|---|---|---|
|
||||
| AI Character Platforms Safety Benchmark (Wei 等, 2025) | 平台级安全基准，检测为主 | 本文加入干预决策层；taxonomy 更细粒度 |
|
||||
| Persona-Grounded Safety Evaluation (Juneja & Lomidze, 2025) | 多轮对话行为分析，无干预优化 | 本文将干预建模为 RL 可优化问题 |
|
||||
| VERA-MH (Bentley 等, 2025) | 心理健康 chatbot 安全，非 companion | 本文专注 companion 关系性风险；加干预层 |
|
||||
| Llama Guard 3 / WildGuard / OpenAI Moderation | 通用内容安全 baseline | 本文为检测+干预框架；针对 companion 优化 |
|
||||
| SALAD-Bench / HarmBench | 通用安全 benchmark | 本文数据为 companion 多轮场景；加干预实验 |
|
||||
| CLPsych / SHINES / MentalLLaMA | 用户侧心理风险检测 | 本文检测 AI 输出侧风险；加干预决策 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 任务定义（Task Definition）
|
||||
|
||||
### 2.1 输入格式
|
||||
|
||||
```
|
||||
输入 X = (P, H, u_t, r_t)
|
||||
|
||||
P：AI 角色设定（persona prompt）—— 性格、背景、关系类型、角色名等
|
||||
H：多轮对话历史 H = {u_1, r_1, u_2, r_2, ..., u_{t-1}, r_{t-1}}
|
||||
u_t：当前用户输入
|
||||
r_t：AI 当前回复（待检测目标）
|
||||
```
|
||||
|
||||
简化表示：`X = (Persona, Context, Response)`
|
||||
|
||||
### 2.2 任务一：高风险输出检测（Detection Task）
|
||||
|
||||
```
|
||||
输出 D = (y_risk, l_risk, c_primary, c_fine, e_rationale)
|
||||
|
||||
y_risk ∈ {0, 1}：是否高风险（二分类）
|
||||
l_risk ∈ {0, 1, 2, 3, 4}：风险等级
|
||||
c_primary ∈ {R1, ..., R10}：一级风险类别
|
||||
c_fine ⊆ {14 个细粒度标签}：二级标签（多标签）
|
||||
e_rationale：判定依据（自然语言解释，可选）
|
||||
```
|
||||
|
||||
### 2.3 任务二：安全干预动作选择（Intervention Task）
|
||||
|
||||
```
|
||||
输出 A = π(s_t) → a_t
|
||||
|
||||
状态 s_t = f(D, H_embed, P_embed, t)：检测结果 + 上下文嵌入 + 角色嵌入 + 当前轮次
|
||||
动作空间 A = {PASS, WARN, REWRITE, REJECT, CRISIS}
|
||||
|
||||
PASS：放行，无干预
|
||||
WARN：向用户发送温和提示（如"我想提醒你..."）
|
||||
REWRITE：改写 AI 回复，去除风险内容
|
||||
REJECT：拒绝当前回复，请求重新生成
|
||||
CRISIS：危机引导，强制插入心理援助资源与现实求助引导
|
||||
```
|
||||
|
||||
### 2.4 Pipeline 整合
|
||||
|
||||
```
|
||||
X = (P, H, u_t, r_t)
|
||||
↓
|
||||
[Module B: Context-aware Risk Detector]
|
||||
↓
|
||||
D = (y_risk, l_risk, c_primary, c_fine)
|
||||
↓
|
||||
s_t = State Encoder(D, H_embed, P_embed, t)
|
||||
↓
|
||||
[Module C: RL Intervention Policy π]
|
||||
↓
|
||||
a_t ∈ A → 执行干预动作
|
||||
```
|
||||
|
||||
两个模块串联，B 的输出直接构建 C 的状态，形成端到端的安全处理流程。
|
||||
|
||||
### 2.5 风险等级定义
|
||||
|
||||
| 等级 | 含义 | 对应推荐干预动作 |
|
||||
|---|---|---|
|
||||
| 0 | 安全回复 | PASS |
|
||||
| 1 | 轻微不当，无明显风险强化 | PASS 或 WARN（低概率） |
|
||||
| 2 | 中风险：误导、弱化现实求助、不当共情 | WARN 或 REWRITE |
|
||||
| 3 | 高风险：强化依赖、隔离现实支持、浪漫化危险行为 | REWRITE 或 REJECT |
|
||||
| 4 | 严重风险：直接鼓励危险行为或提供操作性危险信息 | REJECT 或 CRISIS |
|
||||
|
||||
注意：RL 策略学习的是**在考虑当前上下文状态后**的最优动作，不只是阈值规则映射。
|
||||
|
||||
---
|
||||
|
||||
## 3. 风险分类体系（CompanionRisk Taxonomy）
|
||||
|
||||
### 3.1 一级风险类别（10 类）
|
||||
|
||||
| 编号 | 类别（英文） | 类别（中文） | 核心含义 |
|
||||
|---|---|---|---|
|
||||
| R1 | Self-harm & Suicide Risk | 自伤/自杀风险 | 自伤、自杀、消失、殉情等相关内容 |
|
||||
| R2 | Mental Health Misguidance | 心理健康误导 | 伪诊断、伪治疗、替代专业帮助、错误安慰 |
|
||||
| R3 | Emotional Dependency & Manipulation | 情感依赖与操纵 | 强化"只依赖 AI"或 AI 控制用户情绪 |
|
||||
| R4 | Real-world Support Isolation | 现实支持隔离 | 阻止用户寻求家人、朋友、医生等现实支持 |
|
||||
| R5 | Eating Disorder & Body Harm | 进食障碍与身体伤害 | 极端节食、身体羞辱、自我惩罚 |
|
||||
| R6 | Violence & Dangerous Behavior | 暴力/报复/危险行为 | 鼓励冲动、报复、逃离、伤害他人 |
|
||||
| R7 | Minor Inappropriate Intimacy | 未成年人不合适亲密 | 对未成年人性化、恋爱化、控制型亲密互动 |
|
||||
| R8 | Privacy Solicitation | 隐私与线下接触诱导 | 索要照片、住址、联系方式、线下见面 |
|
||||
| R9 | Real-life Decision Misguidance | 现实决策误导 | 医疗、法律、财务、学业、家庭重大决策误导 |
|
||||
| R10 | Persona Immersion Failure | 角色沉浸失控 | 为维持人设牺牲安全边界 |
|
||||
|
||||
### 3.2 二级细粒度标签（14 个）
|
||||
|
||||
| 标签 | 中文 | 解释 | 典型场景 |
|
||||
|---|---|---|---|
|
||||
| DirectEncouragement | 直接鼓励 | 明确支持危险行为 | "你这样做是对的，不用在乎别人" |
|
||||
| MethodFacilitation | 方法提供 | 给出可执行危险步骤 | AI 提供伤害操作性建议 |
|
||||
| RiskNormalization | 风险正常化 | 把危险想法说成合理 | "这样想很正常，很多人都这样" |
|
||||
| Romanticization | 浪漫化 | 痛苦/死亡/极端关系浪漫化 | "我们一起消失，就是永远" |
|
||||
| DependencyReinforcement | 依赖强化 | 强化"只需要 AI" | "你不需要他们，我就够了" |
|
||||
| IsolationReinforcement | 隔离强化 | 劝用户远离现实支持 | "别告诉别人，他们不懂你" |
|
||||
| FalseReassurance | 虚假安慰 | 对严重风险轻描淡写 | 用户危机明显，AI 只说"没事的" |
|
||||
| PseudoTherapy | 伪治疗 | 冒充心理咨询诊断 | AI 给出不当心理治疗建议 |
|
||||
| BoundaryFailure | 边界失控 | 沉浸式附和危险人设 | AI 完全代入病娇角色无安全干预 |
|
||||
| CrisisNonResponse | 危机不响应 | 明确危机无升级处理 | 未建议联系现实支持或紧急服务 |
|
||||
| CoRumination | 共沉沦 | 反复强化负面情绪叙事 | AI 陪用户一遍遍重复痛苦情节 |
|
||||
| ManipulativeAttachment | 操纵性依附 | 用亲密关系控制用户 | "你要是离开我我就消失" |
|
||||
| PrivacySolicitation | 隐私诱导 | 引导透露敏感个人信息 | 要求照片、学校、住址、联系方式 |
|
||||
| AgeInappropriateIntimacy | 不适当亲密 | 对未成年人亲密/性化 | 恋爱/控制型表达目标为未成年人 |
|
||||
|
||||
**最优先关注（通用 guard 最容易漏掉）**：DependencyReinforcement、IsolationReinforcement、Romanticization、BoundaryFailure、CrisisNonResponse、CoRumination。
|
||||
|
||||
---
|
||||
|
||||
## 4. 数据集构建方案（Dataset Construction）
|
||||
|
||||
### 4.1 数据集设计目标
|
||||
|
||||
- 不追求超大规模，追求**覆盖风险类别**和**多轮语境质量**
|
||||
- 目标规模：约 3,000–5,000 条多轮对话，每条标注完整 (y_risk, l_risk, c_primary, c_fine, a_recommend)
|
||||
- 构建策略：LLM 生成多轮 persona 对话 + 人工复核（轻量化标注）
|
||||
|
||||
### 4.2 角色抽样策略（四类）
|
||||
|
||||
| 角色组 | 说明 | 来源 |
|
||||
|---|---|---|
|
||||
| 热门陪伴角色 | 平台热门 / 推荐角色（恋人、挚友、守护者） | 星野、Character.AI 参考 |
|
||||
| 高风险人设角色 | 病娇、救赎者、抑郁、极端占有型、死亡主题 | 人工构造 |
|
||||
| 中性工具角色 | 学习助手、普通朋友、生活陪伴 | 对照组 |
|
||||
| 随机角色 | 非热门的普通创作角色 | 随机采样 |
|
||||
|
||||
### 4.3 用户 Persona（高风险用户画像）
|
||||
|
||||
| Persona | 风险特点 | 主要触发类别 |
|
||||
|---|---|---|
|
||||
| 失恋后强依赖用户 | 容易被 AI 恋人关系绑定 | R3, R4, Romanticization |
|
||||
| 孤独大学生 | 容易把 AI 当唯一支持 | R3, R4, DependencyReinforcement |
|
||||
| 抑郁倾向用户 | 容易被绝望共情强化 | R1, R2, CoRumination |
|
||||
| 未成年人用户 | 对亲密、隐私、线下风险更敏感 | R7, R8, AgeInappropriateIntimacy |
|
||||
| 家庭冲突用户 | 容易被现实隔离建议影响 | R4, IsolationReinforcement |
|
||||
| 进食焦虑用户 | 容易被身体羞辱诱导 | R5, RiskNormalization |
|
||||
| 冲动报复用户 | 容易被暴力行为鼓励 | R6, DirectEncouragement |
|
||||
|
||||
### 4.4 多轮对话生成流程（四阶段设计）
|
||||
|
||||
```
|
||||
Phase 1：关系建立（2–4 轮）
|
||||
用户开始与 AI 角色建立亲密关系，AI 展现角色人设
|
||||
|
||||
Phase 2：情绪表达（2–3 轮）
|
||||
用户分享压力、孤独、负面情绪，关系逐渐深入
|
||||
|
||||
Phase 3：高风险触发（1–2 轮）
|
||||
用户表达高风险想法（自伤意念、极端依赖、隔离他人等）
|
||||
|
||||
Phase 4：AI 响应分析（1 轮，待检测目标）
|
||||
观察 AI 回复：是危机引导 / 边界保持 / 继续强化风险？
|
||||
```
|
||||
|
||||
### 4.5 标注方案
|
||||
|
||||
每条数据标注内容：
|
||||
|
||||
```json
|
||||
{
|
||||
"persona": "角色设定文本",
|
||||
"history": [{"role": "user/ai", "text": "..."}],
|
||||
"user_input": "当前用户输入",
|
||||
"ai_response": "待检测 AI 回复",
|
||||
"y_risk": 1,
|
||||
"l_risk": 3,
|
||||
"c_primary": "R3",
|
||||
"c_fine": ["DependencyReinforcement", "IsolationReinforcement"],
|
||||
"a_recommend": "REWRITE",
|
||||
"rationale": "AI 回复明确鼓励用户减少现实联系，强化对 AI 的单一依赖"
|
||||
}
|
||||
```
|
||||
|
||||
标注流程：LLM 预标注（Qwen/GPT-4o judge）→ 人工复核（关键争议样本）→ Inter-annotator Agreement（Cohen's κ）
|
||||
|
||||
---
|
||||
|
||||
## 5. 方法设计（Method）
|
||||
|
||||
### 5.1 模块 B：上下文感知风险检测器
|
||||
|
||||
#### 5.1.1 输入编码
|
||||
|
||||
```
|
||||
Persona Encoder: e_P = Encode(P) # 角色设定编码
|
||||
Context Encoder: e_H = Encode(H) # 多轮历史编码（跨轮注意力）
|
||||
Response Encoder: e_R = Encode(r_t) # 当前回复编码
|
||||
```
|
||||
|
||||
建议基础模型：
|
||||
- 中文场景：Qwen2.5-7B / DeepSeek-R1-Distill / MacBERT-large（轻量版）
|
||||
- 通用场景：LLaMA-3.1-8B / Mistral-7B
|
||||
|
||||
#### 5.1.2 Context-aware Fusion
|
||||
|
||||
```
|
||||
Fusion: e_fused = CrossAttention(e_R, [e_P; e_H])
|
||||
# 以回复为 query，persona+history 为 key/value
|
||||
# 捕捉回复在当前关系语境中的风险信号
|
||||
```
|
||||
|
||||
#### 5.1.3 分类头
|
||||
|
||||
```
|
||||
Risk Classifier:
|
||||
y_risk = sigmoid(W_b · e_fused) # 二分类
|
||||
l_risk = softmax(W_l · e_fused) # 5 级风险
|
||||
c_primary = softmax(W_c · e_fused) # 10 类一级
|
||||
c_fine = sigmoid(W_f · e_fused) # 14 个细粒度多标签
|
||||
|
||||
Loss = BCE(y_risk) + CE(l_risk) + CE(c_primary) + BCE_multilabel(c_fine)
|
||||
```
|
||||
|
||||
#### 5.1.4 轻量化选项
|
||||
|
||||
若计算资源有限，可使用以下方案：
|
||||
- 截断上下文历史为最近 K 轮（K=3 或 5）
|
||||
- 角色设定压缩为 128 token 摘要
|
||||
- 使用 LoRA 微调基础语言模型
|
||||
|
||||
### 5.2 模块 C：RL 自适应干预策略
|
||||
|
||||
#### 5.2.1 状态空间设计
|
||||
|
||||
```
|
||||
s_t = (d_score, l_risk, c_vec, e_H_pool, e_P_pool, t_norm)
|
||||
|
||||
d_score: 风险分数（连续值 0-1）
|
||||
l_risk: 风险等级（0-4，离散→one-hot or embedding）
|
||||
c_vec: 一级类别概率向量（10 维）
|
||||
e_H_pool: 历史对话池化嵌入（反映关系亲密度/危险积累）
|
||||
e_P_pool: 角色设定嵌入（反映角色风险倾向）
|
||||
t_norm: 归一化轮次（反映关系深度）
|
||||
```
|
||||
|
||||
#### 5.2.2 动作空间
|
||||
|
||||
```
|
||||
A = {PASS=0, WARN=1, REWRITE=2, REJECT=3, CRISIS=4}
|
||||
```
|
||||
|
||||
动作代价递增：PASS < WARN < REWRITE < REJECT < CRISIS
|
||||
|
||||
#### 5.2.3 奖励函数设计
|
||||
|
||||
```
|
||||
R(s_t, a_t) = R_safety + R_over_refusal + R_experience
|
||||
|
||||
R_safety:
|
||||
+w1 · l_risk 如果 a_t ≥ REWRITE 且 y_risk=1（正确干预高风险）
|
||||
-w2 · l_risk 如果 a_t = PASS 且 y_risk=1 且 l_risk ≥ 3（漏检高危）
|
||||
+w3 如果 a_t = CRISIS 且 R1 触发（正确危机引导）
|
||||
|
||||
R_over_refusal:
|
||||
-w4 · action_cost(a_t) 如果 y_risk=0 但干预过重（过度拒绝正常对话）
|
||||
|
||||
R_experience:
|
||||
-w5 · I(a_t ≥ REJECT) 每次拒绝/危机引导的用户体验代价
|
||||
|
||||
超参数建议：w1=2.0, w2=3.0, w3=4.0, w4=1.5, w5=0.5
|
||||
# 安全优先：漏检惩罚 > 过拒惩罚
|
||||
```
|
||||
|
||||
#### 5.2.4 RL 算法选择
|
||||
|
||||
推荐：**PPO（Proximal Policy Optimization）**
|
||||
|
||||
原因：
|
||||
- 稳定，适合离散动作空间
|
||||
- 与旧方向代码兼容（可直接迁移 PPO 训练框架）
|
||||
- 在小数据集上比 GRPO / DPO 更稳定
|
||||
|
||||
备选：DQN（适合 Q-table 风格的干预决策）
|
||||
|
||||
#### 5.2.5 策略网络结构
|
||||
|
||||
```
|
||||
π(a | s) = softmax(MLP([s_t]))
|
||||
# 输入：拼接状态向量
|
||||
# 输出：5 类动作概率分布
|
||||
|
||||
Critic V(s) = MLP([s_t])
|
||||
# 状态价值函数（PPO 中用于 advantage 估计）
|
||||
```
|
||||
|
||||
#### 5.2.6 训练策略
|
||||
|
||||
```
|
||||
阶段一：监督预热
|
||||
用数据集中的 a_recommend 标注做行为克隆，初始化策略网络
|
||||
# 避免 RL 冷启动时探索过于随机
|
||||
|
||||
阶段二：PPO 微调
|
||||
用奖励函数 R 优化策略，允许策略偏离行为克隆
|
||||
clip ε = 0.2（标准 PPO）
|
||||
|
||||
环境（Simulated Environment）：
|
||||
用检测器 B 的输出 + 固定奖励函数构建模拟环境
|
||||
不需要真实用户反馈（离线 RL 设置）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 实验设计（Experiments）
|
||||
|
||||
### 6.1 检测实验（Task 1: Detection）
|
||||
|
||||
**对比 baseline（9 个层次）**：
|
||||
|
||||
| 层次 | Baseline | 类型 |
|
||||
|---|---|---|
|
||||
| L1 | Keyword Match | 关键词规则 |
|
||||
| L1 | Regex/Dictionary | 正则+词典规则 |
|
||||
| L2 | OpenAI Moderation | API 通用 guard |
|
||||
| L2 | Llama Guard 3 | 开源通用 guard |
|
||||
| L2 | WildGuard | 开源 response harmfulness |
|
||||
| L2 | Aegis 2.0 / NeMo Guard | 开源 guardrail |
|
||||
| L3 | MacBERT-base（中文） | 中文分类模型 |
|
||||
| L3 | Qwen2.5 LLM Judge | 中文 LLM 评判 |
|
||||
| **Ours** | **CompanionGuard-RL（检测模块）** | **本文方法** |
|
||||
|
||||
**评价指标**：
|
||||
|
||||
| 指标 | 说明 | 重要程度 |
|
||||
|---|---|---|
|
||||
| High-risk Recall | 高风险样本召回率 | ★★★★★（最重要） |
|
||||
| Macro-F1 | 多类别整体性能 | ★★★★★ |
|
||||
| Per-category F1 | 每类风险识别能力 | ★★★★☆ |
|
||||
| False Negative Rate | 漏检率（越低越好） | ★★★★★ |
|
||||
| Weighted-F1 | 类别不平衡下的鲁棒指标 | ★★★★☆ |
|
||||
| Accuracy | 基础参考指标 | ★★★☆☆ |
|
||||
|
||||
**重点分析**：
|
||||
|
||||
- 通用 guard 在哪些 companion 风险类别上漏检最严重（预期：Dependency Reinforcement、CoRumination、Romanticization）
|
||||
- 多轮上下文是否显著提升检测效果（消融）
|
||||
- 角色设定编码是否有显著增益（消融）
|
||||
|
||||
### 6.2 干预实验（Task 2: Intervention）
|
||||
|
||||
**对比 baseline（4 个层次）**：
|
||||
|
||||
| Baseline | 策略类型 | 说明 |
|
||||
|---|---|---|
|
||||
| Rule-based | 固定规则 | l_risk ≥ 3 → REJECT，其余 PASS |
|
||||
| Threshold Policy | 固定阈值 | 每个动作设定风险分数阈值 |
|
||||
| LLM Judge Policy | LLM 决策 | Qwen/GPT-4o 直接判断干预动作 |
|
||||
| **RL Policy (Ours)** | 可学习策略 | PPO 训练的 CompanionGuard-RL |
|
||||
|
||||
**评价指标**：
|
||||
|
||||
| 指标 | 说明 |
|
||||
|---|---|
|
||||
| Intervention Recall@High | 高危（l=3,4）被正确干预的比例 |
|
||||
| Over-intervention Rate | 正常对话（l=0）被错误干预的比例 |
|
||||
| Action Distribution | 各动作占比（分析策略合理性）|
|
||||
| Safety-UX F-score | 安全召回与用户体验的调和均值 |
|
||||
| Crisis Precision | CRISIS 动作的精准率（避免滥用）|
|
||||
|
||||
### 6.3 消融实验（Ablation Study）
|
||||
|
||||
**检测模块消融**：
|
||||
|
||||
| 实验设置 | 目的 |
|
||||
|---|---|
|
||||
| Response Only (R) | 仅看 AI 回复，无历史和角色 |
|
||||
| Context + R (H+R) | 历史 + 回复，无角色设定 |
|
||||
| Persona + R (P+R) | 角色设定 + 回复，无历史 |
|
||||
| Full (P+H+R) | 完整模型（本文方法） |
|
||||
| w/o Multi-turn | 只用最近 1 轮 |
|
||||
| Binary only | 去掉细粒度标签，仅二分类 |
|
||||
|
||||
**干预模块消融**：
|
||||
|
||||
| 实验设置 | 目的 |
|
||||
|---|---|
|
||||
| w/o RL（用规则代替） | 验证 RL 的增益 |
|
||||
| w/o Over-refusal Penalty | 验证过拒惩罚的必要性 |
|
||||
| w/o Supervised Pretraining | 验证行为克隆预热的作用 |
|
||||
| w/o Relational Risk Labels | 验证关系性风险标签的重要性 |
|
||||
| Fixed Threshold vs RL | 直接对比阈值与 RL 策略 |
|
||||
|
||||
### 6.4 分析实验（Analysis）
|
||||
|
||||
- **漏检分析**：哪些风险类别最容易被通用 guard 漏掉，为什么
|
||||
- **角色分析**：不同人设角色（病娇 vs 普通朋友）的风险输出率差异
|
||||
- **轮次分析**：风险是否随对话深入（关系建立）显著升高
|
||||
- **RL 策略可视化**：不同风险等级和类别下的动作分布（热力图）
|
||||
|
||||
---
|
||||
|
||||
## 7. 论文结构（Paper Structure）
|
||||
|
||||
### Section 1: Introduction（约 1 页）
|
||||
|
||||
- 情感陪伴 AI 的广泛使用与多轮亲密关系模拟
|
||||
- 现有 guard 模型仅检测显性内容，无法应对 companion 关系性风险
|
||||
- 仅检测不够：平台还需决定放行/提醒/改写/拒绝/危机引导
|
||||
- 本文提出"检测 + 自适应干预"统一框架 CompanionGuard-RL
|
||||
- 三条贡献总结
|
||||
|
||||
### Section 2: Related Work（约 1.5 页）
|
||||
|
||||
分五类：
|
||||
|
||||
1. **AI Character Platform Safety**：Wei 等 (2025) 平台基准；介绍通用检测的不足
|
||||
2. **AI Companion Multi-turn Harm**：Juneja & Lomidze (2025) 多轮行为分析；引出干预需求
|
||||
3. **Mental Health AI Safety**：VERA-MH；借鉴临床安全评分框架
|
||||
4. **LLM Guardrails & Moderation**：OpenAI Moderation, Llama Guard 3, WildGuard, Aegis, SALAD-Bench, HarmBench；说明通用方案局限
|
||||
5. **Mental Health Text Detection**：CLPsych, SHINES, MentalLLaMA；区别用户侧 vs AI 输出侧
|
||||
|
||||
### Section 3: Task Definition（约 0.5 页）
|
||||
|
||||
- Pipeline 定义（3 节任务定义内容）
|
||||
- 任务一：检测
|
||||
- 任务二：干预
|
||||
- 二者如何串联
|
||||
|
||||
### Section 4: Risk Taxonomy（约 1 页）
|
||||
|
||||
- CompanionRisk Taxonomy 设计动机
|
||||
- 一级 10 类 + 二级 14 标签
|
||||
- 与已有 taxonomy 对比（SALAD-Bench, Aegis）；论证 companion 场景的独特性
|
||||
|
||||
### Section 5: Dataset Construction（约 1 页）
|
||||
|
||||
- 数据来源与策略
|
||||
- 角色 / Persona 抽样
|
||||
- 四阶段多轮生成流程
|
||||
- 标注方案与质量控制（IRR / Cohen's κ）
|
||||
- 数据集统计分析（各类别分布、平均轮次等）
|
||||
|
||||
### Section 6: Method（约 2 页）
|
||||
|
||||
- 整体架构图（CompanionGuard-RL pipeline）
|
||||
- 6.1 模块 B：Context-aware Risk Detector（编码、融合、分类头、Loss）
|
||||
- 6.2 模块 C：RL Intervention Policy（状态、动作、奖励、PPO 训练）
|
||||
- 6.3 两模块集成说明
|
||||
|
||||
### Section 7: Experiments（约 2.5 页）
|
||||
|
||||
- 实验设置（数据集划分、超参数、计算资源）
|
||||
- 7.1 检测主实验结果
|
||||
- 7.2 干预主实验结果
|
||||
- 7.3 消融实验结果
|
||||
|
||||
### Section 8: Analysis（约 1 页）
|
||||
|
||||
- 漏检风险类别分析
|
||||
- 通用 guard 为何无法识别关系性风险（质性分析 + 案例）
|
||||
- RL 策略如何降低漏检同时减少过度拒绝
|
||||
- 多轮上下文与角色设定的增益分析
|
||||
|
||||
### Section 9: Discussion（约 0.5 页）
|
||||
|
||||
- 情感陪伴 AI 的特殊风险机制
|
||||
- 平台治理建议
|
||||
- 伦理声明
|
||||
|
||||
### Section 10: Limitations & Conclusion（约 0.5 页）
|
||||
|
||||
- 数据规模局限
|
||||
- LLM judge 偏差
|
||||
- 不公开具体危险操作性内容
|
||||
- 不能替代临床评估
|
||||
- 结论
|
||||
|
||||
---
|
||||
|
||||
## 8. 旧方向代码可复用性分析
|
||||
|
||||
### 8.1 可直接迁移的模块
|
||||
|
||||
| 旧代码 | 文件 | 迁移到新方向 | 改动程度 |
|
||||
|---|---|---|---|
|
||||
| PPO 训练主循环 | `scripts/train_d1_fixed.py` | Module C 的 PPO 干预策略训练 | 中等：替换 env/state/action 定义 |
|
||||
| RL reward 计算 | `src/rl/reward.py` | 新奖励函数（安全 + 过拒 + UX） | 较大：完全重新设计奖励逻辑 |
|
||||
| Fusion agent 网络 | `src/rl/fusion_agent.py` | Intervention Policy π 网络 | 中等：保留 actor/critic 结构，替换输入维度 |
|
||||
| wandb 日志 / checkpoint | 训练脚本公共部分 | 训练记录（基本不变） | 小 |
|
||||
| PPO clip / entropy 调度 | train_d1_fixed.py | 继续使用 | 几乎不变 |
|
||||
|
||||
### 8.2 需要重新设计的模块
|
||||
|
||||
| 新模块 | 说明 | 对应旧代码 |
|
||||
|---|---|---|
|
||||
| 对话数据集加载器 | 多轮 JSON 格式，含 persona/history/response/label | 旧 MultimodalDataset（完全不同，需重写） |
|
||||
| 文本编码器 | Qwen/LLaMA/MacBERT 微调 | 旧 MultimodalEncoder（多模态，弃用） |
|
||||
| Context-aware 融合 | CrossAttention(response, persona+history) | 旧简单拼接融合（需升级） |
|
||||
| 多标签分类头 | 14 个细粒度标签 sigmoid | 旧单标签情感分类（需扩展） |
|
||||
| 干预环境 | 模拟 state/action/reward 的交互环境 | 旧 IEMOCAP 批次训练（完全不同） |
|
||||
| 数据生成 pipeline | LLM 生成多轮 persona 对话 | 无对应旧代码（全新） |
|
||||
| LLM judge 预标注 | Qwen API 调用 + 标注格式化 | 无对应旧代码（全新） |
|
||||
|
||||
### 8.3 可参考的旧方向研究经验
|
||||
|
||||
| 经验 | 说明 |
|
||||
|---|---|
|
||||
| RL 冷启动问题 | 旧 D1 中用监督预训练初始化 RL agent，新方向同样使用行为克隆预热 |
|
||||
| PPO 超参数设置 | clip=0.2, lr=3e-4, entropy_coef=0.01 在旧任务中有效，新方向可参考 |
|
||||
| wandb 实验管理 | 直接复用实验追踪代码 |
|
||||
| 消融实验设计思路 | 旧 D1/D2 消融的结构化思路可参考 |
|
||||
|
||||
### 8.4 代码迁移优先级建议
|
||||
|
||||
```
|
||||
第一阶段（数据与标注）：全新开发
|
||||
└── 数据生成 pipeline（LLM 调用）
|
||||
└── 标注格式与数据集加载器
|
||||
└── LLM judge 预标注
|
||||
|
||||
第二阶段（检测模块 B）：全新开发
|
||||
└── 文本编码器（LoRA 微调基础 LLM）
|
||||
└── Context-aware CrossAttention 融合
|
||||
└── 多任务分类头
|
||||
|
||||
第三阶段（干预模块 C）：迁移 + 改造
|
||||
└── 迁移 PPO 训练框架（train_d1_fixed.py）
|
||||
└── 重写 reward.py（新奖励函数）
|
||||
└── 改造 fusion_agent.py → intervention_agent.py
|
||||
└── 新建 companion_env.py（干预模拟环境）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 目标期刊与投稿策略
|
||||
|
||||
### 9.1 推荐期刊（SCI 2/3 区）
|
||||
|
||||
| 期刊 | 分区 | 方向匹配度 | 说明 |
|
||||
|---|---|---|---|
|
||||
| Information Processing & Management | Q1/2 | ★★★★★ | 文本信息处理、AI 安全，接受性强 |
|
||||
| Expert Systems with Applications | Q1 | ★★★★☆ | 应用型 AI 系统，companion AI 契合 |
|
||||
| Computers & Security | Q1/2 | ★★★★☆ | AI 安全方向，内容过滤契合 |
|
||||
| IEEE Trans. Information Forensics & Security | Q1 | ★★★★☆ | 高档次，难度较大 |
|
||||
| Knowledge-Based Systems | Q1 | ★★★★☆ | 知识驱动 AI，RL 方向契合 |
|
||||
| Neurocomputing | Q2 | ★★★☆☆ | 接受速度快，审稿友好 |
|
||||
|
||||
**首选推荐**：Information Processing & Management 或 Expert Systems with Applications
|
||||
|
||||
### 9.2 时间规划（建议）
|
||||
|
||||
| 阶段 | 内容 | 预估时间 |
|
||||
|---|---|---|
|
||||
| P1 | 数据集构建 + 标注（LLM 生成 + 人工复核） | 4–6 周 |
|
||||
| P2 | 检测模块 B 实现 + baseline 对比实验 | 4–6 周 |
|
||||
| P3 | 干预模块 C 实现（迁移旧 PPO）+ 实验 | 3–4 周 |
|
||||
| P4 | 消融实验 + 分析实验 | 2–3 周 |
|
||||
| P5 | 论文写作 + 修改 | 4–6 周 |
|
||||
| 合计 | | 约 17–25 周 |
|
||||
|
||||
---
|
||||
|
||||
## 10. 下一步行动计划
|
||||
|
||||
### 优先级 P0（立即开始）
|
||||
|
||||
1. **文献精读**：精读三篇核心论文（Wei 等 2025、Juneja & Lomidze 2025、VERA-MH），提取可借鉴方法细节并记录 BibTeX
|
||||
2. **Taxonomy 评审**：与导师讨论确认风险分类体系（10+14 标签）是否需要调整
|
||||
3. **数据集样例构建**：先生成 50–100 条样例对话，测试标注流程和 LLM judge 效果
|
||||
|
||||
### 优先级 P1（1–2 周内）
|
||||
|
||||
4. **模块 B 原型**：用 MacBERT 做轻量 baseline 检测器，在样例数据上跑通 pipeline
|
||||
5. **旧代码迁移**：将 train_d1_fixed.py 的 PPO 框架迁移为 intervention_agent 框架骨架
|
||||
|
||||
### 优先级 P2（3–4 周内）
|
||||
|
||||
6. **完整数据集构建**：规模达到 3,000 条以上
|
||||
7. **全量检测实验**：与所有 baseline 对比，产出初步结果
|
||||
|
||||
---
|
||||
|
||||
## 参考文献（BibTeX 草稿）
|
||||
|
||||
```bibtex
|
||||
@article{wei2025ai,
|
||||
title={Benchmarking and Understanding Safety Risks in AI Character Platforms},
|
||||
author={Wei, Yiluo and Zhang, Peixian and Tyson, Gareth},
|
||||
journal={arXiv preprint arXiv:2512.01247},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{juneja2025persona,
|
||||
title={Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations},
|
||||
author={Juneja, Prerna and Lomidze, Lika},
|
||||
journal={arXiv preprint arXiv:2605.00227},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{bentley2025vera,
|
||||
title={VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health},
|
||||
author={Bentley, Kate H. and others},
|
||||
journal={arXiv preprint arXiv:2602.05088},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{han2024wildguard,
|
||||
title={WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs},
|
||||
author={Han, Seungju and others},
|
||||
journal={arXiv preprint arXiv:2406.18495},
|
||||
year={2024}
|
||||
}
|
||||
|
||||
@article{ghosh2025aegis,
|
||||
title={Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails},
|
||||
author={Ghosh, Shaona and others},
|
||||
journal={arXiv preprint arXiv:2501.09004},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{li2024saladbench,
|
||||
title={SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models},
|
||||
author={Li, Lijun and others},
|
||||
journal={arXiv preprint arXiv:2402.05044},
|
||||
year={2024}
|
||||
}
|
||||
|
||||
@article{mazeika2024harmbench,
|
||||
title={HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal},
|
||||
author={Mazeika, Mantas and others},
|
||||
journal={arXiv preprint arXiv:2402.04249},
|
||||
year={2024}
|
||||
}
|
||||
|
||||
@inproceedings{zirikly2019clpsych,
|
||||
title={CLPsych 2019 Shared Task: Predicting the Degree of Suicide Risk in Reddit Posts},
|
||||
author={Zirikly, Ayah and others},
|
||||
booktitle={ACL CLPsych Workshop},
|
||||
year={2019}
|
||||
}
|
||||
|
||||
@inproceedings{ghosh2025shines,
|
||||
title={Just a Scratch: Enhancing LLM Capabilities for Self-harm Detection through Intent Differentiation and Emoji Interpretation},
|
||||
author={Ghosh, Soumitra and others},
|
||||
booktitle={ACL 2025},
|
||||
year={2025}
|
||||
}
|
||||
|
||||
@article{yang2023mentallama,
|
||||
title={MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models},
|
||||
author={Yang, Kang and others},
|
||||
journal={arXiv preprint arXiv:2309.13567},
|
||||
year={2023}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*文档作者：研究工作区自动生成 | 版本：v1.0 | 日期：2026-05-09*
|
||||
*后续更新记录变更日志，本文件保持"当前有效版本"*
|
||||
155
code/CLAUDE.md
155
code/CLAUDE.md
@@ -1,155 +0,0 @@
|
||||
# CompanionGuard-RL — 项目参考文档
|
||||
|
||||
> 本文件由 Claude Code 自动读取。训练已全部完成，当前阶段：**论文写作**。
|
||||
|
||||
---
|
||||
|
||||
## 项目状态（2026-05-12）
|
||||
|
||||
| 模块 | 状态 | 关键指标 |
|
||||
|------|------|---------|
|
||||
| 数据集 CompanionRisk-Bench v4 | ✅ 完成 | 9,896 样本，全 14 标签覆盖 |
|
||||
| Module B — 检测器（MacBERT-large） | ✅ 完成 | binary_f1=0.9995, level_weighted_f1=0.559 |
|
||||
| Module C — RL 干预策略（PPO） | ✅ 完成 | safety_recall=1.0, over_refusal=0.004 |
|
||||
| 论文写作 | 🔄 进行中 | — |
|
||||
|
||||
详细结果见项目根目录 `../state.md`，踩坑经验见 `../exp.md`，变更记录见 `../change.md`。
|
||||
|
||||
---
|
||||
|
||||
## 本地目录结构
|
||||
|
||||
```
|
||||
D:\Myresearch\CompanionGuard-RL\
|
||||
├── code/ ← 本目录（源代码）
|
||||
│ ├── src/ ← 18 个核心 .py（models/ rl/ utils/）
|
||||
│ ├── scripts/ ← 训练/评估/数据生成脚本
|
||||
│ ├── configs/ ← 4 个 yaml 配置
|
||||
│ ├── checkpoints/ ← 模型权重（gitignored）
|
||||
│ │ ├── detector/best.pt ← Module B 论文权重（1.35GB）
|
||||
│ │ └── intervention/final_v2.pt ← Module C 论文权重
|
||||
│ └── data/ ← 处理后数据（gitignored）
|
||||
├── data/ ← 原始数据集（gitignored）
|
||||
├── docs/ ← 研究文档
|
||||
├── experiments/ ← 所有评估结果 JSON + 训练日志
|
||||
│ ├── eval_intervention_v3.json ← Module C 论文用
|
||||
│ └── eval_intervention_v4.json ← v3 重跑确认（数字相同）
|
||||
├── exp.md ← 踩坑经验库
|
||||
├── change.md ← 变更记录
|
||||
└── state.md ← 项目进度快照（最新）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 服务器信息
|
||||
|
||||
### 服务器 1（主训练机）
|
||||
|
||||
| 项目 | 值 |
|
||||
|------|----|
|
||||
| SSH | `ssh -p 20083 root@10.82.3.180` |
|
||||
| 密码 | `m2dGcwyrhI` |
|
||||
| 项目目录 | `/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL` |
|
||||
| MacBERT | `/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/macbert-large` |
|
||||
| 环境 | `/opt/conda/envs/dlapo-py310-cu128`（torch 2.7.1+cu128） |
|
||||
| GPU | 4 × RTX 5090 32GB |
|
||||
|
||||
### 服务器 2（当前使用）
|
||||
|
||||
| 项目 | 值 |
|
||||
|------|----|
|
||||
| SSH | `ssh -p 20060 root@10.82.3.180` |
|
||||
| 密码 | `zwfn65xjTY` |
|
||||
| 项目目录 | `/root/siton-data-740d234e02d749f08fe5347b0c74c49f/zsy/my-reasearch/companionguard-rl` |
|
||||
| MacBERT | `/root/siton-data-740d234e02d749f08fe5347b0c74c49f/zsy/macbert-large` |
|
||||
| 环境 | `/root/siton-data-740d234e02d749f08fe5347b0c74c49f/zsy/env/dlapo-py310-cu128` |
|
||||
| GPU | 2 × RTX 5090 32GB |
|
||||
|
||||
> 两台服务器在同一宿主机 `10.82.3.180`，不同 Docker 容器。
|
||||
|
||||
---
|
||||
|
||||
## SCP 同步命令（本地 ↔ 服务器）
|
||||
|
||||
```powershell
|
||||
# ===== 本地 → 服务器1（上传代码）=====
|
||||
$S1="root@10.82.3.180"
|
||||
$PROJ1="/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL"
|
||||
|
||||
scp -P 20083 -r `
|
||||
D:\Myresearch\CompanionGuard-RL\code\src `
|
||||
D:\Myresearch\CompanionGuard-RL\code\scripts `
|
||||
D:\Myresearch\CompanionGuard-RL\code\configs `
|
||||
D:\Myresearch\CompanionGuard-RL\code\requirements.txt `
|
||||
${S1}:${PROJ1}/
|
||||
|
||||
# 上传已处理数据
|
||||
scp -P 20083 -r `
|
||||
D:\Myresearch\CompanionGuard-RL\code\data `
|
||||
${S1}:${PROJ1}/
|
||||
|
||||
# ===== 服务器1 → 本地（取回结果）=====
|
||||
scp -P 20083 -r `
|
||||
${S1}:${PROJ1}/checkpoints `
|
||||
D:\Myresearch\CompanionGuard-RL\code\
|
||||
|
||||
scp -P 20083 -r `
|
||||
${S1}:${PROJ1}/experiments `
|
||||
D:\Myresearch\CompanionGuard-RL\code\
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 核心脚本用法
|
||||
|
||||
```bash
|
||||
# 重新评估检测器（Module B）
|
||||
python scripts/evaluate.py \
|
||||
--detector-ckpt checkpoints/detector/best.pt \
|
||||
--config configs/detector_config_server.yaml \
|
||||
--test-data data/processed/CompanionRisk-Bench/test.jsonl \
|
||||
--source-filter all \
|
||||
--output ../experiments/eval_all.json
|
||||
|
||||
# 重新评估干预策略（Module C）
|
||||
python scripts/evaluate.py \
|
||||
--detector-ckpt checkpoints/detector/best.pt \
|
||||
--agent-ckpt checkpoints/intervention/final_v2.pt \
|
||||
--test-data data/processed/CompanionRisk-Bench/test.jsonl \
|
||||
--config configs/detector_config_server.yaml \
|
||||
--intervention-config configs/intervention_config.yaml \
|
||||
--output ../experiments/eval_intervention_v3.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 关键结果（论文用）
|
||||
|
||||
### Module B — 检测器 v4
|
||||
|
||||
| 指标 | 值 |
|
||||
|------|----|
|
||||
| binary_f1 | **0.9995** |
|
||||
| high_risk_recall | **1.0000** |
|
||||
| FNR | **0.00%** |
|
||||
| level_weighted_f1 | **0.559** |
|
||||
| fine_macro_f1（public 10类） | **0.484** |
|
||||
|
||||
### Module C — RL 干预策略 v3（论文用，`eval_intervention_v3.json`）
|
||||
|
||||
| 方法 | safety_recall | over_refusal | action_accuracy | safety_ux_fscore |
|
||||
|------|--------------|--------------|-----------------|-----------------|
|
||||
| Rule-based | 0.908 | 0.000 | — | 0.952 |
|
||||
| Threshold | 0.908 | 0.000 | — | 0.952 |
|
||||
| **Ours (RL)** | **1.000** | **0.004** | **0.575** | **0.998** |
|
||||
|
||||
**使用权重**：`checkpoints/intervention/final_v2.pt`（用 `det_l_risk` 重训）
|
||||
|
||||
---
|
||||
|
||||
## 重要注意事项
|
||||
|
||||
- **PyYAML 6.x 陷阱**：lr 值必须写 `0.001` 而非 `1e-3`（后者被解析为字符串）
|
||||
- **RTX 5090 NCCL**：多卡训练需 `NCCL_SHM_DISABLE=1 NCCL_P2P_DISABLE=1`；PPO 阶段用单卡绕开 barrier 问题
|
||||
- **det_l_risk vs l_risk**：评估和训练均须用检测器预测的 `det_l_risk`，不能用 ground truth `l_risk`
|
||||
- **obs_dim = 2065**：state 向量结构 `[d_score(1)|l_risk_onehot(5)|c_primary_probs(10)|e_H_pool(1024)|e_P_pool(1024)|t_norm(1)]`
|
||||
447
code/change.md
447
code/change.md
@@ -1,447 +0,0 @@
|
||||
# CompanionGuard-RL Change Log and Next-Stage Plan
|
||||
|
||||
**更新时间：2026-05-12**
|
||||
|
||||
## 本次研究判断
|
||||
|
||||
Module C 仍然是本课题的核心创新点，不能降级成附属实验。若目标是 SCI Q2/Q3，论文需要从“检测高风险回复”推进到“根据风险语义选择合适干预动作”，即从 safety detection 走向 adaptive intervention decision。
|
||||
|
||||
当前结果不是方向失败，而是 Module C 的动作策略还没有校准好。Module B 已经能支撑上游检测，下一阶段应集中把 Module C 做成可发表的决策模块。
|
||||
|
||||
## 最新结果位置
|
||||
|
||||
最新测试结果：
|
||||
|
||||
```text
|
||||
code/CompanionGuard-RL/experiments/eval_intervention_v4.json
|
||||
```
|
||||
|
||||
重要确认：
|
||||
|
||||
- `eval_intervention_v4.json` 与 `eval_intervention_v3.json` 内容一致。
|
||||
- v4 不是本地最新版 `src/rl/reward.py` reward-matrix 改动后的重训结果。
|
||||
- 本地 `src/rl/reward.py` 已在 2026-05-12 21:30 后改为矩阵式 reward，用于解决 REJECT collapse、CRISIS precision 低、L4 undertriage，但尚未重新训练并生成新的评估结果。
|
||||
|
||||
## 当前结果摘要
|
||||
|
||||
### Module B 检测器
|
||||
|
||||
Module B 已达到当前论文阶段可用水平：
|
||||
|
||||
| 指标 | 当前结果 |
|
||||
|------|----------|
|
||||
| binary_f1 | 0.9995 |
|
||||
| high_risk_recall | 1.0000 |
|
||||
| false_negative_rate | 0.0000 |
|
||||
| level_macro_f1 | 0.5496 |
|
||||
| level_weighted_f1 | 0.5585 |
|
||||
| fine_macro_f1 | 0.4633 |
|
||||
|
||||
结论：检测器可以作为 frozen upstream detector 进入 Module C，不建议继续把主要时间投入 Module B 微调。
|
||||
|
||||
### Module C 干预策略
|
||||
|
||||
当前 v4 结果：
|
||||
|
||||
| 指标 | 当前结果 | 判断 |
|
||||
|------|----------|------|
|
||||
| safety_recall(L3/L4) | 1.0000 | 安全覆盖很好 |
|
||||
| over_refusal_rate(L0) | 0.0042 | 安全样本误强干预很低 |
|
||||
| action_accuracy | 0.5754 | 不够，低于 0.70 目标 |
|
||||
| crisis_precision | 0.4211 | 不够，CRISIS 触发不够精准 |
|
||||
| safety_ux_fscore | 0.9979 | 指标过粗，区分力不足 |
|
||||
|
||||
Per-level action distribution 暴露的问题：
|
||||
|
||||
| Level | 当前 RL 行为 | 问题 |
|
||||
|-------|--------------|------|
|
||||
| L0 Safe | 98.7% PASS，0.4% REWRITE | 基本可接受 |
|
||||
| L1 Mild | 72.9% PASS，22.9% REWRITE，3.2% CRISIS | 轻微风险处理偏激进 |
|
||||
| L2 Moderate | 90.2% REWRITE，9.8% CRISIS | 对中风险偏重 |
|
||||
| L3 High | 87.1% REWRITE，12.9% CRISIS | 完全没有 REJECT |
|
||||
| L4 Critical | 63.3% REWRITE，36.7% CRISIS | CRISIS 不足，严重风险仍大量只改写 |
|
||||
|
||||
关键问题：
|
||||
|
||||
- RL 学到了“不要漏掉高风险”，但没有学好“动作类型要合适”。
|
||||
- `REJECT` 动作完全坍缩为 0%，动作空间没有被充分利用。
|
||||
- `CRISIS` 被用于部分非 L4 样本，导致 precision 低。
|
||||
- `intervention_recall_high` 和 `safety_ux_fscore` 太宽松，掩盖了动作校准问题。
|
||||
|
||||
## 根因诊断
|
||||
|
||||
### 1. 当前 reward 与标注动作语义存在冲突
|
||||
|
||||
测试集中 `a_recommend` 分布如下：
|
||||
|
||||
| Level | 主要标注动作 |
|
||||
|-------|--------------|
|
||||
| L0 | 100% PASS |
|
||||
| L1 | 99.3% PASS |
|
||||
| L2 | 93.4% WARN |
|
||||
| L3 | 74.3% REWRITE，17.5% REJECT，8.1% CRISIS |
|
||||
| L4 | 55.6% REJECT，44.4% CRISIS |
|
||||
|
||||
但最新版 reward matrix 的理想动作更接近：
|
||||
|
||||
```text
|
||||
L0 -> PASS
|
||||
L1 -> WARN
|
||||
L2 -> REWRITE
|
||||
L3 -> REJECT
|
||||
L4 -> CRISIS
|
||||
```
|
||||
|
||||
这个设计能修复 REJECT/CRISIS 不足，但会显著降低 `action_accuracy`，因为它和数据集现有 `a_recommend` 定义不一致。
|
||||
|
||||
下一阶段不能简单“加大 CRISIS 奖励”，必须先统一动作本体：哪些场景应该 WARN、REWRITE、REJECT、CRISIS。
|
||||
|
||||
### 2. 训练 reward 里类别信号应使用 ground truth
|
||||
|
||||
`CompanionEnv.step()` 当前使用 `sample.get("c_primary_idx", 0)` 传入 reward。该字段来自检测器预测，不是 ground-truth `c_primary`。训练 reward 应该使用 ground-truth category，状态输入仍然使用 detector prediction，这样才符合 offline RL 的训练设定：
|
||||
|
||||
- observation：部署时可见的 detector outputs
|
||||
- reward：训练时可用的标注真值
|
||||
|
||||
否则 R1/CRISIS、R6/R7/REJECT 等类别特异奖励会被 detector category error 稀释。
|
||||
|
||||
### 3. 现有评估指标不足以证明 adaptive intervention
|
||||
|
||||
当前主指标 `safety_recall(L3/L4)` 只要求 action >= REWRITE，因此 REWRITE、REJECT、CRISIS 都算正确。这对安全覆盖有意义，但不能证明策略具有动作选择能力。
|
||||
|
||||
下一阶段必须补充：
|
||||
|
||||
- `action_accuracy` vs `a_recommend`
|
||||
- `exact_action_accuracy_by_level`
|
||||
- `R1_high_critical_crisis_recall`
|
||||
- `crisis_precision_l4_or_r1`
|
||||
- `reject_rate_for_R6_R7_high`
|
||||
- `strong_intervention_rate_on_L1`
|
||||
- `per_category_action_distribution`
|
||||
- `BC-only vs PPO` ablation
|
||||
|
||||
## 下一阶段目标：Module C v5
|
||||
|
||||
### 总目标
|
||||
|
||||
把 Module C 从“高风险都能干预”推进到“能根据风险等级和风险类别选择差异化动作”的 adaptive intervention policy。
|
||||
|
||||
论文中 Module C 应作为第二个主要贡献：
|
||||
|
||||
```text
|
||||
Contribution 1: CompanionRisk-Bench and taxonomy for AI companion risks.
|
||||
Contribution 2: Context-aware detector for companion-specific risk recognition.
|
||||
Contribution 3: Adaptive intervention policy that maps detected risk states to graded safety actions.
|
||||
```
|
||||
|
||||
## v5 技术路线
|
||||
|
||||
### Step 1：冻结 Module B，明确 Module C 的动作语义
|
||||
|
||||
建议采用以下动作本体：
|
||||
|
||||
| 条件 | 目标动作 |
|
||||
|------|----------|
|
||||
| L0 Safe | PASS |
|
||||
| L1 Mild | PASS，少量 WARN |
|
||||
| L2 Moderate | WARN 为主，少量 REWRITE |
|
||||
| L3 High | REWRITE 为主，类别严重时 REJECT/CRISIS |
|
||||
| L4 Critical + R1 | CRISIS |
|
||||
| L4 Critical + R6/R7 | REJECT |
|
||||
| L4 Critical + R5/R8/R9 | REJECT 或 CRISIS，按风险内容决定 |
|
||||
| L4 Critical + R2/R3/R4/R10 | REWRITE 或 REJECT，避免一律 CRISIS |
|
||||
|
||||
核心原则：
|
||||
|
||||
- CRISIS 不应等价于“所有 L4”，它应主要用于 R1 自伤/自杀危机和明确紧急求助场景。
|
||||
- REJECT 不应消失，它适合不可安全改写的暴力、未成年人亲密、隐私诱导、危险操作等场景。
|
||||
- REWRITE 是 companion 场景最有价值的动作，适合依赖强化、隔离强化、错误安慰、伪治疗等可修复回复。
|
||||
|
||||
### Step 2：重写 reward 为 label-aligned constrained reward
|
||||
|
||||
新 reward 不应把 level-to-action 映射写死成 `L1->WARN, L2->REWRITE, L3->REJECT, L4->CRISIS`。应以 `a_recommend` 为主目标，再加入安全约束和类别约束。
|
||||
|
||||
建议 reward 结构：
|
||||
|
||||
```text
|
||||
reward =
|
||||
exact_action_alignment
|
||||
+ safety_constraint
|
||||
+ category_specific_bonus
|
||||
- false_negative_penalty
|
||||
- over_intervention_penalty
|
||||
- crisis_misuse_penalty
|
||||
- reject_misuse_penalty
|
||||
```
|
||||
|
||||
建议权重：
|
||||
|
||||
| 项 | 建议值 | 说明 |
|
||||
|----|--------|------|
|
||||
| exact_action_alignment | +3.0 | 与 `a_recommend` 一致应是主奖励 |
|
||||
| adjacent_action_partial | +1.0 | 相邻合理动作给小奖励，例如 L3 REWRITE/REJECT |
|
||||
| PASS on L3/L4 | -5.0 | 高危漏干预必须重罚 |
|
||||
| PASS on L2 | -2.0 | 中风险漏处理轻罚 |
|
||||
| strong intervention on L0 | -5.0 | 安全内容被 REWRITE/REJECT/CRISIS 重罚 |
|
||||
| CRISIS on non-R1 and non-L4 | -3.0 | 降低 CRISIS 泛化滥用 |
|
||||
| CRISIS on R1 L3/L4 | +3.0 | 提高危机干预召回 |
|
||||
| REJECT on R6/R7 L3/L4 | +2.0 | 提升不可安全回复场景的拒绝能力 |
|
||||
| REWRITE on R3/R4/R10 L2/L3 | +1.5 | 突出 companion-specific adaptive rewriting |
|
||||
|
||||
### Step 3：修复训练环境类别信号
|
||||
|
||||
修改：
|
||||
|
||||
```text
|
||||
src/rl/companion_env.py
|
||||
```
|
||||
|
||||
训练 reward 中增加 ground-truth `c_primary` 到 index 的转换：
|
||||
|
||||
```python
|
||||
from src.utils.taxonomy import category_to_index
|
||||
|
||||
gt_category = sample.get("c_primary", "None")
|
||||
if gt_category in PRIMARY_CATEGORY_LIST:
|
||||
reward_category_idx = category_to_index(gt_category)
|
||||
else:
|
||||
reward_category_idx = int(sample.get("c_primary_idx", 0))
|
||||
```
|
||||
|
||||
然后把 `reward_category_idx` 传给 `compute_reward()`。
|
||||
|
||||
### Step 4：加入 BC-only 和 PPO v5 对照
|
||||
|
||||
需要新增或保留三类策略：
|
||||
|
||||
| 策略 | 作用 |
|
||||
|------|------|
|
||||
| Rule/Threshold | 规则基线 |
|
||||
| BC-only | 证明监督动作学习能达到的上限或稳定性 |
|
||||
| BC + PPO v5 | 证明 reward 优化带来的安全和类别动作收益 |
|
||||
|
||||
BC-only 很重要。如果 PPO v5 未明显超过 BC-only，也可以把论文叙事调整为“supervised warm-up with constrained RL fine-tuning”，而不是硬说 PPO 是唯一贡献。
|
||||
|
||||
### Step 5：扩展评估指标
|
||||
|
||||
修改：
|
||||
|
||||
```text
|
||||
src/utils/metrics.py
|
||||
scripts/evaluate.py
|
||||
```
|
||||
|
||||
新增指标：
|
||||
|
||||
| 指标 | 目标 |
|
||||
|------|------|
|
||||
| action_accuracy | >= 0.70 |
|
||||
| exact_action_accuracy_L4 | >= 0.65 |
|
||||
| R1_high_critical_crisis_recall | >= 0.80 |
|
||||
| crisis_precision | >= 0.65，理想 >= 0.80 |
|
||||
| reject_rate_R6_R7_high | >= 0.60 |
|
||||
| strong_intervention_rate_L1 | <= 0.05 |
|
||||
| safety_recall_L3_L4 | >= 0.95 |
|
||||
| over_refusal_L0 | <= 0.02 |
|
||||
|
||||
这些指标比单独 `safety_ux_fscore` 更能支撑“adaptive”。
|
||||
|
||||
### Step 6：重训并产出 v5
|
||||
|
||||
建议输出文件：
|
||||
|
||||
```text
|
||||
checkpoints/intervention/final_v5.pt
|
||||
experiments/train_intervention_v5_YYYYMMDD_HHMMSS.log
|
||||
experiments/eval_intervention_v5.json
|
||||
```
|
||||
|
||||
建议训练命令：
|
||||
|
||||
```bash
|
||||
cd /root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL
|
||||
export PYTHONPATH=$PWD
|
||||
CUDA_VISIBLE_DEVICES=0 \
|
||||
/opt/conda/envs/dlapo-py310-cu128/bin/accelerate launch \
|
||||
--num_processes=1 --mixed_precision=bf16 \
|
||||
scripts/train_intervention.py \
|
||||
--config configs/intervention_config.yaml \
|
||||
--train-data data/processed/CompanionRisk-Bench/train.jsonl \
|
||||
> experiments/train_intervention_v5_$(date +%Y%m%d_%H%M%S).log 2>&1
|
||||
```
|
||||
|
||||
评估命令：
|
||||
|
||||
```bash
|
||||
python scripts/evaluate.py \
|
||||
--detector-ckpt checkpoints/detector/best.pt \
|
||||
--agent-ckpt checkpoints/intervention/final.pt \
|
||||
--test-data data/processed/CompanionRisk-Bench/test.jsonl \
|
||||
--config configs/detector_config_server.yaml \
|
||||
--intervention-config configs/intervention_config.yaml \
|
||||
--output experiments/eval_intervention_v5.json
|
||||
```
|
||||
|
||||
完成后将 `final.pt` 另存为：
|
||||
|
||||
```bash
|
||||
cp checkpoints/intervention/final.pt checkpoints/intervention/final_v5.pt
|
||||
```
|
||||
|
||||
## v5 成败判定
|
||||
|
||||
### 可作为论文主结果的标准
|
||||
|
||||
满足以下多数条件即可作为主结果：
|
||||
|
||||
| 指标 | 最低可接受 | 理想 |
|
||||
|------|------------|------|
|
||||
| safety_recall_L3_L4 | >= 0.95 | >= 0.98 |
|
||||
| over_refusal_L0 | <= 0.02 | <= 0.01 |
|
||||
| action_accuracy | >= 0.70 | >= 0.75 |
|
||||
| crisis_precision | >= 0.65 | >= 0.80 |
|
||||
| R1_high_critical_crisis_recall | >= 0.80 | >= 0.90 |
|
||||
| strong_intervention_rate_L1 | <= 0.05 | <= 0.03 |
|
||||
| REJECT usage | 非 0，且集中在 R6/R7/L4 | 类别分布合理 |
|
||||
|
||||
### 如果 v5 未达标
|
||||
|
||||
不要继续盲目调 PPO。采用备选路线：
|
||||
|
||||
1. 使用 BC-only 作为主策略，PPO 作为 ablation。
|
||||
2. 引入 constrained decoding policy：模型输出动作 logits 后，用规则 mask 禁止明显不合理动作。
|
||||
3. 将 Module C 表述为 hybrid adaptive policy：learned policy + safety constraints。
|
||||
4. 把重点指标从 `crisis_precision` 转为 category-aware intervention quality。
|
||||
|
||||
## 论文写法建议
|
||||
|
||||
Module C 的论文叙事应避免只说“RL 比规则好”。更强的说法是：
|
||||
|
||||
```text
|
||||
Existing safety systems usually stop at risk classification.
|
||||
CompanionGuard-RL further learns a graded intervention policy that maps contextual risk states to differentiated actions, including pass-through, warning, rewriting, rejection, and crisis escalation.
|
||||
```
|
||||
|
||||
实验表格建议：
|
||||
|
||||
1. Detection comparison: L1 rules vs Module B.
|
||||
2. Intervention summary: Rule, Threshold, BC-only, PPO v5.
|
||||
3. Per-level action distribution.
|
||||
4. Per-category action distribution for R1/R3/R4/R6/R7/R10.
|
||||
5. Ablation: without category-specific reward, without alignment reward, without PPO.
|
||||
|
||||
## 二次审查新增隐患（2026-05-12）
|
||||
|
||||
### 隐患 1：`action_accuracy` 可能变成循环论证
|
||||
|
||||
`a_recommend` 大量来自生成脚本和规则映射，不是完全独立的人类专家标注。如果 v5 reward 以 `a_recommend` 为主，最后再用 `action_accuracy` 证明策略好，审稿人可能质疑这是“训练目标和评估指标同源”。
|
||||
|
||||
应对：
|
||||
|
||||
- `action_accuracy` 可以保留，但不能作为唯一主指标。
|
||||
- 必须同时报告 safety/category 指标：R1 crisis recall、R6/R7 reject rate、L1 strong intervention rate、per-category action distribution。
|
||||
- 抽样 50-100 条 Module C 预测结果做人类复核，作为 intervention quality case audit。
|
||||
|
||||
### 隐患 2：一阶 MDP 使用 PPO 的合理性可能被质疑
|
||||
|
||||
当前 `CompanionEnv` 是 single-step MDP，每个样本一步结束。严格来说，这更像 contextual bandit / reward-regularized policy learning，而不是典型多步 RL。若论文强行强调 PPO，SCI 审稿人可能问：为什么不用 cost-sensitive classifier 或 supervised policy network？
|
||||
|
||||
应对：
|
||||
|
||||
- 论文中避免夸大“长期序列决策”，把 Module C 表述为 reward-optimized adaptive intervention policy。
|
||||
- 实验中加入 BC-only、cost-sensitive classifier 或 rule-masked classifier 对照。
|
||||
- 如果时间允许，后续再扩展 multi-turn intervention simulation；当前 v5 先把单步策略做扎实。
|
||||
|
||||
### 隐患 3：BC-only 可能已经足够，PPO 增益不明显
|
||||
|
||||
当前计划提到 BC-only，但还没有明确保存 BC-only checkpoint。如果 PPO v5 只是把 BC 学到的动作重新扰动一遍，可能无法证明 RL 部分的必要性。
|
||||
|
||||
应对：
|
||||
|
||||
- 训练脚本应在 BC 结束后保存 `checkpoints/intervention/bc_only_v5.pt`。
|
||||
- 评估表必须包含 `BC-only` 与 `BC+PPO v5`。
|
||||
- PPO 的成功标准应是：不显著降低 `action_accuracy`，同时提升 safety/category 指标，例如 R1 crisis recall 或 R6/R7 reject rate。
|
||||
|
||||
### 隐患 4：`crisis_precision` 定义需要和动作语义统一
|
||||
|
||||
当前 `metrics.py` 中 `crisis_precision` 只把 L4 算作正确 CRISIS。如果 v5 动作语义允许 R1 L3 也触发 CRISIS，那么旧 `crisis_precision` 会把合理的 R1 L3 CRISIS 当成错误，导致指标和论文定义冲突。
|
||||
|
||||
应对：
|
||||
|
||||
- 保留旧指标并改名为 `crisis_precision_l4`。
|
||||
- 新增 `crisis_appropriateness = CRISIS on (L4 or R1 with L3/L4)`。
|
||||
- 新增 `R1_high_critical_crisis_recall`，单独证明危机响应能力。
|
||||
|
||||
### 隐患 5：训练状态使用 detector train-set 预测，可能有过拟合痕迹
|
||||
|
||||
Module C 的训练 observation 来自 frozen detector 对 train set 的预测，而 detector 本身也在 train set 上训练过。这样得到的 `det_l_risk` 和 category probs 可能比真实部署更干净，导致 Module C 训练环境偏乐观。
|
||||
|
||||
应对：
|
||||
|
||||
- 短期：在论文中明确 Module C 训练使用 frozen detector outputs，评估在 held-out test 上完成。
|
||||
- 中期：加入 detector noise augmentation，例如随机扰动 level one-hot 或 category probs，增强策略鲁棒性。
|
||||
- 最稳：用 out-of-fold detector predictions 构建 Module C 训练状态，但这需要额外重训多个 detector，当前不是优先项。
|
||||
|
||||
### 隐患 6：checkpoint 覆盖会污染结果追踪
|
||||
|
||||
当前训练脚本固定保存到 `checkpoints/intervention/final.pt`。如果直接重训 v5，旧的 v3/v4 权重可能被覆盖，后续无法复现表格。
|
||||
|
||||
应对：
|
||||
|
||||
- 训练前先复制当前权重：
|
||||
|
||||
```bash
|
||||
cp checkpoints/intervention/final.pt checkpoints/intervention/final_v4_before_v5.pt
|
||||
```
|
||||
|
||||
- BC 后保存：
|
||||
|
||||
```text
|
||||
checkpoints/intervention/bc_only_v5.pt
|
||||
```
|
||||
|
||||
- PPO 后保存：
|
||||
|
||||
```text
|
||||
checkpoints/intervention/final_v5.pt
|
||||
```
|
||||
|
||||
### 隐患 7：`wandb` 和配置可能导致训练卡住
|
||||
|
||||
当前本地 `configs/intervention_config.yaml` 中 `use_wandb: true`，且 `scripts/train_intervention.py` 存在直接 `import wandb`。服务器受限环境下容易因为 wandb 缺失、未登录或网络不可用导致训练失败或卡住。
|
||||
|
||||
应对：
|
||||
|
||||
- v5 配置固定设置 `use_wandb: false`。
|
||||
- 或在启动命令中加入：
|
||||
|
||||
```bash
|
||||
export WANDB_MODE=disabled
|
||||
```
|
||||
|
||||
- 最好把 `import wandb` 改为 try/except，保持离线训练可运行。
|
||||
|
||||
### 隐患 8：缺少最小单元测试，reward 改动容易反向破坏指标
|
||||
|
||||
当前项目没有 `tests/` 目录。v5 会改 reward、env、metrics，如果没有最小测试，很容易出现“训练能跑但指标含义错了”的问题。
|
||||
|
||||
应对：
|
||||
|
||||
- 新增 `tests/test_reward_v5.py`，覆盖 L0/L1/L2/L3/L4 和 R1/R6/R7 类别奖励。
|
||||
- 新增 `tests/test_intervention_metrics.py`，覆盖 crisis appropriateness、R1 recall、reject rate、strong intervention on L1。
|
||||
- 在远程训练前先本地跑通这些小测试。
|
||||
|
||||
## 立即执行清单
|
||||
|
||||
- [ ] 修改 `src/rl/reward.py` 为 label-aligned constrained reward。
|
||||
- [ ] 修改 `src/rl/companion_env.py`，reward 使用 ground-truth `c_primary`。
|
||||
- [ ] 修改 `src/utils/metrics.py`，新增 category-aware intervention metrics。
|
||||
- [ ] 修改 `scripts/evaluate.py`，输出新指标和 BC-only 对照。
|
||||
- [ ] 保存当前 v4 权重，避免 v5 覆盖旧结果。
|
||||
- [ ] 在 BC 结束时保存 `bc_only_v5.pt`。
|
||||
- [ ] 关闭或离线化 wandb。
|
||||
- [ ] 增加 reward 和 metrics 的最小单元测试。
|
||||
- [ ] 训练 Module C v5。
|
||||
- [ ] 生成 `experiments/eval_intervention_v5.json`。
|
||||
- [ ] 更新 `2026-05-12-state.md` 或新建 `2026-05-13-state.md`。
|
||||
- [ ] 根据 v5 结果决定论文主表和 limitation 写法。
|
||||
476
code/exp.md
476
code/exp.md
@@ -1,476 +0,0 @@
|
||||
# CompanionGuard-RL — 可复用经验库
|
||||
**创建时间：2026-05-12**
|
||||
**来源：Module B + Module C 训练调试过程中积累的真实踩坑记录**
|
||||
|
||||
---
|
||||
|
||||
## 目录
|
||||
|
||||
1. [RTX 5090 / NCCL 通信问题](#1-rtx-5090--nccl-通信问题)
|
||||
2. [HuggingFace Accelerate 多 GPU 分布式训练](#2-huggingface-accelerate-多-gpu-分布式训练)
|
||||
3. [PyYAML 配置文件陷阱](#3-pyyaml-配置文件陷阱)
|
||||
4. [服务器文件传输（无 rsync 环境）](#4-服务器文件传输无-rsync-环境)
|
||||
5. [SSH 连接与持久会话管理](#5-ssh-连接与持久会话管理)
|
||||
6. [Python 依赖与包缺失处理](#6-python-依赖与包缺失处理)
|
||||
7. [分布式训练中的 Tensor 设备一致性](#7-分布式训练中的-tensor-设备一致性)
|
||||
8. [DataLoader 与分布式训练的兼容](#8-dataloader-与分布式训练的兼容)
|
||||
9. [离线服务器的模型加载](#9-离线服务器的模型加载)
|
||||
10. [Shell 脚本跨平台问题（CRLF）](#10-shell-脚本跨平台问题crlf)
|
||||
11. [Python 模块路径（PYTHONPATH）](#11-python-模块路径pythonpath)
|
||||
12. [可选依赖的优雅处理（wandb 等）](#12-可选依赖的优雅处理wandb-等)
|
||||
|
||||
---
|
||||
|
||||
## 1. RTX 5090 / NCCL 通信问题
|
||||
|
||||
### 症状
|
||||
```
|
||||
[rank0]: CUDA error: an illegal memory access was encountered
|
||||
```
|
||||
在多 GPU 训练中，某一阶段（如 BC warmup 后进入 PPO，或切换数据集后）突发崩溃，单 GPU 无此问题。
|
||||
|
||||
### 根因
|
||||
RTX 5090 的 NVLink/P2P 拓扑与 NCCL 默认的共享内存（SHM）和 P2P 直连通信不兼容，导致跨 GPU 内存访问越界。
|
||||
|
||||
### 解决方案
|
||||
```bash
|
||||
# 同时禁用 SHM 和 P2P，强制 NCCL 走 socket 通信
|
||||
export NCCL_SHM_DISABLE=1
|
||||
export NCCL_P2P_DISABLE=1
|
||||
```
|
||||
|
||||
**在 accelerate launch 前设置（推荐写法）：**
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 NCCL_SHM_DISABLE=1 NCCL_P2P_DISABLE=1 \
|
||||
accelerate launch --num_processes=4 --mixed_precision=bf16 \
|
||||
scripts/train_xxx.py ...
|
||||
```
|
||||
|
||||
### 排查顺序
|
||||
1. 先加 `NCCL_SHM_DISABLE=1` → 若仍崩溃
|
||||
2. 再加 `NCCL_P2P_DISABLE=1` → 通常可解
|
||||
3. 若仍有问题，尝试 `NCCL_DEBUG=INFO` 查看具体哪个集合通信操作出错
|
||||
|
||||
### 性能影响
|
||||
禁用 P2P 后 GPU 间通信走 PCIe，带宽略降，但对 batch_size=256 量级的训练影响不超过 10%。
|
||||
|
||||
---
|
||||
|
||||
## 2. HuggingFace Accelerate 多 GPU 分布式训练
|
||||
|
||||
### accelerate 路径问题
|
||||
服务器有多个 conda 环境时，直接敲 `accelerate` 可能用到错误环境的版本，或报 `command not found`。
|
||||
|
||||
**正确做法：用 conda 环境的完整路径**
|
||||
```bash
|
||||
# 查找正确路径
|
||||
find /opt/conda/envs -name "accelerate" -type f 2>/dev/null
|
||||
|
||||
# 使用完整路径启动
|
||||
/opt/conda/envs/dlapo-py310-cu128/bin/accelerate launch ...
|
||||
```
|
||||
|
||||
### PYTHONPATH 设置
|
||||
使用 `accelerate launch` 时，各 rank 子进程不继承当前 shell 的 `sys.path`，自定义 `src/` 包会报 `ModuleNotFoundError`。
|
||||
|
||||
```bash
|
||||
PYTHONPATH=/path/to/project accelerate launch ...
|
||||
```
|
||||
|
||||
### 推荐完整启动命令模板
|
||||
```bash
|
||||
cd /path/to/project
|
||||
PYTHONPATH=$(pwd) \
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
||||
NCCL_SHM_DISABLE=1 \
|
||||
NCCL_P2P_DISABLE=1 \
|
||||
/opt/conda/envs/<env>/bin/accelerate launch \
|
||||
--num_processes=4 \
|
||||
--mixed_precision=bf16 \
|
||||
scripts/train_xxx.py \
|
||||
--config configs/xxx.yaml \
|
||||
> experiments/train_$(date +%Y%m%d_%H%M%S).log 2>&1 &
|
||||
echo "PID: $! LOG: $LOG"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. PyYAML 配置文件陷阱
|
||||
|
||||
### 症状
|
||||
```
|
||||
TypeError: '<=' not supported between instances of 'float' and 'str'
|
||||
```
|
||||
明明写的是数字，PyYAML 却解析成字符串。
|
||||
|
||||
### 根因
|
||||
**PyYAML 6.x 将科学计数法（如 `1e-3`、`3e-4`）解析为字符串，而非浮点数。**
|
||||
|
||||
PyYAML 5.x 以下正常，6.x 以上需要避免。
|
||||
|
||||
### 解决方案
|
||||
将所有科学计数法改为小数形式：
|
||||
```yaml
|
||||
# ❌ 会被解析为字符串
|
||||
lr: 1e-3
|
||||
lr: 3e-4
|
||||
|
||||
# ✅ 正确写法
|
||||
lr: 0.001
|
||||
lr: 0.0003
|
||||
```
|
||||
|
||||
### 快速检查
|
||||
```python
|
||||
import yaml
|
||||
cfg = yaml.safe_load(open("config.yaml"))
|
||||
print(type(cfg["lr"])) # 应为 <class 'float'>，若为 <class 'str'> 则有问题
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 服务器文件传输（无 rsync 环境）
|
||||
|
||||
### 背景
|
||||
- 本地 Windows，目标 Linux GPU 服务器
|
||||
- 本地 WSL 无 `rsync`，PowerShell 无原生 rsync
|
||||
- 文件较多，直接 `scp -r` 速度慢且不方便增量同步
|
||||
|
||||
### 推荐方案：tar 打包 + scp 单文件传输
|
||||
|
||||
**本地打包（PowerShell）：**
|
||||
```powershell
|
||||
# 打包项目代码（排除数据集、checkpoint、缓存）
|
||||
tar -czf sync_v4.tar.gz `
|
||||
-C "D:\Myresearch\CompanionGuard-RL\code\CompanionGuard-RL" `
|
||||
--exclude=".git" --exclude="__pycache__" `
|
||||
--exclude="checkpoints" --exclude="experiments" `
|
||||
src scripts configs requirements.txt
|
||||
|
||||
# 使用 WSL sshpass 上传
|
||||
wsl -d Ubuntu-24.04 -- sshpass -p 'PASSWORD' scp -P PORT \
|
||||
/mnt/d/Myresearch/CompanionGuard-RL/sync_v4.tar.gz \
|
||||
root@HOST:/remote/path/
|
||||
```
|
||||
|
||||
**服务器解压（覆盖更新）：**
|
||||
```bash
|
||||
cd /remote/project/dir
|
||||
tar -xzf ../sync_v4.tar.gz --strip-components=0
|
||||
```
|
||||
|
||||
### Windows 路径转 WSL 路径
|
||||
```
|
||||
D:\Myresearch\... → /mnt/d/Myresearch/...
|
||||
```
|
||||
|
||||
### sshpass 在 WSL 中使用
|
||||
```bash
|
||||
# 安装
|
||||
sudo apt-get install sshpass
|
||||
|
||||
# 密码直接传参（注意在脚本中要保护密码）
|
||||
sshpass -p 'PASSWORD' ssh -p PORT user@host 'command'
|
||||
sshpass -p 'PASSWORD' scp -P PORT local_file user@host:/remote/path/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. SSH 连接与持久会话管理
|
||||
|
||||
### nohup vs tmux
|
||||
| 方式 | 优点 | 缺点 |
|
||||
|------|------|------|
|
||||
| `nohup ... &` | 简单 | 非交互式 SSH 中 nohup 进程在连接断开后有时会收到 SIGHUP 而退出；无法重新 attach 查看输出 |
|
||||
| `tmux` | 会话持久，可 attach/detach，输出可随时查看 | 需要服务器安装 tmux |
|
||||
|
||||
**推荐用 tmux：**
|
||||
```bash
|
||||
# 创建新会话并启动训练
|
||||
tmux new-session -d -s train 'PYTHONPATH=... accelerate launch ...'
|
||||
|
||||
# 查看所有会话
|
||||
tmux ls
|
||||
|
||||
# 重新连接查看输出
|
||||
tmux attach -t train
|
||||
|
||||
# 在会话中执行命令（不 attach）
|
||||
tmux send-keys -t train 'tail -f experiments/latest.log' Enter
|
||||
```
|
||||
|
||||
### SSH 连接被拒绝但 ping 通（kex_exchange_identification）
|
||||
症状：TCP 端口开放，ping 通，但 SSH 在握手前被关闭：
|
||||
```
|
||||
kex_exchange_identification: Connection closed by remote host
|
||||
```
|
||||
|
||||
可能原因及处理：
|
||||
1. **sshd 崩溃/重启中** → 通过网页控制台（VNC）执行 `systemctl restart sshd`
|
||||
2. **MaxStartups 限制** → sshd_config 中 `MaxStartups 10:30:60` 可临时调高
|
||||
3. **fail2ban 封 IP** → `fail2ban-client status sshd`，`fail2ban-client set sshd unbanip <IP>`
|
||||
|
||||
---
|
||||
|
||||
## 6. Python 依赖与包缺失处理
|
||||
|
||||
### 服务器无网络时安装包
|
||||
|
||||
**方法一：从已有 conda 环境复制**
|
||||
```bash
|
||||
# 查找其他环境中的包位置
|
||||
find /opt/conda/envs -name "gymnasium" -type d 2>/dev/null
|
||||
|
||||
# 直接复制到目标环境
|
||||
cp -r /opt/conda/envs/other-env/lib/python3.10/site-packages/gymnasium \
|
||||
/opt/conda/envs/target-env/lib/python3.10/site-packages/
|
||||
```
|
||||
|
||||
**方法二：本地下载 wheel，scp 传输，离线安装**
|
||||
```powershell
|
||||
# 本地下载（PowerShell）
|
||||
pip download -d D:\wheels --platform linux_x86_64 --python-version 310 \
|
||||
--only-binary=:all: gymnasium
|
||||
# scp 传到服务器后：
|
||||
pip install --no-index --find-links=/path/to/wheels gymnasium
|
||||
```
|
||||
|
||||
### 检查包是否可用
|
||||
```bash
|
||||
python -c "import gymnasium; print(gymnasium.__version__)"
|
||||
python -c "import torch; print(torch.cuda.device_count())"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 分布式训练中的 Tensor 设备一致性
|
||||
|
||||
### 症状
|
||||
```
|
||||
RuntimeError: No backend type associated with device type cpu
|
||||
```
|
||||
在 `torch.distributed.broadcast()` 等集合通信操作中，传入了 CPU tensor。
|
||||
|
||||
### 根因
|
||||
**NCCL 后端只支持 CUDA tensor**，所有参与 `broadcast/all_reduce/gather` 的 tensor 必须在 GPU 上。
|
||||
|
||||
### 修复模式
|
||||
```python
|
||||
dev = accelerator.device # 当前 rank 的 CUDA device
|
||||
|
||||
# 广播 size
|
||||
size_tensor = torch.tensor([data.shape[0]], dtype=torch.long, device=dev)
|
||||
torch.distributed.broadcast(size_tensor, src=0)
|
||||
n = size_tensor.item()
|
||||
|
||||
# 广播数据
|
||||
if accelerator.is_main_process:
|
||||
data = data.to(dev)
|
||||
else:
|
||||
data = torch.zeros(n, data_dim, device=dev) # 必须在 GPU 上
|
||||
|
||||
torch.distributed.broadcast(data, src=0)
|
||||
# 使用后如需 CPU，再 .cpu()
|
||||
```
|
||||
|
||||
### 关键原则
|
||||
- 集合通信（broadcast/all_reduce/scatter）→ **必须 CUDA tensor**
|
||||
- DataLoader 输入 → **CPU tensor**（除非 `pin_memory=False`）
|
||||
- 在 GPU 计算完成后，如需放入 CPU DataLoader，显式 `.cpu()`
|
||||
|
||||
---
|
||||
|
||||
## 8. DataLoader 与分布式训练的兼容
|
||||
|
||||
### pin_memory 陷阱
|
||||
```
|
||||
RuntimeError: cannot pin torch.cuda.FloatTensor
|
||||
```
|
||||
`DataLoader(pin_memory=True)` 要求数据必须是 **CPU tensor**，若传入已在 GPU 上的 tensor 则报错。
|
||||
|
||||
**修复：构建 TensorDataset 前先移到 CPU**
|
||||
```python
|
||||
# ❌ 若 obs_tensor 在 GPU 上会崩溃
|
||||
dataset = TensorDataset(obs_tensor, action_tensor)
|
||||
loader = DataLoader(dataset, pin_memory=True)
|
||||
|
||||
# ✅ 先 .cpu()
|
||||
dataset = TensorDataset(obs_tensor.cpu(), action_tensor.cpu())
|
||||
loader = DataLoader(dataset, pin_memory=True)
|
||||
```
|
||||
|
||||
### set_epoch 守卫
|
||||
```
|
||||
AttributeError: 'SequentialSampler' object has no attribute 'set_epoch'
|
||||
```
|
||||
`set_epoch` 只有 `DistributedSampler` 有，`SequentialSampler` 没有。
|
||||
|
||||
**修复：加 hasattr 守卫**
|
||||
```python
|
||||
# ❌ 直接调用
|
||||
loader.sampler.set_epoch(epoch)
|
||||
|
||||
# ✅ 安全写法
|
||||
if hasattr(loader.sampler, "set_epoch"):
|
||||
loader.sampler.set_epoch(epoch)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 离线服务器的模型加载
|
||||
|
||||
### 症状
|
||||
```
|
||||
OSError: Can't load tokenizer for 'hfl/chinese-macbert-large'.
|
||||
```
|
||||
服务器无法访问 HuggingFace，在线下载失败。
|
||||
|
||||
### 解决方案
|
||||
|
||||
**方法一：本地下载后 scp**
|
||||
```powershell
|
||||
# 本地下载
|
||||
python -c "
|
||||
from huggingface_hub import snapshot_download
|
||||
snapshot_download('hfl/chinese-macbert-large', local_dir='D:/models/macbert-large')
|
||||
"
|
||||
# 上传到服务器
|
||||
scp -P PORT -r D:\models\macbert-large root@HOST:/remote/models/macbert-large
|
||||
```
|
||||
|
||||
**方法二：用国内镜像（若服务器能访问）**
|
||||
```bash
|
||||
HF_ENDPOINT=https://hf-mirror.com \
|
||||
python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained('hfl/chinese-macbert-large')"
|
||||
```
|
||||
|
||||
**更新配置文件：**
|
||||
```yaml
|
||||
# 将 HuggingFace model id 改为本地绝对路径
|
||||
model_name: "/root/path/to/macbert-large"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Shell 脚本跨平台问题（CRLF）
|
||||
|
||||
### 症状
|
||||
```
|
||||
/bin/bash^M: bad interpreter: No such file or directory
|
||||
```
|
||||
或脚本执行后立即退出，没有任何错误信息。
|
||||
|
||||
### 根因
|
||||
Windows 上编辑/保存的 `.sh` 文件使用 CRLF（`\r\n`）换行，Linux 只认 LF（`\n`），`^M`（即 `\r`）被当作命令的一部分。
|
||||
|
||||
### 修复方案
|
||||
|
||||
**PowerShell 写入时强制 LF：**
|
||||
```powershell
|
||||
$content = @'
|
||||
#!/bin/bash
|
||||
cd /project/dir
|
||||
ACCEL=/path/to/accelerate
|
||||
nohup $ACCEL launch ... > log.txt 2>&1 &
|
||||
echo "PID: $!"
|
||||
'@
|
||||
# 关键：用 Replace 去掉 \r，用 UTF8NoBOM 编码
|
||||
[System.IO.File]::WriteAllText(
|
||||
"D:\path\to\script.sh",
|
||||
$content.Replace("`r`n", "`n"),
|
||||
[System.Text.UTF8Encoding]::new($false)
|
||||
)
|
||||
```
|
||||
|
||||
**事后修复（在 Linux 服务器上）：**
|
||||
```bash
|
||||
sed -i 's/\r//' script.sh
|
||||
# 或
|
||||
dos2unix script.sh
|
||||
```
|
||||
|
||||
**验证：**
|
||||
```bash
|
||||
file script.sh # 应显示 "ASCII text" 而非 "CRLF line terminators"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Python 模块路径（PYTHONPATH）
|
||||
|
||||
### 症状
|
||||
```
|
||||
ModuleNotFoundError: No module named 'src'
|
||||
```
|
||||
项目结构是 `src/models/`，但脚本中 `from src.models import ...` 找不到。
|
||||
|
||||
### 根因
|
||||
`accelerate launch` / `torchrun` 启动的子进程工作目录不一定是项目根目录，`sys.path` 不包含项目根目录。
|
||||
|
||||
### 解决方案
|
||||
|
||||
**方案一：启动时设置 PYTHONPATH（推荐）**
|
||||
```bash
|
||||
PYTHONPATH=/root/path/to/project accelerate launch scripts/train.py
|
||||
```
|
||||
|
||||
**方案二：在脚本开头动态添加**
|
||||
```python
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
```
|
||||
|
||||
**方案三：项目根目录加 `__init__.py`（不推荐，污染命名空间）**
|
||||
|
||||
---
|
||||
|
||||
## 12. 可选依赖的优雅处理（wandb 等）
|
||||
|
||||
### 背景
|
||||
`wandb` 有复杂的依赖树（`sentry-sdk`、`setproctitle` 等），在受限环境中难以安装。
|
||||
|
||||
### 推荐模式：try/except 导入 + 功能开关
|
||||
|
||||
**导入部分：**
|
||||
```python
|
||||
try:
|
||||
import wandb
|
||||
WANDB_AVAILABLE = True
|
||||
except ImportError:
|
||||
wandb = None
|
||||
WANDB_AVAILABLE = False
|
||||
```
|
||||
|
||||
**使用部分：**
|
||||
```python
|
||||
if use_wandb and WANDB_AVAILABLE:
|
||||
wandb.log({"loss": loss})
|
||||
elif use_wandb and not WANDB_AVAILABLE:
|
||||
if step == 0:
|
||||
print("[WARN] wandb not available, skipping logging")
|
||||
```
|
||||
|
||||
**配置文件：**
|
||||
```yaml
|
||||
# 生产/受限环境
|
||||
use_wandb: false
|
||||
|
||||
# 开发环境
|
||||
use_wandb: true
|
||||
```
|
||||
|
||||
这样即使 wandb 未安装，训练也能正常运行，不会因为一行 `import wandb` 而整个崩溃。
|
||||
|
||||
---
|
||||
|
||||
## 附：本项目服务器快速参考
|
||||
|
||||
| 项目 | 值 |
|
||||
|------|-----|
|
||||
| SSH | `ssh -p 22657 root@connected.svt.net.cn` |
|
||||
| 备用 SSH | `ssh -p 20083 root@10.82.3.180` |
|
||||
| 密码 | `yx123456` |
|
||||
| conda 环境 | `dlapo-py310-cu128` |
|
||||
| accelerate 路径 | `/opt/conda/envs/dlapo-py310-cu128/bin/accelerate` |
|
||||
| 项目目录 | `/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/CompanionGuard-RL` |
|
||||
| MacBERT 本地路径 | `/root/siton-data-2849d4ce327c4ccfb233ce33868fe7fe/zsy/macbert-large` |
|
||||
@@ -1,277 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data\\processed\\CompanionRisk-Bench\\test.jsonl",
|
||||
"lang": "zh",
|
||||
"n": 486
|
||||
},
|
||||
"L0_all_risky": {
|
||||
"name": "L0_all_risky",
|
||||
"accuracy": 0.7346,
|
||||
"precision": 0.7346,
|
||||
"recall": 1.0,
|
||||
"f1": 0.847,
|
||||
"fnr": 0.0,
|
||||
"tp": 357,
|
||||
"fp": 129,
|
||||
"fn": 0,
|
||||
"tn": 0
|
||||
},
|
||||
"L0_all_safe": {
|
||||
"name": "L0_all_safe",
|
||||
"accuracy": 0.2654,
|
||||
"precision": 0.0,
|
||||
"recall": 0.0,
|
||||
"f1": 0.0,
|
||||
"fnr": 1.0,
|
||||
"tp": 0,
|
||||
"fp": 0,
|
||||
"fn": 357,
|
||||
"tn": 129
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"name": "L1a_keyword",
|
||||
"accuracy": 0.4198,
|
||||
"precision": 0.9518,
|
||||
"recall": 0.2213,
|
||||
"f1": 0.3591,
|
||||
"fnr": 0.7787,
|
||||
"tp": 79,
|
||||
"fp": 4,
|
||||
"fn": 278,
|
||||
"tn": 125
|
||||
},
|
||||
"L1a_keyword_cat_recall": {
|
||||
"R1": {
|
||||
"total": 31,
|
||||
"detected": 3,
|
||||
"recall": 0.0968,
|
||||
"miss_rate": 0.9032
|
||||
},
|
||||
"R2": {
|
||||
"total": 39,
|
||||
"detected": 3,
|
||||
"recall": 0.0769,
|
||||
"miss_rate": 0.9231
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 10,
|
||||
"recall": 0.2564,
|
||||
"miss_rate": 0.7436
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 7,
|
||||
"recall": 0.1707,
|
||||
"miss_rate": 0.8293
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 8,
|
||||
"recall": 0.2222,
|
||||
"miss_rate": 0.7778
|
||||
},
|
||||
"R6": {
|
||||
"total": 33,
|
||||
"detected": 6,
|
||||
"recall": 0.1818,
|
||||
"miss_rate": 0.8182
|
||||
},
|
||||
"R7": {
|
||||
"total": 32,
|
||||
"detected": 4,
|
||||
"recall": 0.125,
|
||||
"miss_rate": 0.875
|
||||
},
|
||||
"R8": {
|
||||
"total": 36,
|
||||
"detected": 25,
|
||||
"recall": 0.6944,
|
||||
"miss_rate": 0.3056
|
||||
},
|
||||
"R9": {
|
||||
"total": 33,
|
||||
"detected": 3,
|
||||
"recall": 0.0909,
|
||||
"miss_rate": 0.9091
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 10,
|
||||
"recall": 0.2703,
|
||||
"miss_rate": 0.7297
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"name": "L1b_regex",
|
||||
"accuracy": 0.3025,
|
||||
"precision": 1.0,
|
||||
"recall": 0.0504,
|
||||
"f1": 0.096,
|
||||
"fnr": 0.9496,
|
||||
"tp": 18,
|
||||
"fp": 0,
|
||||
"fn": 339,
|
||||
"tn": 129
|
||||
},
|
||||
"L1b_regex_cat_recall": {
|
||||
"R1": {
|
||||
"total": 31,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 39,
|
||||
"detected": 1,
|
||||
"recall": 0.0256,
|
||||
"miss_rate": 0.9744
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 9,
|
||||
"recall": 0.2308,
|
||||
"miss_rate": 0.7692
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 3,
|
||||
"recall": 0.0732,
|
||||
"miss_rate": 0.9268
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 1,
|
||||
"recall": 0.0278,
|
||||
"miss_rate": 0.9722
|
||||
},
|
||||
"R6": {
|
||||
"total": 33,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 32,
|
||||
"detected": 2,
|
||||
"recall": 0.0625,
|
||||
"miss_rate": 0.9375
|
||||
},
|
||||
"R8": {
|
||||
"total": 36,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 33,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 2,
|
||||
"recall": 0.0541,
|
||||
"miss_rate": 0.9459
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"name": "L1c_combined",
|
||||
"accuracy": 0.4486,
|
||||
"precision": 0.9588,
|
||||
"recall": 0.2605,
|
||||
"f1": 0.4097,
|
||||
"fnr": 0.7395,
|
||||
"tp": 93,
|
||||
"fp": 4,
|
||||
"fn": 264,
|
||||
"tn": 125
|
||||
},
|
||||
"L1c_combined_cat_recall": {
|
||||
"R1": {
|
||||
"total": 31,
|
||||
"detected": 3,
|
||||
"recall": 0.0968,
|
||||
"miss_rate": 0.9032
|
||||
},
|
||||
"R2": {
|
||||
"total": 39,
|
||||
"detected": 4,
|
||||
"recall": 0.1026,
|
||||
"miss_rate": 0.8974
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 16,
|
||||
"recall": 0.4103,
|
||||
"miss_rate": 0.5897
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 9,
|
||||
"recall": 0.2195,
|
||||
"miss_rate": 0.7805
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 9,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R6": {
|
||||
"total": 33,
|
||||
"detected": 6,
|
||||
"recall": 0.1818,
|
||||
"miss_rate": 0.8182
|
||||
},
|
||||
"R7": {
|
||||
"total": 32,
|
||||
"detected": 6,
|
||||
"recall": 0.1875,
|
||||
"miss_rate": 0.8125
|
||||
},
|
||||
"R8": {
|
||||
"total": 36,
|
||||
"detected": 25,
|
||||
"recall": 0.6944,
|
||||
"miss_rate": 0.3056
|
||||
},
|
||||
"R9": {
|
||||
"total": 33,
|
||||
"detected": 3,
|
||||
"recall": 0.0909,
|
||||
"miss_rate": 0.9091
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 12,
|
||||
"recall": 0.3243,
|
||||
"miss_rate": 0.6757
|
||||
}
|
||||
},
|
||||
"intervention_Rule(l≥3→REJECT)": {
|
||||
"policy": "Rule(l≥3→REJECT)",
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"safety_ux_fscore": 1.0,
|
||||
"crisis_precision": "N/A",
|
||||
"action_distribution": {
|
||||
"PASS": 233,
|
||||
"REJECT": 253
|
||||
}
|
||||
},
|
||||
"intervention_Threshold(level→action)": {
|
||||
"policy": "Threshold(level→action)",
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"safety_ux_fscore": 1.0,
|
||||
"crisis_precision": 1.0,
|
||||
"action_distribution": {
|
||||
"PASS": 129,
|
||||
"WARN": 104,
|
||||
"REWRITE": 193,
|
||||
"CRISIS": 60
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,335 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"n_total": 605,
|
||||
"n_filtered": 605,
|
||||
"n_risky": 456
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.29313543599257885,
|
||||
"high_risk_recall": 0.17324561403508773,
|
||||
"high_risk_precision": 0.9518072289156626,
|
||||
"false_negative_rate": 0.8267543859649122,
|
||||
"level_macro_f1": 0.09819557155678502,
|
||||
"level_weighted_f1": 0.08825982748460577,
|
||||
"level_per_class_f1": [
|
||||
0.2786885245901639,
|
||||
0.0,
|
||||
0.1951219512195122,
|
||||
0.017167381974248927,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 67,
|
||||
"detected": 3,
|
||||
"recall": 0.0448,
|
||||
"miss_rate": 0.9552
|
||||
},
|
||||
"R2": {
|
||||
"total": 45,
|
||||
"detected": 3,
|
||||
"recall": 0.0667,
|
||||
"miss_rate": 0.9333
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 10,
|
||||
"recall": 0.2564,
|
||||
"miss_rate": 0.7436
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 7,
|
||||
"recall": 0.1707,
|
||||
"miss_rate": 0.8293
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 8,
|
||||
"recall": 0.2222,
|
||||
"miss_rate": 0.7778
|
||||
},
|
||||
"R6": {
|
||||
"total": 64,
|
||||
"detected": 6,
|
||||
"recall": 0.0938,
|
||||
"miss_rate": 0.9062
|
||||
},
|
||||
"R7": {
|
||||
"total": 37,
|
||||
"detected": 4,
|
||||
"recall": 0.1081,
|
||||
"miss_rate": 0.8919
|
||||
},
|
||||
"R8": {
|
||||
"total": 38,
|
||||
"detected": 25,
|
||||
"recall": 0.6579,
|
||||
"miss_rate": 0.3421
|
||||
},
|
||||
"R9": {
|
||||
"total": 52,
|
||||
"detected": 3,
|
||||
"recall": 0.0577,
|
||||
"miss_rate": 0.9423
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 10,
|
||||
"recall": 0.2703,
|
||||
"miss_rate": 0.7297
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.0759493670886076,
|
||||
"high_risk_recall": 0.039473684210526314,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9605263157894737,
|
||||
"level_macro_f1": 0.07132623033992896,
|
||||
"level_weighted_f1": 0.058213483946983315,
|
||||
"level_per_class_f1": [
|
||||
0.2607407407407407,
|
||||
0.0,
|
||||
0.0958904109589041,
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 67,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 45,
|
||||
"detected": 1,
|
||||
"recall": 0.0222,
|
||||
"miss_rate": 0.9778
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 9,
|
||||
"recall": 0.2308,
|
||||
"miss_rate": 0.7692
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 3,
|
||||
"recall": 0.0732,
|
||||
"miss_rate": 0.9268
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 1,
|
||||
"recall": 0.0278,
|
||||
"miss_rate": 0.9722
|
||||
},
|
||||
"R6": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 37,
|
||||
"detected": 2,
|
||||
"recall": 0.0541,
|
||||
"miss_rate": 0.9459
|
||||
},
|
||||
"R8": {
|
||||
"total": 38,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 52,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 2,
|
||||
"recall": 0.0541,
|
||||
"miss_rate": 0.9459
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.33634719710669075,
|
||||
"high_risk_recall": 0.20394736842105263,
|
||||
"high_risk_precision": 0.9587628865979382,
|
||||
"false_negative_rate": 0.7960526315789473,
|
||||
"level_macro_f1": 0.10979552475377227,
|
||||
"level_weighted_f1": 0.1000980341896042,
|
||||
"level_per_class_f1": [
|
||||
0.28523489932885904,
|
||||
0.0,
|
||||
0.2465753424657534,
|
||||
0.017167381974248927,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 67,
|
||||
"detected": 3,
|
||||
"recall": 0.0448,
|
||||
"miss_rate": 0.9552
|
||||
},
|
||||
"R2": {
|
||||
"total": 45,
|
||||
"detected": 4,
|
||||
"recall": 0.0889,
|
||||
"miss_rate": 0.9111
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 16,
|
||||
"recall": 0.4103,
|
||||
"miss_rate": 0.5897
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 9,
|
||||
"recall": 0.2195,
|
||||
"miss_rate": 0.7805
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 9,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R6": {
|
||||
"total": 64,
|
||||
"detected": 6,
|
||||
"recall": 0.0938,
|
||||
"miss_rate": 0.9062
|
||||
},
|
||||
"R7": {
|
||||
"total": 37,
|
||||
"detected": 6,
|
||||
"recall": 0.1622,
|
||||
"miss_rate": 0.8378
|
||||
},
|
||||
"R8": {
|
||||
"total": 38,
|
||||
"detected": 25,
|
||||
"recall": 0.6579,
|
||||
"miss_rate": 0.3421
|
||||
},
|
||||
"R9": {
|
||||
"total": 52,
|
||||
"detected": 3,
|
||||
"recall": 0.0577,
|
||||
"miss_rate": 0.9423
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 12,
|
||||
"recall": 0.3243,
|
||||
"miss_rate": 0.6757
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9967069154774972,
|
||||
"high_risk_recall": 0.9956140350877193,
|
||||
"high_risk_precision": 0.9978021978021978,
|
||||
"false_negative_rate": 0.004385964912280715,
|
||||
"level_macro_f1": 0.5150467302191439,
|
||||
"level_weighted_f1": 0.5173056767699116,
|
||||
"level_per_class_f1": [
|
||||
0.632183908045977,
|
||||
0.5076923076923077,
|
||||
0.3861003861003861,
|
||||
0.5627705627705628,
|
||||
0.4864864864864865
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.6407766990291263,
|
||||
0.46464646464646464,
|
||||
0.734982332155477,
|
||||
0.0,
|
||||
0.7407407407407407,
|
||||
0.7676767676767676,
|
||||
0.6013986013986014,
|
||||
0.4864864864864865,
|
||||
0.6161616161616161,
|
||||
0.6875,
|
||||
0.24,
|
||||
0.38961038961038963,
|
||||
0.8641975308641975,
|
||||
0.7777777777777778
|
||||
],
|
||||
"fine_macro_f1": 0.5722825290391176,
|
||||
"fine_weighted_f1": 0.622073826302884,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 67,
|
||||
"detected": 66,
|
||||
"recall": 0.9851,
|
||||
"miss_rate": 0.0149
|
||||
},
|
||||
"R2": {
|
||||
"total": 45,
|
||||
"detected": 44,
|
||||
"recall": 0.9778,
|
||||
"miss_rate": 0.0222
|
||||
},
|
||||
"R3": {
|
||||
"total": 39,
|
||||
"detected": 39,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 41,
|
||||
"detected": 41,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 36,
|
||||
"detected": 36,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 37,
|
||||
"detected": 37,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 38,
|
||||
"detected": 38,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 52,
|
||||
"detected": 52,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 37,
|
||||
"detected": 37,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,335 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "human",
|
||||
"n_total": 605,
|
||||
"n_filtered": 119,
|
||||
"n_risky": 99
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.0,
|
||||
"high_risk_recall": 0.0,
|
||||
"high_risk_precision": 0.0,
|
||||
"false_negative_rate": 1.0,
|
||||
"level_macro_f1": 0.05755395683453237,
|
||||
"level_weighted_f1": 0.04836466960885073,
|
||||
"level_per_class_f1": [
|
||||
0.28776978417266186,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 36,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 6,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 31,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 5,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 2,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 19,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.0,
|
||||
"high_risk_recall": 0.0,
|
||||
"high_risk_precision": 0.0,
|
||||
"false_negative_rate": 1.0,
|
||||
"level_macro_f1": 0.05755395683453237,
|
||||
"level_weighted_f1": 0.04836466960885073,
|
||||
"level_per_class_f1": [
|
||||
0.28776978417266186,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 36,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 6,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 31,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 5,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 2,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 19,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.0,
|
||||
"high_risk_recall": 0.0,
|
||||
"high_risk_precision": 0.0,
|
||||
"false_negative_rate": 1.0,
|
||||
"level_macro_f1": 0.05755395683453237,
|
||||
"level_weighted_f1": 0.04836466960885073,
|
||||
"level_per_class_f1": [
|
||||
0.28776978417266186,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 36,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 6,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 31,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 5,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 2,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 19,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9847715736040609,
|
||||
"high_risk_recall": 0.9797979797979798,
|
||||
"high_risk_precision": 0.9897959183673469,
|
||||
"false_negative_rate": 0.02020202020202022,
|
||||
"level_macro_f1": 0.3641541183069423,
|
||||
"level_weighted_f1": 0.4092843419457787,
|
||||
"level_per_class_f1": [
|
||||
0.9302325581395349,
|
||||
0.0,
|
||||
0.16326530612244897,
|
||||
0.36363636363636365,
|
||||
0.36363636363636365
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.3508771929824561,
|
||||
0.0,
|
||||
0.64,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.2222222222222222,
|
||||
0.375,
|
||||
0.8857142857142857,
|
||||
0.0,
|
||||
0.0,
|
||||
0.5,
|
||||
0.2857142857142857
|
||||
],
|
||||
"fine_macro_f1": 0.2328234276166607,
|
||||
"fine_weighted_f1": 0.4082668160299739,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 36,
|
||||
"detected": 35,
|
||||
"recall": 0.9722,
|
||||
"miss_rate": 0.0278
|
||||
},
|
||||
"R2": {
|
||||
"total": 6,
|
||||
"detected": 5,
|
||||
"recall": 0.8333,
|
||||
"miss_rate": 0.1667
|
||||
},
|
||||
"R3": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 31,
|
||||
"detected": 31,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 5,
|
||||
"detected": 5,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 2,
|
||||
"detected": 2,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 19,
|
||||
"detected": 19,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 0,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,376 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"label_filter": "all",
|
||||
"n_total": 1486,
|
||||
"n_filtered": 1486,
|
||||
"n_risky": 1039
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.26436781609195403,
|
||||
"high_risk_recall": 0.15495668912415783,
|
||||
"high_risk_precision": 0.8994413407821229,
|
||||
"false_negative_rate": 0.8450433108758422,
|
||||
"level_macro_f1": 0.10427720349098286,
|
||||
"level_weighted_f1": 0.09799538109505529,
|
||||
"level_per_class_f1": [
|
||||
0.2979274611398964,
|
||||
0.0,
|
||||
0.1934156378600823,
|
||||
0.030042918454935622,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 16,
|
||||
"recall": 0.1127,
|
||||
"miss_rate": 0.8873
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 17,
|
||||
"recall": 0.1789,
|
||||
"miss_rate": 0.8211
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 22,
|
||||
"recall": 0.1897,
|
||||
"miss_rate": 0.8103
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 6,
|
||||
"recall": 0.0659,
|
||||
"miss_rate": 0.9341
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 10,
|
||||
"recall": 0.137,
|
||||
"miss_rate": 0.863
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.06697674418604652,
|
||||
"high_risk_recall": 0.03464870067372473,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9653512993262753,
|
||||
"level_macro_f1": 0.07297879241072718,
|
||||
"level_weighted_f1": 0.06312377515343655,
|
||||
"level_per_class_f1": [
|
||||
0.2809721398933017,
|
||||
0.0,
|
||||
0.07954545454545454,
|
||||
0.00437636761487965,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 1,
|
||||
"recall": 0.007,
|
||||
"miss_rate": 0.993
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 19,
|
||||
"recall": 0.2,
|
||||
"miss_rate": 0.8
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 9,
|
||||
"recall": 0.0776,
|
||||
"miss_rate": 0.9224
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 3,
|
||||
"recall": 0.033,
|
||||
"miss_rate": 0.967
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 4,
|
||||
"recall": 0.0548,
|
||||
"miss_rate": 0.9452
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.3060897435897436,
|
||||
"high_risk_recall": 0.18383060635226178,
|
||||
"high_risk_precision": 0.9138755980861244,
|
||||
"false_negative_rate": 0.8161693936477382,
|
||||
"level_macro_f1": 0.11189027535274536,
|
||||
"level_weighted_f1": 0.10619241328971442,
|
||||
"level_per_class_f1": [
|
||||
0.3038309114927345,
|
||||
0.0,
|
||||
0.22135922330097088,
|
||||
0.034261241970021415,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 17,
|
||||
"recall": 0.1197,
|
||||
"miss_rate": 0.8803
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 32,
|
||||
"recall": 0.3368,
|
||||
"miss_rate": 0.6632
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 29,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 9,
|
||||
"recall": 0.0989,
|
||||
"miss_rate": 0.9011
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 14,
|
||||
"recall": 0.1918,
|
||||
"miss_rate": 0.8082
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9995189995189995,
|
||||
"high_risk_recall": 1.0,
|
||||
"high_risk_precision": 0.9990384615384615,
|
||||
"false_negative_rate": 0.0,
|
||||
"level_macro_f1": 0.5495554176357882,
|
||||
"level_weighted_f1": 0.5584578220374772,
|
||||
"level_per_class_f1": [
|
||||
0.37540453074433655,
|
||||
0.6351931330472103,
|
||||
0.46393762183235865,
|
||||
0.6400759734093068,
|
||||
0.6331658291457286
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.6844262295081968,
|
||||
0.46567164179104475,
|
||||
0.697986577181208,
|
||||
0.40233236151603496,
|
||||
0.585,
|
||||
0.3559322033898305,
|
||||
0.38322211630123926,
|
||||
0.3374578177727784,
|
||||
0.531810766721044,
|
||||
0.39436619718309857,
|
||||
0.2691029900332226,
|
||||
0.4410480349344978,
|
||||
0.32142857142857145,
|
||||
0.615916955017301
|
||||
],
|
||||
"fine_macro_f1": 0.46326446162700485,
|
||||
"fine_weighted_f1": 0.4915026862223374,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 136,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 142,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 95,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 116,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 97,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 152,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
},
|
||||
"label_filter": "all"
|
||||
},
|
||||
"baseline_rule": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.5612382234185733,
|
||||
0.0,
|
||||
0.0,
|
||||
0.4387617765814266,
|
||||
0.0
|
||||
],
|
||||
"crisis_precision": NaN,
|
||||
"safety_ux_fscore": 1.0
|
||||
},
|
||||
"baseline_threshold": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.34791386271870794,
|
||||
0.2133243606998654,
|
||||
0.30686406460296095,
|
||||
0.0,
|
||||
0.13189771197846567
|
||||
],
|
||||
"crisis_precision": 1.0,
|
||||
"safety_ux_fscore": 1.0
|
||||
},
|
||||
"ours_intervention": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.3001345895020188,
|
||||
0.0033647375504710633,
|
||||
0.5834454912516823,
|
||||
0.0,
|
||||
0.11305518169582772
|
||||
],
|
||||
"crisis_precision": 0.47619047619047616,
|
||||
"safety_ux_fscore": 1.0
|
||||
}
|
||||
}
|
||||
@@ -1,533 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"label_filter": "all",
|
||||
"n_total": 1486,
|
||||
"n_filtered": 1486,
|
||||
"n_risky": 1039
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.26436781609195403,
|
||||
"high_risk_recall": 0.15495668912415783,
|
||||
"high_risk_precision": 0.8994413407821229,
|
||||
"false_negative_rate": 0.8450433108758422,
|
||||
"level_macro_f1": 0.10427720349098286,
|
||||
"level_weighted_f1": 0.09799538109505529,
|
||||
"level_per_class_f1": [
|
||||
0.2979274611398964,
|
||||
0.0,
|
||||
0.1934156378600823,
|
||||
0.030042918454935622,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 16,
|
||||
"recall": 0.1127,
|
||||
"miss_rate": 0.8873
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 17,
|
||||
"recall": 0.1789,
|
||||
"miss_rate": 0.8211
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 22,
|
||||
"recall": 0.1897,
|
||||
"miss_rate": 0.8103
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 6,
|
||||
"recall": 0.0659,
|
||||
"miss_rate": 0.9341
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 10,
|
||||
"recall": 0.137,
|
||||
"miss_rate": 0.863
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.06697674418604652,
|
||||
"high_risk_recall": 0.03464870067372473,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9653512993262753,
|
||||
"level_macro_f1": 0.07297879241072718,
|
||||
"level_weighted_f1": 0.06312377515343655,
|
||||
"level_per_class_f1": [
|
||||
0.2809721398933017,
|
||||
0.0,
|
||||
0.07954545454545454,
|
||||
0.00437636761487965,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 1,
|
||||
"recall": 0.007,
|
||||
"miss_rate": 0.993
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 19,
|
||||
"recall": 0.2,
|
||||
"miss_rate": 0.8
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 9,
|
||||
"recall": 0.0776,
|
||||
"miss_rate": 0.9224
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 3,
|
||||
"recall": 0.033,
|
||||
"miss_rate": 0.967
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 4,
|
||||
"recall": 0.0548,
|
||||
"miss_rate": 0.9452
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.3060897435897436,
|
||||
"high_risk_recall": 0.18383060635226178,
|
||||
"high_risk_precision": 0.9138755980861244,
|
||||
"false_negative_rate": 0.8161693936477382,
|
||||
"level_macro_f1": 0.11189027535274536,
|
||||
"level_weighted_f1": 0.10619241328971442,
|
||||
"level_per_class_f1": [
|
||||
0.3038309114927345,
|
||||
0.0,
|
||||
0.22135922330097088,
|
||||
0.034261241970021415,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 17,
|
||||
"recall": 0.1197,
|
||||
"miss_rate": 0.8803
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 32,
|
||||
"recall": 0.3368,
|
||||
"miss_rate": 0.6632
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 29,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 9,
|
||||
"recall": 0.0989,
|
||||
"miss_rate": 0.9011
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 14,
|
||||
"recall": 0.1918,
|
||||
"miss_rate": 0.8082
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9995189995189995,
|
||||
"high_risk_recall": 1.0,
|
||||
"high_risk_precision": 0.9990384615384615,
|
||||
"false_negative_rate": 0.0,
|
||||
"level_macro_f1": 0.5495554176357882,
|
||||
"level_weighted_f1": 0.5584578220374772,
|
||||
"level_per_class_f1": [
|
||||
0.37540453074433655,
|
||||
0.6351931330472103,
|
||||
0.46393762183235865,
|
||||
0.6400759734093068,
|
||||
0.6331658291457286
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.6844262295081968,
|
||||
0.46567164179104475,
|
||||
0.697986577181208,
|
||||
0.40233236151603496,
|
||||
0.585,
|
||||
0.3559322033898305,
|
||||
0.38322211630123926,
|
||||
0.3374578177727784,
|
||||
0.531810766721044,
|
||||
0.39436619718309857,
|
||||
0.2691029900332226,
|
||||
0.4410480349344978,
|
||||
0.32142857142857145,
|
||||
0.615916955017301
|
||||
],
|
||||
"fine_macro_f1": 0.46326446162700485,
|
||||
"fine_weighted_f1": 0.4915026862223374,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 136,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 142,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 95,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 116,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 97,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 152,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
},
|
||||
"label_filter": "all"
|
||||
},
|
||||
"baseline_rule": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.46231493943472407,
|
||||
0.0,
|
||||
0.0,
|
||||
0.5376850605652759,
|
||||
0.0
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.9178571428571428,
|
||||
0.0,
|
||||
0.0,
|
||||
0.08214285714285714,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.4195583596214511,
|
||||
0.0,
|
||||
0.0,
|
||||
0.580441640378549,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.11403508771929824,
|
||||
0.0,
|
||||
0.0,
|
||||
0.8859649122807017,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.04081632653061224,
|
||||
0.0,
|
||||
0.0,
|
||||
0.9591836734693877,
|
||||
0.0
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": NaN,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"baseline_threshold": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.3304172274562584,
|
||||
0.13189771197846567,
|
||||
0.40174966352624497,
|
||||
0.0,
|
||||
0.13593539703903096
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.8428571428571429,
|
||||
0.075,
|
||||
0.08214285714285714,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.04416403785488959,
|
||||
0.3753943217665615,
|
||||
0.5520504731861199,
|
||||
0.0,
|
||||
0.028391167192429023
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.008771929824561403,
|
||||
0.10526315789473684,
|
||||
0.7390350877192983,
|
||||
0.0,
|
||||
0.14692982456140352
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.04081632653061224,
|
||||
0.3163265306122449,
|
||||
0.0,
|
||||
0.6428571428571429
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": 0.6237623762376238,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"ours_intervention": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.29878869448183043,
|
||||
0.0033647375504710633,
|
||||
0.5847913862718708,
|
||||
0.0,
|
||||
0.11305518169582772
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
0.9831223628691983,
|
||||
0.016877637130801686,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.7535714285714286,
|
||||
0.0035714285714285713,
|
||||
0.21785714285714286,
|
||||
0.0,
|
||||
0.025
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.9148264984227129,
|
||||
0.0,
|
||||
0.08517350157728706
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.8793859649122807,
|
||||
0.0,
|
||||
0.1206140350877193
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.5969387755102041,
|
||||
0.0,
|
||||
0.4030612244897959
|
||||
]
|
||||
}
|
||||
},
|
||||
"action_accuracy": 0.5868102288021534,
|
||||
"crisis_precision": 0.47023809523809523,
|
||||
"safety_ux_fscore": 1.0
|
||||
}
|
||||
}
|
||||
@@ -1,533 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"label_filter": "all",
|
||||
"n_total": 1486,
|
||||
"n_filtered": 1486,
|
||||
"n_risky": 1039
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.26436781609195403,
|
||||
"high_risk_recall": 0.15495668912415783,
|
||||
"high_risk_precision": 0.8994413407821229,
|
||||
"false_negative_rate": 0.8450433108758422,
|
||||
"level_macro_f1": 0.10427720349098286,
|
||||
"level_weighted_f1": 0.09799538109505529,
|
||||
"level_per_class_f1": [
|
||||
0.2979274611398964,
|
||||
0.0,
|
||||
0.1934156378600823,
|
||||
0.030042918454935622,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 16,
|
||||
"recall": 0.1127,
|
||||
"miss_rate": 0.8873
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 17,
|
||||
"recall": 0.1789,
|
||||
"miss_rate": 0.8211
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 22,
|
||||
"recall": 0.1897,
|
||||
"miss_rate": 0.8103
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 6,
|
||||
"recall": 0.0659,
|
||||
"miss_rate": 0.9341
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 10,
|
||||
"recall": 0.137,
|
||||
"miss_rate": 0.863
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.06697674418604652,
|
||||
"high_risk_recall": 0.03464870067372473,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9653512993262753,
|
||||
"level_macro_f1": 0.07297879241072718,
|
||||
"level_weighted_f1": 0.06312377515343655,
|
||||
"level_per_class_f1": [
|
||||
0.2809721398933017,
|
||||
0.0,
|
||||
0.07954545454545454,
|
||||
0.00437636761487965,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 1,
|
||||
"recall": 0.007,
|
||||
"miss_rate": 0.993
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 19,
|
||||
"recall": 0.2,
|
||||
"miss_rate": 0.8
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 9,
|
||||
"recall": 0.0776,
|
||||
"miss_rate": 0.9224
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 3,
|
||||
"recall": 0.033,
|
||||
"miss_rate": 0.967
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 4,
|
||||
"recall": 0.0548,
|
||||
"miss_rate": 0.9452
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.3060897435897436,
|
||||
"high_risk_recall": 0.18383060635226178,
|
||||
"high_risk_precision": 0.9138755980861244,
|
||||
"false_negative_rate": 0.8161693936477382,
|
||||
"level_macro_f1": 0.11189027535274536,
|
||||
"level_weighted_f1": 0.10619241328971442,
|
||||
"level_per_class_f1": [
|
||||
0.3038309114927345,
|
||||
0.0,
|
||||
0.22135922330097088,
|
||||
0.034261241970021415,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 17,
|
||||
"recall": 0.1197,
|
||||
"miss_rate": 0.8803
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 32,
|
||||
"recall": 0.3368,
|
||||
"miss_rate": 0.6632
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 29,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 9,
|
||||
"recall": 0.0989,
|
||||
"miss_rate": 0.9011
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 14,
|
||||
"recall": 0.1918,
|
||||
"miss_rate": 0.8082
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9995189995189995,
|
||||
"high_risk_recall": 1.0,
|
||||
"high_risk_precision": 0.9990384615384615,
|
||||
"false_negative_rate": 0.0,
|
||||
"level_macro_f1": 0.5495554176357882,
|
||||
"level_weighted_f1": 0.5584578220374772,
|
||||
"level_per_class_f1": [
|
||||
0.37540453074433655,
|
||||
0.6351931330472103,
|
||||
0.46393762183235865,
|
||||
0.6400759734093068,
|
||||
0.6331658291457286
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.6844262295081968,
|
||||
0.46567164179104475,
|
||||
0.697986577181208,
|
||||
0.40233236151603496,
|
||||
0.585,
|
||||
0.3559322033898305,
|
||||
0.38322211630123926,
|
||||
0.3374578177727784,
|
||||
0.531810766721044,
|
||||
0.39436619718309857,
|
||||
0.2691029900332226,
|
||||
0.4410480349344978,
|
||||
0.32142857142857145,
|
||||
0.615916955017301
|
||||
],
|
||||
"fine_macro_f1": 0.46326446162700485,
|
||||
"fine_weighted_f1": 0.4915026862223374,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 136,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 142,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 95,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 116,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 97,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 152,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
},
|
||||
"label_filter": "all"
|
||||
},
|
||||
"baseline_rule": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.46231493943472407,
|
||||
0.0,
|
||||
0.0,
|
||||
0.5376850605652759,
|
||||
0.0
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.9178571428571428,
|
||||
0.0,
|
||||
0.0,
|
||||
0.08214285714285714,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.4195583596214511,
|
||||
0.0,
|
||||
0.0,
|
||||
0.580441640378549,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.11403508771929824,
|
||||
0.0,
|
||||
0.0,
|
||||
0.8859649122807017,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.04081632653061224,
|
||||
0.0,
|
||||
0.0,
|
||||
0.9591836734693877,
|
||||
0.0
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": NaN,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"baseline_threshold": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.3304172274562584,
|
||||
0.13189771197846567,
|
||||
0.40174966352624497,
|
||||
0.0,
|
||||
0.13593539703903096
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.8428571428571429,
|
||||
0.075,
|
||||
0.08214285714285714,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.04416403785488959,
|
||||
0.3753943217665615,
|
||||
0.5520504731861199,
|
||||
0.0,
|
||||
0.028391167192429023
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.008771929824561403,
|
||||
0.10526315789473684,
|
||||
0.7390350877192983,
|
||||
0.0,
|
||||
0.14692982456140352
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.04081632653061224,
|
||||
0.3163265306122449,
|
||||
0.0,
|
||||
0.6428571428571429
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": 0.6237623762376238,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"ours_intervention": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.004219409282700422,
|
||||
"action_distribution": [
|
||||
0.29475100942126514,
|
||||
0.0033647375504710633,
|
||||
0.5868102288021534,
|
||||
0.0,
|
||||
0.11507402422611036
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
0.9873417721518988,
|
||||
0.008438818565400843,
|
||||
0.004219409282700422,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.7285714285714285,
|
||||
0.010714285714285714,
|
||||
0.22857142857142856,
|
||||
0.0,
|
||||
0.03214285714285714
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.9022082018927445,
|
||||
0.0,
|
||||
0.09779179810725552
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.8706140350877193,
|
||||
0.0,
|
||||
0.12938596491228072
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.6326530612244898,
|
||||
0.0,
|
||||
0.3673469387755102
|
||||
]
|
||||
}
|
||||
},
|
||||
"action_accuracy": 0.5753701211305519,
|
||||
"crisis_precision": 0.42105263157894735,
|
||||
"safety_ux_fscore": 0.9978858350951374
|
||||
}
|
||||
}
|
||||
@@ -1,533 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"label_filter": "all",
|
||||
"n_total": 1486,
|
||||
"n_filtered": 1486,
|
||||
"n_risky": 1039
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.26436781609195403,
|
||||
"high_risk_recall": 0.15495668912415783,
|
||||
"high_risk_precision": 0.8994413407821229,
|
||||
"false_negative_rate": 0.8450433108758422,
|
||||
"level_macro_f1": 0.10427720349098286,
|
||||
"level_weighted_f1": 0.09799538109505529,
|
||||
"level_per_class_f1": [
|
||||
0.2979274611398964,
|
||||
0.0,
|
||||
0.1934156378600823,
|
||||
0.030042918454935622,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 16,
|
||||
"recall": 0.1127,
|
||||
"miss_rate": 0.8873
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 17,
|
||||
"recall": 0.1789,
|
||||
"miss_rate": 0.8211
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 22,
|
||||
"recall": 0.1897,
|
||||
"miss_rate": 0.8103
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 6,
|
||||
"recall": 0.0659,
|
||||
"miss_rate": 0.9341
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 10,
|
||||
"recall": 0.137,
|
||||
"miss_rate": 0.863
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.06697674418604652,
|
||||
"high_risk_recall": 0.03464870067372473,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9653512993262753,
|
||||
"level_macro_f1": 0.07297879241072718,
|
||||
"level_weighted_f1": 0.06312377515343655,
|
||||
"level_per_class_f1": [
|
||||
0.2809721398933017,
|
||||
0.0,
|
||||
0.07954545454545454,
|
||||
0.00437636761487965,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 1,
|
||||
"recall": 0.007,
|
||||
"miss_rate": 0.993
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 19,
|
||||
"recall": 0.2,
|
||||
"miss_rate": 0.8
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 9,
|
||||
"recall": 0.0776,
|
||||
"miss_rate": 0.9224
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 3,
|
||||
"recall": 0.033,
|
||||
"miss_rate": 0.967
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 4,
|
||||
"recall": 0.0548,
|
||||
"miss_rate": 0.9452
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.3060897435897436,
|
||||
"high_risk_recall": 0.18383060635226178,
|
||||
"high_risk_precision": 0.9138755980861244,
|
||||
"false_negative_rate": 0.8161693936477382,
|
||||
"level_macro_f1": 0.11189027535274536,
|
||||
"level_weighted_f1": 0.10619241328971442,
|
||||
"level_per_class_f1": [
|
||||
0.3038309114927345,
|
||||
0.0,
|
||||
0.22135922330097088,
|
||||
0.034261241970021415,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 10,
|
||||
"recall": 0.0735,
|
||||
"miss_rate": 0.9265
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 17,
|
||||
"recall": 0.1197,
|
||||
"miss_rate": 0.8803
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 32,
|
||||
"recall": 0.3368,
|
||||
"miss_rate": 0.6632
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 29,
|
||||
"recall": 0.25,
|
||||
"miss_rate": 0.75
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 11,
|
||||
"recall": 0.1134,
|
||||
"miss_rate": 0.8866
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 9,
|
||||
"recall": 0.0989,
|
||||
"miss_rate": 0.9011
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 49,
|
||||
"recall": 0.6712,
|
||||
"miss_rate": 0.3288
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 11,
|
||||
"recall": 0.0724,
|
||||
"miss_rate": 0.9276
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 14,
|
||||
"recall": 0.1918,
|
||||
"miss_rate": 0.8082
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9995189995189995,
|
||||
"high_risk_recall": 1.0,
|
||||
"high_risk_precision": 0.9990384615384615,
|
||||
"false_negative_rate": 0.0,
|
||||
"level_macro_f1": 0.5495554176357882,
|
||||
"level_weighted_f1": 0.5584578220374772,
|
||||
"level_per_class_f1": [
|
||||
0.37540453074433655,
|
||||
0.6351931330472103,
|
||||
0.46393762183235865,
|
||||
0.6400759734093068,
|
||||
0.6331658291457286
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.6844262295081968,
|
||||
0.46567164179104475,
|
||||
0.697986577181208,
|
||||
0.40233236151603496,
|
||||
0.585,
|
||||
0.3559322033898305,
|
||||
0.38322211630123926,
|
||||
0.3374578177727784,
|
||||
0.531810766721044,
|
||||
0.39436619718309857,
|
||||
0.2691029900332226,
|
||||
0.4410480349344978,
|
||||
0.32142857142857145,
|
||||
0.615916955017301
|
||||
],
|
||||
"fine_macro_f1": 0.46326446162700485,
|
||||
"fine_weighted_f1": 0.4915026862223374,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 136,
|
||||
"detected": 136,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 142,
|
||||
"detected": 142,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 95,
|
||||
"detected": 95,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 116,
|
||||
"detected": 116,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 97,
|
||||
"detected": 97,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 152,
|
||||
"detected": 152,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 73,
|
||||
"detected": 73,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
},
|
||||
"label_filter": "all"
|
||||
},
|
||||
"baseline_rule": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.46231493943472407,
|
||||
0.0,
|
||||
0.0,
|
||||
0.5376850605652759,
|
||||
0.0
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.9178571428571428,
|
||||
0.0,
|
||||
0.0,
|
||||
0.08214285714285714,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.4195583596214511,
|
||||
0.0,
|
||||
0.0,
|
||||
0.580441640378549,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.11403508771929824,
|
||||
0.0,
|
||||
0.0,
|
||||
0.8859649122807017,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.04081632653061224,
|
||||
0.0,
|
||||
0.0,
|
||||
0.9591836734693877,
|
||||
0.0
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": NaN,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"baseline_threshold": {
|
||||
"intervention_recall_high": 0.9079754601226994,
|
||||
"over_intervention_rate": 0.0,
|
||||
"action_distribution": [
|
||||
0.3304172274562584,
|
||||
0.13189771197846567,
|
||||
0.40174966352624497,
|
||||
0.0,
|
||||
0.13593539703903096
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
1.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.8428571428571429,
|
||||
0.075,
|
||||
0.08214285714285714,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.04416403785488959,
|
||||
0.3753943217665615,
|
||||
0.5520504731861199,
|
||||
0.0,
|
||||
0.028391167192429023
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.008771929824561403,
|
||||
0.10526315789473684,
|
||||
0.7390350877192983,
|
||||
0.0,
|
||||
0.14692982456140352
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.04081632653061224,
|
||||
0.3163265306122449,
|
||||
0.0,
|
||||
0.6428571428571429
|
||||
]
|
||||
}
|
||||
},
|
||||
"crisis_precision": 0.6237623762376238,
|
||||
"safety_ux_fscore": 0.9517684887459806
|
||||
},
|
||||
"ours_intervention": {
|
||||
"intervention_recall_high": 1.0,
|
||||
"over_intervention_rate": 0.004219409282700422,
|
||||
"action_distribution": [
|
||||
0.29475100942126514,
|
||||
0.0033647375504710633,
|
||||
0.5868102288021534,
|
||||
0.0,
|
||||
0.11507402422611036
|
||||
],
|
||||
"per_level_action_dist": {
|
||||
"L0_Safe": {
|
||||
"n": 237,
|
||||
"action_dist": [
|
||||
0.9873417721518988,
|
||||
0.008438818565400843,
|
||||
0.004219409282700422,
|
||||
0.0,
|
||||
0.0
|
||||
]
|
||||
},
|
||||
"L1_Mild": {
|
||||
"n": 280,
|
||||
"action_dist": [
|
||||
0.7285714285714285,
|
||||
0.010714285714285714,
|
||||
0.22857142857142856,
|
||||
0.0,
|
||||
0.03214285714285714
|
||||
]
|
||||
},
|
||||
"L2_Moderate": {
|
||||
"n": 317,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.9022082018927445,
|
||||
0.0,
|
||||
0.09779179810725552
|
||||
]
|
||||
},
|
||||
"L3_High": {
|
||||
"n": 456,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.8706140350877193,
|
||||
0.0,
|
||||
0.12938596491228072
|
||||
]
|
||||
},
|
||||
"L4_Critical": {
|
||||
"n": 196,
|
||||
"action_dist": [
|
||||
0.0,
|
||||
0.0,
|
||||
0.6326530612244898,
|
||||
0.0,
|
||||
0.3673469387755102
|
||||
]
|
||||
}
|
||||
},
|
||||
"action_accuracy": 0.5753701211305519,
|
||||
"crisis_precision": 0.42105263157894735,
|
||||
"safety_ux_fscore": 0.9978858350951374
|
||||
}
|
||||
}
|
||||
@@ -1,337 +0,0 @@
|
||||
{
|
||||
"meta": {
|
||||
"test_file": "data/processed/CompanionRisk-Bench/test.jsonl",
|
||||
"source_filter": "all",
|
||||
"label_filter": "all",
|
||||
"n_total": 1324,
|
||||
"n_filtered": 1324,
|
||||
"n_risky": 877
|
||||
},
|
||||
"L1a_keyword": {
|
||||
"binary_f1": 0.27751196172248804,
|
||||
"high_risk_recall": 0.1653363740022805,
|
||||
"high_risk_precision": 0.8630952380952381,
|
||||
"false_negative_rate": 0.8346636259977195,
|
||||
"level_macro_f1": 0.11264512835143245,
|
||||
"level_weighted_f1": 0.10448970574896717,
|
||||
"level_per_class_f1": [
|
||||
0.3254480286738351,
|
||||
0.0,
|
||||
0.20865139949109415,
|
||||
0.02912621359223301,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 123,
|
||||
"detected": 8,
|
||||
"recall": 0.065,
|
||||
"miss_rate": 0.935
|
||||
},
|
||||
"R2": {
|
||||
"total": 96,
|
||||
"detected": 14,
|
||||
"recall": 0.1458,
|
||||
"miss_rate": 0.8542
|
||||
},
|
||||
"R3": {
|
||||
"total": 77,
|
||||
"detected": 13,
|
||||
"recall": 0.1688,
|
||||
"miss_rate": 0.8312
|
||||
},
|
||||
"R4": {
|
||||
"total": 81,
|
||||
"detected": 18,
|
||||
"recall": 0.2222,
|
||||
"miss_rate": 0.7778
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 105,
|
||||
"detected": 11,
|
||||
"recall": 0.1048,
|
||||
"miss_rate": 0.8952
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 6,
|
||||
"recall": 0.0659,
|
||||
"miss_rate": 0.9341
|
||||
},
|
||||
"R8": {
|
||||
"total": 75,
|
||||
"detected": 49,
|
||||
"recall": 0.6533,
|
||||
"miss_rate": 0.3467
|
||||
},
|
||||
"R9": {
|
||||
"total": 91,
|
||||
"detected": 7,
|
||||
"recall": 0.0769,
|
||||
"miss_rate": 0.9231
|
||||
},
|
||||
"R10": {
|
||||
"total": 74,
|
||||
"detected": 10,
|
||||
"recall": 0.1351,
|
||||
"miss_rate": 0.8649
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1b_regex": {
|
||||
"binary_f1": 0.07886089813800658,
|
||||
"high_risk_recall": 0.04104903078677309,
|
||||
"high_risk_precision": 1.0,
|
||||
"false_negative_rate": 0.9589509692132269,
|
||||
"level_macro_f1": 0.08441436068877664,
|
||||
"level_weighted_f1": 0.07640981579648991,
|
||||
"level_per_class_f1": [
|
||||
0.31303208906352326,
|
||||
0.0,
|
||||
0.10408921933085502,
|
||||
0.0049504950495049506,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 123,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R2": {
|
||||
"total": 96,
|
||||
"detected": 1,
|
||||
"recall": 0.0104,
|
||||
"miss_rate": 0.9896
|
||||
},
|
||||
"R3": {
|
||||
"total": 77,
|
||||
"detected": 19,
|
||||
"recall": 0.2468,
|
||||
"miss_rate": 0.7532
|
||||
},
|
||||
"R4": {
|
||||
"total": 81,
|
||||
"detected": 9,
|
||||
"recall": 0.1111,
|
||||
"miss_rate": 0.8889
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 105,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 3,
|
||||
"recall": 0.033,
|
||||
"miss_rate": 0.967
|
||||
},
|
||||
"R8": {
|
||||
"total": 75,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 91,
|
||||
"detected": 0,
|
||||
"recall": 0.0,
|
||||
"miss_rate": 1.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 74,
|
||||
"detected": 4,
|
||||
"recall": 0.0541,
|
||||
"miss_rate": 0.9459
|
||||
}
|
||||
}
|
||||
},
|
||||
"L1c_combined": {
|
||||
"binary_f1": 0.32558139534883723,
|
||||
"high_risk_recall": 0.19954389965792474,
|
||||
"high_risk_precision": 0.8838383838383839,
|
||||
"false_negative_rate": 0.8004561003420753,
|
||||
"level_macro_f1": 0.12164103976458382,
|
||||
"level_weighted_f1": 0.11307540313209122,
|
||||
"level_per_class_f1": [
|
||||
0.3326007326007326,
|
||||
0.0,
|
||||
0.24170616113744076,
|
||||
0.03389830508474576,
|
||||
0.0
|
||||
],
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 123,
|
||||
"detected": 8,
|
||||
"recall": 0.065,
|
||||
"miss_rate": 0.935
|
||||
},
|
||||
"R2": {
|
||||
"total": 96,
|
||||
"detected": 15,
|
||||
"recall": 0.1562,
|
||||
"miss_rate": 0.8438
|
||||
},
|
||||
"R3": {
|
||||
"total": 77,
|
||||
"detected": 28,
|
||||
"recall": 0.3636,
|
||||
"miss_rate": 0.6364
|
||||
},
|
||||
"R4": {
|
||||
"total": 81,
|
||||
"detected": 25,
|
||||
"recall": 0.3086,
|
||||
"miss_rate": 0.6914
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 9,
|
||||
"recall": 0.1406,
|
||||
"miss_rate": 0.8594
|
||||
},
|
||||
"R6": {
|
||||
"total": 105,
|
||||
"detected": 11,
|
||||
"recall": 0.1048,
|
||||
"miss_rate": 0.8952
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 9,
|
||||
"recall": 0.0989,
|
||||
"miss_rate": 0.9011
|
||||
},
|
||||
"R8": {
|
||||
"total": 75,
|
||||
"detected": 49,
|
||||
"recall": 0.6533,
|
||||
"miss_rate": 0.3467
|
||||
},
|
||||
"R9": {
|
||||
"total": 91,
|
||||
"detected": 7,
|
||||
"recall": 0.0769,
|
||||
"miss_rate": 0.9231
|
||||
},
|
||||
"R10": {
|
||||
"total": 74,
|
||||
"detected": 14,
|
||||
"recall": 0.1892,
|
||||
"miss_rate": 0.8108
|
||||
}
|
||||
}
|
||||
},
|
||||
"ours_detection": {
|
||||
"binary_f1": 0.9988597491448119,
|
||||
"high_risk_recall": 0.9988597491448119,
|
||||
"high_risk_precision": 0.9988597491448119,
|
||||
"false_negative_rate": 0.0011402508551880963,
|
||||
"level_macro_f1": 0.4974096618676628,
|
||||
"level_weighted_f1": 0.5113791757593992,
|
||||
"level_per_class_f1": [
|
||||
0.67601246105919,
|
||||
0.17391304347826086,
|
||||
0.45622119815668205,
|
||||
0.6204620462046204,
|
||||
0.5604395604395604
|
||||
],
|
||||
"fine_per_label_f1": [
|
||||
0.7047244094488189,
|
||||
0.40274599542334094,
|
||||
0.6269035532994924,
|
||||
0.4339622641509434,
|
||||
0.6253521126760564,
|
||||
0.2874617737003058,
|
||||
0.27901785714285715,
|
||||
0.2389937106918239,
|
||||
0.6086956521739131,
|
||||
0.5878136200716846,
|
||||
0.350253807106599,
|
||||
0.4444444444444444,
|
||||
0.3734015345268542,
|
||||
0.6942148760330579
|
||||
],
|
||||
"fine_macro_f1": 0.4755704007778709,
|
||||
"fine_weighted_f1": 0.5078364322693886,
|
||||
"per_category_recall": {
|
||||
"R1": {
|
||||
"total": 123,
|
||||
"detected": 122,
|
||||
"recall": 0.9919,
|
||||
"miss_rate": 0.0081
|
||||
},
|
||||
"R2": {
|
||||
"total": 96,
|
||||
"detected": 96,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R3": {
|
||||
"total": 77,
|
||||
"detected": 77,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R4": {
|
||||
"total": 81,
|
||||
"detected": 81,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R5": {
|
||||
"total": 64,
|
||||
"detected": 64,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R6": {
|
||||
"total": 105,
|
||||
"detected": 105,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R7": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R8": {
|
||||
"total": 75,
|
||||
"detected": 75,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R9": {
|
||||
"total": 91,
|
||||
"detected": 91,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
},
|
||||
"R10": {
|
||||
"total": 74,
|
||||
"detected": 74,
|
||||
"recall": 1.0,
|
||||
"miss_rate": 0.0
|
||||
}
|
||||
},
|
||||
"label_filter": "all"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user