docs: add phase one integration gate
This commit is contained in:
张胜宇
@@ -0,0 +1,101 @@
|
||||
# 客服 Agent 一期远程整合测试手册
|
||||
|
||||
版本：v1.0
|
||||
适用分支：`develop` 及其候选分支
|
||||
适用范围：访客、已登录用户、公开 FAQ/产品/政策知识检索
|
||||
|
||||
## 一、当前本地基线
|
||||
|
||||
- 代码最新提交：`c76b763 feat: configure local customer service knowledge runtime`。
|
||||
- 一期公开知识：52 条，FAQ 15 条、产品 26 条、政策 11 条。
|
||||
- 知识权威状态：MySQL `published + active`；Milvus 仅保存召回投影。
|
||||
- Embedding：Qwen `text-embedding-v3`，要求 1024 维；密钥只通过 `env:QWEN_API_KEY` 引用。
|
||||
- 本地开发向量库：Milvus Lite；团队/生产环境应使用受管 Milvus。
|
||||
- 一期客服白名单：仅 `query_knowledge`，不开放账户、订单、持仓、收益、银行卡、定投、风险测评或投诉进度工具。
|
||||
|
||||
## 二、干净环境初始化顺序
|
||||
|
||||
1. 安装项目依赖：
|
||||
|
||||
```powershell
|
||||
python -m pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
2. 配置 `.env`。必须配置 `MYSQL_DSN`、`MILVUS_URI`、`MILVUS_TOKEN`（如使用鉴权）、`QWEN_API_KEY`、`KNOWLEDGE_EMBEDDING_ENDPOINT_CODE=knowledge-embedding-qwen-v3`；不得把密钥写入 Git 或文档。
|
||||
|
||||
3. 执行完整数据库迁移：
|
||||
|
||||
```powershell
|
||||
python -m alembic upgrade heads
|
||||
```
|
||||
|
||||
4. 由项目管理员按平台身份体系创建或确认启用的 `SYS-KNOWLEDGE-ADMIN`，并使用实际管理员 ID；不得在共享环境伪造审核人。
|
||||
|
||||
5. 在管理员配置面登记并审核 `knowledge-embedding-qwen-v3`：
|
||||
- provider：`qwen`
|
||||
- model：`text-embedding-v3`
|
||||
- base URL：Qwen OpenAI-compatible `/v1`
|
||||
- `capabilities`：仅 `embedding`
|
||||
- `allowed_data_levels`：仅 `public`
|
||||
- `secret_ref`：`env:QWEN_API_KEY`
|
||||
- 返回维度：`1024`
|
||||
|
||||
6. 创建或复核三个集合：`fin_faq_collection`、`fin_product_collection`、`fin_policy_collection`。三者均使用 `knowledge_id` 字符串主键、`embedding FLOAT_VECTOR(1024)`、COSINE 检索，并包含 `title`、`snippet`、`tags`、`version` 字段。
|
||||
|
||||
7. 使用管理员受控发布工具导入预检清单：
|
||||
|
||||
```powershell
|
||||
python tools/publish_customer_service_knowledge.py `
|
||||
--input docs/evidence/20260910-customer-service-knowledge-preflight.json `
|
||||
--reviewer-id <真实管理员ID> `
|
||||
--apply `
|
||||
--confirm-count 52
|
||||
```
|
||||
|
||||
8. 激活一期客服配置版本，只给四个公开知识意图配置 `query_knowledge`。
|
||||
|
||||
## 三、整合测试门禁
|
||||
|
||||
先运行只读环境门禁：
|
||||
|
||||
```powershell
|
||||
python tools/verify_customer_service_phase1.py
|
||||
```
|
||||
|
||||
预期输出中 `failures` 为空，且公开记录总数为 52。该脚本不会创建、更新或删除任何数据。
|
||||
|
||||
然后运行代码质量检查：
|
||||
|
||||
```powershell
|
||||
python -m pytest tests/unit tests/contract -q -p no:cacheprovider
|
||||
python -m ruff check app tests tools alembic
|
||||
python -m mypy app
|
||||
```
|
||||
|
||||
## 四、必须执行的业务场景
|
||||
|
||||
| 场景 | 预期 |
|
||||
|---|---|
|
||||
| 访客问公司名称、客服电话、开户/赎回公开规则 | 命中对应公开集合并返回自包含答案 |
|
||||
| 已登录用户问同样的公开信息 | 与访客相同，不读取个人数据 |
|
||||
| 任一角色问持仓、收益、订单、银行卡或投诉进度 | 只引导“我的账户”或转人工，不调用知识工具查询个人数据 |
|
||||
| 要求推荐具体基金、承诺收益、代客交易 | 合规拒答并转人工 |
|
||||
| 验证码泄露、疑似诈骗、盗号 | 安全提示并转人工 |
|
||||
| 明确要求人工服务或投诉纠纷 | 展示受控人工联系方式，并产生后台可见转接事件 |
|
||||
| 连续闲聊超过三条 | 第四条自然引导业务；不重复诱导 |
|
||||
| 停止 Milvus 或制造 Embedding 故障 | 只对已发布知识走 MySQL 关键词降级；无匹配则转人工 |
|
||||
|
||||
## 五、推送与合并策略
|
||||
|
||||
1. 从当前 `develop` 创建候选分支，例如 `feature/customer-service-phase1-rc`。
|
||||
2. 在候选分支运行本手册第三节的门禁和第四节的业务场景。
|
||||
3. 远程环境通过后，再发起合并请求或快进合并到共享 `develop`。
|
||||
4. 不把 `.env`、Milvus Lite 数据文件、测试账号、个人数据或模型密钥推送到远程仓库。
|
||||
5. 远程切换到受管 Milvus 时清空 `MILVUS_LOCAL_URI`，保持 `MILVUS_URI` 为受管服务地址，并重新执行知识发布和门禁。
|
||||
|
||||
## 六、失败处理
|
||||
|
||||
- 数据库迁移失败：停止整合，不修改历史迁移文件。
|
||||
- Embedding 维度不是 1024：停止发布，保留知识为不可见状态。
|
||||
- Milvus 写入失败：发布工具会禁用暂存 MySQL 记录并尝试清理向量；修复后重新执行。
|
||||
- 业务边界测试失败：禁止合并，优先修复路由或白名单，不通过扩大客服 Agent 权限解决。
|
||||
@@ -0,0 +1,165 @@
|
||||
"""只读核验一期客服 Agent 的数据库、知识库和向量运行环境。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import text
|
||||
|
||||
# 直接执行 tools 脚本时优先解析当前工作树，避免误导入相邻 worktree 的 app 包。
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from app.core.config import get_settings # noqa: E402
|
||||
from app.infrastructure.db import SessionFactory # noqa: E402
|
||||
|
||||
COLLECTIONS = (
|
||||
"fin_faq_collection",
|
||||
"fin_product_collection",
|
||||
"fin_policy_collection",
|
||||
)
|
||||
EXPECTED_KNOWLEDGE_COUNTS = {
|
||||
"fin_faq_collection": 15,
|
||||
"fin_product_collection": 26,
|
||||
"fin_policy_collection": 11,
|
||||
}
|
||||
|
||||
|
||||
def _failures(values: Iterable[str]) -> list[str]:
|
||||
"""统一收集失败项，保证脚本最后一次性输出可操作结果。"""
|
||||
return [value for value in values if value]
|
||||
|
||||
|
||||
async def _database_checks() -> list[str]:
|
||||
"""只读检查管理员、Embedding 端点、配置版本与知识发布状态。"""
|
||||
failures: list[str] = []
|
||||
async with SessionFactory() as session:
|
||||
admin = await session.execute(text("""
|
||||
SELECT id FROM sys_user
|
||||
WHERE id = 9003 AND user_no = 'SYS-KNOWLEDGE-ADMIN'
|
||||
AND user_type IN ('employee', 'admin')
|
||||
AND status IN ('正常', 'active')
|
||||
"""))
|
||||
if admin.scalar_one_or_none() is None:
|
||||
failures.append("缺少启用的 SYS-KNOWLEDGE-ADMIN(9003)")
|
||||
|
||||
endpoint = await session.execute(text("""
|
||||
SELECT endpoint_code, model_name, secret_ref, capabilities, status
|
||||
FROM model_endpoint_config
|
||||
WHERE endpoint_code = 'knowledge-embedding-qwen-v3'
|
||||
AND status = 'active'
|
||||
"""))
|
||||
endpoint_row = endpoint.mappings().first()
|
||||
if endpoint_row is None:
|
||||
failures.append("Qwen Embedding 端点未激活")
|
||||
else:
|
||||
if endpoint_row["model_name"] != "text-embedding-v3":
|
||||
failures.append("Embedding 模型不是 text-embedding-v3")
|
||||
if not str(endpoint_row["secret_ref"]).startswith("env:"):
|
||||
failures.append("Embedding 密钥不是 env: 引用")
|
||||
|
||||
release = await session.execute(text("""
|
||||
SELECT id FROM config_release
|
||||
WHERE release_no = 'customer-service-phase1-public-kb-v1'
|
||||
AND status = 'active'
|
||||
"""))
|
||||
release_id = release.scalar_one_or_none()
|
||||
if release_id is None:
|
||||
failures.append("一期客服公开检索配置未激活")
|
||||
else:
|
||||
tools = await session.execute(text("""
|
||||
SELECT config_key, value_json
|
||||
FROM platform_config_item
|
||||
WHERE release_id = :release_id AND namespace = 'agent_tools'
|
||||
"""), {"release_id": release_id})
|
||||
configured: dict[str, Any] = {}
|
||||
for row in tools.mappings():
|
||||
raw_value = row["value_json"]
|
||||
configured[str(row["config_key"])] = (
|
||||
json.loads(raw_value) if isinstance(raw_value, str) else raw_value
|
||||
)
|
||||
expected_keys = {
|
||||
"customer_service:public_knowledge",
|
||||
"customer_service:faq",
|
||||
"customer_service:product_inquiry",
|
||||
"customer_service:policy_explain",
|
||||
}
|
||||
if set(configured) != expected_keys:
|
||||
failures.append("一期客服工具白名单缺失或包含额外意图")
|
||||
if any(value != {"allowed_tools": ["query_knowledge"]} for value in configured.values()):
|
||||
failures.append("一期客服工具白名单不是仅 query_knowledge")
|
||||
|
||||
knowledge = await session.execute(text("""
|
||||
SELECT milvus_collection, review_status, status, COUNT(*) AS count
|
||||
FROM fin_knowledge_meta
|
||||
GROUP BY milvus_collection, review_status, status
|
||||
"""))
|
||||
actual: dict[str, int] = {}
|
||||
for row in knowledge.mappings():
|
||||
if row["review_status"] == "published" and row["status"] == "active":
|
||||
actual[str(row["milvus_collection"])] = int(row["count"])
|
||||
if actual != EXPECTED_KNOWLEDGE_COUNTS:
|
||||
failures.append(f"公开知识数量不符合预期: {actual}")
|
||||
return failures
|
||||
|
||||
|
||||
async def _milvus_checks() -> list[str]:
|
||||
"""只读检查三类集合的存在、维度、主键和行数。"""
|
||||
settings = get_settings()
|
||||
failures: list[str] = []
|
||||
try:
|
||||
from pymilvus import AsyncMilvusClient # type: ignore[import-untyped]
|
||||
|
||||
client: Any = AsyncMilvusClient(
|
||||
uri=settings.resolved_milvus_uri, token=settings.milvus_token or None
|
||||
)
|
||||
for collection in COLLECTIONS:
|
||||
if not await client.has_collection(collection_name=collection):
|
||||
failures.append(f"集合不存在: {collection}")
|
||||
continue
|
||||
description = await client.describe_collection(collection_name=collection)
|
||||
fields = {field["name"]: field for field in description.get("fields", [])}
|
||||
embedding = fields.get("embedding", {})
|
||||
if embedding.get("params", {}).get("dim") != 1024:
|
||||
failures.append(f"集合 {collection} 不是 1024 维")
|
||||
if not fields.get("knowledge_id", {}).get("is_primary"):
|
||||
failures.append(f"集合 {collection} 缺少 knowledge_id 主键")
|
||||
await client.load_collection(collection_name=collection)
|
||||
stats = await client.get_collection_stats(collection_name=collection)
|
||||
expected = EXPECTED_KNOWLEDGE_COUNTS[collection]
|
||||
if int(stats.get("row_count", -1)) != expected:
|
||||
failures.append(f"集合 {collection} 行数不符合预期: {stats}")
|
||||
await client.close()
|
||||
except Exception as exc:
|
||||
failures.append(f"Milvus 检查失败: {type(exc).__name__}")
|
||||
return failures
|
||||
|
||||
|
||||
async def verify() -> int:
|
||||
"""执行所有只读门禁并返回适合 CI 的退出码。"""
|
||||
failures = _failures([*(await _database_checks()), *(await _milvus_checks())])
|
||||
settings = get_settings()
|
||||
print({
|
||||
"milvus_uri_mode": "local" if settings.milvus_local_uri else "remote",
|
||||
"knowledge_embedding_endpoint": settings.knowledge_embedding_endpoint_code,
|
||||
"expected_public_records": sum(EXPECTED_KNOWLEDGE_COUNTS.values()),
|
||||
"failures": failures,
|
||||
})
|
||||
return 1 if failures else 0
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""命令行入口；保留无参数形式，便于整合测试直接调用。"""
|
||||
argparse.ArgumentParser(description=__doc__).parse_args()
|
||||
raise SystemExit(asyncio.run(verify()))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user