diff --git a/docs/superpowers/plans/2026-09-10-foundation-safe-migration.md b/docs/superpowers/plans/2026-09-10-foundation-safe-migration.md new file mode 100644 index 0000000..f296a1e --- /dev/null +++ b/docs/superpowers/plans/2026-09-10-foundation-safe-migration.md @@ -0,0 +1,353 @@ +# 新底座无损迁移 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 在不改变现有工作区和当前运行数据库的前提下,将 `qyqy_develop` 升级为项目底座并完整保留场外基金、NL2SQL、访客、客服 RAG 与人工转接。 + +**Architecture:** 整合分支以 `qyqy_develop` 为第一父提交,保留其错误信封、限流、embedding、记忆、Worker 租约、Outbox、配置发布和知识引用签名;再语义合并现有业务。客服只经 `BaseAgent`、`AgentFactory`、`ToolExecutor`、`PlatformGovernance` 和 `WorkerRuntime` 运行。数据库仅在独立副本验证,使用 Alembic merge revision 收敛迁移图。 + +**Tech Stack:** Python 3.13、FastAPI、SQLAlchemy Async、Alembic、MySQL、Redis、Milvus、PyMilvus、pytest、Ruff、mypy。 + +**Spec:** `docs/superpowers/specs/2026-09-10-foundation-safe-migration-design.md` + +## Global Constraints + +- 不修改 `develop`、`qyqy_develop`、`feature/customer-service-rag` 或其工作目录。 +- 不停止或重启当前服务;不向当前 MySQL、Redis、Milvus 执行写入、迁移、删除或清理。 +- 客服仅服务 `visitor` 与 `customer`,绝不返回持仓、收益、订单、银行卡或投诉进度。 +- 访客只使用最小 `visitor` 上下文,不查询正式 RBAC、不写客户记忆、不读取客户数据。 +- 真实密钥、密码和 JWT 私钥只留在本地 `.env`,不进入代码、证据、日志或 Git。 +- 不重命名、删除、复用既有数据库表和字段;客服检索只返回已发布、启用、有效知识。 +- 每项代码变更先写并观察失败测试,再实现最小代码;任务完成后运行测试并提交。 + +--- + +### Task 1: 固化迁移前状态并设立停止门禁 + +**Files:** +- Create: `tools/foundation_migration_preflight.py` +- Create: `tests/unit/tools/test_foundation_migration_preflight.py` +- Create: `docs/evidence/foundation-migration-preflight.json` + +**Interfaces:** +- Produces: `GitRunner = Callable[[Path, str, *str], str]` 与 `run_git(worktree: Path, *args: str) -> str`;`run_git` 只能调用 `git -C ` 的只读子命令。 +- Produces: `collect_workspace_state(worktree: Path) -> dict[str, object]`,仅包含 `path`、`branch`、`head`、`status`。 +- Produces: `write_preflight_report(target: Path, states: list[dict[str, object]]) -> None`。 + +- [ ] **Step 1: 写失败测试,证明采集器保留未跟踪文件状态且不读取环境变量。** + +```python +def test_collect_workspace_state_records_untracked_paths_without_environment_values(tmp_path: Path): + state = collect_workspace_state(tmp_path, runner=fake_git_runner) + assert state["branch"] == "feature/customer-service-rag" + assert "app/service/agent/customer_service_agent.py" in state["status"] + assert "MYSQL_PASSWORD" not in json.dumps(state) +``` + +- [ ] **Step 2: 运行测试确认失败。** + +Run: `python -m pytest tests/unit/tools/test_foundation_migration_preflight.py -q -p no:cacheprovider` + +Expected: FAIL,因为采集器尚不存在。 + +- [ ] **Step 3: 实现只读采集器与报告写入。** + +```python +def collect_workspace_state(worktree: Path, runner: GitRunner = run_git) -> dict[str, object]: + return { + "path": str(worktree.resolve()), + "branch": runner(worktree, "branch", "--show-current").strip(), + "head": runner(worktree, "rev-parse", "HEAD").strip(), + "status": runner(worktree, "status", "--short").splitlines(), + } +``` + +`write_preflight_report` 使用 `json.dumps(..., ensure_ascii=False, indent=2)`,输入只能来自 Git 输出。 + +- [ ] **Step 4: 验证、生成报告并提交。** + +Run: `python -m pytest tests/unit/tools/test_foundation_migration_preflight.py -q -p no:cacheprovider` + +Expected: PASS;报告列出三个原工作区状态。提交 `tools`、测试和证据,消息为 `test: record pre-migration workspace evidence`。 + +### Task 2: 语义合并已提交的场外基金和 NL2SQL 功能 + +**Files:** +- Modify: `.env.example`、`app/api/dependencies/auth.py`、`app/core/config.py`、`app/infrastructure/db.py`、`app/main.py` +- Modify: `app/service/agent/bootstrap.py`、`app/service/model_gateway.py`、`app/service/tool_executor.py`、`app/worker/runtime.py` +- Create/Modify: 现有已提交历史中的场外基金、NL2SQL、邮件 Worker 和测试文件。 + +**Interfaces:** +- Consumes: 已提交的 `feature/customer-service-rag` 历史;不改变该工作区的未提交客服文件。 +- Produces: 保留新底座公共链且继续注册场外基金、NL2SQL 的应用。 + +- [ ] **Step 1: 记录新底座合并前测试基线。** + +Run: `python -m pytest tests/unit tests/contract -q -p no:cacheprovider` + +Expected: PASS;失败时记录输出并停止合并。 + +- [ ] **Step 2: 在整合分支进行无提交合并。** + +```powershell +git merge --no-commit --no-ff feature/customer-service-rag +``` + +Expected: 冲突仅出现在整合工作区,原工作区状态不变。 + +- [ ] **Step 3: 语义解决 13 个公共文件冲突。** + +保留新底座的错误信封、限流、追踪标识、embedding、配置发布、Outbox、Worker 租约和记忆链路;恢复场外基金/NL2SQL 的路由、工具注册、邮件 Worker、健康检查和非敏感配置。禁止整文件使用 `--ours` 或 `--theirs` 覆盖。 + +- [ ] **Step 4: 先写路由失败测试再完成注册。** + +```python +def test_app_registers_platform_and_offsite_routes(): + paths = {route.path for route in create_app().routes} + assert "/api/v1/agent-runs" in paths + assert any(path.startswith("/api/v1/offsite") for path in paths) +``` + +Run: `python -m pytest tests/unit/api/test_controller_routing_contract.py -q -p no:cacheprovider` + +Expected: 先 FAIL,路由注册后 PASS。 + +- [ ] **Step 5: 运行业务回归并提交。** + +Run: `python -m pytest tests/unit/api tests/unit/service tests/contract -q -p no:cacheprovider` + +Expected: 无导入错误,场外基金、NL2SQL、模型、工具、错误信封和限流测试通过。提交语义合并,消息为 `merge: preserve existing business features on new foundation`。 + +### Task 3: 迁移访客最小上下文与客服运行生命周期 + +**Files:** +- Modify: `app/core/contracts.py`、`app/core/security.py`、`app/api/dependencies/auth.py` +- Modify: `app/service/agent/base.py`、`app/service/agent/factory.py`、`app/service/agent_run_application_service.py`、`app/worker/runtime.py` +- Test: `tests/unit/api/test_visitor_tokens.py`、`tests/integration/test_agent_run_acceptance.py`、`tests/integration/test_worker_runtime_mysql.py` + +**Interfaces:** +- Produces: `RequestContext(roles=("visitor",), permissions=("agent:run",), data_scope="public")`,仅由服务器验证的访客 token 产生。 +- Produces: `AgentDefinition.requires_model_intent_classification: bool = True`;客服设为 `False`。 +- Produces: `WorkerRuntime.restore_context(actor_type: str, actor_id: str, trace_id: str) -> RequestContext`;仅当 `actor_type == "visitor"` 时跳过 `resolve_identity`。 + +- [ ] **Step 1: 写失败测试,Worker 恢复访客时不得查询身份仓库。** + +```python +async def test_worker_restores_visitor_without_identity_repository_call(): + runtime = WorkerRuntime(resolve_identity=fail_if_called) + context = await runtime.restore_context(actor_type="visitor", actor_id="visitor:test") + assert context.roles == ("visitor",) + assert context.data_scope == "public" +``` + +- [ ] **Step 2: 运行测试确认失败。** + +Run: `python -m pytest tests/integration/test_worker_runtime_mysql.py -k visitor -q -p no:cacheprovider` + +Expected: FAIL,因为新版运行时尚无访客恢复分支。 + +- [ ] **Step 3: 迁移访客实现。** + +访客 token 只含受限 `sub`、`roles=["visitor"]`、`portal="api"` 和短过期时间;客户端不能传角色。Outbox 只保存已验证 `actor_type="visitor"` 和不可关联账户的 actor id。访客不得触发 `IdentityService.resolve()`、客户记忆或客户范围查询。 + +- [ ] **Step 4: 先写客服跳过模型分类测试并实现条件。** + +```python +async def test_customer_service_skips_model_intent_classification(): + assert CustomerServiceAgent.definition.requires_model_intent_classification is False +``` + +`BaseAgent.classify_intent()` 在 `requires_model_intent_classification` 为 `False` 时直接返回 `None`,其它 Agent 使用默认 `True`。 + +- [ ] **Step 5: 验证和提交。** + +Run: `python -m pytest tests/unit/api/test_visitor_tokens.py tests/integration/test_agent_run_acceptance.py tests/integration/test_worker_runtime_mysql.py -q -p no:cacheprovider` + +Expected: 访客只能运行公开客服;正式用户仍在执行期刷新 RBAC。提交消息为 `feat: preserve isolated visitor agent runtime`。 + +### Task 4: 迁移客服公开知识检索与权威降级 + +**Files:** +- Create: `app/core/knowledge_contracts.py`、`app/infrastructure/milvus_knowledge_adapter.py`、`app/model/knowledge.py` +- Create: `app/service/knowledge_config.py`、`app/service/knowledge_authority.py`、`app/service/knowledge_retrieval_service.py`、`app/service/knowledge_tool_service.py` +- Modify: `app/core/config.py`、`app/service/model_gateway.py`、`app/service/agent/bootstrap.py` +- Test: `tests/unit/core/test_knowledge_contracts.py`、`tests/unit/infrastructure/test_milvus_knowledge_adapter.py`、`tests/unit/service/test_knowledge_config.py`、`tests/unit/service/test_knowledge_authority.py`、`tests/unit/service/test_knowledge_retrieval.py` + +**Interfaces:** +- Produces: `KnowledgeQuery(query, intents, top_k)`;调用方不得传集合名。 +- Produces: `KnowledgeRetrievalService.search(query, context) -> KnowledgeSearchResult`。 +- Produces: 只读工具 `query_knowledge`,权限码 `knowledge:query`。 + +- [ ] **Step 1: 写失败测试,FAQ 只能访问 FAQ 集合。** + +```python +async def test_search_uses_faq_collection_for_faq_only(): + result = await service.search(KnowledgeQuery(query="开户", intents=("faq",)), visitor_context) + assert vector_store.calls == [("fin_faq_collection", 3)] + assert result.searched_collections == ("fin_faq_collection",) +``` + +- [ ] **Step 2: 运行测试确认失败。** + +Run: `python -m pytest tests/unit/service/test_knowledge_retrieval.py -q -p no:cacheprovider` + +Expected: FAIL,因为检索服务尚未迁入。 + +- [ ] **Step 3: 迁移受控契约、配置、Milvus 和 MySQL 权威层。** + +路由固定为 `faq -> fin_faq_collection, top_k=3`、`product_inquiry -> fin_product_collection, top_k=5`、`policy_explain -> fin_policy_collection, top_k=5`。维度使用 `vector_dim`,默认 1024,COSINE 检索。Milvus 只允许搜索,不得写集合或向量。 + +- [ ] **Step 4: 写失败测试,Milvus 故障只能降级到有效知识。** + +```python +async def test_milvus_failure_falls_back_to_published_active_unexpired_knowledge(): + result = await service.search(query, visitor_context) + assert result.degraded is True + assert [hit.knowledge_id for hit in result.hits] == ["101"] + assert result.hits[0].answer == "已审核的标准答复" +``` + +- [ ] **Step 5: 实现降级过滤、工具注册、测试和提交。** + +Milvus 命中按阈值去重后,必须回查 `fin_knowledge_meta` 的 `published`、`active`、开始和结束日期。Embedding 数量或维度不符抛 `RecoverableAgentError`;检索失败时仅返回 MySQL 降级结果或客服转人工,禁止猜测答案。 + +Run: `python -m pytest tests/unit/core/test_knowledge_contracts.py tests/unit/infrastructure/test_milvus_knowledge_adapter.py tests/unit/service/test_knowledge_config.py tests/unit/service/test_knowledge_authority.py tests/unit/service/test_knowledge_retrieval.py -q -p no:cacheprovider` + +Expected: 三集合准确路由,未发布、失效、低分知识均不可回答。提交消息为 `feat: add governed public knowledge retrieval`。 + +### Task 5: 迁移客服 Agent、人工转接与同源联调页 + +**Files:** +- Create: `app/service/agent/customer_service_routing.py`、`app/service/agent/customer_service_agent.py`、`app/static/index.html` +- Modify: `app/main.py`、`app/service/agent/bootstrap.py`、`app/service/agent/governance.py` +- Test: `tests/unit/service/test_customer_service_agent.py`、`tests/unit/api/test_customer_service_test_page.py` + +**Interfaces:** +- Produces: `CustomerServiceIntentRouter.classify(message) -> CustomerServiceRoute`。 +- Produces: `CustomerServiceAgent.handle(request, context) -> CoreResult`。 + +- [ ] **Step 1: 写失败测试,账户问题只能返回前端账户入口。** + +```python +async def test_customer_account_question_only_returns_account_entry(): + result = await agent.handle(request("查一下我的订单"), customer_context) + assert result.text == "我无法查询账户数据,请前往“我的账户”查看相关状态。" +``` + +- [ ] **Step 2: 运行测试确认失败。** + +Run: `python -m pytest tests/unit/service/test_customer_service_agent.py -q -p no:cacheprovider` + +Expected: FAIL,因为客服 Agent 尚未注册。 + +- [ ] **Step 3: 迁移固定路由、电话治理和 Agent 注册。** + +优先级固定为安全风险、合规拒答、账户入口、人工转接、闲聊、公开知识。安全风险、保证收益、代客交易、投诉和人工问题不得进入知识检索。第四条连续闲聊只引导一次。客服电话只从受控 `customer_service_phone` 读取,治理层仅保留该号码,其余手机号继续脱敏。 + +- [ ] **Step 4: 写失败测试,检索失败必须转人工。** + +```python +async def test_public_knowledge_failure_requires_human_transfer(): + result = await agent.handle(request("基金怎么开户"), visitor_context) + assert result.transfer_required is True + assert result.transfer_reason == "knowledge_unavailable" +``` + +- [ ] **Step 5: 挂载页面、验证并提交。** + +Run: `python -m pytest tests/unit/service/test_customer_service_agent.py tests/unit/api/test_customer_service_test_page.py -q -p no:cacheprovider` + +Expected: 访客和客户只获得一期公开服务;账户问题不含数据;人工转接含受控电话;页面返回 200。提交消息为 `feat: migrate scoped customer service agent`。 + +### Task 6: 收敛 Alembic 迁移图并验证独立副本库 + +**Files:** +- Create: `alembic/versions/20260910_merge_foundation_offsite.py` +- Create: `tests/integration/test_migration_graph.py` +- Create: `docs/evidence/foundation-migration-database-verification.md` + +**Interfaces:** +- Produces: 唯一 Alembic head `20260910_merge_foundation_offsite`。 +- Produces: `down_revision = ("20260910_drop_review_separation", "20260910_offsite_worker")` 的无 DDL merge revision。 +- Produces: 测试辅助函数 `current_heads() -> list[str]`,用 `ScriptDirectory.from_config(Config("alembic.ini")).get_heads()` 返回排序后的 revision 列表。 + +- [ ] **Step 1: 写失败测试,当前迁移图存在两个 head。** + +```python +def test_alembic_history_has_one_head_after_merge_revision(): + assert current_heads() == ["20260910_merge_foundation_offsite"] +``` + +- [ ] **Step 2: 运行测试确认失败。** + +Run: `python -m pytest tests/integration/test_migration_graph.py -q -p no:cacheprovider` + +Expected: FAIL,显示 `20260910_drop_review_separation` 和 `20260910_offsite_worker`。 + +- [ ] **Step 3: 创建无 DDL merge revision。** + +```python +revision = "20260910_merge_foundation_offsite" +down_revision = ("20260910_drop_review_separation", "20260910_offsite_worker") + +def upgrade() -> None: + pass + +def downgrade() -> None: + pass +``` + +- [ ] **Step 4: 在独立副本库执行迁移、审计和记录。** + +Run: `alembic heads; alembic upgrade head; python tools/migration_state_check.py; python tools/audit_schema.py; python tools/audit_constraints.py` + +Expected: 仅一个 head,副本库升级和结构审计通过。副本连接地址和数据库名必须与当前运行数据库不同。 + +- [ ] **Step 5: 提交迁移图收敛。** + +提交 revision、迁移图测试和副本验证证据,消息为 `chore: merge foundation and offsite migration heads`。 + +### Task 7: 全量回归、原工作区一致性核验与交付 + +**Files:** +- Create: `tests/integration/test_foundation_migration_regression.py` +- Create: `docs/evidence/foundation-migration-regression.md` +- Modify: `docs/20-第一版到当前版本变更与迁移指南.md` + +**Interfaces:** +- Produces: 同时覆盖底座、场外基金、NL2SQL、访客客服、已登录客服和迁移图的回归证据。 + +- [ ] **Step 1: 写失败测试,场外能力与客服页必须同时存在。** + +```python +async def test_regression_keeps_offsite_and_customer_service_available(client): + assert (await client.get("/health")).status_code == 200 + assert (await client.get("/customer-service-test/")).status_code == 200 + assert any(route.path.startswith("/api/v1/offsite") for route in client.app.routes) +``` + +- [ ] **Step 2: 运行测试确认失败或暴露缺失模块。** + +Run: `python -m pytest tests/integration/test_foundation_migration_regression.py -q -p no:cacheprovider` + +Expected: FAIL,直到场外路由与客服页面均已注册。 + +- [ ] **Step 3: 执行全量测试和静态检查。** + +Run in order: `python -m pytest tests/unit tests/contract -q -p no:cacheprovider`; `python -m pytest tests/integration -q -p no:cacheprovider`; `python -m ruff check app tests tools alembic`; `python -m mypy app`. + +Expected: 全部适用测试通过,Ruff 与 mypy 无错误;集成测试仅使用独立副本库。 + +- [ ] **Step 4: 重跑预检工具验证原工作区完全未变。** + +Run: `python tools/foundation_migration_preflight.py --verify docs/evidence/foundation-migration-preflight.json` + +Expected: 三个原工作区的 branch、head 与 status 和迁移前一致;有差异立即停止交付。 + +- [ ] **Step 5: 提交证据和迁移指南。** + +提交回归测试、证据和指南,消息为 `docs: record safe foundation migration verification`。 + +## Execution Order and Stop Conditions + +严格按 Task 1 至 Task 7 执行。出现下列任一情况立即停止:合并不能保留既有功能、访客触发正式身份查询、未发布知识可见、副本库审计失败、全量回归失败或原工作区状态变化。停止时只保留整合分支供诊断,不合并、不推送、不发布数据库迁移。