fischer-agentkit/tests/integration
chiguyong 0a8f6eebef feat(U5): E2E integration test for PLAN_EXEC lifecycle
Add tests/integration/test_plan_exec_e2e.py covering the full PLAN_EXEC
path through a scripted LLM mock (deterministic, no real API call).

Mock boundary: LLMGateway.chat_stream yields scripted StreamChunk
objects. Real ReActEngine, real PhasePolicy (default_policy()), real
AdvancePhaseTool, real chat._handle_chat_message WS handler.

Test scenarios (7 tests, all passing):
- Happy path: PLANNING (search) → advance_phase → BUILDING (write_file)
  → advance_phase → VERIFICATION (shell ls tests/unit/) → advance_phase
  → DELIVERY (final answer). Asserts final_answer, tool dispatch counts,
  no phase_violation events, engine ends at DELIVERY.
- Negative path: write_file in PLANNING blocked → phase_violation event
  emitted with violation_kind=tool_not_allowed → LLM calls advance_phase
  → write_file in BUILDING succeeds. Asserts exactly 1 violation, tool
  NOT dispatched during PLANNING (write_file.call_count==1 after recovery).
- Edge cases:
  - auto_advance_after_steps=2: engine transitions out of PLANNING
    after 2 LLM calls without explicit advance_phase.
  - policy_from_config(enabled=False) returns None (PLAN_EXEC disabled).
  - policy_from_config({}) returns None (opt-out, fall back to default).
- Error path: chat_stream raises RuntimeError → exception propagates,
  phase state unchanged (still PLANNING), tool not dispatched.
- WS handler integration: full _handle_chat_message path emits both
  phase_violation (from engine) and phase_changed (from WS handler's
  transition detection) to the client WebSocket.

Notes:
- Loop detector threshold bumped to 99 for happy/negative/auto-advance
  tests (3 legitimate advance_phase calls with {} args would trigger
  the default threshold=2; this is a known PLAN_EXEC production concern
  tracked separately).
- VERIFICATION-phase shell command uses `ls tests/unit/` instead of
  plan's `pytest tests/unit/ -q` — pytest is not in
  ShellTool._SAFE_COMMAND_PREFIXES and would be flagged dangerous by
  the default policy's bash filter. Using ls (whitelisted) keeps the
  test focused on lifecycle validation rather than policy tuning.

Verification: python3 -m pytest tests/integration/test_plan_exec_e2e.py -v
passes (7/7). Full regression: 116 tests pass across U1-U5 test files.
Ruff check + format clean.

Refs: R34, R27. Plan: docs/plans/2026-06-30-001-feat-agent-wave4-plan-exec-hardening-plan.md
2026-06-30 11:36:02 +08:00
..
admin fix(review): CLI field names, Pydantic validation, exception chaining 2026-06-22 15:24:31 +08:00
auth fix(review): U7 refresh token hash verification on whoami 2026-06-22 16:55:20 +08:00
__init__.py feat: initial fischer-agentkit package with unified agent architecture 2026-06-04 22:24:06 +08:00
conftest.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_agent_lifecycle.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_agent_v2_lifecycle.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_chat_adaptive_e2e.py test: add Phase 8 integration tests for Chat + Adaptive + Multi-Agent (U8) 2026-06-08 01:17:04 +08:00
test_coding_harness_pipeline.py test(pipeline): add coding harness integration tests 2026-06-12 09:42:21 +08:00
test_document_e2e.py feat(documents): add document processing capability (U1-U9) 2026-06-23 15:05:01 +08:00
test_evolution_loop.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_expert_team.py test(integration): U9 重写集成测试覆盖流水线模式 2026-06-18 02:26:59 +08:00
test_gap_closure.py feat: gap closure sprint — dark theme, @-mention, LocalComputerUse, tests 2026-06-14 16:16:50 +08:00
test_geo_compression.py fix(agentkit): resolve all P0/P1/P2/P3 issues from code review 2026-06-07 22:05:18 +08:00
test_geo_e2e.py feat(geo): U4 GEO skill tool binding with BaiduSearch and E2E tests 2026-06-07 17:25:37 +08:00
test_goal_driven_scenario.py fix(security): resolve all P0/P1 findings from code review 2026-06-10 07:12:41 +08:00
test_marketplace_e2e.py fix(marketplace): address code review findings 2026-06-10 19:21:40 +08:00
test_mcp_roundtrip.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_merged_router.py feat: accumulated frontend enhancements, docs, and static assets 2026-06-14 16:35:01 +08:00
test_p0_hardening.py feat(llm): U17 — LiteLLM 语义缓存替换 + per-user/ACL scope 安全隔离 2026-06-25 22:49:59 +08:00
test_parallel_tools.py feat: accumulated frontend enhancements, docs, and static assets 2026-06-14 16:35:01 +08:00
test_plan_exec_e2e.py feat(U5): E2E integration test for PLAN_EXEC lifecycle 2026-06-30 11:36:02 +08:00
test_react_loop.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00
test_reflexion_loop.py fix: resolve code review issues from deferred improvements 2026-06-11 13:49:02 +08:00
test_rewoo_configurable_fallback.py feat: accumulated frontend enhancements, docs, and static assets 2026-06-14 16:35:01 +08:00
test_rewoo_fallback.py fix: resolve code review issues from deferred improvements 2026-06-11 13:49:02 +08:00
test_router_engine_chain.py fix(review): comprehensive P0-P2 code review fixes 2026-06-12 22:18:25 +08:00
test_server_e2e.py fix(test): adapt health check assertion to Phase 4 status value change 2026-06-06 21:56:30 +08:00
test_soul_evolution_trigger.py fix: resolve code review issues from deferred improvements 2026-06-11 13:49:02 +08:00
test_tool_composition.py feat(agentkit): v2 Phase 1 - ReAct/LLM Gateway/Skill/Server + review fixes 2026-06-05 23:32:16 +08:00