Commit Graph

301 Commits

Author SHA1 Message Date
chiguyong 5c15238a5a fix(calendar): 修复 agent 创建日历事件后 UI 不刷新 + 文档化三根因三部曲
Test / backend-test (pull_request) Has been cancelled Details
Test / frontend-unit (pull_request) Has been cancelled Details
Test / api-e2e (pull_request) Has been cancelled Details
Test / frontend-e2e (pull_request) Has been cancelled Details
代码修复 (ce-debug):
- CalendarService.create_event 注入 notify_callback,成功后广播 calendar_event_created WS 消息
- app.py 调整 _calendar_ws_sender 闭包定义顺序,注入 CalendarService(与 ReminderScheduler 共享)
- tauri-auth.ts keychain fallback 修复(localStorage 始终作为备份)
- 新增 2 个广播回归测试

文档 (ce-compound + ce-compound-refresh):
- 新增 docs/solutions/ui-bugs/calendar-agent-create-no-refresh.md(第三根因:WS 广播缺失)
- 更新 calendar-capability-and-ui-fixes.md:刷新 test count + 加 Related Issues 前向引用
- 更新 jwt-secret-dev-mode-user-id-mismatch.md:扩展 e2e bullet + 加第三个根因引用
- CONCEPTS.md 新增 Service Broadcast Callback 条目 (Real-Time Fan-Out 节)

测试:
- 新增 E2E 测试套件 (admin/auth-persistence/bitable/calendar/conversation/documents/evolution/settings/skills)
- 新增 tests/e2e/test_api_coverage.py
- CI: .gitea/.github workflows/test.yml
2026-06-29 02:20:33 +08:00
chiguyong d27681a93c fix(portal-auth): 修复 dev mode JWT 验证误激活 + README 文档同步
## Portal 401 根因修复

问题:AGENTKIT_JWT_SECRET 未设置时,jwt_utils 生成 ephemeral 非空 secret,
该 secret 被传给 AuthMiddleware 后 _is_dev_mode() 返回 False(not "" = False),
导致无 JWT/API key 的请求被拒为 401(17 个 portal 测试失败)。

修复:分离 explicit_jwt_secret 与 jwt_secret —
- explicit_jwt_secret = get_jwt_secret()  # None when env unset
- jwt_secret = explicit_jwt_secret or get_or_create_jwt_secret()  # for signing
- AuthMiddleware(jwt_secret=explicit_jwt_secret or "")  # only explicit activates JWT verify

ephemeral secret 仅供 token 签名 routes,不激活 middleware 的 JWT 验证。
生产环境(AGENTKIT_JWT_SECRET 已设置)行为不变。

验证:
- _is_dev_mode(): False → True
- GET /api/v1/portal/conversations: 401 → 200
- 27 个 portal 测试全部通过(之前 17 失败)
- 232 个测试通过 (portal + auth + calendar),0 失败

## README 文档同步

代码中 CostAwareRouter / RegexRules / HeuristicClassifier / SemanticRouter / LLMClassifier
类已完全删除,仅 RequestPreprocessor 存在。README.md 6 处过时引用同步:

- 第 4 节"意图路由"改为引用 RequestPreprocessor(详见第 7 节)
- 第 7 节重写为"请求预处理(RequestPreprocessor)",按 AGENTS.md 架构描述
- 第 8 节"语义路由"删除(合并入第 7 节历史说明)
- 架构图 CostAwareRouter → RequestPreprocessor,22→28 路由模块
- 模块详解 chat/skill_routing + chat/semantic_router 合并为 chat/request_preprocessor
- 模块详解 router/intent 描述更新为"未接入 chat 流程"
- 目录注释 CostAwareRouter → RequestPreprocessor
- 章节重新编号 1-16 连续(原 1-17 跳过 9)
2026-06-28 15:26:42 +08:00
chiguyong c9ce15fa4b fix(code-review): 修复走查发现的 13 High + Medium 安全/可靠性问题
代码修复(8 High + 9 Medium):
- portal.py — C1 IDOR 文档 / C2 类型修复 / C3 WS 连接上限 16 / C4 ws_user_id 早初始化 / M silent swallow 日志化
- auth/middleware.py — C5 WS sid 补齐
- calendar_tool.py — C6 偏移量 ±43200 双向校验 + reminder_channels 类型/白名单校验
- sqlite_conversation_store.py — C7 DELETE 事务回滚
- chat.ts (Pinia) — C8 deleteConversation 清理 pending 缓存
- app.py — M except: pass → logger.debug(exc_info=True)
- Scene6Error.vue — M onUnmounted 清理 setTimeout
- DocumentsTab.vue — M Invalid Date 守卫
- ChatSidebar/RightPanel/TopNav.vue — M aria-label 无障碍标签
- SystemMonitorPanel.vue — M v-else 兜底 + active 边框色 + tablist 键盘导航
- CalendarDrawer.vue — M overflow-y: auto
- CalendarGrid.vue — M ResizeObserver 反馈循环防护
- SkillsTab.vue — M onMounted 始终 fetchSkills

文档修复(5 High + 6 Medium):
- portal-platform-security-reliability-fixes.md — D2 测试路径 / D3 Root Cause+Impact 章节 / D4 severity: mixed / 标题中文化 / 12 处绝对路径转相对 / P2 #12 数字口径
- AGENTS.md — D5 路由表 22→28 / 专家模板 5→15 / LiteLLM U15 迁移 / 配置查找 fallback
- README.md — 8 处端口 8000→8001

新增测试:
- tests/unit/calendar/test_calendar_tool.py — ponytail 自检断言

验证:
- ruff check (5 文件) — All checks passed
- vue-tsc --noEmit — exit 0
- git stash baseline 验证 — portal 17 个 401 失败为预存在问题

已知限制(预存在):
- 17 个 portal 测试 401 失败 — 需另起 ce-debug 调查
- README.md 7 处 CostAwareRouter 引用过时 — 文档同步另起任务
2026-06-28 15:06:41 +08:00
chiguyong 8ae8ed4e9b Merge branch 'feat/calendar-ui-fixes' — 日历能力缺失修复 + UI布局优化 + 会话404处理 + ce-code-review 修复 2026-06-28 14:25:44 +08:00
chiguyong 43e9025c6d fix(calendar): 日历能力缺失修复 + UI 布局优化 + 会话404处理
P0: calendar_tool reminder_rules 未传入 create_event,提醒功能完全失效。P1: chat.ts deleteConversation 未清理 pending + 404 递归保护。P2: app.py 系统提示重复段落 + gui_mode F821 + SystemMonitorPanel flex 布局。P3: portal send_json 快照 + WS connected 清除 is_local + 移除死代码。验证: ruff+pytest 98passed+typecheck 通过。
2026-06-28 14:24:58 +08:00
chiguyong 31c65e01b8 fix(security): P0 安全加固 + 多实例部署一致性 (U1-U4 + U5c)
Deploy to Production / deploy (push) Has been cancelled Details
U1: LLM gateway KB 缓存 fail-closed — 异常时默认禁用缓存防止 KB 数据泄漏
U2: MCP 危险工具黑名单过滤 — 6+1 端点覆盖,防止绕过 chat confirmation
U3: SecretsStore Redis 迁移 — 多 worker 共享凭证,内存降级保留开发模式
U4: channels webhook Redis 状态 — ZSET 滑动窗口限流 + nonce dedup + backpressure
U5c: ce-code-review 修复批次:
  - P0: 统一 MCP 黑名单与 publisher.py 一致 (terminal_execute -> terminal, +file_read)
  - P1: ZSET 限流 member 加 uuid 后缀避免同时间戳碰撞
  - P1: SecretsStore redis 参数 Any -> aioredis.Redis | None (AGENTS.md 合规)
  - P1: Redis client 添加 socket_timeout 防止单点故障请求挂死

测试: 171 scoped tests pass, ruff clean
2026-06-26 04:05:33 +08:00
chiguyong c62d435c43 Merge branch 'feat/portal-platform-evolution' — portal platform evolution (U1-U17 + RAG + channels + LiteLLM + ce-code-review fixes + ce-compound doc)
Deploy to Production / deploy (push) Waiting to run Details
2026-06-26 01:48:19 +08:00
chiguyong 75e9b58e46 docs(ce-compound): 记录 portal-platform 安全/可靠性修复批次
记录 ce-code-review 修复批次(commit 53faa60)的 10 个 P1/P2/P3 修复:
- P1: WeCom 重放、缓存跨用户泄漏、webhook 异常风暴、shutdown 泄漏
- P2: Feishu TTL、无界任务集、配额 N+1、冗余 SHA-256、未用参数
- P3: DIRECT_CHAT 去重

新增 docs/solutions/security-issues/portal-platform-security-reliability-fixes.md
CONCEPTS.md 补充 3 个领域术语:Per-User Cache Namespace、Webhook Signature Freshness、Webhook Backpressure
2026-06-26 01:47:57 +08:00
chiguyong 53faa60472 fix(review): ce-code-review P1+P2 修复 — 安全/可靠性/性能
P1 安全与可靠性(4 项):
- wecom: verify_signature 增加时间戳新鲜度校验(5 分钟窗口防重放)
- cache: should_cache 在 per_user_namespace 开启时拒绝 user_id=None
  匿名请求,避免跨用户缓存泄漏(安全要求 a/e)
- channels: webhook receive_message 异常兜底,防止 500 触发平台重试风暴
- app: shutdown 调用 close_all_adapters + await _pending_webhook_tasks,
  防止 httpx 连接泄漏和丢失 IM 回复

P2 效率与可维护性(5 项):
- feishu: _TOKEN_CACHE_TTL 300 → 6900(2h 减 5min 余量,避免 24x 过频刷新)
- channels: _pending_webhook_tasks 有界化(2x 并发上限时 429 拒绝)
- gateway: quota 检查每 period 单次 get_usage,复用 summary 检查 token+cost
- cache_key: generate_cache_key 合并为单次 SHA-256(消除 8-10 次冗余哈希)
- config: ProviderConfig.get_api_key 移除未用的 secrets_store 参数

P3 去重(1 项):
- channels: _process_inbound_message DIRECT_CHAT 路径提取 _direct_chat 辅助函数

测试:
- test_wecom: 时间戳改用 int(time.time()),新增 test_expired_timestamp_rejected
- test_cache: should_cache 测试覆盖匿名拒绝 + namespace_off 兼容
- test_config_migration: get_api_key 测试适配新签名
- channels/config_migration/quota_enforcement 测试全部通过
2026-06-26 01:40:31 +08:00
chiguyong 1ccaf56b9a refactor: ce-simplify-code 审查修复 — 去重 + 效率 + 死代码清理
3 个审查代理(复用/质量/效率)发现 15 个问题,全部修复:

效率与安全(6 项):
- MCPClient 缓存 MultiServerMCPClient 单例 + aclose(),修复连接/子进程泄漏
- _rate_limits 清理空 IP 条目,修复 X-Forwarded-For 欺骗下内存泄漏
- _seen_nonces 改用 OrderedDict,O(1) 摊销过期清理
- webhook 后台任务加 Semaphore(20) + 任务引用追踪,限制无界并发
- _build_adapter 用 asyncio.gather 并行解密 secrets
- 适配器实例缓存(_adapter_cache),token TTL 缓存跨请求命中

去重(4 项):
- header_get 提取到 channels/base.py,4 个适配器统一 import
- _get_client/close() 移入 MessageAdapter 基类,子类继承
- URLVerificationChallenge 统一到 base.py,feishu/slack/wecom 共用
- Transport ABC 添加 endpoint_url 属性,from_transport 不再访问私有字段

死代码与类型安全(5 项):
- detect_cache_hit 死方法替换为 record_cache_result 公开 API
- execution_mode.value == "direct_chat" 改用枚举比较
- 删除 yielded_any 死变量、重复 from fastapi import Request、
  多余 getattr 防御

453 tests passed, ruff clean(预存 F841 非本次引入)
2026-06-25 23:54:14 +08:00
chiguyong 793476cafa feat(llm): U17 — LiteLLM 语义缓存替换 + per-user/ACL scope 安全隔离
- 新增 LitellmCacheManager:配置 litellm.cache 全局,三级后端 fallback
  (RedisSemanticCache -> RedisCache -> InMemoryCache),redisvl lazy import
- cache_key 扩展 user_id + kb_acl_hash 参数(安全要求 a/b/e)
- gateway 集成:读取 KB caching_disabled flag(安全要求 c),构建带 scope
  的 cache_key,命中时 cost=0
- LLMResponse 新增 cache_hit 字段;LLMRequest 新增 cache 参数
- litellm_provider 透传 cache 参数 + 检测 _hidden_params 缓存命中
- 33 个新测试覆盖 13 场景(含 User A != User B 缓存隔离)
- 旧 InMemoryLLMCache/RedisLLMCache 保留向后兼容
2026-06-25 22:49:59 +08:00
chiguyong 86541d7172 feat(mcp): U16 — langchain-mcp-adapters client replacement + transport deprecation
- 重写 MCPClient:URL scheme 自动检测(stdio/http/sse)→ langchain config
- 旧 Transport 注入路径保留(DeprecationWarning),向后兼容
- transport.py 模块级弃用警告
- 28 个新测试覆盖 URL 检测、list_tools、call_tool、legacy 路径、ImportError
- 修复 manager.py / transport.py 预存 F401/F841
2026-06-25 22:04:37 +08:00
chiguyong 069dbc22b1 feat(llm): U15 — LiteLLM unified provider + api_key encrypted secrets migration 2026-06-25 21:41:15 +08:00
chiguyong 13c516a54f feat(mcp): U14 — Skill/Team MCP publish with admin auth + dangerous-tool opt-in 2026-06-25 21:10:06 +08:00
chiguyong 16c33be295 feat(mcp): U13 — refactor MCPServer to route factory + mount at /api/v1/mcp with auth 2026-06-25 20:58:41 +08:00
chiguyong 8998f94c42 feat(channels): U12 — DingTalk/WeCom/Slack adapters + multi-channel webhook dispatch 2026-06-25 20:45:43 +08:00
chiguyong 4b58e8f661 feat(channels): U11 — Feishu IM adapter end-to-end (webhook + signature + AES-CBC decrypt + chat integration) 2026-06-25 20:24:21 +08:00
chiguyong 5572387c01 feat(channels): U10 — message adapter ABC + AES-256-GCM secrets store + channel CRUD routes 2026-06-25 20:13:37 +08:00
chiguyong af96cb49bd docs(plan): deepen portal platform evolution plan — KTD5/7/8/9 expanded, KTD11 added 2026-06-25 20:13:27 +08:00
chiguyong 864bb95a30 feat(server): wire rag_platform components to app.state lifespan
Initialize in lifespan() (after bitable, before yield):
- KBStore + ensure_tables() → app.state.kb_store (if database_url available)
- RetrievalEngine + vector_store → app.state.retrieval_engine (if database_url available)
- HitProcessor → app.state.hit_processor (with llm_gateway)
- TaskManager → app.state.task_manager (degraded mode, InMemoryTaskStore)
- KBSettingsStore → app.state.kb_settings_store (singleton)

Each component wrapped in try/except — failures logged but don't block startup.
Follows same pattern as episodic memory initialization.
2026-06-25 20:02:01 +08:00
chiguyong 1f691ca178 feat(frontend): U9 — KB management extension with segment preview, status display, settings
New: SegmentPreview.vue, KBSettings.vue
Extended: DocumentUpload.vue (status badges, retry, preview), SearchTest.vue (3 modes), SourceConfig.vue (ACL), KnowledgeBaseView.vue (settings + task history tabs)
API+Store: kb.ts new types/methods, knowledge.ts new state/actions

typecheck: passed
2026-06-25 13:14:58 +08:00
chiguyong e3ae2f3a56 feat(rag_platform): U8 — TaskIQ async task integration
Add tasks.py: TaskManager with vectorize/batch_index tasks, per-user concurrency limits, degraded mode (sync execution without broker), WorkerSweeper for timeout detection, error message sanitization
Add taskiq>=0.11 and taskiq-redis>=0.5 to pyproject.toml
Task parameter schema validation (VectorizeTaskParams, BatchIndexTaskParams)

Tests: 41 new tests, 289 total passing
2026-06-25 12:58:51 +08:00
chiguyong d026a91f43 feat(rag_platform): U6 — hit processing mode + KB settings
Add hit_processing.py: HitProcessor with model_opt (LLM-generated) and direct (concatenated chunks) modes, with in-process cache
Add settings.py: KBSettings/KBSettingsUpdate models, KBSettingsStore with async CRUD
Add KB settings endpoints to kb_management.py: GET/PUT /kb-management/kbs/{kb_id}/settings with owner-only modification

Tests: 43 new tests (25 hit_processing + 18 settings), 293 total passing
2026-06-25 12:44:47 +08:00
chiguyong 5c562dbff3 feat(rag_platform): U5 — rerank + question generation + termbase
Add rerank.py: Reranker with Cohere/BGE provider support, data export risk annotation, graceful degradation
Add question_gen.py: LLM-based question generation following ContextualChunker pattern, with caching
Add termbase.py: jieba custom dictionary management, add/remove/load terms

Tests: 58 new tests (14 rerank + 19 question_gen + 25 termbase), 205 total passing
2026-06-25 12:31:43 +08:00
chiguyong fb9f16d6e5 feat(rag_platform): U4 — dual-index retrieval (pgvector semantic + PG fulltext jieba)
Add fulltext.py: jieba tokenization + tsvector write/query
Add retrieval.py: RetrievalEngine with embedding/keywords/blend modes
Update models.py: add RetrievalRequest model
Tests: 35 new tests, 147 total passing
2026-06-25 12:20:48 +08:00
chiguyong 3f9588e673 feat(rag_platform): U3+U7 — rewrite upload endpoint with sanitization + pipeline
Rewrite upload_document() to use rag_platform sanitize + DocumentProcessor:
- File type whitelist validation (8 allowed types, reject .exe/.sh)
- File size limit (50MB) + zip bomb detection for ZIP-based formats
- DocumentProcessor.parse() (with content sanitization) + segment()
- Return chunks preview, status="segmenting" (pending vectorization)

Add POST /kb-management/documents/preview endpoint:
- Pre-upload preview with adjustable chunk_size/chunk_overlap
- Same security validation as upload, no document record created

Add POST /kb-management/documents/{id}/vectorize placeholder:
- Returns 503 — full async vectorization deferred to U8 (TaskIQ)

Test: update test_upload_document assertion (status "indexed" → "segmenting")
2026-06-25 12:06:16 +08:00
chiguyong b55c896794 feat(rag_platform): U3+U7 — document processing pipeline + upload security
U3: Document processing pipeline (document_processor.py)
- DocumentProcessor class wrapping parse → segment → vectorize
- parse() uses memory/document_loader.py for multi-format extraction
- segment() uses LlamaIndex SentenceSplitter
- preview() returns chunks for read-only preview (no vectorization)
- vectorize() embeds chunks and stores in pgvector (all-or-nothing)
- process() orchestrates full pipeline with status transitions:
  pending → parsing → segmenting → vectorizing → indexed | failed

U7: Upload security & content sanitization (sanitize.py)
- ALLOWED_FILE_TYPES whitelist (pdf/docx/xlsx/pptx/txt/md/csv/html)
- MAX_FILE_SIZE 50MB limit
- validate_file_type() / validate_file_size() guards
- check_zip_bomb() for ZIP-based formats (ratio > 100:1 or > 500MB)
- check_image_bomb() for pixel count > 100MP (PNG/JPEG/GIF header parsing)
- is_safe_ip() SSRF protection (loopback/RFC1918/link-local/ULA denied)
- sanitize_markdown() removes dangerous HTML tags (script/iframe/object/embed)
- sanitize_content() main entry point for text format sanitization
- parse_xml_safe() XXE protection (forbid_dtd/forbid_entities/forbid_external)

Preview API (preview.py)
- PreviewChunk / PreviewResult Pydantic models
- generate_preview() returns read-only segmentation preview

Tests: 112 tests passing (45 new + 67 existing)
- test_sanitize.py: file type/size, markdown sanitization, SSRF, zip/image bomb
- test_document_processor.py: parse/segment, preview, vectorize, failure status
2026-06-25 11:21:42 +08:00
chiguyong c1a21f57a1 feat(rag_platform): U2 — KB persistence + per-KB ACL
Add PostgreSQL-backed KB store replacing in-memory KnowledgeSourceStore:
- models.py: ORM models (KBModel, DocumentModel, KBAclModel) using
  SQLAlchemy 2 DeclarativeBase + Mapped style
- store.py: KBStore with async CRUD for KBs and documents,
  create_kb creates owner ACL in same transaction
- acl.py: filter_kb_by_user_acl(), grant_access(), revoke_access(),
  list_acl() — follows filter_kb_sources_by_department pattern

Schema: rag_platform_kbs, rag_platform_documents, rag_platform_kb_acl
with FK CASCADE on kb_id. UniqueConstraint on (kb_id, user_id).

Tests: 23 unit tests covering KB CRUD, document operations, ACL
filtering, grant/revoke. All 37 rag_platform tests pass.
2026-06-25 11:01:04 +08:00
chiguyong 27d0184392 feat(rag_platform): U1 — RAG platform skeleton + LlamaIndex integration
Create src/agentkit/rag_platform/ module with:
- models.py: Pydantic domain models (KB, Document, Chunk, QueryResult)
- indexing.py: PGVectorStore wrapper with explicit table name
  (rag_platform_kb_chunks) for schema isolation from episodic_memory
- pipeline.py: RAGPipeline wrapping LlamaIndex IngestionPipeline
  (SentenceSplitter + embedding + vector store)

Add dependencies: llama-index-core, llama-index-vector-stores-postgres,
llama-index-embeddings-openai, pgvector, jieba.

Tests: 14 unit tests covering models, indexing (URL conversion, table
name isolation, embed_dim), and pipeline (ingest, query, chunk params).
2026-06-25 10:49:35 +08:00
chiguyong 22c89763e2 docs: add long-horizon reliability fixes learning + scrub CONCEPTS.md
- New solution doc: logic-errors/long-horizon-reliability-code-review-fixes.md
  Documents 13 code-review fixes (2 P0, 5 P1, 6 P2) across U1-U7
  long-horizon reliability features (disclosure_level default, resume
  plan_id mismatch, middleware dataclass compat, state offload readback,
  checkpoint dedup, dynamic phase persistence, debate count restore,
  loop detection reset, concurrent resume lock, FAILED phase handling,
  checkpoint cleanup, offload type guard).

- CONCEPTS.md: add Expert Orchestration cluster (Disclosure Level,
  State Offloading, Pipeline Checkpoint, Debate Phase, Resume).
  Scrub Bitable entries to remove implementation specifics per
  vocabulary rules (API paths, library calls, SQL syntax, class names,
  enum values).
2026-06-25 02:40:22 +08:00
chiguyong 71eaf8dc7c docs: add bitable security/reliability patterns solution doc + CONCEPTS.md
Deploy to Production / deploy (push) Has been cancelled Details
- docs/solutions/architecture-patterns/bitable-companion-service-security-reliability-patterns.md
  Knowledge-track doc capturing 10 security/reliability patterns from the
  bitable companion service (SSRF prevention, SQL injection, IDOR, atomic
  task claiming, cache invalidation, composite cursor, batch ops, async
  I/O safety, OOM prevention, internal token auth)

- CONCEPTS.md
  Seeded with 3 core domain nouns: Bitable, Field Ownership, Recalc

- AGENTS.md
  Added discoverability tips for docs/solutions/ and CONCEPTS.md
2026-06-25 01:25:06 +08:00
chiguyong bbbf9cd40a feat(bitable): add bitable companion service with full P0-P2 fixes
Bitable is a multi-dimensional table companion service that runs alongside
the main AgentKit server. It provides structured data storage with formula
fields, views, and ingestion pipelines.

Major components:
- Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask
- SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema
- Formula engine: AST parser, DAG, Kahn topological sort, safe eval
- RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order
  processing, stale-threshold reaper for crash recovery
- REST API (/api/v1/bitable): tables, fields, records, views, files
- BitableTool: agent-facing tool with batch chunking (500/batch)
- CLI: agentkit bitable subcommands (create, list, import-excel, etc.)
- Frontend: Vue 3 + vxe-table grid with field management, views, filters
- Ingestion: Excel (openpyxl), database reflection, API collector

Security fixes (ce-code-review P0 + ce-debug P1):
- SQL injection prevention (field_id validation, parameterized queries)
- IDOR protection (_check_table_ownership on all table-level endpoints)
- SSRF prevention (URL scheme + private IP validation in parse_excel_url)
- OOM prevention (streaming file upload, batch delete, batch insert)
- Atomic recalc task claiming (FOR UPDATE SKIP LOCKED)
- Formula engine cache invalidation on field changes
- Composite cursor pagination for non-id sort orders
- Batch upsert (eliminates N+1 queries)
- Sync I/O offloaded to thread pool in async contexts
- Internal token auth (X-Internal-Token, hmac.compare_digest)
- PK unique index enforcement

Test coverage: 88 unit tests (95 skipped without Docker)
2026-06-25 01:09:59 +08:00
chiguyong 567cbc9c9b refactor: simplify code across U1-U7 (bug fix + efficiency + reuse + quality) 2026-06-24 22:35:52 +08:00
chiguyong 0847c0e086 fix(checkpoint): add TTL expiration for memory fallback mode
内存降级模式之前没有 TTL 过期机制,长期运行进程会导致内存泄漏。
现在 list_checkpoints 和 load_plan 在内存模式下会过滤/清除过期数据。

- list_checkpoints: 内存降级分支过滤过期 checkpoint
- load_plan: 内存降级分支检查 TTL 过期,过期则清除并返回 None
- 新增 _is_expired 方法检查 saved_at 是否超过 TTL
- _memory_plans 类型改为 tuple(plan_dict, timestamp) 以支持 TTL
- 新增 5 个 TTL 过期测试覆盖内存模式和 Redis 降级场景
2026-06-24 22:04:55 +08:00
chiguyong fa152e24ac feat(skills): add progressive skill loading with disclosure_level=0 (U5)
When disclosure_level=0, system prompt only injects skill name + description
(summary mode). SkillDetailTool is injected into the tool set, allowing the
LLM to load full instructions on-demand via skill_detail(query). This reduces
context window consumption when many skills are registered.
2026-06-24 21:49:00 +08:00
chiguyong dfd188b1a4 feat(orchestrator): add pipeline checkpoint and crash recovery (U7)
Add PipelineCheckpoint for stage-level crash recovery with Redis-first
+ memory fallback. TeamOrchestrator saves checkpoints after each phase
finalizes and supports resume(plan_id) to continue from the last
completed phase. New POST /api/v1/tasks/{id}/resume endpoint recreates
the team from saved plan and calls resume.
2026-06-24 21:04:18 +08:00
chiguyong 3dfda904d7 feat(core): add middleware pipeline architecture with onion model
U6: Unified middleware protocol (before/after) with MiddlewareChain
implementing onion model execution. Parallel integration (KTD1) —
middleware path controlled by presence of middleware_chain parameter,
existing ReActEngine path unchanged when None.

- New core/middleware.py: RequestContext, Middleware protocol,
  MiddlewareChain (onion model: before outer→inner, after inner→outer)
- 3 example middlewares: SummarizationMiddleware (U3 headroom compression),
  TokenUsageMiddleware, LoopDetectionMiddleware (request-level audit)
- ReActEngine.__init__ accepts middleware_chain parameter
- execute() branches: middleware path when chain present, existing path otherwise
- 22 tests covering ordering, error handling, state passing, backward compat
2026-06-24 20:52:15 +08:00
chiguyong ef84e3fd53 feat(experts): add SharedWorkspace state offloading for long-horizon runs
U4: ExpertTeam accepts redis_client, passes to SharedWorkspace. After phase
completion, full result is written to workspace and in-memory phase.result
is replaced with a 500-char summary + _ref_key. Dependency output reading
resolves offloaded content from workspace on demand, with graceful fallback
to summary on read failure.

Tests: 8 scenarios (offload creation, short content, dependency resolution,
workspace failure fallback, non-offloaded passthrough, redis_client wiring,
memory dict fallback, pipeline integration) — all pass.
2026-06-24 20:32:10 +08:00
chiguyong 122173ec2c feat(core): add headroom-based compression trigger
U3: ContextCompressor now accepts model_context_limit, headroom_threshold,
and min_tokens. should_compress() triggers when token ratio exceeds 0.8 of
model limit OR exceeds min_tokens (8000 fallback). ReActEngine._should_compress
delegates to compressor when available, checks is_available() first.

Tests: 6 scenarios (headroom trigger, min_tokens guard, small model,
unavailable compressor, delegation, fallback) — all pass.
2026-06-24 20:28:14 +08:00
chiguyong 717aad1303 feat(experts): add concurrency limit to TeamOrchestrator parallel phases
U2: Add asyncio.Semaphore to bound concurrent phase execution and debate
argument generation. Default limit=3, configurable via max_concurrent_phases.
Prevents LLM rate-limit spikes when many phases run in the same layer.

Tests: 5 scenarios (happy path, 5-phase edge case, serial mode, failure
release, debate integration) — all pass.
2026-06-24 20:23:30 +08:00
chiguyong 018b342d96 feat(react): add loop detection to prevent repeated identical tool calls
U1: Sliding window hash detection in ReAct loop. When the same tool is
called with identical arguments >= threshold times (default 2), injects
a correction message first, then raises LoopDetectedError if the LLM
doesn't change strategy. Covers both _execute_loop and execute_stream.
2026-06-24 20:12:35 +08:00
chiguyong a312e584ae Merge branch 'feat/expert-team-pm-collaboration' — PM 协同模式 + 代码审查全量修复
Deploy to Production / deploy (push) Waiting to run Details
# Conflicts:
#	src/agentkit/server/frontend/components.d.ts
2026-06-24 18:57:37 +08:00
chiguyong 20a4c55d5b feat(skills): SkillHarness 前置条件 + 风险守卫学习增强
- cli/skill.py: skill learn 子命令增强
- evolution/risk_guard_learner.py: 风险守卫学习改进
- memory/models.py: 记忆模型扩展
- skills/base.py + loader.py: SkillHarness 前置条件支持
- 对应测试更新
2026-06-24 18:56:51 +08:00
chiguyong 574db8458f fix(experts): PM 协同代码审查全量修复
P0: 跨阶段契约状态同步 — _notify_collaborators 更新接收方契约状态为 received
P0: 4 个 PM 事件加入 _VALID_TEAM_EVENT_TYPES 白名单

P1: 验收 fail-open 改标注降级原因
P1: 返工失败抛 RuntimeError 而非返回 dict
P1: 验收 prompt injection 防护 — 专家输出用 XML 标签包裹
P1: 契约字段校验 _EXPERT_NAME_RE
P1: bool("false") 修复 — 显式比较避免字符串真值陷阱
P1: _parse_risk_flags(None) 防御

P2: _notify_collaborators 移到验收通过后
P2: SharedWorkspace 写入移到验收通过后
P2: 验收贪婪正则修复
P2: 风险标记数量上限 MAX_RISK_FLAGS=10
P2: 返工 feedback 截断
P2: 前端会话隔离 — 切换会话时清除/恢复 collaborationState
P2: 前端契约状态更新 — collaboration_notice 时标记 delivered
P2: CLI 死代码标注 + 异常改 debug 日志
P2: 模块级 _RISK_FLAG_RE 预编译
2026-06-24 18:56:27 +08:00
chiguyong 6016c087fe feat(cli): U6 CLI 协同事件 Rich 渲染
- chat.py 新增 _render_collaboration_contracts 和 _render_pm_collaboration_event
- 4 种 PM 协同事件渲染:
  collaboration_contract_defined (cyan Panel)
  collaboration_notice (蓝→品红 文本)
  review_result (passed=green / failed=red Panel)
  risk_flagged (yellow Panel)
- plan_update 中提取 collaboration_contracts 并渲染
- _print_help 更新项目经理模式说明
- 优雅降级:字段缺失回退到 ?,空契约不输出,整体 try/except 不中断编排
- 新增 11 个测试(TestPMCollaborationRendering 9 + TestPrintHelpPMMode 2)
- ruff 通过,pytest 23 passed
2026-06-24 14:57:49 +08:00
chiguyong 34a4164430 feat(frontend): U5 前端协作关系图 + 验收/风险卡片
- types.ts 新增 ICollaborationContract/ICollaborationNotice/IReviewResult/IRiskFlag 等接口
- chat.ts 新增 collaborationState ref,处理 4 种协同事件
  (collaboration_contract_defined/collaboration_notice/review_result/risk_flagged)
  并在 plan_update 中提取 contracts,team_formed/dissolved 清理状态
- CollaborationGraphCard.vue SVG 协作关系图:
  圆形布局节点(专家首字),实线=契约,虚线动画=数据流向
  节点颜色编码验收状态(绿=通过,红=返工/失败),橙色!标记风险
- ReviewResultCard.vue 验收结果卡片(passed/failed + feedback)
- RiskFlagCard.vue 风险标记卡片(专家 + 风险描述)
- useMessageRenderer.ts 新增 3 个视图类型和渲染规格
- index.ts 导出 3 个新组件
- 遵循 U5 辩论可视化 BoardState 模式
- typecheck 通过
2026-06-24 14:42:00 +08:00
chiguyong 5487cca199 feat(experts): U4 专家风险标记 + risk_flagged 事件
- orchestrator 新增 _parse_risk_flags 静态方法,正则解析 [RISK: ...] 标记
- _execute_execution_phase 在协作通知后、验收前解析风险标记
- 风险标记通过 risk_flagged 事件广播,供前端/CLI 渲染
- 无风险标记时行为不变,向后兼容
- 新增 TestRiskFlagging 7 个测试(单/多/无/格式错误/事件发出/内容/兼容)
2026-06-24 14:17:58 +08:00
chiguyong 62fcbc0feb feat(experts): U3 Lead 验收环节 + 返工机制
- PlanPhase 添加 rework_count 和 review_feedback 字段
- 添加 _review_phase_output 方法,Lead 用 LLM 验收阶段输出
- _execute_execution_phase 重构为返工循环(MAX_REWORKS=2)
- 验收通过/返工/失败三种路径,发出 review_result 事件
- LLM 不可用时优雅降级直接通过
- 6 个新测试,全套 449 passed 无回归
2026-06-24 14:09:18 +08:00
chiguyong fef7ecea39 feat(skills): SkillHarness 激活前置条件 + 风险守卫学习
基于 SkillHarness 论文(arXiv:2606.20636)与 Agent Skills 综述
(arXiv:2602.12430)引入激活前置条件(preconditions)与来源标记
(provenance),并新增从失败轨迹学习风险守卫建议的能力。

变更内容:
- U1: SkillConfig 新增 v7 preconditions/provenance 字段(base.py)
- U2: build_skill_system_prompt 注入 preconditions 软检查段落
- U3: SkillLoader 三路径记录 provenance + entry_points 危险能力告警
- U4: 10 个业务 Skill YAML 补充 preconditions(2-4 条中文短句)
- U5: RiskGuardLearner 从失败轨迹学习风险守卫建议(人工审查,不自动应用)
- U6: CLI 命令 agentkit skill learn-risk-guards

关键决策:
- KTD1: preconditions 通过 system_prompt 注入(软检查),不做硬 LLM 调用
- KTD2: RiskGuardLearner 不自动应用,需人工审查(论文显示 75% 自动学习不安全)
- KTD3: provenance 为轻量字符串,不加 hash/签名(无合规需求)

测试:39 个新增单元测试全部通过,ruff 检查通过。
2026-06-24 13:56:37 +08:00
chiguyong c46cf06f6d feat(experts): U2 协作契约执行 — 专家可见 + 主动通知
- _execute_execution_phase 按协作契约读取相关专家输出(可见性)
- 添加 _notify_collaborators 方法,完成后通知相关专家(可协助)
- 发出 collaboration_notice 事件,契约状态更新为 delivered
- 7 个新测试,全套 443 passed 无回归
2026-06-24 13:54:38 +08:00