Commit Graph

164 Commits

Author SHA1 Message Date
chiguyong bbbf9cd40a feat(bitable): add bitable companion service with full P0-P2 fixes
Bitable is a multi-dimensional table companion service that runs alongside
the main AgentKit server. It provides structured data storage with formula
fields, views, and ingestion pipelines.

Major components:
- Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask
- SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema
- Formula engine: AST parser, DAG, Kahn topological sort, safe eval
- RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order
  processing, stale-threshold reaper for crash recovery
- REST API (/api/v1/bitable): tables, fields, records, views, files
- BitableTool: agent-facing tool with batch chunking (500/batch)
- CLI: agentkit bitable subcommands (create, list, import-excel, etc.)
- Frontend: Vue 3 + vxe-table grid with field management, views, filters
- Ingestion: Excel (openpyxl), database reflection, API collector

Security fixes (ce-code-review P0 + ce-debug P1):
- SQL injection prevention (field_id validation, parameterized queries)
- IDOR protection (_check_table_ownership on all table-level endpoints)
- SSRF prevention (URL scheme + private IP validation in parse_excel_url)
- OOM prevention (streaming file upload, batch delete, batch insert)
- Atomic recalc task claiming (FOR UPDATE SKIP LOCKED)
- Formula engine cache invalidation on field changes
- Composite cursor pagination for non-id sort orders
- Batch upsert (eliminates N+1 queries)
- Sync I/O offloaded to thread pool in async contexts
- Internal token auth (X-Internal-Token, hmac.compare_digest)
- PK unique index enforcement

Test coverage: 88 unit tests (95 skipped without Docker)
2026-06-25 01:09:59 +08:00
chiguyong 567cbc9c9b refactor: simplify code across U1-U7 (bug fix + efficiency + reuse + quality) 2026-06-24 22:35:52 +08:00
chiguyong 0847c0e086 fix(checkpoint): add TTL expiration for memory fallback mode
内存降级模式之前没有 TTL 过期机制,长期运行进程会导致内存泄漏。
现在 list_checkpoints 和 load_plan 在内存模式下会过滤/清除过期数据。

- list_checkpoints: 内存降级分支过滤过期 checkpoint
- load_plan: 内存降级分支检查 TTL 过期,过期则清除并返回 None
- 新增 _is_expired 方法检查 saved_at 是否超过 TTL
- _memory_plans 类型改为 tuple(plan_dict, timestamp) 以支持 TTL
- 新增 5 个 TTL 过期测试覆盖内存模式和 Redis 降级场景
2026-06-24 22:04:55 +08:00
chiguyong fa152e24ac feat(skills): add progressive skill loading with disclosure_level=0 (U5)
When disclosure_level=0, system prompt only injects skill name + description
(summary mode). SkillDetailTool is injected into the tool set, allowing the
LLM to load full instructions on-demand via skill_detail(query). This reduces
context window consumption when many skills are registered.
2026-06-24 21:49:00 +08:00
chiguyong dfd188b1a4 feat(orchestrator): add pipeline checkpoint and crash recovery (U7)
Add PipelineCheckpoint for stage-level crash recovery with Redis-first
+ memory fallback. TeamOrchestrator saves checkpoints after each phase
finalizes and supports resume(plan_id) to continue from the last
completed phase. New POST /api/v1/tasks/{id}/resume endpoint recreates
the team from saved plan and calls resume.
2026-06-24 21:04:18 +08:00
chiguyong 3dfda904d7 feat(core): add middleware pipeline architecture with onion model
U6: Unified middleware protocol (before/after) with MiddlewareChain
implementing onion model execution. Parallel integration (KTD1) —
middleware path controlled by presence of middleware_chain parameter,
existing ReActEngine path unchanged when None.

- New core/middleware.py: RequestContext, Middleware protocol,
  MiddlewareChain (onion model: before outer→inner, after inner→outer)
- 3 example middlewares: SummarizationMiddleware (U3 headroom compression),
  TokenUsageMiddleware, LoopDetectionMiddleware (request-level audit)
- ReActEngine.__init__ accepts middleware_chain parameter
- execute() branches: middleware path when chain present, existing path otherwise
- 22 tests covering ordering, error handling, state passing, backward compat
2026-06-24 20:52:15 +08:00
chiguyong ef84e3fd53 feat(experts): add SharedWorkspace state offloading for long-horizon runs
U4: ExpertTeam accepts redis_client, passes to SharedWorkspace. After phase
completion, full result is written to workspace and in-memory phase.result
is replaced with a 500-char summary + _ref_key. Dependency output reading
resolves offloaded content from workspace on demand, with graceful fallback
to summary on read failure.

Tests: 8 scenarios (offload creation, short content, dependency resolution,
workspace failure fallback, non-offloaded passthrough, redis_client wiring,
memory dict fallback, pipeline integration) — all pass.
2026-06-24 20:32:10 +08:00
chiguyong 122173ec2c feat(core): add headroom-based compression trigger
U3: ContextCompressor now accepts model_context_limit, headroom_threshold,
and min_tokens. should_compress() triggers when token ratio exceeds 0.8 of
model limit OR exceeds min_tokens (8000 fallback). ReActEngine._should_compress
delegates to compressor when available, checks is_available() first.

Tests: 6 scenarios (headroom trigger, min_tokens guard, small model,
unavailable compressor, delegation, fallback) — all pass.
2026-06-24 20:28:14 +08:00
chiguyong 717aad1303 feat(experts): add concurrency limit to TeamOrchestrator parallel phases
U2: Add asyncio.Semaphore to bound concurrent phase execution and debate
argument generation. Default limit=3, configurable via max_concurrent_phases.
Prevents LLM rate-limit spikes when many phases run in the same layer.

Tests: 5 scenarios (happy path, 5-phase edge case, serial mode, failure
release, debate integration) — all pass.
2026-06-24 20:23:30 +08:00
chiguyong 018b342d96 feat(react): add loop detection to prevent repeated identical tool calls
U1: Sliding window hash detection in ReAct loop. When the same tool is
called with identical arguments >= threshold times (default 2), injects
a correction message first, then raises LoopDetectedError if the LLM
doesn't change strategy. Covers both _execute_loop and execute_stream.
2026-06-24 20:12:35 +08:00
chiguyong a312e584ae Merge branch 'feat/expert-team-pm-collaboration' — PM 协同模式 + 代码审查全量修复
Deploy to Production / deploy (push) Waiting to run Details
# Conflicts:
#	src/agentkit/server/frontend/components.d.ts
2026-06-24 18:57:37 +08:00
chiguyong 20a4c55d5b feat(skills): SkillHarness 前置条件 + 风险守卫学习增强
- cli/skill.py: skill learn 子命令增强
- evolution/risk_guard_learner.py: 风险守卫学习改进
- memory/models.py: 记忆模型扩展
- skills/base.py + loader.py: SkillHarness 前置条件支持
- 对应测试更新
2026-06-24 18:56:51 +08:00
chiguyong 574db8458f fix(experts): PM 协同代码审查全量修复
P0: 跨阶段契约状态同步 — _notify_collaborators 更新接收方契约状态为 received
P0: 4 个 PM 事件加入 _VALID_TEAM_EVENT_TYPES 白名单

P1: 验收 fail-open 改标注降级原因
P1: 返工失败抛 RuntimeError 而非返回 dict
P1: 验收 prompt injection 防护 — 专家输出用 XML 标签包裹
P1: 契约字段校验 _EXPERT_NAME_RE
P1: bool("false") 修复 — 显式比较避免字符串真值陷阱
P1: _parse_risk_flags(None) 防御

P2: _notify_collaborators 移到验收通过后
P2: SharedWorkspace 写入移到验收通过后
P2: 验收贪婪正则修复
P2: 风险标记数量上限 MAX_RISK_FLAGS=10
P2: 返工 feedback 截断
P2: 前端会话隔离 — 切换会话时清除/恢复 collaborationState
P2: 前端契约状态更新 — collaboration_notice 时标记 delivered
P2: CLI 死代码标注 + 异常改 debug 日志
P2: 模块级 _RISK_FLAG_RE 预编译
2026-06-24 18:56:27 +08:00
chiguyong 6016c087fe feat(cli): U6 CLI 协同事件 Rich 渲染
- chat.py 新增 _render_collaboration_contracts 和 _render_pm_collaboration_event
- 4 种 PM 协同事件渲染:
  collaboration_contract_defined (cyan Panel)
  collaboration_notice (蓝→品红 文本)
  review_result (passed=green / failed=red Panel)
  risk_flagged (yellow Panel)
- plan_update 中提取 collaboration_contracts 并渲染
- _print_help 更新项目经理模式说明
- 优雅降级:字段缺失回退到 ?,空契约不输出,整体 try/except 不中断编排
- 新增 11 个测试(TestPMCollaborationRendering 9 + TestPrintHelpPMMode 2)
- ruff 通过,pytest 23 passed
2026-06-24 14:57:49 +08:00
chiguyong 5487cca199 feat(experts): U4 专家风险标记 + risk_flagged 事件
- orchestrator 新增 _parse_risk_flags 静态方法,正则解析 [RISK: ...] 标记
- _execute_execution_phase 在协作通知后、验收前解析风险标记
- 风险标记通过 risk_flagged 事件广播,供前端/CLI 渲染
- 无风险标记时行为不变,向后兼容
- 新增 TestRiskFlagging 7 个测试(单/多/无/格式错误/事件发出/内容/兼容)
2026-06-24 14:17:58 +08:00
chiguyong 62fcbc0feb feat(experts): U3 Lead 验收环节 + 返工机制
- PlanPhase 添加 rework_count 和 review_feedback 字段
- 添加 _review_phase_output 方法,Lead 用 LLM 验收阶段输出
- _execute_execution_phase 重构为返工循环(MAX_REWORKS=2)
- 验收通过/返工/失败三种路径,发出 review_result 事件
- LLM 不可用时优雅降级直接通过
- 6 个新测试,全套 449 passed 无回归
2026-06-24 14:09:18 +08:00
chiguyong fef7ecea39 feat(skills): SkillHarness 激活前置条件 + 风险守卫学习
基于 SkillHarness 论文(arXiv:2606.20636)与 Agent Skills 综述
(arXiv:2602.12430)引入激活前置条件(preconditions)与来源标记
(provenance),并新增从失败轨迹学习风险守卫建议的能力。

变更内容:
- U1: SkillConfig 新增 v7 preconditions/provenance 字段(base.py)
- U2: build_skill_system_prompt 注入 preconditions 软检查段落
- U3: SkillLoader 三路径记录 provenance + entry_points 危险能力告警
- U4: 10 个业务 Skill YAML 补充 preconditions(2-4 条中文短句)
- U5: RiskGuardLearner 从失败轨迹学习风险守卫建议(人工审查,不自动应用)
- U6: CLI 命令 agentkit skill learn-risk-guards

关键决策:
- KTD1: preconditions 通过 system_prompt 注入(软检查),不做硬 LLM 调用
- KTD2: RiskGuardLearner 不自动应用,需人工审查(论文显示 75% 自动学习不安全)
- KTD3: provenance 为轻量字符串,不加 hash/签名(无合规需求)

测试:39 个新增单元测试全部通过,ruff 检查通过。
2026-06-24 13:56:37 +08:00
chiguyong c46cf06f6d feat(experts): U2 协作契约执行 — 专家可见 + 主动通知
- _execute_execution_phase 按协作契约读取相关专家输出(可见性)
- 添加 _notify_collaborators 方法,完成后通知相关专家(可协助)
- 发出 collaboration_notice 事件,契约状态更新为 delivered
- 7 个新测试,全套 443 passed 无回归
2026-06-24 13:54:38 +08:00
chiguyong f219c5f016 feat(experts): U1 协作契约数据模型 + Lead 生成契约
- PlanPhase 添加 collaboration_contracts 字段(CollaborationContract dataclass)
- 修改 _decompose_task prompt,要求 Lead 分解任务时定义协作契约
- 修改 _parse_phases 解析 LLM 返回的协作契约信息
- plan_update 事件自动包含协作契约(通过 to_dict 序列化)
- 71 + 9 = 80 个新测试,全套 436 passed 无回归
2026-06-24 13:44:50 +08:00
chiguyong b86100a0a1 feat(cli): U6 CLI 多 Agent 入口 + 辩论 Rich 渲染
- 新增 _execute_team_cli() 处理 @team 前缀,运行 ExpertTeam 流水线
- Rich 事件渲染:team_formed/plan_update/phase_*/debate_*/team_synthesis
- 干预循环使用 select.select() 非阻塞轮询 stdin(Unix-only,ponytail 标注)
- 支持 /debate 手动触发辩论、/stop 终止团队、纯文本作为上下文注入
- 扩展 _print_help() 增加 Multi-Agent 与 Interventions 说明
- 新增 12 个单元测试覆盖路由、帮助文档、函数返回值、干预基础设施
2026-06-24 13:03:57 +08:00
chiguyong c831e925b6 feat(experts): U4 用户干预通道 + 手动辩论触发
建立 @team 执行期间的用户干预通道,支持 /stop、/debate <topic>、
普通文本追加上下文。

ExpertTeam (src/agentkit/experts/team.py):
- 新增 _interventions: asyncio.Queue (maxsize=64) 干预队列
- add_user_intervention(msg): 广播 + 入队
- consume_user_interventions(): 排空并返回待处理干预
- broadcast_user_message 现在同时入队干预队列

TeamOrchestrator (src/agentkit/experts/orchestrator.py):
- 新增 _user_context: list[str] 累积普通文本干预
- 新增 _process_interventions(lead, plan) 在每层执行前调用:
  * /stop → 终止执行,广播 plan_update(stopped_by_user)
  * /debate <topic> → 动态插入 DEBATE phase(受 MAX_DEBATES 限制)
  * 普通文本 → 累积到 _user_context
- _synthesize_results 将 _user_context 追加到 synthesis prompt

WS 路由 (src/agentkit/server/routes/chat.py):
- 模块级 _active_teams dict 跟踪每个 session 的活跃团队
- _execute_team_collab 执行前注册、finally 注销
- WS 消息循环:若 session 有活跃团队,message 路由为干预而非新任务
- 新增 team_intervention_ack 确认消息

测试:tests/unit/experts/test_team_intervention.py(20 测试),
覆盖队列基础、/stop、/debate、普通文本、混合消息、synthesis 影响。
同步更新 test_orchestrator_debate.py 的干预通道兼容性测试
(U4 已实现 consume_user_interventions)。

全部 418 experts 测试 + 325 server 测试通过。
2026-06-24 12:17:09 +08:00
chiguyong 5b5bd44ac4 test(calendar): 7 integration flow tests (lifecycle, recurrence, tags, types, invitations, authz, ICS) 2026-06-24 12:04:42 +08:00
chiguyong 3fdee65979 fix(calendar): code review fixes - 23 issues (2 critical, 15 major, 6 minor) 2026-06-24 11:29:23 +08:00
chiguyong ac26d417b3 feat(experts): U3 分歧检测 + 方案评审辩论自动触发
在 TeamOrchestrator 中新增 4 个方法实现自动辩论触发:

- _maybe_add_plan_review_debate: 任务分解后可选插入方案评审 DEBATE
  phase(phases > 2 且 LLM 判断需要时),所有执行阶段依赖它
- _detect_divergence: 每层执行后用 LLM 判断已完成阶段产出是否与其他
  阶段存在分歧,偏好 false negative
- _insert_debate_phase: 动态插入 DEBATE phase 并重 wiring 依赖
  (原依赖 trigger 的 phase 现在依赖 DEBATE)
- _check_divergence_and_insert_debates: 每层完成后的协调入口,
  受 MAX_DEBATES=3 上限保护

主循环从 `for layer in layers` 改为 `while True` + 重新计算
topological_sort(),以支持动态插入 DEBATE phase 后的依赖分层。

测试:tests/unit/experts/test_divergence_detection.py(21 测试),
覆盖 happy path / 边界 / 错误路径 / 集成分层。同步修复
test_team_orchestrator.py 的 mock gateway 以适配 U3 的额外 LLM 调用。

全部 398 测试通过。
2026-06-24 11:09:53 +08:00
chiguyong fbe08cb1e2 feat(experts): add debate phase executor to TeamOrchestrator (U2)
Implement _execute_debate_phase() with Lead-facilitated structured debate:
- Lead opens with divergence point + dependency context
- Experts argue in parallel per round (asyncio.gather)
- Lead summarizes each round, then adjudicates final verdict
- Verdict produces decision (adopt/compromise/shelve/inconclusive) + conclusion
- Conclusion written to SharedWorkspace for downstream phases

Escape hatches:
- debate_config.skip=true short-circuits with template text
- MAX_DEBATE_ROUNDS=4 hard cap on rounds
- User /stop intervention ends debate early (U4-compatible via getattr fallback)
- LLM unavailable falls back to template verdict, no crash

New events: debate_started, expert_argument, debate_round_summary,
debate_resolved (plus existing phase_completed for consistency).

Phase dispatcher (_execute_phase) routes by phase_type:
EXECUTION to _execute_execution_phase, DEBATE to _execute_debate_phase.

36 new tests in test_orchestrator_debate.py covering happy path (2 rounds,
2 experts), max_rounds=1 boundary, empty participants, user stop, skip
escape hatch, LLM unavailable, SharedWorkspace integration, event
broadcasting, intervention channel compatibility, and helper methods.
All 377 expert tests pass.

Also includes planning artifacts (brainstorm requirements + implementation
plan with 6 units U1-U6).
2026-06-24 10:54:51 +08:00
chiguyong e539122314 feat(experts): add PhaseType enum and debate_config to PlanPhase
U1: Data model foundation for structured debate collaboration.
- Add PhaseType enum (EXECUTION | DEBATE)
- Add phase_type and debate_config fields to PlanPhase
- Update to_dict/from_dict for serialization with backward compatibility
- Add tests for PhaseType, debate phase creation, serialization, and
  mixed EXECUTION+DEBATE topological sort
2026-06-24 10:42:11 +08:00
chiguyong 8d4145ddf9 feat(calendar): U7 Outlook sync via Microsoft Graph API
OutlookSyncProvider implementing AbstractSyncProvider for
bidirectional Outlook Calendar sync. Uses Graph API delta query
for incremental pull, auto-refreshes OAuth tokens on 401, and
converts Outlook recurrence patterns to RRULE. Same conflict
resolution as CalDAV (last-write-wins + WS notification).

- src/agentkit/calendar/sync/outlook_provider.py — OutlookSyncProvider
- tests/unit/calendar/test_sync_outlook.py — 8 tests
2026-06-23 23:49:24 +08:00
chiguyong 40d326cd3f feat(calendar): U6 CalDAV sync provider and SyncManager
AbstractSyncProvider interface with CalDAVSyncProvider implementation
for bidirectional Apple Calendar sync. SyncManager orchestrates all
providers (G8) — sync_all/sync_provider/resolve_conflict with
last-write-wins + WS notification on conflicts (G4). caldav library
calls wrapped in asyncio.to_thread for non-blocking operation.

- src/agentkit/calendar/sync/base.py — AbstractSyncProvider ABC
- src/agentkit/calendar/sync/caldav_provider.py — CalDAVSyncProvider
- src/agentkit/calendar/sync/manager.py — SyncManager (G8)
- pyproject.toml — added caldav>=1.3 dependency
- tests — 12 tests (9 CalDAV + 3 SyncManager)
2026-06-23 22:52:29 +08:00
chiguyong ffb184acc7 feat(calendar): U8 iCal/ICS import and export
ICSProvider parses .ics files (icalendar library) and creates local
CalendarEvents, skipping duplicate UIDs. Export builds an iCalendar
from events in a date range, deduplicating recurring event
occurrences back to a single VEVENT with RRULE. REST endpoints:
POST /import-ics (multipart upload), GET /export-ics (download).

- src/agentkit/calendar/sync/__init__.py — sync subpackage init
- src/agentkit/calendar/sync/ics_provider.py — ICSProvider (import/export)
- src/agentkit/calendar/db.py — added get_event_by_external_id() for dedup
- src/agentkit/server/routes/calendar.py — import-ics and export-ics endpoints
- pyproject.toml — added icalendar>=5.0 dependency
- tests/unit/calendar/test_ics_provider.py — 8 tests
2026-06-23 22:20:07 +08:00
chiguyong 26efbb51db feat(calendar): U5 reminder subsystem with scheduler and multi-channel dispatch
ReminderScheduler scans upcoming events every 60s, matches reminder
rules, and dispatches via client (WS), email (SMTP), or webhook
channels. Idempotent delivery (no duplicates on re-scan), retry with
exponential backoff (up to 3 attempts). Follows task_store.py
start/stop asyncio loop pattern (KTD-2 — conscious deviation from
APScheduler).

- src/agentkit/calendar/scheduler.py — ReminderScheduler (start/stop/scan_once)
- src/agentkit/calendar/reminders.py — ReminderDispatcher (strategy per channel)
- src/agentkit/calendar/db.py — added list_all_events_in_time_range() for scheduler
- tests/unit/calendar/test_scheduler.py — 8 tests
- tests/unit/calendar/test_reminders.py — 9 tests
2026-06-23 22:19:57 +08:00
chiguyong ddcedb57b2 feat(calendar): U4 post-processing extractor with keyword gating
Adds PostProcessingExtractor — a zero-LLM keyword gate (Chinese +
English time words) followed by LLM extraction for ambiguous cases.
Events created from extraction carry source="post_extract" so the UI
can style them distinctly (R33). LLM gateway is optional to keep the
constructor testable without a live provider.

- src/agentkit/calendar/extraction.py — PostProcessingExtractor
- tests/unit/calendar/test_extraction.py — 13 tests with MockLLMGateway
2026-06-23 21:56:20 +08:00
chiguyong 42fe7bcbc9 feat(calendar): U3 agent calendar tool for ReAct integration
Adds CalendarTool implementing the Tool ABC so the ReAct engine can
create, query, update, and delete events autonomously. Resolves
event_type_name and tag_names (look up or create), sets
source="agent" to distinguish agent-created events from manual ones.

- src/agentkit/tools/calendar_tool.py — CalendarTool(Tool)
- tests/unit/tools/test_calendar_tool.py — 13 tests covering all actions
2026-06-23 21:56:08 +08:00
chiguyong d36e45bbe7 feat(calendar): U2 backend service & REST API
Add CalendarService business logic layer and 14 REST endpoints:
- service.py: event CRUD with RRULE expansion, event types, tags,
  invitations, non-admin user search (G5/A3), type-level default
  reminder rule cloning
- routes/calendar.py: JWT-authenticated endpoints for events, types,
  tags, invitations, user search — with ownership checks
- 17 new tests (12 service + 5 routes), 33 total calendar tests passing
2026-06-23 21:43:39 +08:00
chiguyong 2ea799f6c4 feat(calendar): U1 backend data model, storage & RRULE expansion
Add calendar subsystem foundation mirroring documents/ pattern:
- models.py: 8 dataclasses (CalendarEvent with is_invited, EventType,
  Tag, EventTag, ReminderRule, ReminderDelivery, ExternalCalendarConfig,
  Invitation)
- db.py: aiosqlite bare-connection CRUD for all 8 tables with WAL mode
- recurrence.py: RRULE expansion via dateutil.rrule (RFC 5545)
- 16 unit tests covering DB CRUD and RRULE edge cases (DST, UNTIL, range)
- Add python-dateutil>=2.9 to pyproject.toml
2026-06-23 21:30:39 +08:00
chiguyong 3337589395 fix(review): document-processing code review fixes — validation, tests, formatting
Deploy to Production / deploy (push) Waiting to run Details
- SkillConfig._validate_v2: validate fallback_strategies against
  ReWOOEngine.VALID_STRATEGIES (lazy import, #20)
- test_skill_config: +4 tests for fallback_strategies validation
- test_document_loader: +8 xlsx edge case tests (empty workbook,
  malformed bytes, column mismatch, row/cell truncation, multi-sheet,
  file size limit, None cells, #16)
- test_execution_modes: fix ReWOOEngine patch path (lazy import ->
  patch at source) + FakeReWOOEngine.execute return .output attribute
- config_driven: ruff formatting (quotes, blank lines after imports)
- project_rules: remove stale "known failing test" note (now passes)
2026-06-23 20:21:19 +08:00
chiguyong a672dddc9a feat(skills): distinguish agent templates from business skills in UI
Deploy to Production / deploy (push) Waiting to run Details
The skills tab mixed generic execution-engine templates (react/direct/
rewoo/...) with business-domain skills (monitor/geo_optimizer/...) with
no visual or data distinction. Adds a derived `category` field to the
SkillInfo/SkillDetail API models and groups the frontend display.

Backend:
- SkillInfo/SkillDetail: add category (Literal), agent_type, execution_mode,
  task_mode fields
- _skill_to_info: derive category from explicit _ENGINE_TEMPLATE_NAMES set
  (not name suffix — trend_agent/deai_agent are business skills despite
  the _agent suffix)
- Simplify repetitive hasattr pattern with getattr

Frontend:
- ISkillInfo/ISkillDetail: add category + mode fields
- skills store: agentTemplates/businessSkills computed getters
  (businessSkills is defensive: anything not explicitly engine template)
- SkillsView: group into 执行引擎 / 业务技能 sections with counts
- SkillCard: type badge (引擎/技能), category-based icon, mode display,
  dark-mode-aware accent color

Tests:
- test_category_derived_from_name_suffix: verifies field exposure
- test_category_no_orphans: invariant — every skill has a valid category
- test_trend_agent_classified_as_business_skill: regression guard for
  the _agent suffix misclassification bug

Code review (ce-code-review): 2 P1 + 5 P2 findings applied.
2026-06-23 15:55:59 +08:00
chiguyong 47f3bfecfc feat(documents): add document processing capability (U1-U9)
Implements end-to-end document generation, template filling, and reading:

- DocumentService: unified business layer for create/query/download
- Renderers: Word (Markdown->docx), Excel (Markdown/JSON->xlsx),
  PDF (Markdown->pdf with CJK font), Template (Jinja2 sandbox .docx fill)
- DocumentLoader: read PDF/Word/Excel/Markdown/HTML/text -> Document
- DocumentTool: Agent tool with action=create|read
- REST API: /api/v1/documents (create, upload-template, list, download)
- Frontend: DocumentPanel, DocumentCard, documents Pinia store,
  chat store tool_result detection
- Security: path traversal guard (Path.resolve + relative_to),
  SSTI guard (SandboxedEnvironment), API key auth, 50MB upload limit
- Bug fixes: template path traversal (400 not 500), TemplateRenderer
  lazy-load (no external registration dependency)
- Tests: 168 tests (unit + security + E2E F1/F2/F3 + bug hunt)
- Docs: README section 17, requirements + plan + test-plan docs

Requirements R1-R28 verified, F1-F3 user flows pass.
2026-06-23 15:05:01 +08:00
chiguyong 4f261523c2 fix(review): U3 atomic file writes for YAML + .env + skill config
All config file writes now use the write-temp + fsync + os.replace
pattern (KTD-4) so a crash mid-write leaves the original file intact.

- Add src/agentkit/server/utils/atomic_write.py with write_text_atomic
- settings.py: _write_yaml_config and _write_env_var use atomic write
- skill_service.py: import_skill uses atomic write
- skill_service.py: update_skill_config uses atomic write + fcntl.flock
  around the read-modify-write cycle to serialize concurrent updates
- Add 11 unit tests covering happy path, crash safety, concurrency, errors
2026-06-22 17:03:27 +08:00
chiguyong 698a8fafba fix(review): U7 refresh token hash verification on whoami
The whoami route accepted rotated/old refresh tokens for cold-start
because it only checked session revocation status, not the token hash.
Now when token_type == "refresh", the route computes hash_token(token)
and compares it with the session's stored refresh_token_hash using
hmac.compare_digest (constant-time). Mismatch returns 401.

- Add SessionService.get_stored_refresh_hash(session_id) helper
- Add hash verification in whoami route (R9)
- Add TestWhoamiTokenHash with 5 integration tests
2026-06-22 16:55:20 +08:00
chiguyong 278d76b381 fix(review): U6 frontend field alignment + CLI top-users field fix 2026-06-22 16:28:44 +08:00
chiguyong 00c8386939 fix(review): U1 Redis quota enforcement — key construction + fail-closed + degradation recovery + async 2026-06-22 16:22:33 +08:00
chiguyong abe2a66436 fix(review): CLI field names, Pydantic validation, exception chaining 2026-06-22 15:24:31 +08:00
chiguyong 5e977539c7 test(admin): U10 — E2E + security isolation + quota enforcement tests
23 integration tests across 3 files:
- test_e2e_admin_flow: 5 end-to-end lifecycle tests (department, user,
  LLM config, skill management, usage dashboard)
- test_security_isolation: 7 department isolation tests + non-admin 403
  tests (cross-dept skill/KB access, multi-dept union, admin sees all,
  removed user loses access, disabled dept, API key client)
- test_quota_enforcement: 10 quota tests (token/cost/whitelist limits,
  multi-dept strictest-wins, real gateway integration, usage recording)

418 admin tests pass, no regressions.
2026-06-21 19:57:49 +08:00
chiguyong 2dd0091bda feat(admin): U8 — CLI admin command group
AdminHttpClient: sync HTTP client with JWT/API key auth, config file
support (~/.agentkit/admin_config.yaml), env var fallback.

35+ CLI commands across 7 groups: login, department (CRUD + bind/unbind
skill/KB + quotas), user (CRUD + reset-password + assign/remove dept),
llm (providers + api-key + fallbacks), skill (list/enable/disable/
import/reload), kb (documents CRUD + sync/rebuild), usage (summary/
timeseries/by-model/top-users/export).

All commands support --server-url, --token, --api-key, --json flags.
Rich table output by default, raw JSON with --json. Friendly error
handling for connection/auth/not-found/conflict errors.

64 new tests, 102 CLI tests pass, no regressions.
2026-06-21 18:56:14 +08:00
chiguyong 09feca3307 feat(admin): U7 — usage dashboard + quota enforcement
UsageRecord extended with user_id + department_id (backward compatible).
UsageStore Protocol extended: record() accepts user_id/department_id,
get_usage() accepts filters, new get_usage_by_user/department methods.
RedisUsageStore uses versioned keys (v2) for new records.

LLMGateway.chat()/chat_stream() accept user_id, department_ids, db_path.
Quota check before provider call: model whitelist + token limit + cost
limit (daily). Multi-department uses strictest-wins (any exceed → reject).
QuotaExceededError → 429 at route layer.

UsageService: summary, timeseries, by-model, top-users, export (CSV/JSON).
5 new admin endpoints under /admin/usage/*.

llm_gateway.py routes pass DepartmentContext + db_path to gateway,
catch QuotaExceededError → 429 (JSON for /chat, SSE error for /stream).

84 new tests. 441 admin+usage tests pass, no regressions.
2026-06-21 17:23:20 +08:00
chiguyong fd7f6816b8 feat(admin): U6 — Skill & KB management endpoints + department binding
SkillService: enable/disable (persisted in skill_states table, schema
v4), import from YAML (with path traversal + name validation), reload
from file, update config. GET /skills now filters disabled skills.

KbService: list/upload/delete documents with department_id binding.
Added department_id field to KnowledgeSource + UploadedDocument.
Department visibility: (bound to user depts) ∪ (global = None).

10 new admin endpoints: skill enable/disable/import/reload/update,
KB documents CRUD, source sync/rebuild. All guarded by _require_admin.

Implemented reload stub in skill_management.py (was no-op).

54 new tests (26 unit + 28 integration). Fixed 4 pre-existing lint
errors. 357 admin tests pass, no regressions.
2026-06-21 16:19:51 +08:00
chiguyong 980919fc95 feat(admin): U5 — LLM config admin endpoints + department quotas
QuotaService: set/get/list/delete quotas, check_quota (hard reject),
is_model_allowed. JSON-serialized limit_value, upsert with ON CONFLICT.

LlmConfigService: provider CRUD + set_api_key + fallback management.
fcntl.flock file lock prevents concurrent YAML writes. Reuses
settings.py helpers (_read_yaml_config, _write_yaml_config,
_write_env_var, _mask_api_key).

11 new admin endpoints: provider CRUD, api-key, fallback CRUD,
department quotas CRUD. All guarded by _require_admin.

93 new tests (30 quota unit + 32 llm-config unit + 31 integration).
2026-06-21 15:03:38 +08:00
chiguyong ad65f7a8d7 feat(admin): U1+U2+U4 — schema v3, department service, context filtering
U1: Bump _SCHEMA_VERSION to 3, add 5 department tables (departments,
user_departments, department_skill_bindings, department_kb_bindings,
department_quotas) + 5 ORM models + helpers.

U2: DepartmentService (12 async methods: CRUD + bind/unbind skill/KB +
count_users). Mount admin_router in app.py. 36 unit + 28 integration tests.

U4: DepartmentContext FastAPI dependency (per-route, admin bypasses
filtering). filter_skills_by_department / filter_kb_sources_by_department
helpers. Applied to GET /skills and GET /kb-management/* routes.
15 integration tests for department isolation.

Also includes brainstorm + plan docs. 108 new tests, all pass.
2026-06-21 15:03:27 +08:00
chiguyong 6dca9ba4f2 feat(admin): U3 — user CRUD + password reset + multi-department
Add create_user method to LocalAuthProvider (bcrypt hash + INSERT,
raises ValueError on duplicate username/email).

Add UserService with 9 async methods: create/list/get/update/delete
(soft)/reset_password/assign_department/remove_department/list_user_departments. reset_password revokes all sessions via SessionService.
delete_user is soft (is_active=0, row preserved).

Add 9 user endpoints to routes/admin.py: POST/GET/PATCH/DELETE users,
reset-password, assign/remove department, list departments. All
guarded by _require_admin.

Tests: 40 unit + 37 integration = 77 new tests. Full admin suite
170 tests pass, no regressions.
2026-06-21 13:45:42 +08:00
chiguyong 67c0d67262 fix(auth,chat): P0 security fixes + stop-generation button + doc sync
U1: whoami cold-start security — add is_active check (disabled users
now get 401, not 200) and replace create_token_pair with create_access_token
to avoid minting a discarded refresh token (token-amplification risk).

U2: list_active_by_provider now filters expired sessions (expires_at > now)
matching its docstring promise; previously only checked revoked = 0.

U3: Fix asyncio.run() crash in test_revoke_other_user_session_returns_404
(converted to async). Add U1/U2 verification tests (disabled-user whoami,
no-refresh-leak, expired-session filtering, provider filtering) and
strengthen admin route tests (404 boundary, non-admin 403 on /admin/sessions).

U4: Update CLAUDE.md/AGENTS.md Request Flow — CostAwareRouter 3-layer
diagram replaced with actual RequestPreprocessor architecture (@board/@team
prefix intercepts then @skill: prefix then trivial-input regex then default
REACT). ExecutionMode list expanded to all 7 values.

U5: Frontend stop-generation button — ChatInput.vue shows a stop button
when isGenerating is true; chat store gains stopGeneration() that sends
{type:"cancel"} over WebSocket (backend portal.py already handles cancel).

Tests: 120 auth tests pass (unit + integration). ruff clean. vue-tsc clean.
2026-06-21 11:36:58 +08:00