fischer
/
fischer-agentkit
1
0
Fork 0
Code Issues Pull Requests Actions 85 Packages Projects Releases Wiki Activity
311 Commits 1 Branch 0 Tags 22 MiB
Commit Graph

89 Commits

Author SHA1 Message Date
chiguyong a6e1bf5884 feat(bitable): 多维表格文件层 + 默认字段 + 表内字段操作 + ce-code-review 修复 (Stage 1)
Test / backend-test (pull_request) Has been cancelled Details
Test / frontend-unit (pull_request) Has been cancelled Details
Test / api-e2e (pull_request) Has been cancelled Details
Test / frontend-e2e (pull_request) Has been cancelled Details 
实现多维表格 UI 完整性 Stage 1（U1-U6），补齐飞书/twenty 对齐缺失的文件层、
默认字段与表内字段操作能力，并修复 ce-code-review 走查发现的 P0/P1 级问题。

后端（U1-U2）:
- 新增 BitableFile 实体（models/db/repository/service/routes），三级层级：文件→数据表→字段/记录
- Schema V2 迁移：bitable_files 表 + tables.file_id 列，幂等（IF NOT EXISTS），保留 V1 孤儿表
- 新建数据表自动创建 5 个默认字段（标题/状态/日期/创建人/创建时间）
- agent-owned 字段在 create_record 时自动填充（按 type+owner 匹配，传 actor_user_id）
- 7 个文件 REST 端点 + IDOR ownership 检查（404-before-403，internal token 旁路）

前端（U3-U5）:
- 文件列表页（FileCard 网格 + 新建/重命名/删除）+ 文件详情页（侧栏表格列表 + vxe-table 网格）
- Vue Router 嵌套路由 /bitable → /bitable/:fileId → /bitable/:fileId/:tableId
- 列头菜单（编辑/隐藏/删除字段）+ 末尾 + 列新增字段
- select/multiselect 字段自定义单元格编辑器 + Tag 展示
- Pinia store 扩展 file 状态与动作，深链直访回退 getFile，fileId 切换 watch

测试（U6）:
- 文件 CRUD（12 例）+ 默认字段（10 例）单元测试
- 3 个 E2E spec（视图加载、文件流、字段操作），后端不可用时优雅跳过

ce-code-review 修复（P0/P1）:
- P0 路由冲突：GET /files/{file_id} 遮蔽下载端点 → 下载改 /uploads/{filename}
- P0 IDOR：update/delete field/record/view 五端点补 ownership 检查
- P1 is_initialized property 缺失致二次初始化崩溃
- P1 直接 URL 导航失效（files 数组为空）→ selectFile 回退 getFile
- P1 fileId 切换不重载 → 增加 watch
- P1 轮询丢弃最终公式值（wasCalculating 守卫）+ 复用视图 filters
- P1 测试断言 200→201；test_db 无 URL 用例解除 postgres 标记得以执行
- P2 _check_table_ownership 403→404；输入长度校验；upload field-table 一致性校验
- P2 multiselect 浅比较 → 深比较；E2E bitable-view 补 waitForServer 守卫

验证：ruff check 通过；pytest 91 passed/116 skipped；vue-tsc --noEmit 通过。
 2026-06-29 04:07:45 +08:00
chiguyong c9ce15fa4b fix(code-review): 修复走查发现的 13 High + Medium 安全/可靠性问题 
代码修复（8 High + 9 Medium）：
- portal.py — C1 IDOR 文档 / C2 类型修复 / C3 WS 连接上限 16 / C4 ws_user_id 早初始化 / M silent swallow 日志化
- auth/middleware.py — C5 WS sid 补齐
- calendar_tool.py — C6 偏移量 ±43200 双向校验 + reminder_channels 类型/白名单校验
- sqlite_conversation_store.py — C7 DELETE 事务回滚
- chat.ts (Pinia) — C8 deleteConversation 清理 pending 缓存
- app.py — M except: pass → logger.debug(exc_info=True)
- Scene6Error.vue — M onUnmounted 清理 setTimeout
- DocumentsTab.vue — M Invalid Date 守卫
- ChatSidebar/RightPanel/TopNav.vue — M aria-label 无障碍标签
- SystemMonitorPanel.vue — M v-else 兜底 + active 边框色 + tablist 键盘导航
- CalendarDrawer.vue — M overflow-y: auto
- CalendarGrid.vue — M ResizeObserver 反馈循环防护
- SkillsTab.vue — M onMounted 始终 fetchSkills

文档修复（5 High + 6 Medium）：
- portal-platform-security-reliability-fixes.md — D2 测试路径 / D3 Root Cause+Impact 章节 / D4 severity: mixed / 标题中文化 / 12 处绝对路径转相对 / P2 #12 数字口径
- AGENTS.md — D5 路由表 22→28 / 专家模板 5→15 / LiteLLM U15 迁移 / 配置查找 fallback
- README.md — 8 处端口 8000→8001

新增测试：
- tests/unit/calendar/test_calendar_tool.py — ponytail 自检断言

验证：
- ruff check (5 文件) — All checks passed
- vue-tsc --noEmit — exit 0
- git stash baseline 验证 — portal 17 个 401 失败为预存在问题

已知限制（预存在）：
- 17 个 portal 测试 401 失败 — 需另起 ce-debug 调查
- README.md 7 处 CostAwareRouter 引用过时 — 文档同步另起任务
 2026-06-28 15:06:41 +08:00
chiguyong 43e9025c6d fix(calendar): 日历能力缺失修复 + UI 布局优化 + 会话404处理 
P0: calendar_tool reminder_rules 未传入 create_event，提醒功能完全失效。P1: chat.ts deleteConversation 未清理 pending + 404 递归保护。P2: app.py 系统提示重复段落 + gui_mode F821 + SystemMonitorPanel flex 布局。P3: portal send_json 快照 + WS connected 清除 is_local + 移除死代码。验证: ruff+pytest 98passed+typecheck 通过。
 2026-06-28 14:24:58 +08:00
chiguyong 31c65e01b8 fix(security): P0 安全加固 + 多实例部署一致性 (U1-U4 + U5c)
Deploy to Production / deploy (push) Has been cancelled Details 
U1: LLM gateway KB 缓存 fail-closed — 异常时默认禁用缓存防止 KB 数据泄漏
U2: MCP 危险工具黑名单过滤 — 6+1 端点覆盖，防止绕过 chat confirmation
U3: SecretsStore Redis 迁移 — 多 worker 共享凭证，内存降级保留开发模式
U4: channels webhook Redis 状态 — ZSET 滑动窗口限流 + nonce dedup + backpressure
U5c: ce-code-review 修复批次:
  - P0: 统一 MCP 黑名单与 publisher.py 一致 (terminal_execute -> terminal, +file_read)
  - P1: ZSET 限流 member 加 uuid 后缀避免同时间戳碰撞
  - P1: SecretsStore redis 参数 Any -> aioredis.Redis | None (AGENTS.md 合规)
  - P1: Redis client 添加 socket_timeout 防止单点故障请求挂死

测试: 171 scoped tests pass, ruff clean
 2026-06-26 04:05:33 +08:00
chiguyong 53faa60472 fix(review): ce-code-review P1+P2 修复 — 安全/可靠性/性能 
P1 安全与可靠性（4 项）：
- wecom: verify_signature 增加时间戳新鲜度校验（5 分钟窗口防重放）
- cache: should_cache 在 per_user_namespace 开启时拒绝 user_id=None
  匿名请求，避免跨用户缓存泄漏（安全要求 a/e）
- channels: webhook receive_message 异常兜底，防止 500 触发平台重试风暴
- app: shutdown 调用 close_all_adapters + await _pending_webhook_tasks，
  防止 httpx 连接泄漏和丢失 IM 回复

P2 效率与可维护性（5 项）：
- feishu: _TOKEN_CACHE_TTL 300 → 6900（2h 减 5min 余量，避免 24x 过频刷新）
- channels: _pending_webhook_tasks 有界化（2x 并发上限时 429 拒绝）
- gateway: quota 检查每 period 单次 get_usage，复用 summary 检查 token+cost
- cache_key: generate_cache_key 合并为单次 SHA-256（消除 8-10 次冗余哈希）
- config: ProviderConfig.get_api_key 移除未用的 secrets_store 参数

P3 去重（1 项）：
- channels: _process_inbound_message DIRECT_CHAT 路径提取 _direct_chat 辅助函数

测试：
- test_wecom: 时间戳改用 int(time.time())，新增 test_expired_timestamp_rejected
- test_cache: should_cache 测试覆盖匿名拒绝 + namespace_off 兼容
- test_config_migration: get_api_key 测试适配新签名
- channels/config_migration/quota_enforcement 测试全部通过
 2026-06-26 01:40:31 +08:00
chiguyong 1ccaf56b9a refactor: ce-simplify-code 审查修复 — 去重 + 效率 + 死代码清理 
3 个审查代理（复用/质量/效率）发现 15 个问题，全部修复：

效率与安全（6 项）：
- MCPClient 缓存 MultiServerMCPClient 单例 + aclose()，修复连接/子进程泄漏
- _rate_limits 清理空 IP 条目，修复 X-Forwarded-For 欺骗下内存泄漏
- _seen_nonces 改用 OrderedDict，O(1) 摊销过期清理
- webhook 后台任务加 Semaphore(20) + 任务引用追踪，限制无界并发
- _build_adapter 用 asyncio.gather 并行解密 secrets
- 适配器实例缓存（_adapter_cache），token TTL 缓存跨请求命中

去重（4 项）：
- header_get 提取到 channels/base.py，4 个适配器统一 import
- _get_client/close() 移入 MessageAdapter 基类，子类继承
- URLVerificationChallenge 统一到 base.py，feishu/slack/wecom 共用
- Transport ABC 添加 endpoint_url 属性，from_transport 不再访问私有字段

死代码与类型安全（5 项）：
- detect_cache_hit 死方法替换为 record_cache_result 公开 API
- execution_mode.value == "direct_chat" 改用枚举比较
- 删除 yielded_any 死变量、重复 from fastapi import Request、
  多余 getattr 防御

453 tests passed, ruff clean（预存 F841 非本次引入）
 2026-06-25 23:54:14 +08:00
chiguyong 13c516a54f feat(mcp): U14 — Skill/Team MCP publish with admin auth + dangerous-tool opt-in 2026-06-25 21:10:06 +08:00
chiguyong 8998f94c42 feat(channels): U12 — DingTalk/WeCom/Slack adapters + multi-channel webhook dispatch 2026-06-25 20:45:43 +08:00
chiguyong 4b58e8f661 feat(channels): U11 — Feishu IM adapter end-to-end (webhook + signature + AES-CBC decrypt + chat integration) 2026-06-25 20:24:21 +08:00
chiguyong 5572387c01 feat(channels): U10 — message adapter ABC + AES-256-GCM secrets store + channel CRUD routes 2026-06-25 20:13:37 +08:00
chiguyong d026a91f43 feat(rag_platform): U6 — hit processing mode + KB settings 
Add hit_processing.py: HitProcessor with model_opt (LLM-generated) and direct (concatenated chunks) modes, with in-process cache
Add settings.py: KBSettings/KBSettingsUpdate models, KBSettingsStore with async CRUD
Add KB settings endpoints to kb_management.py: GET/PUT /kb-management/kbs/{kb_id}/settings with owner-only modification

Tests: 43 new tests (25 hit_processing + 18 settings), 293 total passing
 2026-06-25 12:44:47 +08:00
chiguyong 3f9588e673 feat(rag_platform): U3+U7 — rewrite upload endpoint with sanitization + pipeline 
Rewrite upload_document() to use rag_platform sanitize + DocumentProcessor:
- File type whitelist validation (8 allowed types, reject .exe/.sh)
- File size limit (50MB) + zip bomb detection for ZIP-based formats
- DocumentProcessor.parse() (with content sanitization) + segment()
- Return chunks preview, status="segmenting" (pending vectorization)

Add POST /kb-management/documents/preview endpoint:
- Pre-upload preview with adjustable chunk_size/chunk_overlap
- Same security validation as upload, no document record created

Add POST /kb-management/documents/{id}/vectorize placeholder:
- Returns 503 — full async vectorization deferred to U8 (TaskIQ)

Test: update test_upload_document assertion (status "indexed" → "segmenting")
 2026-06-25 12:06:16 +08:00
chiguyong bbbf9cd40a feat(bitable): add bitable companion service with full P0-P2 fixes 
Bitable is a multi-dimensional table companion service that runs alongside
the main AgentKit server. It provides structured data storage with formula
fields, views, and ingestion pipelines.

Major components:
- Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask
- SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema
- Formula engine: AST parser, DAG, Kahn topological sort, safe eval
- RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order
  processing, stale-threshold reaper for crash recovery
- REST API (/api/v1/bitable): tables, fields, records, views, files
- BitableTool: agent-facing tool with batch chunking (500/batch)
- CLI: agentkit bitable subcommands (create, list, import-excel, etc.)
- Frontend: Vue 3 + vxe-table grid with field management, views, filters
- Ingestion: Excel (openpyxl), database reflection, API collector

Security fixes (ce-code-review P0 + ce-debug P1):
- SQL injection prevention (field_id validation, parameterized queries)
- IDOR protection (_check_table_ownership on all table-level endpoints)
- SSRF prevention (URL scheme + private IP validation in parse_excel_url)
- OOM prevention (streaming file upload, batch delete, batch insert)
- Atomic recalc task claiming (FOR UPDATE SKIP LOCKED)
- Formula engine cache invalidation on field changes
- Composite cursor pagination for non-id sort orders
- Batch upsert (eliminates N+1 queries)
- Sync I/O offloaded to thread pool in async contexts
- Internal token auth (X-Internal-Token, hmac.compare_digest)
- PK unique index enforcement

Test coverage: 88 unit tests (95 skipped without Docker)
 2026-06-25 01:09:59 +08:00
chiguyong dfd188b1a4 feat(orchestrator): add pipeline checkpoint and crash recovery (U7) 
Add PipelineCheckpoint for stage-level crash recovery with Redis-first
+ memory fallback. TeamOrchestrator saves checkpoints after each phase
finalizes and supports resume(plan_id) to continue from the last
completed phase. New POST /api/v1/tasks/{id}/resume endpoint recreates
the team from saved plan and calls resume.
 2026-06-24 21:04:18 +08:00
chiguyong ef84e3fd53 feat(experts): add SharedWorkspace state offloading for long-horizon runs 
U4: ExpertTeam accepts redis_client, passes to SharedWorkspace. After phase
completion, full result is written to workspace and in-memory phase.result
is replaced with a 500-char summary + _ref_key. Dependency output reading
resolves offloaded content from workspace on demand, with graceful fallback
to summary on read failure.

Tests: 8 scenarios (offload creation, short content, dependency resolution,
workspace failure fallback, non-offloaded passthrough, redis_client wiring,
memory dict fallback, pipeline integration) — all pass.
 2026-06-24 20:32:10 +08:00
chiguyong a312e584ae Merge branch 'feat/expert-team-pm-collaboration' — PM 协同模式 + 代码审查全量修复
Deploy to Production / deploy (push) Waiting to run Details 
# Conflicts:
#	src/agentkit/server/frontend/components.d.ts
 2026-06-24 18:57:37 +08:00
chiguyong 574db8458f fix(experts): PM 协同代码审查全量修复 
P0: 跨阶段契约状态同步 — _notify_collaborators 更新接收方契约状态为 received
P0: 4 个 PM 事件加入 _VALID_TEAM_EVENT_TYPES 白名单

P1: 验收 fail-open 改标注降级原因
P1: 返工失败抛 RuntimeError 而非返回 dict
P1: 验收 prompt injection 防护 — 专家输出用 XML 标签包裹
P1: 契约字段校验 _EXPERT_NAME_RE
P1: bool("false") 修复 — 显式比较避免字符串真值陷阱
P1: _parse_risk_flags(None) 防御

P2: _notify_collaborators 移到验收通过后
P2: SharedWorkspace 写入移到验收通过后
P2: 验收贪婪正则修复
P2: 风险标记数量上限 MAX_RISK_FLAGS=10
P2: 返工 feedback 截断
P2: 前端会话隔离 — 切换会话时清除/恢复 collaborationState
P2: 前端契约状态更新 — collaboration_notice 时标记 delivered
P2: CLI 死代码标注 + 异常改 debug 日志
P2: 模块级 _RISK_FLAG_RE 预编译
 2026-06-24 18:56:27 +08:00
chiguyong c831e925b6 feat(experts): U4 用户干预通道 + 手动辩论触发 
建立 @team 执行期间的用户干预通道，支持 /stop、/debate <topic>、
普通文本追加上下文。

ExpertTeam (src/agentkit/experts/team.py):
- 新增 _interventions: asyncio.Queue (maxsize=64) 干预队列
- add_user_intervention(msg): 广播 + 入队
- consume_user_interventions(): 排空并返回待处理干预
- broadcast_user_message 现在同时入队干预队列

TeamOrchestrator (src/agentkit/experts/orchestrator.py):
- 新增 _user_context: list[str] 累积普通文本干预
- 新增 _process_interventions(lead, plan) 在每层执行前调用：
  * /stop → 终止执行，广播 plan_update(stopped_by_user)
  * /debate <topic> → 动态插入 DEBATE phase（受 MAX_DEBATES 限制）
  * 普通文本 → 累积到 _user_context
- _synthesize_results 将 _user_context 追加到 synthesis prompt

WS 路由 (src/agentkit/server/routes/chat.py):
- 模块级 _active_teams dict 跟踪每个 session 的活跃团队
- _execute_team_collab 执行前注册、finally 注销
- WS 消息循环：若 session 有活跃团队，message 路由为干预而非新任务
- 新增 team_intervention_ack 确认消息

测试：tests/unit/experts/test_team_intervention.py（20 测试），
覆盖队列基础、/stop、/debate、普通文本、混合消息、synthesis 影响。
同步更新 test_orchestrator_debate.py 的干预通道兼容性测试
（U4 已实现 consume_user_interventions）。

全部 418 experts 测试 + 325 server 测试通过。
 2026-06-24 12:17:09 +08:00
chiguyong d4bc79e409 test(calendar): wire calendar router into app.py + test plan 
- Register calendar router in create_app() so /api/v1/calendar/* is reachable
- Initialize CalendarService + ReminderScheduler in lifespan
- Register CalendarTool into tool registry for ReAct integration
- Lazy-import ICSProvider in routes to break circular import
- Add test plan document (5 layers: unit/integration/e2e)
 2026-06-24 11:51:31 +08:00
chiguyong 3fdee65979 fix(calendar): code review fixes - 23 issues (2 critical, 15 major, 6 minor) 2026-06-24 11:29:23 +08:00
chiguyong ffb184acc7 feat(calendar): U8 iCal/ICS import and export 
ICSProvider parses .ics files (icalendar library) and creates local
CalendarEvents, skipping duplicate UIDs. Export builds an iCalendar
from events in a date range, deduplicating recurring event
occurrences back to a single VEVENT with RRULE. REST endpoints:
POST /import-ics (multipart upload), GET /export-ics (download).

- src/agentkit/calendar/sync/__init__.py — sync subpackage init
- src/agentkit/calendar/sync/ics_provider.py — ICSProvider (import/export)
- src/agentkit/calendar/db.py — added get_event_by_external_id() for dedup
- src/agentkit/server/routes/calendar.py — import-ics and export-ics endpoints
- pyproject.toml — added icalendar>=5.0 dependency
- tests/unit/calendar/test_ics_provider.py — 8 tests
 2026-06-23 22:20:07 +08:00
chiguyong d36e45bbe7 feat(calendar): U2 backend service & REST API 
Add CalendarService business logic layer and 14 REST endpoints:
- service.py: event CRUD with RRULE expansion, event types, tags,
  invitations, non-admin user search (G5/A3), type-level default
  reminder rule cloning
- routes/calendar.py: JWT-authenticated endpoints for events, types,
  tags, invitations, user search — with ownership checks
- 17 new tests (12 service + 5 routes), 33 total calendar tests passing
 2026-06-23 21:43:39 +08:00
chiguyong a672dddc9a feat(skills): distinguish agent templates from business skills in UI
Deploy to Production / deploy (push) Waiting to run Details 
The skills tab mixed generic execution-engine templates (react/direct/
rewoo/...) with business-domain skills (monitor/geo_optimizer/...) with
no visual or data distinction. Adds a derived `category` field to the
SkillInfo/SkillDetail API models and groups the frontend display.

Backend:
- SkillInfo/SkillDetail: add category (Literal), agent_type, execution_mode,
  task_mode fields
- _skill_to_info: derive category from explicit _ENGINE_TEMPLATE_NAMES set
  (not name suffix — trend_agent/deai_agent are business skills despite
  the _agent suffix)
- Simplify repetitive hasattr pattern with getattr

Frontend:
- ISkillInfo/ISkillDetail: add category + mode fields
- skills store: agentTemplates/businessSkills computed getters
  (businessSkills is defensive: anything not explicitly engine template)
- SkillsView: group into 执行引擎 / 业务技能 sections with counts
- SkillCard: type badge (引擎/技能), category-based icon, mode display,
  dark-mode-aware accent color

Tests:
- test_category_derived_from_name_suffix: verifies field exposure
- test_category_no_orphans: invariant — every skill has a valid category
- test_trend_agent_classified_as_business_skill: regression guard for
  the _agent suffix misclassification bug

Code review (ce-code-review): 2 P1 + 5 P2 findings applied.
 2026-06-23 15:55:59 +08:00
chiguyong 47f3bfecfc feat(documents): add document processing capability (U1-U9) 
Implements end-to-end document generation, template filling, and reading:

- DocumentService: unified business layer for create/query/download
- Renderers: Word (Markdown->docx), Excel (Markdown/JSON->xlsx),
  PDF (Markdown->pdf with CJK font), Template (Jinja2 sandbox .docx fill)
- DocumentLoader: read PDF/Word/Excel/Markdown/HTML/text -> Document
- DocumentTool: Agent tool with action=create|read
- REST API: /api/v1/documents (create, upload-template, list, download)
- Frontend: DocumentPanel, DocumentCard, documents Pinia store,
  chat store tool_result detection
- Security: path traversal guard (Path.resolve + relative_to),
  SSTI guard (SandboxedEnvironment), API key auth, 50MB upload limit
- Bug fixes: template path traversal (400 not 500), TemplateRenderer
  lazy-load (no external registration dependency)
- Tests: 168 tests (unit + security + E2E F1/F2/F3 + bug hunt)
- Docs: README section 17, requirements + plan + test-plan docs

Requirements R1-R28 verified, F1-F3 user flows pass.
 2026-06-23 15:05:01 +08:00
chiguyong bc424574c7 fix: Tauri reload, multi-conv blocking, skill install, UI polish
Deploy to Production / deploy (push) Waiting to run Details 
1) Tauri reload login: main.ts beginStartup before mount + router guard await waitForStartup

2) Multi-conversation blocking: per-conversation Map/Set tracking

3) Skill install: npx --yes + SKILL.md support + path traversal validation

4) Markdown table rendering + 80ms streaming debounce

5) Agent execution UI: structured IStreamingStep

6) auth.py _ensure_db idempotent

Code review fixes: renderTimer cleanup, counter accumulation, memory leak, WS reconnect stale steps
 2026-06-23 11:03:46 +08:00
chiguyong 4f261523c2 fix(review): U3 atomic file writes for YAML + .env + skill config 
All config file writes now use the write-temp + fsync + os.replace
pattern (KTD-4) so a crash mid-write leaves the original file intact.

- Add src/agentkit/server/utils/atomic_write.py with write_text_atomic
- settings.py: _write_yaml_config and _write_env_var use atomic write
- skill_service.py: import_skill uses atomic write
- skill_service.py: update_skill_config uses atomic write + fcntl.flock
  around the read-modify-write cycle to serialize concurrent updates
- Add 11 unit tests covering happy path, crash safety, concurrency, errors
 2026-06-22 17:03:27 +08:00
chiguyong 698a8fafba fix(review): U7 refresh token hash verification on whoami 
The whoami route accepted rotated/old refresh tokens for cold-start
because it only checked session revocation status, not the token hash.
Now when token_type == "refresh", the route computes hash_token(token)
and compares it with the session's stored refresh_token_hash using
hmac.compare_digest (constant-time). Mismatch returns 401.

- Add SessionService.get_stored_refresh_hash(session_id) helper
- Add hash verification in whoami route (R9)
- Add TestWhoamiTokenHash with 5 integration tests
 2026-06-22 16:55:20 +08:00
chiguyong 206b82740c fix(review): U5 DB migration — always run init_auth_db + busy_timeout 2026-06-22 16:40:30 +08:00
chiguyong 45b6752e0d fix(review): U4 WebSocket quota enforcement + gateway-layer hardening 2026-06-22 16:37:23 +08:00
chiguyong 00c8386939 fix(review): U1 Redis quota enforcement — key construction + fail-closed + degradation recovery + async 2026-06-22 16:22:33 +08:00
chiguyong abe2a66436 fix(review): CLI field names, Pydantic validation, exception chaining 2026-06-22 15:24:31 +08:00
chiguyong 09feca3307 feat(admin): U7 — usage dashboard + quota enforcement 
UsageRecord extended with user_id + department_id (backward compatible).
UsageStore Protocol extended: record() accepts user_id/department_id,
get_usage() accepts filters, new get_usage_by_user/department methods.
RedisUsageStore uses versioned keys (v2) for new records.

LLMGateway.chat()/chat_stream() accept user_id, department_ids, db_path.
Quota check before provider call: model whitelist + token limit + cost
limit (daily). Multi-department uses strictest-wins (any exceed → reject).
QuotaExceededError → 429 at route layer.

UsageService: summary, timeseries, by-model, top-users, export (CSV/JSON).
5 new admin endpoints under /admin/usage/*.

llm_gateway.py routes pass DepartmentContext + db_path to gateway,
catch QuotaExceededError → 429 (JSON for /chat, SSE error for /stream).

84 new tests. 441 admin+usage tests pass, no regressions.
 2026-06-21 17:23:20 +08:00
chiguyong fd7f6816b8 feat(admin): U6 — Skill & KB management endpoints + department binding 
SkillService: enable/disable (persisted in skill_states table, schema
v4), import from YAML (with path traversal + name validation), reload
from file, update config. GET /skills now filters disabled skills.

KbService: list/upload/delete documents with department_id binding.
Added department_id field to KnowledgeSource + UploadedDocument.
Department visibility: (bound to user depts) ∪ (global = None).

10 new admin endpoints: skill enable/disable/import/reload/update,
KB documents CRUD, source sync/rebuild. All guarded by _require_admin.

Implemented reload stub in skill_management.py (was no-op).

54 new tests (26 unit + 28 integration). Fixed 4 pre-existing lint
errors. 357 admin tests pass, no regressions.
 2026-06-21 16:19:51 +08:00
chiguyong 980919fc95 feat(admin): U5 — LLM config admin endpoints + department quotas 
QuotaService: set/get/list/delete quotas, check_quota (hard reject),
is_model_allowed. JSON-serialized limit_value, upsert with ON CONFLICT.

LlmConfigService: provider CRUD + set_api_key + fallback management.
fcntl.flock file lock prevents concurrent YAML writes. Reuses
settings.py helpers (_read_yaml_config, _write_yaml_config,
_write_env_var, _mask_api_key).

11 new admin endpoints: provider CRUD, api-key, fallback CRUD,
department quotas CRUD. All guarded by _require_admin.

93 new tests (30 quota unit + 32 llm-config unit + 31 integration).
 2026-06-21 15:03:38 +08:00
chiguyong ad65f7a8d7 feat(admin): U1+U2+U4 — schema v3, department service, context filtering 
U1: Bump _SCHEMA_VERSION to 3, add 5 department tables (departments,
user_departments, department_skill_bindings, department_kb_bindings,
department_quotas) + 5 ORM models + helpers.

U2: DepartmentService (12 async methods: CRUD + bind/unbind skill/KB +
count_users). Mount admin_router in app.py. 36 unit + 28 integration tests.

U4: DepartmentContext FastAPI dependency (per-route, admin bypasses
filtering). filter_skills_by_department / filter_kb_sources_by_department
helpers. Applied to GET /skills and GET /kb-management/* routes.
15 integration tests for department isolation.

Also includes brainstorm + plan docs. 108 new tests, all pass.
 2026-06-21 15:03:27 +08:00
chiguyong 6dca9ba4f2 feat(admin): U3 — user CRUD + password reset + multi-department 
Add create_user method to LocalAuthProvider (bcrypt hash + INSERT,
raises ValueError on duplicate username/email).

Add UserService with 9 async methods: create/list/get/update/delete
(soft)/reset_password/assign_department/remove_department/list_user_departments. reset_password revokes all sessions via SessionService.
delete_user is soft (is_active=0, row preserved).

Add 9 user endpoints to routes/admin.py: POST/GET/PATCH/DELETE users,
reset-password, assign/remove department, list departments. All
guarded by _require_admin.

Tests: 40 unit + 37 integration = 77 new tests. Full admin suite
170 tests pass, no regressions.
 2026-06-21 13:45:42 +08:00
chiguyong 67c0d67262 fix(auth,chat): P0 security fixes + stop-generation button + doc sync 
U1: whoami cold-start security — add is_active check (disabled users
now get 401, not 200) and replace create_token_pair with create_access_token
to avoid minting a discarded refresh token (token-amplification risk).

U2: list_active_by_provider now filters expired sessions (expires_at > now)
matching its docstring promise; previously only checked revoked = 0.

U3: Fix asyncio.run() crash in test_revoke_other_user_session_returns_404
(converted to async). Add U1/U2 verification tests (disabled-user whoami,
no-refresh-leak, expired-session filtering, provider filtering) and
strengthen admin route tests (404 boundary, non-admin 403 on /admin/sessions).

U4: Update CLAUDE.md/AGENTS.md Request Flow — CostAwareRouter 3-layer
diagram replaced with actual RequestPreprocessor architecture (@board/@team
prefix intercepts then @skill: prefix then trivial-input regex then default
REACT). ExecutionMode list expanded to all 7 values.

U5: Frontend stop-generation button — ChatInput.vue shows a stop button
when isGenerating is true; chat store gains stopGeneration() that sends
{type:"cancel"} over WebSocket (backend portal.py already handles cancel).

Tests: 120 auth tests pass (unit + integration). ruff clean. vue-tsc clean.
 2026-06-21 11:36:58 +08:00
chiguyong aee7362665 feat(auth): U3/U4/U9 logout-others + whoami cold-start + admin UI + integration tests 2026-06-21 09:08:34 +08:00
chiguyong 9328451050 feat(auth): U7-U10 会话管理 UI + admin API + 测试修复 
- U7: 前端 ActiveSessionsPanel + ChangePasswordPanel 组件
- U8: 用户会话管理（查看/撤销/改密）集成到 SettingsView
- U9: 管理员会话管理 API + UserSessionsPanel + AdminApiClient
- U10: 认证中间件支持 sid 会话验证 + legacy client 兼容
- 修复 test_auth.py 测试夹具：注入 SessionService 单例绑定测试 DB
- 修复 wrong-password 断言大小写匹配
- ruff: 清理未使用导入
 2026-06-21 08:48:25 +08:00
TraeAI d245f2e3d8 fix: UI/UX 修复 + 暗色主题 + async generator 防御 
- App.vue: 重构 bootstrapBackend 流程，新增 retryBootstrap 重试入口
- SplashScreen.vue: 错误状态显示「重试」按钮
- system.py: /system/resources 移除 SYSTEM_CONFIG 权限依赖，避免 dev 模式 401
- react.py + gateway.py: 新增 _ensure_async_iterable helper 防御
  'async for requires aiter, got coroutine'
- theme.ts: Ant Design colorTextLightSolid 映射到 --text-inverse
  修复暗色主题下所有 primary 按钮白底白字
- ChatSidebar.vue: 新建对话按钮兜底深色文字
- SystemMonitorPanel.vue: 服务状态区域间距优化
- chat.ts + portal.py + sqlite_conversation_store.py: 会话标题派生修复
  解决点击对话标题变成"对话"的问题
- app.py: Serve 模式自动创建 default agent
- Tauri src-tauri/: 完整 Tauri 客户端配置 (icons, capabilities, Cargo)
 2026-06-20 23:35:57 +08:00
chiguyong cac9c73dd5 fix(routing): U1-U6 路由优化 + 修复方案 + 代码审查修复 
实现 6 个修复单元（U1-U6）并应用 ce-code-review 发现的 5 项安全修复。

## U1: benchmark 超时阈值
- 按 difficulty 分级超时：easy=45s, medium=60s, hard=90s
- 替换原单一 60s 硬编码

## U2: OpenAICompatibleProvider httpx 超时
- 新增 timeout 参数（默认 120s），替换硬编码 60s
- ProviderConfig.timeout 透传到 Provider
- 新增 2 项单元测试

## U3: 激活 QualityGate skill_match 校验
- BaseAgent._build_skill_context() 构造 skill_context
- 在 base.py / tasks.py / runner.py 三处传入 QualityGate.validate()

## U4: 添加 disambiguation_keywords 字段
- IntentConfig 新增 disambiguation_keywords 字段
- 8 个 skill YAML 补充该字段

## U5: 优化 RequestPreprocessor 路由正则
- 拆分 _FACTUAL_RE 为 CN/EN 双正则（中文无空格）
- 新增 _MATH_RE / _TRANSLATION_RE 纯模式
- _TOOL_CONTEXT_RE 排除需要工具的实时查询
- 多行输入守卫 + 结尾标点支持
- 新增 21 项单元测试（共 40 项全通过）

## U6: 重新基准测试
- 真实 LLM benchmark：准确率 60% -> 93.3%
- 4/5 通过，p50=40.8s，一致性=100%
- 旧基线备份至 baseline_2026-06-17_old_arch.json

## ce-code-review 修复（5 项）
- 修复 \s 字符类匹配换行符的安全隐患
- 添加事实/数学正则的结尾标点支持
- 修复 geo_optimizer.yaml 关键词重复
- 修复 _login_with_retry 不可达 return
- 修复 real_llm_server fixture stderr_fh 资源泄漏

测试：tests/unit/chat/ 63 项全通过，ruff 检查通过。
 2026-06-20 19:31:49 +08:00
chiguyong 91f56ca663 feat: 企业级客户端-服务端架构 + 代码审查修复 
## 主要变更

### 新增功能
- 企业级客户端-服务端架构（JWT 认证 + RBAC 权限 + 终端安全）
- Tauri 桌面客户端与服务端配置同步
- 远程 LLM 网关（RemoteLLMProvider，支持 401 token 刷新重试）
- 服务端终端 WebSocket（带管理员审批流程）
- 终端白名单六层防御（黑名单 → shell 操作符检测 → 内置安全 → 全局/用户/会话白名单 → 危险检测）

### 代码审查修复（P0/P1/P2）
- P0: 危险二进制（rm/docker 等）不再加入白名单，compute_whitelist_entry 返回 None
- P1: 终端审批所有权追踪（_approval_owners dict）+ 会话清理防泄漏
- P1: 本地终端 WebSocket URL 补齐 JWT token
- P1: 审计日志支持 terminal_mode 过滤
- P1: /system/resources 端点强制 SYSTEM_CONFIG 权限
- P1: RemoteLLMProvider 增加 401 token 刷新重试机制
- P1: auth/models.py 使用 Mapping[str, object] 替代 Any 类型
- P2: 终端授权依赖检查 is_active 账户状态
- 修复 app.py 未使用的 APIKeyAuthMiddleware 导入

### 文档更新
- README.md: 新增第 16 章「企业级客户端-服务端架构」
- AGENTS.md / CLAUDE.md: 同步模块映射、路由表、前端页面
- 计划文档标记为 completed

Closes: docs/plans/2026-06-19-003-feat-enterprise-client-server-evolution-plan.md
 2026-06-20 06:48:18 +08:00
chiguyong b4ba65b9ca fix(gui): 修复启动报错和对话列表不正确的两个关键Bug 
Bug1: 'async for' requires __aiter__ method, got coroutine
- EventQueue.subscribe() 在 _closed=True 时直接 return，
  Python 将其视为协程而非异步生成器
- 修复: 添加不可达的 yield 语句，确保函数始终为异步生成器

Bug2: 启动时对话列表全显示"对话"，无法识别之前的对话
- list_conversations() 不加载消息，_derive_conversation_title
  遍历空 messages 列表导致标题全为"对话"
- 修复: list_conversations 从 SQLite 加载首条用户消息用于标题推导

Bug2b: WebSocket 不响应前端对话切换
- conv 变量只在首条消息时设置，之后忽略 conversation_id
- 修复: 每条消息都检查 conversation_id，切换时更新 conv
 2026-06-18 16:26:02 +08:00
chiguyong 771756814f fix(review): 修复代码审查发现的 P0/P1/P2 问题 
P0 (Critical):
- orchestrator: plan_update 事件 key 从 phases 改为 plan_phases 匹配前端契约
- orchestrator: team_formed 事件 payload 从 string[] 改为 IExpertInfo[] + plan_phases:[]

P1 (High):
- orchestrator: 新增 phase_failed 事件广播 (3处: gather 失败/_execute_phase 异常/_mark_dependents_failed 级联)
- orchestrator: 新增 team_dissolved 事件广播 (3处: 正常完成/ValueError/Exception)
- orchestrator: _mark_dependents_failed 改为 async 以支持事件广播
- orchestrator: gather 结果检查增加 asyncio.CancelledError (Python 3.11+ BaseException)
- plan: PhaseStatus.RUNNING 值从 running 改为 in_progress 匹配前端联合类型
- team.ts: updatePhaseStatus 增加 plan_phases undefined 防御守卫
- chat.py: 增加 asyncio.CancelledError 处理 + team.dissolve() 移入 finally 块

P2 (Medium):
- orchestrator: _get_isolated_agent 返回类型 Any 改为 ConfigDrivenAgent
- orchestrator: _get_llm_gateway 返回类型 Any 改为 LLMGateway | None
- orchestrator: 依赖输出从 SharedWorkspace 读取改为内存 dep_phase.result (减少冗余 I/O)
- plan: PlanPhase.to_dict() result 序列化为 string 匹配前端 ITeamPlanPhase.result 类型
- types.ts: expert_step.step 类型从 number 改为 string (后端发送 phase ID)

Tests: 377 passed (experts + chat_team + expert_team)
 2026-06-18 13:00:59 +08:00
chiguyong 1e818b507d feat(server): U6 新增 _execute_team_collab 集成 @team 流水线到 WebSocket 2026-06-18 02:08:29 +08:00
chiguyong dddcbd24e3 feat: 私董会讨论模式 + 回测集成 + WS持久化修复 
私董会讨论模式 (Board Meeting Mode):
- BoardRouter: @board 前缀路由, 专家名验证, 模板回退
- BoardTeam: 讨论容器, 状态机 (FORMING->DISCUSSING->CONCLUDING->COMPLETED)
- BoardOrchestrator: 多轮自主循环讨论引擎, 主持人小结, 停止命令检测
- 9个预设名人专家 YAML (马斯克/贝佐斯/张小龙/芒格等)
- 前端 BoardStatusView 群聊式 UI + WebSocket 事件处理
- 后端 chat.py 集成 @board 路由到主聊天流程

回测集成:
- benchmark.py: 新增 board_meeting 维度 (18 tasks, 6 categories)
- benchmark_dataset.py: 新增 BOARD_BENCHMARKS (11 E2E cases)
- test_board_backtest.py: 66 个回测测试 (9 test classes)

Bug 修复:
- resolve_expert_configs: deep-copy 防止 is_lead 修改污染共享模板
- 所有专家名无效时回退到默认模板
- board_router: 非匹配路径 topic 未 strip
- benchmark_dataset: board-name-invalid-001 输入修正

WebSocket 持久化修复:
- chat.py: 三层防御机制确保任务结果不丢失
- chat store: 断线恢复逻辑

部署配置:
- Gitea Actions CI/CD workflow
- docker-compose.deploy.yaml 部署编排
- scripts/deploy.sh 自动化部署脚本

测试结果: 120 单元测试通过, 71 benchmark 测试 100% 通过, ruff 全部通过
 2026-06-17 23:52:53 +08:00
chiguyong 5b5291c7e5 fix: WebSocket task persistence three-layer defense with security hardening 
Fix chat history empty content and task stops on refresh. Implements: result persistence on disconnect, task backgrounding via asyncio + EventQueue, frontend reconnection recovery. Security: fail-closed conversation_id ownership, asyncio.shield on CancelledError cleanup, async TaskStore shim, EventQueue subscriber limit, connection error resilience. 23 tests added.
 2026-06-17 22:11:51 +08:00
chiguyong ecf87391a5 feat: integrate SQ/EQ into portal WebSocket and CLI (Phase 4) 
- app.py: initialize EventQueue + SubmissionQueue in app.state, close on shutdown
- portal.py: emit unified events (task.created/started/completed/failed,
  turn.thinking/tool_call/tool_result/final_answer) to EQ alongside WebSocket messages
- cli/chat.py: optional --event-queue flag for event emission
- EQ is bypass-only: emit failures never affect WebSocket or CLI main flow
- WebSocket message format unchanged (backward compatible)

Tests: 650 passed, 0 failed, 4 skipped
 2026-06-17 11:05:04 +08:00
chiguyong 773a62ead2 refactor: remove IntentRouter from tasks.py, delete legacy ConversationStore 
- tasks.py: replace IntentRouter.route() with default agent fallback (REACT mode)
- app.py: remove IntentRouter import and initialization
- portal.py: delete legacy in-memory ConversationStore class (~120 lines),
  SqliteConversationStore is the sole implementation now
- Remove unused SessionManager import from portal.py

Tests: 622 passed, 0 failed
 2026-06-17 10:50:41 +08:00
chiguyong 200174c5c7 feat: SQLite persistence, verification loop, spec-driven execution 
Phase 2 of architecture optimization (U5/U6/U9):

- U5: SqliteConversationStore with WAL mode + LRU cache (1000 convs)
  Replaces in-memory ConversationStore in portal.py
  Data survives server restarts (ref: Codex Thread persistence)
- U6: VerificationLoop with verify/verify_and_retry
  Default commands: pytest + ruff check
  ReActEngine integration via verification_enabled flag
  New run_tests tool for LLM to invoke verification
- U9: SpecManager for plan-as-contract (ref: Qoder Quest Mode)
  Plans persisted to .agentkit/specs/{spec_id}.yaml
  API: GET/PUT /api/v1/specs, POST /api/v1/specs/{id}/confirm
  PlanExecEngine emits spec_created event after plan generation

Also fixes: portal skill_name routing, app.py SessionManager guard,
test_telemetry CostAwareRouter removal, test_compression_config fixture
 2026-06-17 10:45:20 +08:00