9 Commits

Author	SHA1	Message	Date
Chiguyong	e35b2289ce	fix(bitable): cross-table recalc + merge_record_values + SSRF tests ... Fixes 4 bugs blocking AE1 acceptance scenario (rollup returns 0): 1. Decimal not recognized as numeric: PostgreSQL JSONB returns numbers as Decimal; _evaluate_rollup's isinstance(v, (int,float)) missed them. Added _to_float() helper (also excludes bool, an int subclass). Applied to _evaluate_lookup for JSON serialization safety. 2. add_relation_link clobbers record values: update_record_values does full-replace, so setting the relation field wiped amount/etc. Added merge_record_values (|| operator) and switched all relation link mutations to use it (add + remove + reverse cleanup). 3. enqueue_recalc ON CONFLICT DO NOTHING swallowed re-enqueues: once a task reached 'done', the unique constraint silently dropped new requests for the same (record_id, field_id) — field showed stale data forever. Changed to ON CONFLICT DO UPDATE: reset done/error tasks to pending; leave pending/calculating as-is (no double-process). 4. No cross-table recalc on record update: update_record_values only triggered recalc on the record's own table; rollup on the target table never updated when source values changed. Added _trigger_cross_table_recalc() — finds relation fields on the record's table, triggers recalc on linked target records. Tests: - AE1 acceptance: 3 e2e tests (SUM on insert, update-triggered recalc, AVG aggregation) — all pass against real PG. - Bidirectional relation deletion: verifies _cleanup_reverse_link removes source from target's reverse field. - SSRF guard: 3 tests blocking loopback, private IP, non-HTTP scheme. - Updated test_recalc_deduplication for new ON CONFLICT DO UPDATE semantics (returns existing task, not None). - Fixed test_crash_recovery to use stale_threshold=0 (was always broken with default 600s threshold).	2026-07-06 13:29:54 +08:00
Chiguyong	fae76b7cca	merge: resolve conflict with origin/main (PR #24/#25) ... Conflict in src/agentkit/bitable/service.py delete_view(): - Adopted main's MAINT-2 fix (delete_view_if_not_last atomic op + race check) - Worktree's simpler version replaced by main's race-safe implementation No conflict markers remain. 277 tests pass, ruff clean.	2026-07-06 13:04:53 +08:00
Chiguyong	6d5a34ada2	feat(bitable): formula relations v2 — U1-U8 + review fixes ... Implements bitable v2: formula engine expansion (10->64 functions), cross-table formula references, relation CRUD (1:1/1:N/N:N + bidirectional), lookup/rollup fields, cross-table dependency graph, automation engine (3 triggers + 5 actions + retry + webhook), and REST API + BitableTool actions. Implementation units: - U1: Schema V3 migration (junction table, automations, cross_table_deps) - U2: Formula functions 10->64 (datetime/text/logical/aggregate/lookup) - U3: Cross-table formula {rel.target} syntax (parser 3-tuple, engine resolve) - U4: Relation CRUD + bidirectional + self-reference - U5: Lookup & Rollup field types with recalc worker routing - U6: Cross-table dependency graph + recalc trigger - U7: Automation engine (asyncio queue, 3x retry, execution logs) - U8: REST API (11 endpoints) + BitableTool (17 actions) Review fixes applied (ce-code-review mode:agent): - P1: SSRF guard on outbound webhook (reuse _assert_safe_host from ingestion) - P1: Bidirectional relation removal now cleans reverse field (symmetric with add) - P2: Ownership check on target_table_id in relation field creation (IDOR fix) - P2: Typed AutomationUpdate Pydantic model (prevents table_id injection) - P3: Tracked automation fire-and-forget tasks (error visibility) - P3: Public FormulaEngine.get_cross_table_mapping (remove private attr access) Tests: 267 passed, 153 skipped (PG). Ruff clean. Deferred: generate_formula/generate_automation LLM actions, frontend components.	2026-07-06 12:57:07 +08:00
Chiguyong	c60e96ac32	fix(bitable/team): U3 — Residual P2 findings (DR-1/3/4/5 + #2/#3) ... U3 修复 8 项 P2 design review 与 streaming 残留问题： DR-1 SQL 注入防御 — repository.py 添加 _validate_field_id UUID 校验， 拒绝非 UUID 字符串进入 jsonb_set path 与 WHERE 子句的字符串插值。 ceiling 注释标明仅覆盖 field_id，其余 SQL 结构由 service 层枚举白名单守护。 DR-3 design token 契约 — 新增 bitable-tokens.test.ts (31 tests)， 解析 CSS :root 块并断言所有 --bitable-* token 在 :root 定义， 防止 token 被硬编码到组件外或遗漏定义。 DR-4 TOCTOU 修复 — repository.delete_view_if_not_last 用 SELECT ... FOR UPDATE 单事务原子化 check + delete， 消除 list_views → count → delete 三步分离的竞态。 service.delete_view 委托原子方法，失败抛 LastViewDeletionError。 DR-5 _update_field 静默失败 — type 参数被 Pydantic extra=ignore 静默丢弃，用户以为改了类型实际无效。显式返回 UNSUPPORTED_FIELD_TYPE_CHANGE 错误，避免静默失败的 UX 陷阱。 #2 team_synthesis 孤儿 milestone — 补 error + cancelled 路径测试， 验证内层 except 广播终结事件后 re-raise，外层 except 捕获后 fallback。 #3 synthesis_id 去重附身 — chatStream.ts 移除 || m.synthesis_id === undefined 兜底匹配，避免新 milestone 附身到上一次孤儿 streaming 占位。 测试：DR-1 (7) + DR-4 repo (3) + DR-4 service (3) + DR-5 (2) + #2 (2) + #8 frontend (31) = 48 项通过；U2 无回归 (29 项)。	2026-07-06 03:48:14 +08:00
chiguyong	229dc0b2f3	feat(bitable): U6 R15a BitableTool 4 new actions + DELETE /views endpoint ... Extend BitableTool from 6 to 10 actions (create_view, update_view, update_field, delete_view) and add the DELETE /views/{view_id} backend endpoint with 404-before-403 ownership, 409 last-view protection, and X-Internal-Token passthrough (KTD11). Backend: - repository.py: add delete_view() — DELETE row by view_id, returns rowcount > 0 - service.py: add LastViewDeletionError domain exception + delete_view() with last-view guard (siblings <= 1 → raise → route maps to 409) - routes/bitable.py: add DELETE /views/{view_id} (204 No Content), 404-before-403 ownership pattern, 409 on LastViewDeletionError, X-Internal-Token passthrough via require_bitable_auth - tools/bitable_tool.py: add 4 new actions (_create_view, _update_view, _update_field, _delete_view), register in BOTH handlers dict AND input_schema.action.enum (KTD10 — 10 actions each) Frontend: - api/bitable.ts: add deleteView(viewId): Promise<void> - stores/bitable.ts: add deleteView action — removes from local state, switches to first remaining view if active was deleted, 409 warning - ViewSwitcher.vue: add delete button (a-popconfirm "确认删除此视图？"), hidden when views.length <= 1 (preempt last-view 409) - BitableFileDetailView.vue: handle @delete event from ViewSwitcher Tests: - test_routes.py: 6 new DELETE /views tests (204, 404 missing, 404 non-owner, 409 last-view, internal-token passthrough, internal-token 404) - test_bitable_tool.py: 13 new tests (action count = 10, handlers = 10, 4 action happy paths, missing-field errors, 409 last-view, R3/R4 config parity, X-Internal-Token passthrough on all 4 new actions) - e2e/bitable-agent-parity.spec.ts: 10 scenarios (P1-P10) covering delete button visibility, popconfirm, 204/409/404 flows, tab removal, view switch after delete, create view adds tab Verification: - ruff check: all files pass - pytest: 62 passed, 12 pre-existing failures (unchanged from e931fbe baseline) - typecheck: pass (EXIT_CODE=0) - build:frontend: pass (BUILD_EXIT=0) - action count: ENUM=10, HANDLERS=10, delete_view in both - no blue hex colors in ViewSwitcher.vue Pre-existing test failures (12, unchanged from e931fbe): test_create_table_success, test_create_field_success, test_list_fields, test_create_records_batch, test_upsert_inserts_then_updates, test_upsert_preserves_user_columns, test_create_view_success, test_batch_upsert_1200_records, test_resume_from_partial_failure, test_query_records, test_query_records_with_limit, test_collect_api Constraints honored: - No emojis, no `any` type, no blue hex colors, no pyproject.toml changes - 404-before-403 for non-owned resources (Pattern 4) - X-Internal-Token transparent passthrough (KTD11) - KTD10: actions registered in both handlers dict AND enum	2026-07-03 23:13:46 +08:00
chiguyong	e931fbef2d	feat(bitable): U5 R4 grouping (max 3 fields) + conditional formatting (7 operators) ... - GroupingEditor: multi-select field picker (max 3), per-level direction toggle, reorder buttons, "已知限制：不支持跨分组多选" note, empty state - ConditionalFormatEditor: per-rule enable/field/operator/value/color/bold, 8 color keys, WCAG 1.4.1 bold default true, first-match-wins footer legend - BitableGrid: unified section rendering (grouped/ungrouped via single vxe-grid declaration), group headers as separate divs (CF only on data cells), CF via row-config.className, multi-grid instance map for refresh - groupingRulesUtils: pure functions for CF matching (7 operators), group tree builder, SUM/AVG aggregation, CSS var mappers, self-check on load - view_config.py: Pydantic v2 validation (MAX_GROUP_BY_FIELDS=3, 7 operators, 8 color keys, extra="forbid" on sub-models) - routes/bitable.py: validate_view_config on PATCH (HTTP 422 on error) - stores/bitable.ts: updateViewConfig action (merges U5 sub-keys, preserves filters/sort/hidden_fields) - ViewConfigPanel: grouping + conditional-format tabs - E2E: 8 scenarios (G1-G8: single/multi grouping, collapse/expand, CF equals/between, combined, aggregation) - Tests: 54 unit tests (19 grouping + 35 CF), 2 PG-marked skipped	2026-07-03 22:33:18 +08:00
chiguyong	78a7faa17b	refactor: remove all emoji from agentkit ... Replace emoji across codebase: YAML avatars -> first char, frontend banners -> Ant Design Vue components, CLI status -> OK/FAIL/WARN labels, terminal -> [WARN]/[OK]/[PENDING], Bitable DB default -> table, App.vue font cleanup, test fixtures -> first char letters. shell.avatar type upgraded to string | Component.	2026-07-02 01:33:28 +08:00
chiguyong	a6e1bf5884	feat(bitable): 多维表格文件层 + 默认字段 + 表内字段操作 + ce-code-review 修复 (Stage 1) ... 实现多维表格 UI 完整性 Stage 1（U1-U6），补齐飞书/twenty 对齐缺失的文件层、 默认字段与表内字段操作能力，并修复 ce-code-review 走查发现的 P0/P1 级问题。 后端（U1-U2）: - 新增 BitableFile 实体（models/db/repository/service/routes），三级层级：文件→数据表→字段/记录 - Schema V2 迁移：bitable_files 表 + tables.file_id 列，幂等（IF NOT EXISTS），保留 V1 孤儿表 - 新建数据表自动创建 5 个默认字段（标题/状态/日期/创建人/创建时间） - agent-owned 字段在 create_record 时自动填充（按 type+owner 匹配，传 actor_user_id） - 7 个文件 REST 端点 + IDOR ownership 检查（404-before-403，internal token 旁路） 前端（U3-U5）: - 文件列表页（FileCard 网格 + 新建/重命名/删除）+ 文件详情页（侧栏表格列表 + vxe-table 网格） - Vue Router 嵌套路由 /bitable → /bitable/:fileId → /bitable/:fileId/:tableId - 列头菜单（编辑/隐藏/删除字段）+ 末尾 + 列新增字段 - select/multiselect 字段自定义单元格编辑器 + Tag 展示 - Pinia store 扩展 file 状态与动作，深链直访回退 getFile，fileId 切换 watch 测试（U6）: - 文件 CRUD（12 例）+ 默认字段（10 例）单元测试 - 3 个 E2E spec（视图加载、文件流、字段操作），后端不可用时优雅跳过 ce-code-review 修复（P0/P1）: - P0 路由冲突：GET /files/{file_id} 遮蔽下载端点 → 下载改 /uploads/{filename} - P0 IDOR：update/delete field/record/view 五端点补 ownership 检查 - P1 is_initialized property 缺失致二次初始化崩溃 - P1 直接 URL 导航失效（files 数组为空）→ selectFile 回退 getFile - P1 fileId 切换不重载 → 增加 watch - P1 轮询丢弃最终公式值（wasCalculating 守卫）+ 复用视图 filters - P1 测试断言 200→201；test_db 无 URL 用例解除 postgres 标记得以执行 - P2 _check_table_ownership 403→404；输入长度校验；upload field-table 一致性校验 - P2 multiselect 浅比较 → 深比较；E2E bitable-view 补 waitForServer 守卫 验证：ruff check 通过；pytest 91 passed/116 skipped；vue-tsc --noEmit 通过。	2026-06-29 04:07:45 +08:00
chiguyong	bbbf9cd40a	feat(bitable): add bitable companion service with full P0-P2 fixes ... Bitable is a multi-dimensional table companion service that runs alongside the main AgentKit server. It provides structured data storage with formula fields, views, and ingestion pipelines. Major components: - Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask - SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema - Formula engine: AST parser, DAG, Kahn topological sort, safe eval - RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order processing, stale-threshold reaper for crash recovery - REST API (/api/v1/bitable): tables, fields, records, views, files - BitableTool: agent-facing tool with batch chunking (500/batch) - CLI: agentkit bitable subcommands (create, list, import-excel, etc.) - Frontend: Vue 3 + vxe-table grid with field management, views, filters - Ingestion: Excel (openpyxl), database reflection, API collector Security fixes (ce-code-review P0 + ce-debug P1): - SQL injection prevention (field_id validation, parameterized queries) - IDOR protection (_check_table_ownership on all table-level endpoints) - SSRF prevention (URL scheme + private IP validation in parse_excel_url) - OOM prevention (streaming file upload, batch delete, batch insert) - Atomic recalc task claiming (FOR UPDATE SKIP LOCKED) - Formula engine cache invalidation on field changes - Composite cursor pagination for non-id sort orders - Batch upsert (eliminates N+1 queries) - Sync I/O offloaded to thread pool in async contexts - Internal token auth (X-Internal-Token, hmac.compare_digest) - PK unique index enforcement Test coverage: 88 unit tests (95 skipped without Docker)	2026-06-25 01:09:59 +08:00