Chiguyong
|
6d5a34ada2
|
feat(bitable): formula relations v2 — U1-U8 + review fixes
Test / backend-test (pull_request) Waiting to run
Details
Test / frontend-unit (pull_request) Waiting to run
Details
Test / api-e2e (pull_request) Waiting to run
Details
Test / frontend-e2e (pull_request) Waiting to run
Details
Implements bitable v2: formula engine expansion (10->64 functions),
cross-table formula references, relation CRUD (1:1/1:N/N:N + bidirectional),
lookup/rollup fields, cross-table dependency graph, automation engine
(3 triggers + 5 actions + retry + webhook), and REST API + BitableTool actions.
Implementation units:
- U1: Schema V3 migration (junction table, automations, cross_table_deps)
- U2: Formula functions 10->64 (datetime/text/logical/aggregate/lookup)
- U3: Cross-table formula {rel.target} syntax (parser 3-tuple, engine resolve)
- U4: Relation CRUD + bidirectional + self-reference
- U5: Lookup & Rollup field types with recalc worker routing
- U6: Cross-table dependency graph + recalc trigger
- U7: Automation engine (asyncio queue, 3x retry, execution logs)
- U8: REST API (11 endpoints) + BitableTool (17 actions)
Review fixes applied (ce-code-review mode:agent):
- P1: SSRF guard on outbound webhook (reuse _assert_safe_host from ingestion)
- P1: Bidirectional relation removal now cleans reverse field (symmetric with add)
- P2: Ownership check on target_table_id in relation field creation (IDOR fix)
- P2: Typed AutomationUpdate Pydantic model (prevents table_id injection)
- P3: Tracked automation fire-and-forget tasks (error visibility)
- P3: Public FormulaEngine.get_cross_table_mapping (remove private attr access)
Tests: 267 passed, 153 skipped (PG). Ruff clean.
Deferred: generate_formula/generate_automation LLM actions, frontend components.
|
2026-07-06 12:57:07 +08:00 |
chiguyong
|
bbbf9cd40a
|
feat(bitable): add bitable companion service with full P0-P2 fixes
Bitable is a multi-dimensional table companion service that runs alongside
the main AgentKit server. It provides structured data storage with formula
fields, views, and ingestion pipelines.
Major components:
- Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask
- SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema
- Formula engine: AST parser, DAG, Kahn topological sort, safe eval
- RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order
processing, stale-threshold reaper for crash recovery
- REST API (/api/v1/bitable): tables, fields, records, views, files
- BitableTool: agent-facing tool with batch chunking (500/batch)
- CLI: agentkit bitable subcommands (create, list, import-excel, etc.)
- Frontend: Vue 3 + vxe-table grid with field management, views, filters
- Ingestion: Excel (openpyxl), database reflection, API collector
Security fixes (ce-code-review P0 + ce-debug P1):
- SQL injection prevention (field_id validation, parameterized queries)
- IDOR protection (_check_table_ownership on all table-level endpoints)
- SSRF prevention (URL scheme + private IP validation in parse_excel_url)
- OOM prevention (streaming file upload, batch delete, batch insert)
- Atomic recalc task claiming (FOR UPDATE SKIP LOCKED)
- Formula engine cache invalidation on field changes
- Composite cursor pagination for non-id sort orders
- Batch upsert (eliminates N+1 queries)
- Sync I/O offloaded to thread pool in async contexts
- Internal token auth (X-Internal-Token, hmac.compare_digest)
- PK unique index enforcement
Test coverage: 88 unit tests (95 skipped without Docker)
|
2026-06-25 01:09:59 +08:00 |
Fischer