2 Commits

Author	SHA1	Message	Date
Chiguyong	e35b2289ce	fix(bitable): cross-table recalc + merge_record_values + SSRF tests ... Fixes 4 bugs blocking AE1 acceptance scenario (rollup returns 0): 1. Decimal not recognized as numeric: PostgreSQL JSONB returns numbers as Decimal; _evaluate_rollup's isinstance(v, (int,float)) missed them. Added _to_float() helper (also excludes bool, an int subclass). Applied to _evaluate_lookup for JSON serialization safety. 2. add_relation_link clobbers record values: update_record_values does full-replace, so setting the relation field wiped amount/etc. Added merge_record_values (|| operator) and switched all relation link mutations to use it (add + remove + reverse cleanup). 3. enqueue_recalc ON CONFLICT DO NOTHING swallowed re-enqueues: once a task reached 'done', the unique constraint silently dropped new requests for the same (record_id, field_id) — field showed stale data forever. Changed to ON CONFLICT DO UPDATE: reset done/error tasks to pending; leave pending/calculating as-is (no double-process). 4. No cross-table recalc on record update: update_record_values only triggered recalc on the record's own table; rollup on the target table never updated when source values changed. Added _trigger_cross_table_recalc() — finds relation fields on the record's table, triggers recalc on linked target records. Tests: - AE1 acceptance: 3 e2e tests (SUM on insert, update-triggered recalc, AVG aggregation) — all pass against real PG. - Bidirectional relation deletion: verifies _cleanup_reverse_link removes source from target's reverse field. - SSRF guard: 3 tests blocking loopback, private IP, non-HTTP scheme. - Updated test_recalc_deduplication for new ON CONFLICT DO UPDATE semantics (returns existing task, not None). - Fixed test_crash_recovery to use stale_threshold=0 (was always broken with default 600s threshold).	2026-07-06 13:29:54 +08:00
chiguyong	bbbf9cd40a	feat(bitable): add bitable companion service with full P0-P2 fixes ... Bitable is a multi-dimensional table companion service that runs alongside the main AgentKit server. It provides structured data storage with formula fields, views, and ingestion pipelines. Major components: - Domain models (Pydantic v2): Table, Field, Record, View, RecalcTask - SQLAlchemy 2 async ORM with independent bitable PostgreSQL schema - Formula engine: AST parser, DAG, Kahn topological sort, safe eval - RecalcWorker: atomic task claiming (FOR UPDATE SKIP LOCKED), topo-order processing, stale-threshold reaper for crash recovery - REST API (/api/v1/bitable): tables, fields, records, views, files - BitableTool: agent-facing tool with batch chunking (500/batch) - CLI: agentkit bitable subcommands (create, list, import-excel, etc.) - Frontend: Vue 3 + vxe-table grid with field management, views, filters - Ingestion: Excel (openpyxl), database reflection, API collector Security fixes (ce-code-review P0 + ce-debug P1): - SQL injection prevention (field_id validation, parameterized queries) - IDOR protection (_check_table_ownership on all table-level endpoints) - SSRF prevention (URL scheme + private IP validation in parse_excel_url) - OOM prevention (streaming file upload, batch delete, batch insert) - Atomic recalc task claiming (FOR UPDATE SKIP LOCKED) - Formula engine cache invalidation on field changes - Composite cursor pagination for non-id sort orders - Batch upsert (eliminates N+1 queries) - Sync I/O offloaded to thread pool in async contexts - Internal token auth (X-Internal-Token, hmac.compare_digest) - PK unique index enforcement Test coverage: 88 unit tests (95 skipped without Docker)	2026-06-25 01:09:59 +08:00